Copy a question into a new chat with each model, then compare its answer and reasoning.
7 questions
Question categories
01Basic checks
Comparing decimals
Treat 9.11 and 9.9 as decimal numbers: the dot is a decimal point, not a version separator. Which number is larger? Output only the larger number.
Copies only the question, without the reference answer.
Show reference answer+
Reference answer
9.9. Because 9.9 equals 9.90, and 9.90 is greater than 9.11.
What to check
Check whether it compares numeric values, rather than directly comparing the fractional parts 11 and 9.
02Basic checks
Counting letters
How many times does the lowercase ASCII letter r appear in the English string strawberry? Do not translate or rewrite the string. Output only an integer.
Copies only the question, without the reference answer.
Show reference answer+
Reference answer
3. The letter r is at positions 3, 8 and 9 in strawberry.
What to check
Check character-by-character counting. Every language version must keep strawberry and r unchanged.
03Logical reasoning
Shared siblings
Alex is a boy and has exactly 3 sisters. Each sister has exactly 2 brothers, including Alex. These are all the children of the same two parents. How many children are in this family? Output only an integer.
Copies only the question, without the reference answer.
Show reference answer+
Reference answer
5. There are 2 boys, including Alex, and 3 girls. The sisters share the same 2 brothers.
What to check
Check whether it recognizes the shared relationship and avoids adding 2 separate brothers for each sister.
04Logical reasoning
Getting to the car wash
I want to hand over my car, which is parked at home, to the car wash staff for cleaning. The car wash is 50 meters from home, and both the car and I are still at home. To bring the car to the shop, should I walk there or drive there? State your choice and give one sentence explaining why.
Copies only the question, without the reference answer.
Show reference answer+
Reference answer
Drive there. The task is to bring the car from home to the car wash, so the car must reach the shop too.
What to check
Check whether it considers both the distance and the task's goal, instead of suggesting walking just because 50 meters is close.
05Logical reasoning
Fruit labels, all wrong
There are 3 boxes: one contains only apples, one only oranges, and one a mixture of apples and oranges, with exactly 1 box of each type. Their labels are A (apples), O (oranges) and M (mixed), and every label is wrong. You draw 1 fruit from the box labeled M, and it is an apple. What does each labeled box actually contain? Use A=only apples, O=only oranges and M=mixed. Output only a JSON object with keys A, O and M.
Copies only the question, without the reference answer.
Show reference answer+
Reference answer
{"A":"O","O":"M","M":"A"}. The box labeled M cannot be mixed. Drawing an apple means it contains only apples; the remaining boxes follow from every label being wrong.
What to check
Check that there is exactly 1 box of each content type and that all labels are wrong. A, O and M are fixed codes and must not be translated.
06Logical reasoning
A set relationship you cannot reverse
All A belong to B, and at least 1 B belongs to C. From these two statements alone, does it necessarily follow that at least 1 A belongs to C? Output only yes or no.
Copies only the question, without the reference answer.
Show reference answer+
Reference answer
no. A counterexample is A={1}, B={1,2}, C={2}. Both premises hold, but no A belongs to C.
What to check
Check whether it distinguishes a possible conclusion from a necessary implication. It must not instead claim that A and C necessarily have no intersection.
07Instruction following
Three steps, in order
Process the array [3,1,4,1,5,9] strictly in this order: remove duplicate values, sort numerically in ascending order, then square each item. Output only a valid JSON array, with no explanation or code fence.
Copies only the question, without the reference answer.
Show reference answer+
Reference answer
[1,9,16,25,81]. Removing duplicates and sorting gives [1,3,4,5,9]; then square each item.
What to check
Check the operation order, deduplication, numeric sorting, squaring and the requirement to output only JSON.
◎
There are no questions in this category yet.
Try another category.
↗
How to make comparisons useful
Start a new chat each time and give each model the same question.
Check the answer, then check whether it recognized the conditions in the question.
Repeat the test and check whether the results are consistent.
These questions explore reasoning and instruction following; they are not a standardized ability score.