Reading the number in the image
Statement
A two digit number is written in the middle of the image. You have to print only that number.
This category is where the model does best. Measured, it got 15 out of 16 right even with a bare question. The hard part is not the reading, it is the format. When 42 is written in the image, this is how the output splits.
| What the model printed | Grading |
|---|---|
42 | pass |
42. | fail |
The number shown is 42. | fail |
Giving it only the start of a sentence worked well. Throw a question at the model and it tries to answer. Give it the start of a sentence and it has nothing to do but carry on.
The input to this problem is an image, not text. One image goes into the model per case, and the text you write goes in on the same turn as that image. There is no need to describe the image in words. The model is already looking at it.
Input
One image with a two digit number written on it
The input arrives as one line.
Output
That number as digits. No other characters
Not a single character of the output may differ from the expected value.
Requirements
- At most 16 tokens
- There are 16 cases
- The input is an image. It is not text
- This is chat mode. Your text is wrapped in the chat template, so the model acts as an assistant. You can write the control tokens yourself and build a conversation of several turns.
- The output has to match the expected value down to the character. Only leading and trailing whitespace is ignored.
- There are 16 cases, and 5 of them show the input together with the expected output. The rest show the input only.
- Model output is accepted up to 16 tokens.
- You have to pass all 16 cases to solve this.
Examples
5 public cases. Judging also uses hidden cases. You have to pass all 16 cases to solve this.

42

17

93

58

26
Notes
Background that helps with solving. The rules you have to follow are in the input and output specs and the requirements above.
The image is a 448 pixel square. It holds nothing but a single black number on a white background, so the model has no trouble reading it. The hard part is the format.