The last lesson. The problem removes the spaces from a string with spaces in it. Given c o a x, the answer is coax. Nothing a model cannot do. Yet the pass rate is lower than the problems before it.
There is one reason. Judging on this site allows not one character of difference from the expected output. Only leading and trailing spaces are forgiven. Everything else counts as wrong.
The four that catch people
When the answer is right and it still fails, it is usually one of these.
It sticks something on the end. After
coaxit carries on with입니다or with the next example. Since raw mode looks only as far as the first line, something attached on a new line is fine. Attached on the same line is the problem.It wraps it in quotes.
"coax"is wrong. If the examples have quotes, the model uses quotes.A space is left in the middle. In this problem that is fatal. When you build the examples, leave not a single space.
It is cut off at 16 tokens. Start adding an explanation and it never reaches the answer. The result box shows
cut off at max tokens.
The examples are the format instruction
The surest way to fix the output format in raw mode is to write it in that format in the examples. The model copies the shape of the examples. No quotes used means none used, no space left means none left. Not using them in the examples is followed far better than writing 따옴표를 쓰지 마세요 (do not use quotes) in words.
Do not explain the output format, show it. Examples are stronger than instructions.
How to read a failure
When you judge, public cases show you the text the model actually produced, as it is. Put it side by side with the expected value and compare. If they look identical and it is still wrong, a space or an invisible character got mixed in.
Look at the margin next to the result as well. It is the smallest of the score gaps between first and second place when picking each character. If it passed but the margin is very small, it means it could flip on another machine. A prompt with a comfortable margin is a good prompt.
If you got this far
The five lessons covered four things: how raw carries text on, how to use examples instead of instructions, what happens as a model gets smaller, and the output format. With those four, most of the 100 point problems are solvable. For the rest, solving a lot of problems is the faster way to learn.
Work down the problem list from the lowest score. If you get stuck, write it in that problem's discussion. If you think the problem itself is wrong, write that too. I fix it and credit you as a contributor.