Answers · How models work
Why do models fail at counting letters or handling numbers and code?
In shortModels often fail at letters, digits and code because the text is chopped into pieces before the model sees it, hiding individual characters and place values.
The model is not seeing characters. It sees a sequence of piece indices, and the cutting into pieces happens before any reasoning starts, so counting letters, aligning digits, and reading indentation can fail before the model has a chance to be good or bad at them. The failures look like arithmetic, spelling and code competence because those are the tasks where the mismatch shows up. The mismatch itself is at the input layer.
Digits arrive in groups
Long multiplication needs one digit per column. The model does not receive digits. It receives whatever digit groups the piece list happens to contain, and those groups depend on which digit sequences were frequent in the text used to build the list. The same quantity can be split as 12 and 34, or as 1 and 234, and the pieces carry no marker saying which is hundreds and which is units. If the first number arrives as 12 and 34, the first piece means 12 hundreds, not 12 units. If it arrives as 1 and 234, the first piece means one thousand. A model that has to multiply by 56 must reconstruct that before multiplying, and it must also carry between groups that may not line up between the two numbers.
The natural guess is that the model cannot multiply. That guess is too quick. In the split above, the arithmetic is correct and short. The extra work is recovering place value from pieces that were cut at different points in each number, then carrying across group boundaries. More training on arithmetic does not remove that input mismatch. Changing how the digits are written can, which is why spacing the digits out sometimes rescues a calculation, and why a model that deliberately cuts numbers into single digits is measurably better at arithmetic.
Letters and indentation
Letter counting is the same problem in a smaller space. The word strawberry has ten letters and three letters r, at positions 3, 8 and 9. If the word arrives as one piece, the input contains one item, not ten. There are no letter positions to inspect, so a counting procedure has nothing to count. If it arrives as straw and berry, the r count is 1+2=3, but only after both pieces are opened and merged. Common words are often answered correctly because their spelling was seen often in text; that is recall, not inspection. Unusual words fail because recall has less to work with.
Code puts weight on characters that prose treats as filler. Leading whitespace can encode nesting, and a piece list built on prose has entries for runs of spaces that reflect prose, not code structure. Four spaces may be one piece or two pieces of two spaces, and neither piece says one level of nesting. A code-aware piece list adds entries for common indentation runs for exactly this reason. Without that, the model must infer structure from a representation that has already obscured it.
The test
Change the surface form and keep the question identical. Put spaces between digits, hyphenate the letters of the word, reindent the code, or ask in a better represented language. If the answer improves, the pieces were the binding constraint. If it does not improve, the difficulty is elsewhere. The test is not proof that the cutting was perfect; it says the cutting was not what stopped this answer.
That test also says where the explanation stops. A model can still fail after the digits are spaced, because a long multiplication has many steps and a mistake can happen in any of them. A word can still be miscounted after hyphenation if the model does not follow the instruction. Code with explicit braces already exposes structure more than indentation-only code, so the failure is smaller there. A cutting problem is fixed by changing the form of the input, which is cheap. A capability problem is not fixed that way, and treating one as the other wastes effort in both directions.
Worth keeping
- A model cannot count letters it never receives; if a word arrives as one piece, there are no letter positions to inspect.
- Long multiplication fails more often when digit groups are uneven, because place value must be recovered before any arithmetic starts.
- Changing only the surface form, such as spacing digits or hyphenating letters, is the test that separates a cutting problem from a capability problem.
Where this goes next
This question is one step of How Text Becomes Numbers, a course that works through its subject from the beginning. The page that teaches this part of it is Numbers, Code and Other Languages, and it reads the whole way through whether or not you have an account.
What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.
Read that pageThe whole courseOther questions people ask about this
- What is wrong with just tokenizing text one character at a time?Nothing is ever unknown, but a passage becomes roughly five times longer than it needs to be, and attention cost grows faster than length, so you pay several times over for the same text.
- Why do tokenizers split words into smaller pieces instead of keeping whole words?Tokenisers split words into pieces because a fixed list of whole words is enormous and still cannot handle names, typos or new words, while common fragments can build any word without an unknown entry.
- What does it mean if two words are close together in embedding space?It means training placed their rows in similar positions because the pieces appeared in similar company, so one can often stand in for the other in text; it is not a direct synonym score.
- Why does the same word get a different vector depending on the sentence it appears in?The lookup gives the same word the same starting row every time; the different vector is built later by mixing in the words around it.
- Why does the model get stuck repeating the same sentence over and over?The model is not stuck in a memory loop: once it has written a phrase, that phrase is in its context and makes writing it again more likely, so a decoder that always picks the top word cannot leave.
- Why does diffusion add Gaussian noise instead of any random corruption?Diffusion adds Gaussian noise because Gaussian errors combine exactly: each step keeps the spread fixed and gives a closed-form noisy picture at any later step, which makes the reverse step learnable.