Where Things Get Lost
Last timePutting Retrieved Material In
The same sentence placed in the middle of a long input is used less often than at either end, which makes position a design decision rather than an accident of assembly.
Position changes whether it is used
Take a question whose answer is in exactly one of twenty passages, and ask it
twenty times, moving that passage through every position. The answer should be
found every time, since the material is present in all twenty runs.
It is not. The measured pattern, which has been reproduced across models and
tasks, is a curve that is high at the first position, falls through the middle,
and rises again at the last. The dip is large: on long inputs the difference
between the best and worst position is often twenty or thirty points of
accuracy, and in some settings a needed passage in the middle does worse than
having no passages at all, because the surrounding material is actively
distracting.
The practical consequence is that a failure cannot be diagnosed by checking
whether the material was included. It has to be diagnosed by checking where the
material was, and the same question with the same passages in a different order
is a different experiment.
Repeating at both ends
Given a curve that is high at both ends, the cheapest remedy is to put the
critical thing in both places.
The lesson stops here
5 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents