The Decision Made Before Anything Is Searched

Last timeThe Question a Retrieval System Answers

Nothing is ever retrieved. Pieces of documents are, and the choice of where to cut sets a limit on what can be found that no later stage can lift.

Suppose the collection is a forty page product manual. A question arrives about

the return policy. What should the system hand over?

Not the manual. One representation of forty pages is an average over forty pages,

and the return policy is two sentences of it. Those two sentences contribute

almost nothing to the average, so the manual as a whole does not look much like

a question about returns. Worse, it looks about equally unlike a question about

anything else in it, which means the representation has no useful discriminating

power at all.

FIG 1What happens to a document before any question arrives

Every arrow before the last one happens in advance and is never revisited at question time. That is why a bad cut is expensive to discover and expensive to fix: it means rebuilding the whole index.

What size costs

A piece has some words that bear on the question and some that do not, and the

representation is built from all of them together.

FIG 2How much of a piece is actually about the thing you want

: words in the piece that bear on the question
: the rest of the words in the piece
: the share of the piece that is on topic, which is roughly what its representation is weighted by

The answer is two sentences long whatever you do. So growing the piece grows only the lower term, and the share falls. That is the cost of a larger piece, and it is paid on every single comparison.

Against that sits the gain. The two sentences have to be inside one piece. If a

boundary falls between them, there is no piece anywhere in the index that

contains the answer, and the best possible retrieval returns half of it.

These two effects are not interchangeable. Dilution makes a piece harder to find

and still leaves it findable, since a question about returns will rank a page

mentioning returns above a page that does not. A split answer is not findable at

all, by anything, ever. So the two sides of the trade should not be weighed

equally, which is why real systems sit nearer the large end of the sensible range

than a first guess suggests.

The lesson stops here

3 more paragraphs to go

You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.

The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.

See the plan The contents

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

The Decision Made Before Anything Is Searched

What size costs

3 more paragraphs to go

This is the reading half

Read alongside