The Two Methods Fail on Opposite Questions
Last timeWhat Actually Gets Compared
Matching on meaning finds paraphrases and loses part numbers. Matching on words does the reverse. Running both and merging the lists is close to free and fixes most of each.
A customer writes in: part MX-4471B is making a noise. The manual has a page
about MX-4471B. Meaning-based retrieval frequently fails to find it.
This is not a bug. The piece was reduced to a few hundred numbers chosen to place
it near things about the same subject. Nothing in that process was asked to
preserve one eight character string, and in a collection with four hundred part
numbers of the same shape, it generally does not.
What the older method does
Matching on words is the method that predates all of this, and it is extremely
good at exactly the thing meaning matching is bad at.
- rarity: a word in few pieces of the collection counts for a lot, a word in most of them counts for almost nothing
- frequency, flattened, with a correction b for how long the piece is
The three ingredients are worth stating plainly. How often the word appears, so a
page about the part beats a page mentioning it once. How rare the word is, so
rare words dominate. How long the piece is, so a long piece does not win just by
containing more words in general.
None of these three are learned. They are counted from the collection you
already have, which has two consequences worth noticing. There is no training
step, no model to choose and no domain mismatch to worry about, so this method
works equally well on case law and chip specifications as on web prose. And it
has no notion whatsoever of meaning, so a question answered in different words is
invisible to it no matter how much text you give it.
The lesson stops here
4 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents