Vectors, Distance and Similarity
Dividing the Lengths Out
Last timeOne Number From Two Vectors
Divide the dot product by both lengths and what is left is the cosine of the angle, a similarity between minus one and one that ignores how long either vector is.
The dot product answered a question about direction and quietly included the
lengths in the answer. The fix is to divide them out, and the result is the
single most used similarity measure in practice.
- the dot product, which already contains the two lengths and the cosine multiplied together
- the two lengths, divided out so that only the cosine is left
What the division buys
Multiply any vector by three. Its dot product with everything triples, and its
length triples too, so the ratio does not move at all. That is the property
people are usually after: a long document and a short one about the same subject
should count as similar, and under this measure they do.
| step | length of the stored vector | dot product | cosine | what happened |
|---|---|---|---|---|
| 1 | 1 | 0.8 | 0.8 | The original. |
| 2 | 2 | 1.6 | 0.8 | The same direction, twice as long. The dot product has doubled and the cosine has not moved. |
| 3 | 0.5 | 0.4 | 0.8 | Half as long. Same story in the other direction. |
Reading the number
The value always lies between minus one and one, and it is tempting to read it
as a percentage of similarity. Resist that.
In a real collection of learned vectors the scores do not use the available
range. Unrelated items commonly score well above zero, sometimes above a half,
because the vectors occupy a narrow cone rather than spreading out over all
directions. A score of 0.8 may be excellent in one collection and unremarkable in
another.
The lesson stops here
3 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents