The Same List, Read Two Ways

A vector is a list of numbers, and everything interesting comes from agreeing to read that list as a position in space or as a direction with a length.

A vector is a list of numbers. That is the whole definition, and it is unhelpful

on its own, because the interesting part is not what a vector is but what you are

allowed to do with two of them.

Two readings of the same list

Take the list of three numbers 2, 5, 1. There are two pictures, and both are

used constantly.

FIG 1One list, two pictures

Neither reading is more correct. A search system reading items as points asks what is nearby; one reading them as directions asks what points the same way, and those are different systems with different behaviour.

Nothing about the pictures requires three numbers. A list of three hundred places

a point in a three hundred dimensional space by exactly the same rule, and the

fact that you cannot visualise it changes nothing about the arithmetic. That is

worth accepting early, because almost every vector you will meet in practice has

hundreds of entries.

What actually makes it a vector

The structure comes from two operations, and they are unglamorous.

FIG 2The only two things you are allowed to do

: scaling: every coordinate of u multiplied by the same number
: the same for v, and then the two scaled lists added coordinate by coordinate

Distance, angle, projection and every method later in the course are built from these two moves and nothing else. When a new kind of object turns out to support them, all of this machinery transfers to it unchanged, which is why the same tools work on words, images and users.

FIG 3Adding two short vectors

step	first coordinate	second coordinate	third coordinate	what happened
1	2	5	1	The first vector.
2	3	-1	4	The second.
3	5	4	5	Their sum. Each coordinate minds its own business, which is the whole of the rule.

3 steps

Coordinates never mix. That is why these operations are cheap on long lists, and it is also why everything built on them can be computed by hardware designed for exactly this shape of work.

The coordinates usually mean nothing

Here is the point that saves the most wasted effort. When the numbers come from

counting things you chose, each coordinate has a name and a meaning. When they

come from training, they do not.

FIG 4Two ways the same three documents might be turned into vectors

	count of the first word	count of the second	count of the third
document one	3	0	1
document two	0	4	0
document three	2	1	5

Counted coordinates can be named, and most of them are zero, since most documents contain a small share of the vocabulary. A learned representation for the same documents would have no zeros, a few hundred coordinates, and no name for any of them.

A learned collection of vectors can be rotated as a whole without changing a

single distance or angle between any pair, and such a rotation changes every

coordinate of every vector. So anything that depends on one coordinate alone was

not a real property of the data. Distances, angles and relative positions survive

the rotation, which is a good rule for what is safe to believe.

What to hold on to

A vector is a list of numbers plus an agreement to add and to scale. Read it as a

point and you will ask about distance; read it as an arrow and you will ask about

direction. The coordinates of a learned vector are an accident of training, so

build nothing on them individually, and build everything on the relations between

whole vectors.

Recap

A list of numbers becomes a vector the moment you agree to add lists together and to scale them, and those two operations are the entire structure that everything later is built from.
The same list can be read as a point, which makes distance the natural question, or as an arrow from the origin, which makes direction and angle the natural questions, and different tasks want different readings.
The individual coordinates usually mean nothing on their own in a learned representation, so any conclusion that depends on which coordinate is which is almost certainly an artefact.

This is the reading half

Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.

The contents

NextHow Far Apart →