Attention Cannot See Order, So Order Has to Be Added
Last timeMultiple Heads
Shuffle the input tokens and attention produces the same outputs, shuffled. Position must be injected into the vectors themselves, and how it is done constrains what the model can generalise to.
Here is a property of attention that is easy to state and easy to underrate.
Take a sentence, shuffle the tokens, and run attention. The outputs are the same
vectors as before, in the shuffled order. Not similar. Identical.
The proof, which is short
Scores are dot products between token vectors. A dot product refers to the two
vectors and nothing else, certainly not to where they sit in a sequence. The
softmax normalises over a collection of scores, and a sum does not care about
order. The weighted average of values is another sum.
So for a permutation of the rows:
Nowhere in the computation does an index appear. This is called permutation
equivariance, and it means attention operates on a set of vectors, not a
sequence.
The lesson stops here
11 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents