Basis Lab

Answers · How models work

Why does the same word get a different vector depending on the sentence it appears in?

In shortThe lookup gives the same word the same starting row every time; the different vector is built later by mixing in the words around it.

The lookup gives the same word the same starting row every time. The vector that differs between sentences is built after the lookup, by mixing that row with the rows at the positions around it, so it depends on the sentence rather than on the word alone.

xi(0)=E[id(bank)]in either sentence\mathbf{x}_i^{(0)} = E[\mathrm{id}(\text{bank})] \quad \text{in either sentence}

EE is the table and id(bank)\mathrm{id}(\text{bank}) is one integer. That integer is the same in both sentences, so the starting row xi(0)\mathbf{x}_i^{(0)} is the same. Nothing in the lookup consults the sentence, and no second row exists for the other sense.

The arithmetic of one layer

Take two-dimensional rows so the sums are visible. The bank row is [1,0][1,0]. In one sentence its only neighbour with nonzero weight has row [0,1][0,1]; in another, [0,2][0,2]. Give the word itself weight 0.50.5 and that neighbour weight 0.50.5, so the weights add to 11.

xi(0)=[10],xi+1(0)=[01] in A,xi+1(0)=[02] in B\mathbf{x}_i^{(0)}=\begin{bmatrix}1\\0\end{bmatrix},\quad \mathbf{x}_{i+1}^{(0)}=\begin{bmatrix}0\\1\end{bmatrix}\text{ in A},\quad \mathbf{x}_{i+1}^{(0)}=\begin{bmatrix}0\\2\end{bmatrix}\text{ in B}
hiA=0.5[10]+0.5[01]=[0.50.5],hiB=0.5[10]+0.5[02]=[0.51],hiB−hiA=[00.5]\mathbf{h}_i^{A}=0.5\begin{bmatrix}1\\0\end{bmatrix}+0.5\begin{bmatrix}0\\1\end{bmatrix}=\begin{bmatrix}0.5\\0.5\end{bmatrix},\quad \mathbf{h}_i^{B}=0.5\begin{bmatrix}1\\0\end{bmatrix}+0.5\begin{bmatrix}0\\2\end{bmatrix}=\begin{bmatrix}0.5\\1\end{bmatrix},\quad \mathbf{h}_i^{B}-\mathbf{h}_i^{A}=\begin{bmatrix}0\\0.5\end{bmatrix}

The first coordinate is 0.50.5 in both cases because the bank row contributes 0.5×10.5\times1 and the neighbour contributes 0.5×00.5\times0. The second coordinate is 0.50.5 in the first sentence and 11 in the second because the neighbour's second coordinate is 11 in A and 22 in B. The same row sat at the centre of both sums; the second sentence simply put different numbers beside it. Nothing in that calculation used a second bank row: the only difference between A and B is the row at the neighbouring position and therefore the second term of the sum.

The guess that fails

The natural guess is that the word has two table entries, or that the model notices which sense is meant and swaps in the matching row. Neither happens. There is one row per entry, consulted by identifier alone, and no disambiguation step chooses a sense. The first layer can only see the immediate neighbours, and later layers mix in more of the passage, so two occurrences drift apart as a side effect of repeated mixing. The separation is real, but it is not produced by the lookup.

The layer in general

At position ii, a layer computes the new numbers as a weighted sum over every position in the passage:

hi=∑jwij xj\mathbf{h}_i=\sum_j w_{ij}\,\mathbf{x}_j

The sum includes j=ij=i, which is why a representation keeps some of the row it started with. The weights wijw_{ij} are not fixed by position; they are computed from the content on each pass. If the relevant neighbour is two places away in one sentence and twenty in another, the weights can still find it. That is the mechanism that lets the same starting row end up with different numbers in different surroundings.

A table with one row per entry cannot hold two unrelated senses at once. The best it can do is sit between them, which is a poor representation of either use. The layers do not repair that by editing the table. They rebuild each position from its neighbours until the numbers at that position are specific to that occurrence. The row is then a starting point shared by every appearance of the piece, not the meaning the model acts on.

Where it stops being true

If there are no layers above the lookup, as in a static embedding table, the lookup row is final: the word has one vector and two senses cannot be told apart. If two occurrences happen to have identical surrounding contexts, the built vectors remain identical, because the sums are identical. At the very top of a trained model the numbers are also shaped by the next-piece prediction job, so they are sharpened for prediction rather than neutral description; middle and upper-middle layers are often better for telling senses apart.

How much a representation changes with context varies by layer and by word, so there is no single layer that is correct for every use outside the model. A word used much the same way everywhere moves less than a word whose use varies, and the only reliable test is to compare the two occurrences at the layer that will actually be used.

Worth keeping

  • At the lookup, one identifier selects one row, so two occurrences of the same word start from identical numbers.
  • A contextual vector is a weighted sum of the rows at every position, with weights computed from the content, so different neighbours produce different sums.
  • Static embeddings stop at the lookup, so they cannot separate two senses of one word.

Where this goes next

This question is one step of How Text Becomes Numbers, a course that works through its subject from the beginning. The page that teaches this part of it is The Same Word Twice, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. What is wrong with just tokenizing text one character at a time?Nothing is ever unknown, but a passage becomes roughly five times longer than it needs to be, and attention cost grows faster than length, so you pay several times over for the same text.
  2. Why do models fail at counting letters or handling numbers and code?Models often fail at letters, digits and code because the text is chopped into pieces before the model sees it, hiding individual characters and place values.
  3. Why do tokenizers split words into smaller pieces instead of keeping whole words?Tokenisers split words into pieces because a fixed list of whole words is enormous and still cannot handle names, typos or new words, while common fragments can build any word without an unknown entry.
  4. What does it mean if two words are close together in embedding space?It means training placed their rows in similar positions because the pieces appeared in similar company, so one can often stand in for the other in text; it is not a direct synonym score.
  5. Why does the model get stuck repeating the same sentence over and over?The model is not stuck in a memory loop: once it has written a phrase, that phrase is in its context and makes writing it again more likely, so a decoder that always picks the top word cannot leave.
  6. Why does diffusion add Gaussian noise instead of any random corruption?Diffusion adds Gaussian noise because Gaussian errors combine exactly: each step keeps the spread fixed and gives a closed-form noisy picture at any later step, which makes the reverse step learnable.

Read alongside