Answers · The mathematics underneath

What's the difference between a vector as a point and a vector as a direction?

In shortA vector as a point is a location, while a vector as a direction is an arrow from the origin whose length and orientation matter; the same numbers can be read either way.

A vector as a point is a location. A vector as a direction is an arrow from the origin to that location, and the arrow reading cares about length and orientation rather than position. The list of numbers is identical in both readings, so the difference is not in the list but in the question you put to it: how far apart are two locations, or how aligned are two arrows. Neither reading is more correct; they are two sets of questions applied to the same arithmetic.

The same pair, two different numbers

Take $u=(2,5,1)$ and $v=(3,-1,4)$ . Read as points, the natural question is distance. Subtract coordinate by coordinate: $u-v=(-1,6,-3)$ . Square each entry, add, and take the square root. The result is a single number attached to the pair, not to either point on its own.

\|u-v\| = \sqrt{(-1)^2+6^2+(-3)^2} = \sqrt{46} \approx 6.78

Read as arrows, the natural question is angle. The dot product is $u\cdot v=2\cdot3+5\cdot(-1)+1\cdot4=6-5+4=5$ . The lengths are $\|u\|=\sqrt{4+25+1}=\sqrt{30}\approx5.48$ and $\|v\|=\sqrt{9+1+16}=\sqrt{26}\approx5.10$ . Dividing the dot product by the product of lengths gives the cosine of the angle. Here $5/(\sqrt{30}\sqrt{26})=5/\sqrt{780}\approx0.179$ , so the angle is about $79.7^\circ$ . The dot product is large only when arrows point the same way; this value is small compared with $\sqrt{780}\approx27.9$ , so the arrows are close to perpendicular without being exactly perpendicular.

\cos\theta=\frac{u\cdot v}{\|u\|\,\|v\|}=\frac{5}{\sqrt{30}\sqrt{26}}\approx0.179,\qquad \theta\approx79.7^\circ

Why the point reading cannot answer everything

The natural guess is that the arrow reading is just a picture of the point, so distance and angle should carry the same information. They do not, and scaling shows it cleanly. Double $u$ to get $2u=(4,10,2)$ . As a point it has moved: the distance to $v$ is now $\|2u-v\|=\|(1,11,-2)\|=\sqrt{1+121+4}=\sqrt{126}\approx11.22$ , up from about $6.78$ . As a direction it has not turned: $2u\cdot v=10$ and $\|2u\|=\sqrt{120}\approx10.95$ , so $\cos\theta=10/(\sqrt{120}\sqrt{26})=10/\sqrt{3120}\approx0.179$ , the same angle as before. Positive scaling moves the point along the same ray and leaves the arrow's direction untouched. That is exactly the information the point reading keeps and the arrow reading discards.

\|2u-v\|=\sqrt{126}\approx11.22,\qquad \cos\theta=\frac{10}{\sqrt{120}\sqrt{26}}\approx0.179

What survives a change of coordinates

Directions are defined up to positive scaling: $u$ and $2u$ point the same way, while the point $u$ and the point $2u$ are different places. A zero vector has no direction at all, so the arrow reading is undefined there, while the point reading still gives a location. Negative scaling reverses the arrow and sends an angle $\theta$ to $180^\circ-\theta$ ; the line of the arrow is unchanged but its orientation is not. In a learned list of hundreds of coordinates, no single coordinate has a reliable name. Rotating every vector in the collection changes every coordinate, yet leaves every distance and every angle between pairs unchanged. So anything that depends on coordinate one being large is an artefact of training, not a property of the data. The safe conclusions are the ones built from whole vectors: distances, angles, and relative positions.

Nothing about either reading needs three entries. A list of three hundred numbers places a point in a three hundred dimensional space by the same rule, and the distance is still the square root of the sum of three hundred squared differences. The angle formula is still dot product divided by the product of lengths. What you lose is the ability to draw the arrow, not the arithmetic, and the two readings remain distinct even when you cannot visualise either one.

The point reading is the right one when position matters, as in a map or a counted feature vector where each coordinate is named. The arrow reading is the right one when only relative orientation matters, as in comparing profiles, word embeddings or any representation where multiplying by a positive number should not change the answer. Mixing them is the common error: computing a distance when you meant alignment, or treating an angle as if it located something. If two vectors are the same point, their distance is zero and their angle is undefined, because the zero arrow has no direction. If two vectors point the same way, their angle is zero but their distance can be large.

Worth keeping

The same list can be a place or an arrow, and the distance question and the angle question are different questions.
Positive scaling moves the point but does not turn the arrow.
In learned vectors, individual coordinates are arbitrary; distances and angles between whole vectors are what survive.

Where this goes next

This question is one step of Vectors, Distance and Similarity, a course that works through its subject from the beginning. The page that teaches this part of it is A List of Numbers With a Geometry, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course