When You Cannot Hit the Target
Last timeThe Stretch in Each Direction
Most real systems ask for something the map cannot produce, so the honest answer is the closest reachable point, and that one idea is the whole of fitting.
Suppose you have forty measurements and three unknowns. The map from unknowns to
predicted measurements has three columns, so its reachable set is a flat piece of
at most three directions sitting inside a forty dimensional space. The target, your
actual measurements, sits somewhere in that forty dimensional space, and the chance
that it happens to lie exactly on a three dimensional flat piece is nil.
So the system has no solution. By the previous lessons this is the expected answer
rather than a problem: the target is not in the reachable set. What makes it useful
is that the question can be changed. Instead of asking which input produces the
target exactly, ask which input produces the closest thing to the target that the
map can produce at all.
That question always has an answer, and finding it is the single most used piece of
linear algebra in practice.
Drop a perpendicular
The geometry is the kind of thing a child knows. Given a plane and a point above
it, the nearest point of the plane is directly below, found by dropping a
perpendicular. Any other point of the plane is further away, and the reason is the
right triangle: the line from the outside point to the other candidate is a
hypotenuse, with the perpendicular drop as one leg, and a hypotenuse is always
longer than a leg.
Where the formula comes from
Now transcribe the geometry. The leftover is perpendicular to the reachable set,
and the reachable set is spanned by the columns, so the leftover is perpendicular
to every column. That is one equation per column, which is one equation per
unknown, which is a square system.
The lesson stops here
4 more paragraphs to go
You have read the opening. The rest of the argument, the problems that check whether it landed, and the lines worth keeping at the end all come with a plan.
The first lesson of every course in the library reads the whole way through, free, so you can see exactly what the rest of them are.
See the planThe contentsThis is the reading half
Starting the course gives you your own copy of it. Every idea on every page has problems standing under it, marked with a reason rather than a tick, and any sentence you do not believe can be opened and argued with. None of that can happen on a page nobody owns.
The contents