Answers · The mathematics underneath

What does it mean when a matrix doesn't have full rank?

In shortA matrix lacks full rank when it cannot reach as many independent directions as its shape allows, so it loses information and cannot be reversed.

A matrix lacks full rank when its rank is below the largest value its shape allows: $\min(m,n)$ for an $m\times n$ matrix. For a square $n\times n$ matrix that means $r<n$ : some non-zero input $\mathbf{v}$ is sent to zero, so two different inputs produce the same output and the map has no inverse.

A\mathbf{v}=\mathbf{0}\quad\Longrightarrow\quad A(\mathbf{x}+\mathbf{v})=A\mathbf{x}.

What the rank counts

Rank is the number of independent directions among the columns, which is the same as the number of independent directions in the reachable set. The columns are the destinations the matrix can mix, so every output is a mixture of them. If the columns repeat a direction, the reachable set is smaller than the output space. The inputs sent to zero are the destroyed directions. They form a flat set through the origin, and their number $k$ satisfies $r+k=n$ , where $n$ is the number of columns. Knowing any two of $r$ , $k$ and $n$ gives the third. Full rank means $k=0$ for a square matrix; not full rank means $k>0$ .

r+k=n

Take $A=\begin{pmatrix}1&2\\2&4\end{pmatrix}$ . Its columns are $(1,2)$ and $(2,4)=2(1,2)$ , so there is only one independent column direction: $r=1$ . The input $\mathbf{v}=(2,-1)$ gives $A\mathbf{v}=(1\cdot2+2\cdot(-1),\,2\cdot2+4\cdot(-1))=(0,0)$ , so $k=1$ . And $1+1=2$ , the number of columns. Any output is $x(1,2)+y(2,4)=(x+2y)(1,2)$ , so the reachable set is the line through $(1,2)$ , not the whole plane.

A\begin{pmatrix}3\\1\end{pmatrix}=\begin{pmatrix}5\\10\end{pmatrix}=A\begin{pmatrix}1\\2\end{pmatrix}.

Non-zero rows are not the count

It is tempting to count non-zero rows and call that the rank. This matrix has two non-zero rows, yet its rank is $1$ . The rows are $(1,2)$ and $(2,4)$ , which are also multiples of one another, so they do not add a second independent direction. Counting non-zero rows can overstate rank because a row can be non-zero and still be a combination of the others. Rank is a dimension, not a tally of entries that happen to be non-zero. The determinant makes the same point: $1\cdot4-2\cdot2=0$ , so the two columns do not span the plane.

The loss shows up as duplicate outputs. The two inputs $(3,1)$ and $(1,2)$ differ by $(2,-1)$ , the vector sent to zero, and they land on the same output $(5,10)$ . From $(5,10)$ alone there is no way to tell which input was used. That is why a square system with a non-full-rank matrix either has no solution or infinitely many: the target may lie outside the reachable line, or it may lie on it with a whole line of inputs behind it. If the target is outside the line, no input can hit it; if it is on the line, every solution differs by a destroyed vector. It cannot have exactly one solution.

Nearly full rank

With exact arithmetic, a matrix is either full rank or not. With measured data the distinction can collapse. Two columns may be nearly multiples without being exact multiples, so the exact rank is $2$ and the destroyed set is only the origin, yet one direction is stretched by a tiny factor. If the stretch is $10^{-6}$ , running the map backwards along that direction multiplies noise by $10^6$ . The matrix passes the exact test and fails the useful one. The same matrix can look perfectly invertible in exact arithmetic and still produce answers that swing wildly when a measurement changes in the sixth decimal place. The honest question is how many directions survive by a margin worth trusting, which needs a number attached to each direction, not just a count.

For an $m\times n$ matrix the bookkeeping still uses $n$ , the number of columns. Full rank means $r=\min(m,n)$ ; if the matrix is wide, full rank still leaves $k=n-m>0$ destroyed directions, because there are more inputs than outputs. Not full rank is stronger: $r<\min(m,n)$ , so the loss is larger than the shape alone forces. In that case the columns fail to span as many directions as the shape allows, and the missing directions are exactly the ones the matrix cannot reach.

Worth keeping

Rank is the number of independent directions in the reachable set, not the number of non-zero rows.
For a square n by n matrix, rank plus the number of destroyed directions equals n.
A matrix can be exactly full rank and still be numerically useless if one direction is stretched by a tiny factor.

Where this goes next

This question is one step of Matrices as Maps, a course that works through its subject from the beginning. The page that teaches this part of it is What a Map Throws Away, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course