Answers · The mathematics underneath

What's the difference between singular values and eigenvalues?

In shortSingular values measure stretching along perpendicular directions and exist for every matrix; eigenvalues measure directions a square matrix leaves on their own line.

Singular values measure how much a matrix stretches vectors along a set of perpendicular directions, and every matrix has them. Eigenvalues measure directions that a square matrix sends back along their own line, so they only exist when the matrix is square. The two lists can agree, but that is a special case, not the definition. A singular value belongs to a pair of directions, one going in and one coming out. An eigenvalue belongs to a single direction that survives the map up to a scale factor.

A=U\Sigma V^{\mathsf T},\qquad Av=\lambda v.

One small matrix, both lists

Take \(A=\begin{pmatrix}0&1\\0&0\end{pmatrix}\). It is square, so eigenvalues exist. The eigenvalue equation is \(\det(A-\lambda I)=\det\begin{pmatrix}-\lambda&1\\0&-\lambda\end{pmatrix}=\lambda^2\), so the only eigenvalue is \(0\), twice. The matrix sends the first axis to zero and sends the second axis to the first axis. Now compute \(A^{\mathsf T}A=\begin{pmatrix}0&0\\0&1\end{pmatrix}\). Its eigenvalues solve \(\det\begin{pmatrix}-\mu&0\\0&1-\mu\end{pmatrix}=\mu(\mu-1)=0\), giving \(1\) and \(0\). The singular values are the square roots, \(1\) and \(0\).

A^{\mathsf T}A=\begin{pmatrix}0&0\\0&1\end{pmatrix},\qquad \sigma(A)=\{\sqrt1,\sqrt0\}=\{1,0\}.

The eigenvalue list is \(0,0\). The singular value list is \(1,0\). They are not the same objects. The singular value \(1\) says that some input direction is stretched by factor \(1\), and the singular value \(0\) says that some input direction is destroyed. The eigenvalue \(0\) says that a direction is sent to zero, but it does not report a stretch of \(1\) anywhere. For this matrix the singular value \(1\) is not an eigenvalue at all.

For a non-square example take \(B=\begin{pmatrix}3&0&0\\0&1&0\end{pmatrix}\). It is \(2\) by \(3\), so it has no eigenvalues. It does have singular values. Compute \(BB^{\mathsf T}=\begin{pmatrix}9&0\\0&1\end{pmatrix}\), whose eigenvalues are \(9\) and \(1\), so the singular values are \(3\) and \(1\). There are only two of them because the smaller dimension is \(2\).

FIG 1

Matrix	Shape	Eigenvalues	Singular values
[[0,1],[0,0]]	2 by 2	0, 0	1, 0
[[3,0,0],[0,1,0]]	2 by 3	none	3, 1
[[0,-1],[1,0]]	2 by 2	i, -i	1, 1
[[3,0],[0,-1]]	2 by 2	3, -1	3, 1

Why the natural guess fails

The natural guess is that a singular value is just an eigenvalue of the original matrix, perhaps after taking an absolute value. That guess fails first on shape: \(B\) above is not square, so it has no eigenvalues to compare. It fails again on type: eigenvalues can be negative or complex, while singular values are always real and non-negative. It fails a third time on directions: an eigenvalue uses one direction, \(Av=\lambda v\), while a singular value uses two, \(Av_i=\sigma_i u_i\), and \(u_i\) need not be parallel to \(v_i\).

Take \(R=\begin{pmatrix}0&-1\\1&0\end{pmatrix}\), a quarter turn. Then \(R^{\mathsf T}R=I\), so both singular values are \(\sqrt1=1\). The eigenvalue equation is \(\det(R-\lambda I)=\det\begin{pmatrix}-\lambda&-1\\1&-\lambda\end{pmatrix}=\lambda^2+1=0\), whose roots are \(i\) and \(-i\). A complex eigenvalue is not a stretch factor. The singular values say correctly that the rotation changes no length.

When they coincide, and when they do not

If \(A\) is symmetric with all eigenvalues non-negative, then \(A^{\mathsf T}A=A^2\), and the singular values are exactly the eigenvalues. For \(A=\begin{pmatrix}3&0\\0&1\end{pmatrix}\) the eigenvalues are \(3,1\), while \(A^{\mathsf T}A=\begin{pmatrix}9&0\\0&1\end{pmatrix}\) has eigenvalues \(9,1\), whose square roots are \(3,1\). If the symmetric matrix has a negative eigenvalue, the singular values are the absolute values. For \(A=\begin{pmatrix}3&0\\0&-1\end{pmatrix}\) the eigenvalues are \(3,-1\), while \(A^{\mathsf T}A=\begin{pmatrix}9&0\\0&1\end{pmatrix}\) gives singular values \(3,1\).

Even a square matrix with real eigenvalues can have different singular values. Take \(A=\begin{pmatrix}1&1\\0&1\end{pmatrix}\). Its eigenvalues are both \(1\), since \(\det(A-\lambda I)=(1-\lambda)^2\). But \(A^{\mathsf T}A=\begin{pmatrix}1&1\\1&2\end{pmatrix}\), whose eigenvalues solve \((1-\mu)(2-\mu)-1=\mu^2-3\mu+1=0\), so \(\mu=(3\pm\sqrt5)/2\). The singular values are the square roots, approximately \(1.618\) and \(0.618\), not \(1,1\). The matrix is invertible and has a repeated eigenvalue, but it does not stretch every direction equally.

The count also differs. An \(n\) by \(n\) matrix has \(n\) eigenvalues counted with multiplicity, and they may be complex. A matrix with \(m\) rows and \(n\) columns has \(\min(m,n)\) singular values, all real and non-negative, conventionally sorted from largest to smallest. A singular value always describes a pair: an input direction and the output direction it is sent to. An eigenvalue describes a direction that is sent along its own line. That pairing is the cleanest way to keep the two ideas apart.

Worth keeping

Singular values are always real and non-negative and are defined for every matrix; eigenvalues can be negative or complex and require a square matrix.
For a symmetric positive semidefinite matrix the two lists are the same; for a general square matrix they are not.
An n by n matrix has n eigenvalues counted with multiplicity, while the number of singular values is the smaller of the two dimensions.

Where this goes next

This question is one step of Matrices as Maps, a course that works through its subject from the beginning. The page that teaches this part of it is The Stretch in Each Direction, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that page The whole course