Basis Lab

Answers · The mathematics underneath

Why doesn't matrix multiplication work like regular multiplication where order doesn't matter?

In shortMatrix multiplication means doing one transformation and then another, and reversing that order usually lands a vector somewhere else.

Matrix multiplication is not a rule for grids. It is what happens when two linear maps are performed in sequence, first one and then the other, so ABAB means apply BB and then AA.

(AB)x=A(Bx)(AB)\mathbf{x}=A(B\mathbf{x})

That equation is the definition, not a convention. The matrix on the right is nearest the vector, so it acts first; the matrix on the left acts on whatever came out. Swapping the order asks for B(Ax)B(A\mathbf{x}), which is a different sequence of two maps. The familiar row-column rule follows from this: (AB)ik=∑jAijBjk(AB)_{ik}=\sum_j A_{ij}B_{jk}, where the summed index jj belongs to the intermediate space shared by the two maps. The sum is not decoration; it is the bookkeeping for passing through that intermediate space.

A flatten and a quarter turn

Let FF flatten every vector onto the xx-axis, so F(x,y)=(x,0)F(x,y)=(x,0), and let RR turn every vector a quarter turn anticlockwise, so R(x,y)=(−y,x)R(x,y)=(-y,x). Their matrices are:

F=(1000),R=(0−110),v=(11)F=\begin{pmatrix}1&0\\0&0\end{pmatrix},\quad R=\begin{pmatrix}0&-1\\1&0\end{pmatrix},\quad \mathbf{v}=\begin{pmatrix}1\\1\end{pmatrix}

Now apply the two maps in both orders to the same vector v=(1,1)\mathbf{v}=(1,1). Flattening first gives Fv=(1,0)F\mathbf{v}=(1,0), and the quarter turn then gives R(Fv)=(0,1)R(F\mathbf{v})=(0,1). Turning first gives Rv=(−1,1)R\mathbf{v}=(-1,1), and flattening then gives F(Rv)=(−1,0)F(R\mathbf{v})=(-1,0). The two outputs are different, so the two composites are different matrices.

FIG 1Two routes through the same pair of maps
The same input and the same two maps, applied in opposite orders, finish at different vectors. This is the whole reason need not equal .

The same conclusion is visible in the product matrices. RFRF has first column RR applied to FF's first column (1,0)(1,0), which is (0,1)(0,1), and second column RR applied to FF's second column (0,0)(0,0), which is (0,0)(0,0). So RF=(0010)RF=\begin{pmatrix}0&0\\1&0\end{pmatrix}. Reversing the order gives FR=(0−100)FR=\begin{pmatrix}0&-1\\0&0\end{pmatrix}. Checking one entry: (RF)21=R21F11+R22F21=1⋅1+0⋅0=1(RF)_{21}=R_{21}F_{11}+R_{22}F_{21}=1\cdot1+0\cdot0=1, while (FR)21=F21R11+F22R21=0⋅0+0⋅1=0(FR)_{21}=F_{21}R_{11}+F_{22}R_{21}=0\cdot0+0\cdot1=0.

The natural guess is AB=BAAB=BA because ordinary numbers commute, and the notation looks like ordinary multiplication. That guess treats a matrix as a number rather than as an action. A number changes a size; a matrix rearranges directions, and a rearrangement before a destruction is not the same as the destruction before the rearrangement. In the example, turning first keeps the second coordinate alive until the flattening; flattening first destroys it before the turn. Nothing in the later turn can recover what the flattening threw away. That is why the order is part of the operation and not a harmless label.

When the order does not matter

Commuting is the exception. A uniform scaling of the whole plane commutes with every map, because it stretches every direction by the same amount and cannot change which directions another map treats specially. Two rotations about the same centre commute, because both act on the same set of directions and the angles simply add. Two different powers of one matrix commute, because they are built from the same map repeated. A rotation and a non-uniform flattening do not commute, and two rotations about different axes in three dimensions do not commute, because each disturbs the axis about which the other turns. In one dimension, where a matrix is just a single number, every pair commutes; that is the special case, not the general rule.

Associativity survives even when commutativity does not: (AB)C=A(BC)(AB)C=A(BC), so a chain of maps can be grouped freely without changing the final vector. But grouping changes the amount of arithmetic. Take AA of size 1000×21000\times2, BB of size 2×10002\times1000, and CC of size 1000×11000\times1. Forming ABAB first costs 1000⋅2⋅1000=2,000,0001000\cdot2\cdot1000=2,000,000 multiplications, and then (AB)C(AB)C costs 1000⋅1000⋅1=1,000,0001000\cdot1000\cdot1=1,000,000, for 3,000,0003,000,000 in total. Forming BCBC first costs 2⋅1000⋅1=2,0002\cdot1000\cdot1=2,000, and then A(BC)A(BC) costs 1000⋅2⋅1=2,0001000\cdot2\cdot1=2,000, for 4,0004,000 in total. The answer is the same; the first grouping is 750 times more work.

For square matrices the product is defined in both orders, but ABAB and BABA are generally different matrices. The order on the page runs opposite to the order in time, and that is the source of most of the confusion. Matrix multiplication is associative, not commutative.

Worth keeping

  • Matrix multiplication is composition of transformations, so the rightmost matrix acts first.
  • Two matrices commute exactly when applying them in either order has the same effect on every vector.
  • Associativity lets a product be regrouped freely, but different groupings can differ enormously in cost.

Where this goes next

This question is one step of Matrices as Maps, a course that works through its subject from the beginning. The page that teaches this part of it is One Map After Another, and it reads the whole way through whether or not you have an account.

What an account changes is the practice: problems standing under every idea, marked with a reason rather than a tick, in a copy of the course that is yours.

Read that pageThe whole course

Other questions people ask about this

  1. Why do eigenvectors stay in the same direction after a transformation?An eigenvector is defined as a vector that a matrix sends to a scaled copy of itself, so staying on its line is the condition that selects it, not a property of ordinary vectors.
  2. What's the difference between singular values and eigenvalues?Singular values measure stretching along perpendicular directions and exist for every matrix; eigenvalues measure directions a square matrix leaves on their own line.
  3. What does it mean when a matrix doesn't have full rank?A matrix lacks full rank when it cannot reach as many independent directions as its shape allows, so it loses information and cannot be reversed.
  4. Why do the columns of a matrix tell you where the basis vectors go?A matrix lists where each coordinate axis lands, one destination per column, because feeding in a single axis picks out exactly that column.
  5. What is the difference between variance and standard deviation?They measure the same spread: variance is the average squared distance from the mean, standard deviation is its square root, so you report the standard deviation and calculate with the variance.
  6. How do you make a vague claim precise enough to tell whether it's true?Fix what is being talked about, what is asserted, which cases are covered and the cut-off, then name an observation that would prove it false.

Read alongside