Why do the columns of a matrix tell you where the basis vectors go?

The columns are where the axes land because a matrix acts on a vector by mixing its columns, and a basis vector asks for one column with amount one and all the others with amount zero. In symbols, the $j$-th column of $A$ is $A\mathbf{e}_j$, where $\mathbf{e}_j$ is the vector with a 1 in position $j$ and 0 elsewhere.

Take $\mathbf{x}=\mathbf{e}_j$. Its coordinates are $x_j=1$ and all other $x_i=0$, so the mixture above collapses: every term with a zero amount disappears and the survivor is $\mathbf{a}_j$. That is the whole reason the columns are destinations. The first column is where the first axis goes, the second column is where the second axis goes, and the same sentence continues for as many input axes as there are columns.

Let $A = \begin{pmatrix}2&-1\\1&3\end{pmatrix}$. Its first column is $(2,1)$ and its second column is $(-1,3)$. By hand, $A\mathbf{e}_1$ has first coordinate $2\cdot 1 + (-1)\cdot 0 = 2$ and second coordinate $1\cdot 1 + 3\cdot 0 = 1$, while $A\mathbf{e}_2$ has first coordinate $2\cdot 0 + (-1)\cdot 1 = -1$ and second coordinate $1\cdot 0 + 3\cdot 1 = 3$.

Reading a matrix row by row is not false. The first row pairs with $\mathbf{x}$ to manufacture the first output coordinate: $2x_1-x_2$. The second row manufactures $x_1+3x_2$. But a row is not a destination. If you read rows only, the image of $\mathbf{e}_1$ is split across two rows, 2 in the first and 1 in the second, and you have to know to gather those entries down a column before you see the point $(2,1)$. The natural guess is that a matrix stores a set of rules about output coordinates, so the rows look primary; the better description is that it stores a short list of destination vectors, and the rows are what you see when you ask how each output coordinate is assembled.

The shape is part of the statement. An $m$ by $n$ matrix has $n$ columns, one for each input axis, and $m$ rows, one for each coordinate of a destination, so it maps $\mathbb{R}^n$ to $\mathbb{R}^m$. For $A=\begin{pmatrix}1&0&2\\0&1&-1\end{pmatrix}$ there are three columns and two rows. Feeding in the third axis gives the third column $(2,-1)$, so the third axis lands at $(2,-1)$ in the plane. The row count is 2 because destinations live in $\mathbb{R}^2$; the column count is 3 because there are three input axes.

The statement is about a fixed choice of axes. If you change the input basis, the $j$-th column still records where the $j$-th basis vector goes, but the entries are now coordinates of that destination in the chosen output basis. With a different input basis, column $j$ belongs to that basis vector, not to the old $\mathbf{e}_j$. If the map is affine, with a translation added, the columns give only the linear part; the translation is a separate vector. A single matrix cannot represent a non-linear map at all, so the column statement does not apply without qualification.