Matrices as Transformations
From combinations of vectors to rank, and what a matrix can lose
1 From Vector Combination to Matrix
Previously, we discussed some basic concepts and operations of vectors. Now, let’s start with three vectors:
\[ \vec{u}= \begin{bmatrix}1\\-1\\0\end{bmatrix}, \quad \vec{v}= \begin{bmatrix}0\\1\\-1\end{bmatrix}, \quad \vec{w}= \begin{bmatrix}0\\0\\1\end{bmatrix} \]
Suppose we scale these three vectors by \(x_1\), \(x_2\), and \(x_3\), respectively, and then add them together:
\[ \begin{aligned} x_1\vec{u}+x_2\vec{v}+x_3\vec{w} &= x_1\begin{bmatrix}1\\-1\\0\end{bmatrix} +x_2\begin{bmatrix}0\\1\\-1\end{bmatrix} +x_3\begin{bmatrix}0\\0\\1\end{bmatrix}\\ &= \begin{bmatrix} x_1\\ x_2-x_1\\ x_3-x_2 \end{bmatrix} \end{aligned} \]
This is quite easy to understand. We have three vectors, and we can stretch, shrink, or even reverse their directions by multiplying them by different scalars. Finally, we add them together to obtain a new vector (Figure 1).
But we can rewrite this expression in a simpler form:
\[ \underbrace{ \begin{bmatrix} 1&0&0\\ -1&1&0\\ 0&-1&1 \end{bmatrix}}_A \underbrace{ \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}}_{\vec{x}} = \underbrace{ \begin{bmatrix} x_1\\ x_2-x_1\\ x_3-x_2 \end{bmatrix}}_{\vec{b}} \]
Or simply:
\[ A\vec{x}=\vec{b} \]
Although this expression looks simpler, I think it introduces a very interesting change in perspective.
Previously, \(x_1\), \(x_2\), and \(x_3\) were just three coefficients used to scale the vectors. Now, we combine these three numbers into a new vector, \(\vec{x}\).
From this perspective, the matrix \(A\) performs a linear transformation on the input vector \(\vec{x}\). It is called linear because it respects the two basic operations on vectors: \(A(\vec{x}+\vec{y})=A\vec{x}+A\vec{y}\) and \(A(c\vec{x})=cA\vec{x}\). Adding or scaling the inputs first and then transforming them gives the same result as transforming first and then adding or scaling.
We can observe the transformation directly:
\[ \begin{aligned} x_1&\longrightarrow x_1\\ x_2&\longrightarrow x_2-x_1\\ x_3&\longrightarrow x_3-x_2 \end{aligned} \]
In other words, the original vector \((x_1,x_2,x_3)\) is transformed into another vector. Figure 2 shows this happening to the whole space at once.
For any given input \(\vec{x}\), we can apply the matrix \(A\) and obtain the corresponding output \(\vec{b}\).
Actually, these two perspectives describe exactly the same mathematical operation.
One perspective treats matrix-vector multiplication as a linear combination of the columns of a matrix. The other treats it as a linear transformation applied to an input vector.
There is also a third way to read \(A\vec{x}\), and it is how we usually compute it by hand: each entry of \(\vec{b}\) is the dot product of one row of \(A\) with \(\vec{x}\). The second row of \(A\) is \((-1,1,0)\), and \((-1,1,0)\cdot(x_1,x_2,x_3)=x_2-x_1\), which is exactly the second entry of \(\vec{b}\). So the dot product from the previous post is hiding inside every matrix multiplication.
This is where I started to understand matrices from a geometric perspective: a matrix is not just a collection of numbers. It can also represent a linear transformation in space.
2 Matrix as a Linear Transformation
If a matrix represents a linear transformation, a natural question arises: what actually happens to vectors in space after we apply this transformation?
Instead of considering a single vector, let’s consider an entire set of vectors.
Suppose our input vector \(\vec{x}\) can take any value in three-dimensional space. What kind of set will these vectors form after applying the matrix \(A\)?
Or let’s make the question more specific.
Suppose all input vectors are restricted to a three-dimensional unit ball:
\[ x_1^2+x_2^2+x_3^2\leq1 \]
What will happen to this ball after a linear transformation?
Could it be stretched into an ellipsoid? Could it be compressed onto a two-dimensional plane, becoming an elliptical disk? Could it even collapse into a line segment or a single point?
All of these are possible.1 Figure 3 shows the same idea one dimension lower, where the ball is a disk and its image is an ellipse, a line segment, or a single point.
1 What shapes can a linear transformation produce? My first idea was that a linear transformation could turn a ball into almost any shape, such as a cube or a rectangular box. When I explored this with AI, it pointed out that this is not possible: a ball under a linear transformation can only become an ellipsoid or one of its degenerate forms, such as an elliptical disk, a line segment, or a point. Turning a ball into a cube would need a nonlinear transformation. I haven’t studied why this restriction exists yet, so I’ll leave it for later.
I find this perspective particularly interesting. Instead of studying only how one vector changes, we can investigate how an entire set of vectors is transformed.
From this perspective, matrices can describe various linear transformations, such as stretching, compression, rotation, reflection, and projection.
We can also investigate which properties are preserved during these transformations, such as vector lengths, angles between vectors, and the dimension of the resulting space.
2.1 Some Examples of Linear Transformations
Let’s start with a simple example: the identity matrix.
\[ I= \begin{bmatrix} 1&0&0\\ 0&1&0\\ 0&0&1 \end{bmatrix} \]
For any vector:
\[ I\vec{x}=\vec{x} \]
It does not change any coordinate of the input vector. Whatever the original vector is, it remains exactly the same after the transformation.
Now consider two other matrices:
\[ P_1= \begin{bmatrix} 1&0&0\\ 0&0&0\\ 0&0&0 \end{bmatrix} \]
\[ P_2= \begin{bmatrix} 1&0&0\\ 0&1&0\\ 0&0&0 \end{bmatrix} \]
Their transformations are:
\[ P_1 \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix}x_1\\0\\0\end{bmatrix} \]
\[ P_2 \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix}x_1\\x_2\\0\end{bmatrix} \]
Figure 4 shows what these matrices do to the unit ball.
The first matrix preserves only the \(x_1\) coordinate and removes the other two coordinates.
If the original input set is a three-dimensional ball, the transformed set will lie entirely on the \(x_1\)-axis, forming a line segment.
The second matrix preserves \(x_1\) and \(x_2\) but removes \(x_3\).
The same three-dimensional ball will become a disk on the \(x_1x_2\)-plane.
In other words, these matrices reduce the dimension of the original set and discard some information.
If we modify some entries of the matrix, for example changing one of the nonzero diagonal entries of \(P_2\) from 1 to 2, the resulting disk becomes an elliptical disk (the fourth example in Figure 4).
What if a matrix keeps all three coordinates but scales them by different amounts? For example:
\[ S= \begin{bmatrix} 1.5&0&0\\ 0&1&0\\ 0&0&0.5 \end{bmatrix} \]
Its transformation is:
\[ S \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix}1.5x_1\\x_2\\0.5x_3\end{bmatrix} \]
Pick \(S\) in Figure 4: the ball is stretched along the \(x_1\)-axis and squeezed along the \(x_3\)-axis, so it becomes an ellipsoid. Unlike \(P_1\) and \(P_2\), \(S\) does not remove any coordinate, so the result is still three-dimensional and no information is lost.
Now consider another matrix:
\[ R= \begin{bmatrix} 1&0&0\\ 0&0&-1\\ 0&1&0 \end{bmatrix} \]
It performs the following transformation:
\[ R\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix}x_1\\-x_3\\x_2\end{bmatrix} \]
This transformation has an interesting property.
It does not change the length of the vector:
\[ \begin{aligned} \|R\vec{x}\|^2 &=x_1^2+(-x_3)^2+x_2^2\\ &=x_1^2+x_2^2+x_3^2\\ &=\|\vec{x}\|^2 \end{aligned} \]
Geometrically, this matrix represents a rotation of \(90^\circ\) around the \(x_1\)-axis. Pick \(R\) in Figure 4: the ball looks unchanged, but the arrows show that it has turned.
We can also calculate its determinant:2
2 Determinants, rotations, and scaling. At first I thought that every matrix with determinant 1 represents a rotation. AI pointed out that this is not true. The determinant describes how oriented area or volume is scaled, not how vector length is scaled. The condition for preserving the lengths of all vectors is \[ A^TA=I \] and a matrix satisfying it is called an orthogonal matrix. AI also told me that breaking a matrix into rotations, reflections, and scaling along different directions is related to singular value decomposition (SVD), \(A=U\Sigma V^T\). I haven’t studied any of these yet, so I’ll leave them here and explore them later.
\[ \det(R)=1 \]
2.2 A Thought About Neural Networks and Computation
Understanding matrices as linear transformations also reminds me of neural networks.
If we temporarily ignore the nonlinear components, many computations in neural networks can be understood as a sequence of linear transformations applied to input data using a large number of parameters.
Of course, actual neural networks are much more complicated. We are ignoring bias terms, activation functions, and many other structures for now.
But from this highly simplified perspective, we can imagine an input passing through a series of matrix operations, producing new representations and eventually an output.
There is a catch in this simplification, though. Applying several matrices one after another is itself just one matrix: \(W_3(W_2(W_1\vec{x}))=(W_3W_2W_1)\vec{x}\).3 Without the nonlinear parts, a hundred layers could do nothing that a single layer couldn’t, and they could still only turn a ball into an ellipsoid. The nonlinear activation functions between the layers are what let a network bend space into more complicated shapes, which is the same question as turning a ball into a cube in note 1.4
3 Is the transformation unique? Walking downstairs one day, I had this thought. Suppose a linear transformation takes \(\vec{a}\) to \(\vec{b}\). It might be one matrix \(A\), or several, \(B\), \(C\) and \(D\), and then \(BCD=A\). My first idea was that if \(A\) is invertible, this transformation must be unique, and if \(A\) is not invertible, it might not be. AI pointed out that I was mixing up two questions. A single pair \(\vec{a}\), \(\vec{b}\) never determines a matrix: \(A\) has \(n^2\) unknowns, but \(A\vec{a}=\vec{b}\) gives only \(n\) equations. To determine \(A\), we need many pairs. If the inputs are the columns of \(X\) and the outputs are the columns of \(Y\), then \(AX=Y\) has a unique solution exactly when \(X\) has rank \(n\). So it is the data that must be invertible, not \(A\). And even when \(A\) is unique, its factors are not, because \(BCD=(BM)(M^{-1}C)D\) for any invertible \(M\).
4 Training as a search for a transformation. I then thought that a neural network does the same thing with linear and nonlinear transformations, so training is the search for such a transformation. I assumed that a fully connected network is only linear however many parameters it has, that later networks added some nonlinearity with modest results, and that the Transformer succeeded because its nonlinearity let it fit the data much better. AI pointed out that a fully connected network with activation functions is already nonlinear. A result called the universal approximation theorem says that such a network, made wide enough, can approximate any continuous function on a bounded region. It also told me that fitting the training data is the wrong measure, because a lookup table fits it perfectly. What matters is generalization, how well the network does on inputs it has never seen. And it told me that the Transformer mainly changed two things. Its attention computes its mixing matrix from the input itself, and it can be trained in parallel over long sequences, which made huge scale practical. AI added that this is the common explanation, not a proven one: why these models generalize so well is still not well understood, and some other architectures come fairly close at a similar scale.
So the problem, as I understand it now, is this: given a huge set of pairs \((\vec{a}_i,\vec{b}_i)\), choose a family of functions and search it for an \(f\) whose outputs \(f(\vec{a}_i)\) come close to the \(\vec{b}_i\), in a way that carries over to new inputs. I haven’t studied any of this yet, so I’ll leave it here.
This also makes me think about why GPUs are particularly suitable for these computations.
Matrix multiplication contains a large number of multiply-and-add operations that can be executed in parallel, and GPUs are designed to perform large-scale parallel computation efficiently.5
5 An idea about physical matrix multipliers. I also had an interesting thought: if some physical laws naturally perform addition or linear superposition, could we build a physical device that performs matrix multiplication directly, instead of relying entirely on conventional digital logic circuits? For example, could we use electrical currents, voltages, light, or certain physical properties of materials to perform computations through physical processes?
I later learned that related approaches already exist, such as analog computing, optical computing, and memristor crossbar architectures. In a memristor crossbar the idea is quite literal: each crossing point stores a conductance \(G\), Ohm’s law \(I=GV\) does the multiplication, and Kirchhoff’s current law adds up the currents flowing into each wire.
However, even if some physical operations can be performed with massive parallelism, it does not automatically mean that arbitrary-sized matrix multiplication can achieve \(O(1)\) complexity. Hardware scale, input/output costs, numerical precision, and data movement still matter.
This idea is somewhat outside the main topic of this post, so I may explore it separately in the future.
3 From Forward Transformation to the Inverse Problem
Now let’s return to our original equation:
\[ A\vec{x}=\vec{b} \]
So far, we have assumed that \(\vec{x}\) is known, and our task is to calculate \(\vec{b}\).
We have already discussed two perspectives.
The first is to consider three known vectors \(\vec{u}\), \(\vec{v}\), and \(\vec{w}\), together with three known coefficients \(x_1\), \(x_2\), and \(x_3\), and calculate their linear combination.
The second is to consider a known input vector \(\vec{x}\) and apply a linear transformation represented by the matrix \(A\) to obtain the output vector \(\vec{b}\).
Although these perspectives are different, they produce exactly the same result.
Now, let’s reverse the question.
Suppose the matrix \(A\) is known, and the output vector \(\vec{b}\) is also known. Can we recover the original input vector \(\vec{x}\)?
From the perspective of linear combinations, we are asking:
How should we combine \(\vec{u}\), \(\vec{v}\), and \(\vec{w}\) to produce a specific vector \(\vec{b}\)?
From the perspective of linear transformations, we are asking:
If we know the transformation and its output, can we recover the original input?
For an invertible matrix, we can write:
\[ \vec{x}=A^{-1}\vec{b} \]
But what exactly is \(A^{-1}\)?
Let’s return to our previous example:
\[ \begin{bmatrix} 1&0&0\\ -1&1&0\\ 0&-1&1 \end{bmatrix} \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix}b_1\\b_2\\b_3\end{bmatrix} \]
We can expand this into a system of equations:
\[ \begin{cases} x_1=b_1\\ x_2-x_1=b_2\\ x_3-x_2=b_3 \end{cases} \]
Then solve for the original variables:
\[ \begin{cases} x_1=b_1\\ x_2=b_1+b_2\\ x_3=b_1+b_2+b_3 \end{cases} \]
We can rewrite these equations in matrix form:
\[ \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} = \underbrace{ \begin{bmatrix} 1&0&0\\ 1&1&0\\ 1&1&1 \end{bmatrix}}_{A^{-1}} \begin{bmatrix}b_1\\b_2\\b_3\end{bmatrix} \]
So the inverse matrix is:6
6 Look at what the two matrices do. \(A\) takes differences of neighbouring entries, and \(A^{-1}\) takes running sums: \(x_3=b_1+b_2+b_3\). Summing undoes differencing, the same way integration undoes differentiation, and this pair is a small discrete version of that.
\[ A^{-1}= \begin{bmatrix} 1&0&0\\ 1&1&0\\ 1&1&1 \end{bmatrix} \]
This means that for any given output vector \(\vec{b}\), we can find exactly one corresponding input vector \(\vec{x}\).
Therefore, the matrix \(A\) is invertible.
But does every matrix have this property?
4 Linear Dependence and Independence
Let’s modify our original matrix slightly.
We keep \(\vec{u}\) and \(\vec{v}\) unchanged, but replace the third vector with:
\[ \vec{w}^*= \begin{bmatrix}-1\\0\\1\end{bmatrix} \]
Now we have a new matrix:
\[ C= \begin{bmatrix} 1&0&-1\\ -1&1&0\\ 0&-1&1 \end{bmatrix} \]
The transformation becomes:
\[ C\vec{x}= \begin{bmatrix} x_1-x_3\\ x_2-x_1\\ x_3-x_2 \end{bmatrix} = \begin{bmatrix}b_1\\b_2\\b_3\end{bmatrix} \]
Let’s try to solve this equation:
\[ \begin{cases} x_1-x_3=b_1\\ x_2-x_1=b_2\\ x_3-x_2=b_3 \end{cases} \]
If we add all three equations together, something interesting happens.
Every \(x\) term cancels out:
\[ 0=b_1+b_2+b_3 \]
Therefore:
\[ b_1+b_2+b_3=0 \]
What does this mean?
First, no matter what input vector \(\vec{x}\) we choose, the output of the transformation must satisfy this condition.
Geometrically, all vectors satisfying this equation lie on a plane in three-dimensional space:
\[ b_1+b_2+b_3=0 \]
In other words, although the original input vectors can fill the entire three-dimensional space, their outputs after applying \(C\) are restricted to a two-dimensional plane (Figure 5).
We have lost one dimension.
4.1 What Happens to the Missing Dimension?
Let’s consider a special case:
\[ \vec{b}= \begin{bmatrix}0\\0\\0\end{bmatrix} \]
The equations become:
\[ \begin{cases} x_1-x_3=0\\ x_2-x_1=0\\ x_3-x_2=0 \end{cases} \]
Therefore:
\[ x_1=x_2=x_3=c \]
where \(c\) can be any real number.
This means that every vector of the form:
\[ \vec{x}= c\begin{bmatrix}1\\1\\1\end{bmatrix} \]
will be transformed into the zero vector:
\[ C\begin{bmatrix}c\\c\\c\end{bmatrix} = \begin{bmatrix}0\\0\\0\end{bmatrix} \]
From a geometric perspective, all these vectors lie on a line passing through the origin, in the direction of \((1,1,1)\).
The entire line is collapsed into a single point: the origin.
Does this mean that all information along this direction has been erased?
Actually, there is something even more interesting.
For any input vector \(\vec{x}\), suppose we add an arbitrary amount in the direction \((1,1,1)\):
\[ \vec{x}'= \vec{x} +c\begin{bmatrix}1\\1\\1\end{bmatrix} \]
Then:
\[ C\vec{x}'=C\vec{x} \]
Even though the two input vectors are different, they produce exactly the same output.
The matrix \(C\) can no longer distinguish between these inputs. In Figure 5, this is why 27 different inputs give only 19 different outputs.
This gives us a geometric interpretation of information loss: different inputs are mapped to the same output, so the original input can no longer be uniquely recovered.
In this example, the three-dimensional input space is mapped onto a two-dimensional output plane.
Interestingly, the direction that gets erased is perpendicular to the output plane (Figure 6).7
7 The lost direction and the output plane. I first assumed that the erased direction is always perpendicular to the output plane. AI pointed out that this is not true in general: the null space and the image of a matrix are not necessarily perpendicular. Here it just happens to be the case. I don’t understand why yet, so I’ll leave it for later.
4.2 Looking at the Columns of the Matrix
Now let’s return to the matrix \(C\) itself.
Its three column vectors are:
\[ \vec{u}= \begin{bmatrix}1\\-1\\0\end{bmatrix}, \quad \vec{v}= \begin{bmatrix}0\\1\\-1\end{bmatrix}, \quad \vec{w}^*= \begin{bmatrix}-1\\0\\1\end{bmatrix} \]
We can immediately notice that:
\[ \vec{u}+\vec{v}+\vec{w}^*=\vec{0} \]
Or:
\[ \vec{w}^*=-\vec{u}-\vec{v} \]
The third vector can be expressed as a linear combination of the first two vectors.
This means that although we have three vectors, there are only two independent directions.
The first two vectors already span a two-dimensional plane, and the third vector happens to lie on the same plane (Figure 6).
Therefore, no matter how we scale and combine these three vectors, we can only produce vectors on that plane. We cannot produce any vector outside it.
This is linear dependence.
In contrast, our original three vectors \(\vec{u}\), \(\vec{v}\), and \(\vec{w}\) were linearly independent (switch Figure 6 to \(A\)).
In that example, the third vector did not lie on the plane spanned by the first two vectors. It introduced a new independent direction.
Together, those three vectors could span the entire three-dimensional space.
But in matrix \(C\), the third vector does not provide a new independent direction.
As a result, its columns can only span a two-dimensional plane.
5 Rank and Dimension
From the previous examples, we can begin to understand a very important concept: rank.
For a matrix, we can ask how many dimensions are spanned by its column vectors.
Consider our original matrix:
\[ A= \begin{bmatrix} 1&0&0\\ -1&1&0\\ 0&-1&1 \end{bmatrix} \]
Its three column vectors are linearly independent and span the entire three-dimensional space.
Therefore:
\[ \operatorname{rank}(A)=3 \]
Now consider the modified matrix:
\[ C= \begin{bmatrix} 1&0&-1\\ -1&1&0\\ 0&-1&1 \end{bmatrix} \]
Its three column vectors are linearly dependent and span only a two-dimensional plane.
Therefore:
\[ \operatorname{rank}(C)=2 \]
From the perspective of linear transformations, rank tells us how many independent directions exist in the output space, or equivalently, the dimension of the space that the transformation can reach.
For a transformation from three-dimensional space to three-dimensional space:
- Rank 3 means the outputs can fill the entire three-dimensional space.
- Rank 2 means the outputs are restricted to a two-dimensional plane.
- Rank 1 means the outputs lie on a line.
- Rank 0 means every input is transformed into the zero vector.
So we can think of rank as the number of independent dimensions preserved in the output of a linear transformation.
Of course, this is about the dimension of the entire output space, not the number of coordinates used to represent an individual vector.8
8 Rank–nullity theorem. AI told me that what we saw with \(C\), a two-dimensional output and one input direction erased, is an instance of a general theorem: \[ \operatorname{rank}(A)+\operatorname{nullity}(A)=n \] where \(n\) is the dimension of the input space and the nullity is the dimension of the inputs that get mapped to zero. I haven’t studied it yet.
6 Rank, Linear Dependence, and Invertibility
Now we can connect several concepts that initially seemed unrelated.
For our original matrix \(A\), the three column vectors are linearly independent. Its rank is 3, and its columns span the entire three-dimensional space.
Therefore, for any given \(\vec{b}\), there is exactly one solution:
\[ \vec{x}=A^{-1}\vec{b} \]
This means that \(A\) is invertible.
But the situation is different for matrix \(C\).
Its three column vectors are linearly dependent, and its rank is only 2.
As a result, we encounter two possible situations.
First case: No solution.
If the given output vector \(\vec{b}\) does not satisfy:
\[ b_1+b_2+b_3=0 \]
then it does not lie on the plane spanned by the three column vectors.
No matter how we choose \(x_1\), \(x_2\), and \(x_3\), their linear combination cannot produce this vector.
Therefore, the equation has no solution.
Second case: Infinitely many solutions.
If the given output vector \(\vec{b}\) lies on that plane, then we can find a corresponding input.
However, because the three column vectors are linearly dependent, the third vector can be expressed as a combination of the first two.
This means that the same output can be produced by infinitely many different combinations of coefficients.
For example, for any \(\vec{b}\) satisfying \(b_1+b_2+b_3=0\), we can choose:
\[ \begin{aligned} x_1&=t\\ x_2&=t+b_2\\ x_3&=t-b_1 \end{aligned} \]
where \(t\) can be any real number.
By changing \(t\), we obtain infinitely many different input vectors \(\vec{x}\) that produce exactly the same output \(\vec{b}\).
These solutions have a familiar shape. Written as vectors,
\[ \vec{x}= \begin{bmatrix}0\\b_2\\-b_1\end{bmatrix} +t\begin{bmatrix}1\\1\\1\end{bmatrix}, \]
they are one particular solution plus any point on the line that \(C\) sends to zero, the same line we found earlier. All the solutions differ from each other only along the erased direction. Figure 7 shows both cases in one picture.
This explains why matrix \(C\) is not invertible.
One dimension of information has been lost. Even if we know the output, we cannot uniquely determine the original input.
From this perspective, linear dependence, rank, and invertibility are closely related. They describe different aspects of the same underlying problem.
Linear dependence tells us whether the column vectors of a matrix contain redundant directions.
Rank tells us how many dimensions those column vectors can span.
Invertibility tells us whether we can uniquely recover the original input from the output of a transformation.
For a \(3\times3\) matrix, if its three column vectors are linearly independent, then its rank is 3, and the matrix is invertible.
If the three column vectors are linearly dependent, then its rank is less than 3, and the matrix is not invertible.
We can also use the determinant to determine whether a square matrix is invertible:
\[ \det(A)\neq0 \quad\Longleftrightarrow\quad A\text{ is invertible} \]
If:
\[ \det(A)=0 \]
then the matrix does not have an ordinary inverse.
For our two examples, \(\det(A)=1\) (for a triangular matrix, the determinant is the product of the diagonal entries), and \(\det(C)=0\).
In fact, for an invertible square matrix, we have:
\[ A^{-1}=\frac{\operatorname{adj}(A)}{\det(A)} \]
where \(\operatorname{adj}(A)\) is the adjugate matrix.
If the determinant is zero, the denominator of this formula becomes zero, so the inverse cannot be obtained from this expression.
But the deeper geometric reason for non-invertibility is what we discussed earlier: the transformation maps different inputs to the same output, making it impossible to uniquely recover the original input.
The determinant fits this picture too. As note 2 says, the determinant is the factor by which a transformation scales volume. \(C\) flattens all of three-dimensional space onto a plane, and anything flat has zero volume, so \(\det(C)\) has to be zero. A zero determinant is not a separate rule. It is the same collapse, measured as volume. You can watch this happen in Figure 3: as the two columns line up, the shaded parallelogram gets thinner and its area goes to zero.
This is something I have gradually realized while revisiting linear algebra.
Many concepts that initially appeared unrelated, such as matrices, linear combinations, linear dependence, rank, determinants, and inverse matrices, can actually be understood through the same geometric perspective of linear transformations.
When we stop focusing only on how to calculate formulas and instead try to understand what happens to vectors and entire spaces during a transformation, the connections between these concepts become much clearer.