The Inverse of a Matrix
Elementary matrices, and the Gauss–Jordan algorithm that turns [A | I] into [I | A⁻¹]
§1Why Bother With an Inverse?
For an ordinary number like $5$, the "undo" of multiplying by $5$ is multiplying by $\tfrac{1}{5}$, because $5 \cdot \tfrac{1}{5} = 1$. The number $\tfrac15$ is the multiplicative inverse of $5$. We want the same thing for matrices: given $A$, find a matrix that multiplies with it to give the identity $I$ — the matrix version of the number $1$.
Why do we care so much? Because if $A$ has an inverse $A^{-1}$, then the linear system $A\mathbf{x} = \mathbf{b}$ is solved in one clean stroke: multiply both sides by $A^{-1}$ to get $\mathbf{x} = A^{-1}\mathbf{b}$. The inverse is a master key that unlocks every system with the same coefficient matrix at once.
§2The Multiplicative Inverse of a Matrix
A square matrix $A$ is invertible if there is a matrix $A^{-1}$ (read "$A$ inverse") such that
$$A^{-1}A = AA^{-1} = I.$$
The matrix $A^{-1}$ is the multiplicative inverse of $A$. A matrix that has no inverse is called singular; one that does is non-singular.
Two requirements are hidden in that definition, and both matter:
$A$ must be a square matrix. Only an $n\times n$ matrix can satisfy both $A^{-1}A = I$ and $AA^{-1} = I$ with the same partner.
$A^{-1}$ must have the same order (size) as $A$. If $A$ is $3\times3$, then $A^{-1}$ is also $3\times3$.
§3How Do We Know an Inverse Exists?
Not every matrix has an inverse — just as the number $0$ has no multiplicative inverse (nothing times $0$ gives $1$). So before hunting for $A^{-1}$, we need a test.
A square matrix $A$ has an inverse if and only if it has full rank — equivalently, $A$ is non-singular. If $A$ is rank-deficient (a singular matrix), then $A^{-1}$ does not exist.
§4The 2×2 Shortcut
For the smallest square matrices there is a formula you can memorise. It is worth knowing cold, because $2\times2$ inverses appear everywhere.
For $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$, the inverse is
$$A^{-1} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix},$$
provided $ad - bc \neq 0$. The quantity $ad - bc$ is the determinant of $A$. If it is zero, $A$ is singular and has no inverse.
The recipe in words: swap the diagonal entries ($a$ and $d$), negate the off-diagonal entries ($b$ and $c$), then divide by the determinant.
Find the inverse of $A = \begin{bmatrix} 2 & 1 \\ 3 & 2 \end{bmatrix}$.
Determinant: $ad - bc = (2)(2) - (1)(3) = 4 - 3 = 1$. Since it is nonzero, $A$ is invertible.
$$A^{-1} = \frac{1}{1}\begin{bmatrix} 2 & -1 \\ -3 & 2 \end{bmatrix} = \begin{bmatrix} 2 & -1 \\ -3 & 2 \end{bmatrix}.$$
Check: $A A^{-1} = \begin{bmatrix}2&1\\3&2\end{bmatrix}\begin{bmatrix}2&-1\\-3&2\end{bmatrix} = \begin{bmatrix}1&0\\0&1\end{bmatrix} = I.$ ✓
Try it yourself — edit any entry and watch the determinant and inverse update live. Push the determinant to zero and the inverse disappears:
Change any entry. The instant the determinant hits zero, the inverse vanishes — that is the knife-edge between invertible and singular.
§5Elementary Matrices — the Secret Ingredient
Here is a fact that looks small but changes everything. Every row operation you perform on a matrix can be achieved by multiplying on the left by a special matrix.
An elementary matrix $E$ is what you get by applying a single row operation to the identity matrix $I$. Multiplying any matrix $A$ on the left by $E$ performs that same row operation on $A$.
Let us see it happen. Take $A = \begin{bmatrix}1 & 2\\-1 & 0\end{bmatrix}$ and the row operation $R_2 \to R_2 + R_1$.
Apply $R_2 \to R_2 + R_1$ to $I = \begin{bmatrix}1&0\\0&1\end{bmatrix}$. Row 2 becomes $(0{+}1,\,1{+}0) = (1,1)$:
$$E = \begin{bmatrix}1&0\\1&1\end{bmatrix}.$$
Now left-multiply $A$ by $E$:
$$EA = \begin{bmatrix}1&0\\1&1\end{bmatrix}\begin{bmatrix}1&2\\-1&0\end{bmatrix} = \begin{bmatrix}1&2\\0&2\end{bmatrix}.$$
The result $\begin{bmatrix}1&2\\0&2\end{bmatrix}$ is exactly what you would get by doing $R_2 \to R_2 + R_1$ directly to $A$. The row operation and the matrix multiplication are the same thing.
Row-reducing $A$ all the way to its reduced echelon form is a sequence of row operations. Each one is a left-multiply by an elementary matrix:
$$E_k \cdots E_3 E_2 E_1 A = \text{(reduced echelon form of } A).$$
If $A$ is invertible, its reduced echelon form is the identity $I$. So $E_k \cdots E_2 E_1 A = I$. But that says the product $E_k \cdots E_2 E_1$ is $A^{-1}$! The very operations that reduce $A$ to $I$, applied to $I$, build $A^{-1}$.
§6The Gauss–Jordan Inversion Algorithm
To find $A^{-1}$: write the augmented matrix $[\,A \mid I\,]$ — the matrix $A$ next to an identity of the same size. Row-reduce until the left block becomes $I$. The right block is then $A^{-1}$:
$$[\,A \mid I\,] \xrightarrow{\text{row operations}} [\,I \mid A^{-1}\,].$$
Write $A$ and $I$ side by side as $[A \mid I]$.
Apply row operations to the entire augmented matrix, aiming to turn the left block into $I$.
When the left block is $I$, the right block is $A^{-1}$.
If a full row of zeros appears in the left block, $A$ cannot reach $I$ — it is singular, and no inverse exists.
§7The Full 3×3 Worked Example — Step by Step
Let us invert the matrix from the notes:
$$A = \begin{bmatrix} 1 & 0 & -1 \\ 3 & 2 & 0 \\ -1 & -1 & 0 \end{bmatrix}.$$
We attach the identity and reduce. Each line below shows the augmented matrix after the stated operation. The vertical bar separates $A$'s side (left) from $I$'s side (right).
$$\left[\begin{array}{ccc|ccc} 1 & 0 & -1 & 1 & 0 & 0 \\ 3 & 2 & 0 & 0 & 1 & 0 \\ -1 & -1 & 0 & 0 & 0 & 1 \end{array}\right]$$
$$\left[\begin{array}{ccc|ccc} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 2 & 3 & -3 & 1 & 0 \\ -1 & -1 & 0 & 0 & 0 & 1 \end{array}\right]$$
$$\left[\begin{array}{ccc|ccc} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 2 & 3 & -3 & 1 & 0 \\ 0 & -1 & -1 & 1 & 0 & 1 \end{array}\right]$$
$$\left[\begin{array}{ccc|ccc} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & -1 & -1 & 1 & 0 & 1 \\ 0 & 2 & 3 & -3 & 1 & 0 \end{array}\right]$$
$$\left[\begin{array}{ccc|ccc} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 1 & 1 & -1 & 0 & -1 \\ 0 & 2 & 3 & -3 & 1 & 0 \end{array}\right]$$
$$\left[\begin{array}{ccc|ccc} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 1 & 1 & -1 & 0 & -1 \\ 0 & 0 & 1 & -1 & 1 & 2 \end{array}\right]$$
$$\left[\begin{array}{ccc|ccc} 1 & 0 & 0 & 0 & 1 & 2 \\ 0 & 1 & 1 & -1 & 0 & -1 \\ 0 & 0 & 1 & -1 & 1 & 2 \end{array}\right]$$
$$\left[\begin{array}{ccc|ccc} 1 & 0 & 0 & 0 & 1 & 2 \\ 0 & 1 & 0 & 0 & -1 & -3 \\ 0 & 0 & 1 & -1 & 1 & 2 \end{array}\right]$$
$$A^{-1} = \begin{bmatrix} 0 & 1 & 2 \\ 0 & -1 & -3 \\ -1 & 1 & 2 \end{bmatrix}.$$
A 30-second check catches almost every arithmetic slip. Compute $A A^{-1}$:
$$\begin{bmatrix} 1 & 0 & -1 \\ 3 & 2 & 0 \\ -1 & -1 & 0 \end{bmatrix} \begin{bmatrix} 0 & 1 & 2 \\ 0 & -1 & -3 \\ -1 & 1 & 2 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} = I.$$ ✓
For instance the $(1,1)$ entry is $(1)(0) + (0)(0) + (-1)(-1) = 1$, and the $(1,2)$ entry is $(1)(1) + (0)(-1) + (-1)(1) = 0$. Every entry lands exactly where the identity needs it.
If at any stage a row of the left block becomes all zeros, $A$ does not have full rank. It is singular, the reduction cannot reach $I$, and $A^{-1}$ does not exist. Stop — there is no inverse to find.
§8A Note on Matrix Multiplication
Solve $3X - 2Y = \begin{bmatrix} 3 & -1 \end{bmatrix}$ where $X = \begin{bmatrix} x_1 & x_2 \end{bmatrix}$ and $Y = \begin{bmatrix} y_1 & y_2 \end{bmatrix}$ are $1\times2$ row matrices.
Decide whether $B = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$ is invertible. If so, find $B^{-1}$.
We can now undo a matrix. Next: using the inverse to solve systems instantly, and the properties that make inverses behave.
With $A^{-1}$ in hand, the system $A\mathbf{x} = \mathbf{b}$ solves as $\mathbf{x} = A^{-1}\mathbf{b}$ — no row reduction needed once you have the inverse. We will also meet the rules $(AB)^{-1} = B^{-1}A^{-1}$ and $(A^{-1})^{-1} = A$, and see how inverses connect to determinants. First, though, make sure you are comfortable multiplying matrices — bring questions to the tutorial.