Elementary Matrices & Solving Systems
The three types of elementary matrices, how to invert them by sight, and using inverses to crack linear systems fast
§1Quick Recall — What Is an Elementary Matrix?
Last lecture we met elementary matrices in passing while building the inverse. Today they take centre stage, so let us refresh the idea cleanly before we go deeper.
An elementary matrix $E$ is what you get by applying exactly one elementary row operation to the identity matrix $I$. Left-multiplying any matrix $A$ by $E$ performs that same row operation on $A$: the matrix $EA$ is $A$ with that one operation done to it.
Since there are exactly three kinds of elementary row operation, there are exactly three types of elementary matrix. We will take them one at a time, and for each one learn the single most useful fact: how to write down its inverse instantly, just by reversing the operation.
§2Type I — Swapping Two Rows
The first elementary row operation swaps two rows. Apply it to $I$ and you get a Type I elementary matrix.
Start with $I_3$ and swap rows 1 and 2 ($R_1 \leftrightarrow R_2$):
$$I = \begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix} \xrightarrow{R_1 \leftrightarrow R_2} E_1 = \begin{bmatrix}0&1&0\\1&0&0\\0&0&1\end{bmatrix}.$$
Now $E_1 A$ swaps rows 1 and 2 of any matrix $A$ it multiplies.
What undoes a swap? The same swap again. Swapping rows 1 and 2, then swapping them back, returns the original. So a Type I matrix is its own inverse: $E_1^{-1} = E_1$, equivalently $E_1 E_1 = I$. The reverse operation of "$R_1 \leftrightarrow R_2$" is just "$R_1 \leftrightarrow R_2$".
§3Type II — Multiplying a Row by a Constant
The second operation multiplies one row by a nonzero constant $k$. Apply it to $I$ for a Type II matrix.
Multiply row 2 of $I_3$ by $5$ ($R_2 \to 5R_2$):
$$I = \begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix} \xrightarrow{R_2 \to 5R_2} E_2 = \begin{bmatrix}1&0&0\\0&5&0\\0&0&1\end{bmatrix}.$$
Multiplying $E_2 A$ scales row 2 of $A$ by $5$.
What undoes "multiply row 2 by $5$"? Divide row 2 by $5$ — that is, multiply it by $\tfrac{1}{5}$. So the inverse just replaces $k$ with $\tfrac{1}{k}$ in the same slot:
$$E_2 = \begin{bmatrix}1&0&0\\0&5&0\\0&0&1\end{bmatrix} \quad\Longrightarrow\quad E_2^{-1} = \begin{bmatrix}1&0&0\\0&\tfrac{1}{5}&0\\0&0&1\end{bmatrix}.$$
§4Type III — Adding a Multiple of One Row to Another
The third operation adds (or subtracts) a multiple of one row to another row. This is the workhorse of elimination. Apply it to $I$ for a Type III matrix.
Add $2$ times row 1 to row 3 of $I_3$ ($R_3 \to R_3 + 2R_1$):
$$I = \begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix} \xrightarrow{R_3 \to R_3 + 2R_1} E_3 = \begin{bmatrix}1&0&0\\0&1&0\\2&0&1\end{bmatrix}.$$
The $2$ lands in the $(3,1)$ slot — row 3, column 1 — recording "$2$ of row 1 added into row 3."
What undoes "add $2R_1$ to $R_3$"? Subtract $2R_1$ from $R_3$. So the inverse just flips the sign of the off-diagonal entry:
$$E_3 = \begin{bmatrix}1&0&0\\0&1&0\\2&0&1\end{bmatrix} \quad\Longrightarrow\quad E_3^{-1} = \begin{bmatrix}1&0&0\\0&1&0\\-2&0&1\end{bmatrix}.$$
The three types at a glance
Swap two rows.
Multiply a row by k ≠ 0.
Add k times one row to another.
§5A Quick Detour — Idempotent Matrices
While we are looking at special matrices, here is a small but important family worth knowing.
A square matrix $M$ is idempotent if multiplying it by itself returns itself: $M^2 = M$. Applying it once or a hundred times gives the same result — it "settles" after one step.
(a) The identity: $I^2 = I$. ✓
(b) The zero matrix: $0^2 = 0$. ✓
(c) $\begin{bmatrix}1&0\\0&0\end{bmatrix}$: squaring gives $\begin{bmatrix}1&0\\0&0\end{bmatrix}$. ✓
(d) A less obvious one: $\begin{bmatrix}2&-2\\1&-1\end{bmatrix}^2 = \begin{bmatrix}2&-2\\1&-1\end{bmatrix}$. ✓ (Multiply it out and check!)
§6The Shortcut — Inverses Without the Full Algorithm
Here is the payoff of knowing how to invert each type by sight. It lets us skip the long Gauss–Jordan method entirely in many situations.
Recall from Lecture 6 that reducing $A$ to $I$ is a chain of elementary operations:
$$E_k \cdots E_2 E_1 A = I.$$
This says $E_k \cdots E_2 E_1 = A^{-1}$. But watch what happens if we instead want $A$ back from a known product of elementary matrices.
Suppose $E_1$ and $E_2$ are elementary matrices and $(E_1 E_2) A = I$. Then
$$A = (E_1 E_2)^{-1} = E_2^{-1} E_1^{-1}.$$
Two things to notice. First, the inverse of a product reverses the order: $(E_1 E_2)^{-1} = E_2^{-1} E_1^{-1}$ — socks-then-shoes becomes shoes-then-socks. Second, each $E_i^{-1}$ is found instantly by reversing its operation, using the rules from $\S2$–$\S4$. No row reduction needed.
Suppose $E_1 = \begin{bmatrix}1&0\\-3&1\end{bmatrix}$ (Type III, $R_2 \to R_2 - 3R_1$) and $E_2 = \begin{bmatrix}1&0\\0&2\end{bmatrix}$ (Type II, $R_2 \to 2R_2$), and we know $E_2 E_1 A = I$. Find $A$.
§7Solving Systems Fast With the Inverse
Now the most practical use of everything so far. If $A$ is invertible, every system $A\mathbf{x} = \mathbf{b}$ has a one-line solution.
If $A$ is invertible, the system $A\mathbf{x} = \mathbf{b}$ has the unique solution
$$\mathbf{x} = A^{-1}\mathbf{b}.$$
Multiply both sides of $A\mathbf{x} = \mathbf{b}$ on the left by $A^{-1}$: $A^{-1}A\mathbf{x} = A^{-1}\mathbf{b}$, and since $A^{-1}A = I$, this is $\mathbf{x} = A^{-1}\mathbf{b}$.
Solve $\begin{cases} 2x + 3y = 8 \\ x - y = -1 \end{cases}$ using the inverse.
In matrix form $A\mathbf{x} = \mathbf{b}$ with $A = \begin{bmatrix}2&3\\1&-1\end{bmatrix}$, $\mathbf{b} = \begin{bmatrix}8\\-1\end{bmatrix}$.
Determinant: $\det A = (2)(-1) - (3)(1) = -2 - 3 = -5 \neq 0$, so $A$ is invertible. Using the 2×2 formula:
$$A^{-1} = \frac{1}{-5}\begin{bmatrix}-1 & -3\\-1 & 2\end{bmatrix} = \begin{bmatrix}\tfrac15 & \tfrac35\\[2pt] \tfrac15 & -\tfrac25\end{bmatrix}.$$
Then
$$\mathbf{x} = A^{-1}\mathbf{b} = \begin{bmatrix}\tfrac15 & \tfrac35\\[2pt] \tfrac15 & -\tfrac25\end{bmatrix}\begin{bmatrix}8\\-1\end{bmatrix} = \begin{bmatrix}\tfrac85 - \tfrac35\\[2pt] \tfrac85 + \tfrac25\end{bmatrix} = \begin{bmatrix}1\\2\end{bmatrix}.$$
So $x = 1$, $y = 2$. Check: $2(1)+3(2) = 8$ ✓ and $1 - 2 = -1$ ✓.
Solve $A\mathbf{x} = \mathbf{b}$ where $A = \begin{bmatrix}1&0&-1\\3&2&0\\-1&-1&0\end{bmatrix}$ and $\mathbf{b} = \begin{bmatrix}1\\8\\-3\end{bmatrix}$.
We already found this $A^{-1}$ in Lecture 6 by the $[A \mid I]$ method:
$$A^{-1} = \begin{bmatrix}0&1&2\\0&-1&-3\\-1&1&2\end{bmatrix}.$$
§8Exercises — Nicholson §2.5
Let $A = \begin{bmatrix}1&2\\-1&1\end{bmatrix}$ and $C = \begin{bmatrix}-1&1\\2&1\end{bmatrix}$.
(a) Find elementary matrices $E_1$ and $E_2$ such that $C = E_2 E_1 A$.
(b) Show there is no elementary matrix $E$ such that $C = EA$.
(a) Is $I$ an elementary matrix?
(b) Is $0$ (the zero matrix) an elementary matrix?
(a) $A = \begin{bmatrix}2&1&3\\-1&1&2\end{bmatrix}$, $B = \begin{bmatrix}1&-1&-2\\3&0&1\end{bmatrix}$.
(b) $A = \begin{bmatrix}2&-1&0\\1&1&1\end{bmatrix}$, $B = \begin{bmatrix}3&0&1\\2&-1&0\end{bmatrix}$.
(a) Show that $E^{\mathsf{T}}$ is also elementary, of the same type as $E$.
(b) Show $E^{\mathsf{T}} = E$ if $E$ is of Type I or Type II.
Let $A = \begin{bmatrix}1&2\\1&-3\end{bmatrix}$ and $B = \begin{bmatrix}5&2\\-5&-3\end{bmatrix}$. Find an elementary matrix $F$ such that $AF = B$.
Elementary matrices, inverses, and fast solving are now in your toolkit. Next: determinants — the single number that decides invertibility.
We have leaned on "$\det \neq 0$ means invertible" a few times. Next lecture we define the determinant properly, learn to compute it for any size, and see how it ties together everything — rank, invertibility, and the geometry of how a matrix stretches space.