Row Operations & Gaussian Elimination
How to actually solve a system — and tell whether it has one, none, or infinitely many solutions
Last lecture we built the language — matrices, augmented matrices, and row-echelon form. Today we put it to work. By the end you will be able to take any system of linear equations, however large, and either solve it completely or prove it has no solution — using a single, mechanical procedure that never fails.
§1Three Questions We Must Answer
Everything in this lecture is aimed at three practical questions about a system of linear equations:
You will hear the word consistent constantly from now on, so let us pin it down. A system of equations is consistent if it has at least one solution — the equations agree with one another, and some choice of values satisfies them all. A system is inconsistent if it has no solution — the equations contradict each other, like demanding $x+y=3$ and $x+y=5$ at the same time. "Consistent" does not mean the solution is unique; a consistent system can have one solution or infinitely many. It simply means a solution exists.
§2The Motto of This Course
Here is what this means in practice. Suppose someone hands you the system $x = 3,\; y = 5$. You could dutifully write the augmented matrix, perform row operations, compute the rank, and announce the solution. Or you could just read it: $x=3$, $y=5$. Done. Firing the full Gaussian-elimination cannon at a system that is already solved is the mathematical equivalent of using artillery on an insect.
A second example: to solve $\frac{x}{2} = 4$, you do not need linear algebra at all — multiply both sides by 2. But to solve a tangle of six equations in six unknowns, mental arithmetic will collapse, and then the cannon is exactly right. The skill we are building this term is not just "knowing methods" — it is knowing which method fits.
§3Recall: Row-Echelon Form — and Why We Want It
From Lecture 1: a matrix is in row-echelon form (REF) when (1) all zero rows are at the bottom, (2) the first nonzero entry of each nonzero row is a leading 1, and (3) each leading 1 lies strictly to the right of the one above. Why do we care so much about this shape? Because a system in REF practically solves itself.
Look at this augmented matrix in REF, and convert it back to equations:
$$\left(\begin{array}{ccc|c} 1 & 2 & -1 & 3 \\ 0 & 1 & 4 & 5 \\ 0 & 0 & 1 & 2 \end{array}\right)$$
$$\begin{cases} x + 2y - z = 3 \\ \phantom{x+2}y + 4z = 5 \\ \phantom{x+2y+}z = 2 \end{cases}$$
The bottom equation already hands us $z = 2$. Substitute upward into the middle: $y + 4(2) = 5 \Rightarrow y = -3$. Then the top: $x + 2(-3) - 2 = 3 \Rightarrow x = 11$. No juggling, no guessing — just read from the bottom and climb. This is back-substitution, and it is the whole reason REF is worth chasing.
So here is the plan. Given any system, we will turn its augmented matrix into REF, then back-substitute. The only question left is: how do we transform a matrix into REF without changing the solution of the underlying system?
The Three Elementary Row Operations
We are allowed exactly three moves. Each one rearranges the matrix but leaves the solution set untouched — because each corresponds to something we have always been allowed to do to equations. Let us take one system and apply all three, watching the system and the matrix side by side.
Our starting system and its augmented matrix:
$$\begin{cases} 2x + 4y = 6 \\ x - y = 1 \\ 3x + y = 7 \end{cases}$$
$$\left(\begin{array}{cc|c} 2 & 4 & 6 \\ 1 & -1 & 1 \\ 3 & 1 & 7 \end{array}\right)$$
Swap two rows. Notation: $R_{23}$ means "interchange row 2 and row 3."
Always write the smaller index first: $R_{23}$ is good notation, $R_{32}$ means the same thing but is considered poor form.
Why does this not change the solution? Because swapping two rows of the matrix just means writing the equations in a different order. The system "$2x+4y=6$ and $x-y=1$" is exactly the same system as "$x-y=1$ and $2x+4y=6$." Order never mattered. Apply $R_{12}$ (swap rows 1 and 2) to get a leading 1 at the top:
$$\begin{cases} x - y = 1 \\ 2x + 4y = 6 \\ 3x + y = 7 \end{cases}$$
$$\left(\begin{array}{cc|c} 1 & -1 & 1 \\ 2 & 4 & 6 \\ 3 & 1 & 7 \end{array}\right)$$
Scale every entry of a row by the same nonzero number. Notation: $3R_2$ means "multiply row 2 by 3"; every entry in that row is multiplied.
The constant must be nonzero — multiplying by 0 would erase the equation and destroy information.
Why is the solution preserved? Multiplying an equation by a nonzero number gives an equivalent equation — "$2x + 4y = 6$" and "$x + 2y = 3$" (after $\tfrac{1}{2}R_1$) have exactly the same solutions. Apply $\tfrac{1}{2}R_2$ to the second row:
$$\begin{cases} x - y = 1 \\ x + 2y = 3 \\ 3x + y = 7 \end{cases}$$
$$\left(\begin{array}{cc|c} 1 & -1 & 1 \\ 1 & 2 & 3 \\ 3 & 1 & 7 \end{array}\right)$$
Notation: $R_3 - 2R_1$ means "replace row 3 by (row 3 minus twice row 1)."
Only $R_3$ changes. We compute $2R_1$ and subtract it from $R_3$; rows 1 and 2 are left exactly as they were. The row being replaced is the one written first.
This is the workhorse — it is how we create zeros below a pivot. Why does it preserve the solution? Because any values $(x,y)$ that satisfy both row 1 and row 3 will also satisfy "row 3 minus twice row 1" — you are just combining two true equations into another true one. Apply $R_2 - R_1$ and $R_3 - 3R_1$ to clear column 1 below the top pivot:
$$\begin{cases} x - y = 1 \\ 3y = 2 \\ 4y = 4 \end{cases}$$
$$\left(\begin{array}{cc|c} 1 & -1 & 1 \\ 0 & 3 & 2 \\ 0 & 4 & 4 \end{array}\right)$$
Notice how the matrix is marching toward REF: the first column now has its leading 1 on top and zeros below. We would continue — scale row 2 to get a leading 1, clear below it, and so on — but first let us name what we have just done.
The three moves above are called the elementary row operations:
1. Interchange two rows $(R_{ij})$.
2. Multiply a row by a nonzero constant $(kR_i)$.
3. Add a multiple of one row to another $(R_i + kR_j)$.
Each one leaves the solution set unchanged. And here is the powerful fact that makes the whole method work:
§4Gaussian Elimination
The Gaussian elimination method solves a linear system by: (1) writing its augmented matrix, (2) using elementary row operations to reduce it to row-echelon form, working pivot by pivot from the top-left, and (3) reading the solution by back-substitution. Working column by column: make the pivot a leading 1, then use it to clear every entry below it; move to the next pivot; repeat.
The target, step by step: get a leading 1 in the first pivot position, use it to zero out everything below in that column, move down-right to the next pivot, make it a leading 1, clear below it, and continue until the matrix is in REF. Then back-substitute. Watch the entire process play out on a 4-variable system — click through one step at a time:
Now a fully worked example you can follow on paper:
Solve $\begin{cases} 2x + y - z = 8 \\ -3x - y + 2z = -11 \\ -2x + y + 2z = -3 \end{cases}$
**Augmented matrix:** $\left(\begin{array}{ccc|c} 2 & 1 & -1 & 8 \\ -3 & -1 & 2 & -11 \\ -2 & 1 & 2 & -3 \end{array}\right)$
$\tfrac{1}{2}R_1$ (make first pivot a leading 1): $\left(\begin{array}{ccc|c} 1 & \tfrac12 & -\tfrac12 & 4 \\ -3 & -1 & 2 & -11 \\ -2 & 1 & 2 & -3 \end{array}\right)$
$R_2 + 3R_1$ and $R_3 + 2R_1$ (clear below): $\left(\begin{array}{ccc|c} 1 & \tfrac12 & -\tfrac12 & 4 \\ 0 & \tfrac12 & \tfrac12 & 1 \\ 0 & 2 & 1 & 5 \end{array}\right)$
$2R_2$ (second pivot to 1): $\left(\begin{array}{ccc|c} 1 & \tfrac12 & -\tfrac12 & 4 \\ 0 & 1 & 1 & 2 \\ 0 & 2 & 1 & 5 \end{array}\right)$, then $R_3 - 2R_2$: $\left(\begin{array}{ccc|c} 1 & \tfrac12 & -\tfrac12 & 4 \\ 0 & 1 & 1 & 2 \\ 0 & 0 & -1 & 1 \end{array}\right)$
$-R_3$ (third pivot to 1): $\left(\begin{array}{ccc|c} 1 & \tfrac12 & -\tfrac12 & 4 \\ 0 & 1 & 1 & 2 \\ 0 & 0 & 1 & -1 \end{array}\right)$ — this is REF.
**Back-substitute:** bottom row gives $z = -1$. Middle: $y + (-1) = 2 \Rightarrow y = 3$. Top: $x + \tfrac12(3) - \tfrac12(-1) = 4 \Rightarrow x + 2 = 4 \Rightarrow x = 2$.
**Solution:** $(x, y, z) = (2, 3, -1)$. Check in equation 1: $2(2)+3-(-1) = 4+3+1 = 8$ ✓.
§5Writing Infinitely Many Solutions
Sometimes elimination leaves us with fewer "real" equations than variables. The system is still consistent — but instead of one answer, there is a whole family. We need a clean way to write that family. Consider this augmented matrix, already in REF, with 6 variables $x_1, \dots, x_6$:
$$\left(\begin{array}{cccccc|c} 1 & 2 & 0 & 3 & 1 & 0 & -1 \\ 0 & 0 & 1 & -1 & 1 & 0 & 2 \\ 0 & 0 & 0 & 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array}\right)$$
Before solving, let us count carefully — this counting is the heart of the whole consistency story:
For any system, count the pivot entries in the REF of the augmented matrix. Then:
• (number of variables) $\ge$ (number of pivots) — pivots sit in distinct columns, and there are only as many columns as variables.
• (number of equations) $\ge$ (number of pivots) — each pivot heads a distinct nonzero row, and there are only as many rows as equations.
In our example: variables $= 6$, pivots $= 3$, so variables $> $ pivots. That gap is exactly what produces freedom. The variables whose columns contain a pivot are determined; the rest are free.
The number of free variables is $(\text{variables}) - (\text{pivots})$. In our example, $6 - 3 = 3$ free variables. The pivot columns are 1, 3, 6, so $x_1, x_3, x_6$ are determined; the non-pivot columns are 2, 4, 5, so $x_2, x_4, x_5$ are free — they may take any value.
To write the solution, we assign a parameter to each free variable. In this course we use $r, s, t, u, \dots$. A solution containing free parameters is an infinite family of solutions — each choice of parameter values gives one specific solution. So "infinitely many solutions" simply means the answer carries free parameters.
Assign parameters to the free variables: $x_2 = s,\quad x_4 = t,\quad x_5 = u$.
Read each pivot row as an equation and solve for its pivot variable:
Row 3: $x_6 = 3$.
Row 2: $x_3 - x_4 + x_5 = 2 \Rightarrow x_3 = 2 + x_4 - x_5 = 2 + t - u$.
Row 1: $x_1 + 2x_2 + 3x_4 + x_5 = -1 \Rightarrow x_1 = -1 - 2s - 3t - u$.
**Write the full solution as a column vector** — this is the format we expect in the exam:
$$\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \end{pmatrix} = \begin{pmatrix} -1 - 2s - 3t - u \\ s \\ 2 + t - u \\ t \\ u \\ 3 \end{pmatrix}, \qquad s, t, u \in \mathbb{R}.$$
Every choice of $(s, t, u)$ gives one solution; since there are infinitely many such choices, the system has infinitely many solutions.
§6Rank, and the Complete Consistency Test
The rank of a matrix is the number of nonzero rows in its row-echelon form — equivalently, the number of pivots (leading 1s). It measures how much genuine, non-redundant information the matrix carries.
(a) $\left(\begin{array}{ccc} 1 & 2 & 3 \\ 0 & 1 & 4 \\ 0 & 0 & 1 \end{array}\right)$ — three nonzero rows, three pivots. **Rank $= 3$.**
(b) $\left(\begin{array}{ccc} 1 & 2 & 0 \\ 0 & 1 & 5 \\ 0 & 0 & 0 \end{array}\right)$ — two nonzero rows. **Rank $= 2$.**
(c) $\left(\begin{array}{cccc} 1 & 0 & 3 & 0 \\ 0 & 1 & 2 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{array}\right)$ — three pivots (cols 1, 2, 4). **Rank $= 3$.**
Find the rank of each (all already in REF):
(i) $\left(\begin{array}{ccc} 1 & 3 & 1 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{array}\right)$ \quad (ii) $\left(\begin{array}{cccc} 1 & 0 & 2 & 5 \\ 0 & 1 & 1 & 3 \end{array}\right)$ \quad (iii) $\left(\begin{array}{ccc|c} 1 & 2 & 1 & 0 \\ 0 & 0 & 1 & 4 \\ 0 & 0 & 0 & 1 \end{array}\right)$
Solve $\begin{cases} x + y + z = 2 \\ x + z = 1 \\ 2x + 5y + 2z = 7 \end{cases}$ by Gaussian elimination.
**Augmented:** $\left(\begin{array}{ccc|c} 1 & 1 & 1 & 2 \\ 1 & 0 & 1 & 1 \\ 2 & 5 & 2 & 7 \end{array}\right)$
$R_2 - R_1$, $R_3 - 2R_1$: $\left(\begin{array}{ccc|c} 1 & 1 & 1 & 2 \\ 0 & -1 & 0 & -1 \\ 0 & 3 & 0 & 3 \end{array}\right)$
$-R_2$: $\left(\begin{array}{ccc|c} 1 & 1 & 1 & 2 \\ 0 & 1 & 0 & 1 \\ 0 & 3 & 0 & 3 \end{array}\right)$, then $R_3 - 3R_2$: $\left(\begin{array}{ccc|c} 1 & 1 & 1 & 2 \\ 0 & 1 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{array}\right)$ — REF, with a zero row at the bottom.
**Count:** variables $= 3$, pivots $= 2$ (columns 1, 2). Free variables $= 3 - 2 = 1$. The non-pivot column is 3, so $z$ is free. Set $z = t$.
Row 2: $y = 1$. Row 1: $x + y + z = 2 \Rightarrow x = 2 - 1 - t = 1 - t$.
**Solution:** $\begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 1 - t \\ 1 \\ t \end{pmatrix},\quad t \in \mathbb{R}.$ Consistent, infinitely many solutions.
Change the last constant from 7 to 8: $\begin{cases} x + y + z = 2 \\ x + z = 1 \\ 2x + 5y + 2z = 8 \end{cases}$
The identical row operations now give: $\left(\begin{array}{ccc|c} 1 & 1 & 1 & 2 \\ 0 & 1 & 0 & 1 \\ 0 & 0 & 0 & 1 \end{array}\right)$
Look at the bottom row: $0x + 0y + 0z = 1$, i.e. $0 = 1$. Impossible. Whenever elimination produces a zero row on the left with a nonzero constant on the right, stop immediately — the system has no solution. It is inconsistent.
In rank language: $\operatorname{rank}(A) = 2$ but $\operatorname{rank}(A\mid b) = 3$. Since $\operatorname{rank}(A) < \operatorname{rank}(A\mid b)$, no solution exists.
Recall Example A: it reduced to $\left(\begin{array}{ccc|c} 1 & \tfrac12 & -\tfrac12 & 4 \\ 0 & 1 & 1 & 2 \\ 0 & 0 & 1 & -1 \end{array}\right)$ with solution $(2, 3, -1)$.
Here variables $= 3$ and pivots $= 3$ — every variable has a pivot, no free variables. $\operatorname{rank}(A) = \operatorname{rank}(A\mid b) = 3 = $ number of variables. That is exactly the fingerprint of a unique solution.
For a system $AX = b$ with $n$ variables, compare $\operatorname{rank}(A)$ and $\operatorname{rank}(A\mid b)$:
• $\operatorname{rank}(A) < \operatorname{rank}(A\mid b)$ → inconsistent (no solution).
• $\operatorname{rank}(A) = \operatorname{rank}(A\mid b) = n$ → consistent, unique solution.
• $\operatorname{rank}(A) = \operatorname{rank}(A\mid b) < n$ → consistent, infinitely many solutions ($n - \operatorname{rank}$ free variables).
§7Worked Problem & Exercises
For which values of $a$ and $b$ does $\begin{cases} -x + 3y + 2z = -8 \\ x + z = 2 \\ 3x + 3y + az = b \end{cases}$ have (1) no solution, (2) infinitely many, (3) a unique solution?
Exercises from the textbook (Nicholson §1.2)
Work these by hand first, then reveal the worked solution to check. These are the kind of problems you will see on quizzes and the midterm.
Three Nissans, two Fords, and four Chevrolets rent for \$106/day. Two Nissans, four Fords, three Chevrolets cost \$107/day. Four Nissans, three Fords, two Chevrolets cost \$102/day. Find each rate.
A school has three clubs; each student belongs to exactly one. After switching: Club A keeps $\tfrac{4}{10}$, sends $\tfrac{1}{10}$ to B, $\tfrac{5}{10}$ to C. Club B keeps $\tfrac{7}{10}$, sends $\tfrac{2}{10}$ to A, $\tfrac{1}{10}$ to C. Club C keeps $\tfrac{6}{10}$, sends $\tfrac{2}{10}$ to A, $\tfrac{2}{10}$ to B. If each club's fraction of the population is unchanged, find these fractions.
Three players' scores are lost. We know the totals for players 1&2, for 2&3, and for 3&1. (a) Show the individual scores can be recovered. (b) Is it possible with four players, knowing totals for 1&2, 2&3, 3&4, and 4&1?
A boy finds \$1.05 in dimes (10¢), nickels (5¢), and pennies (1¢). There are 17 coins in all. How many of each type can he have?
Given points $(p_1, q_1), (p_2, q_2), (p_3, q_3)$ with $p_1, p_2, p_3$ distinct, show they lie on a curve $y = a + bx + cx^2$. (Hint: solve for $a, b, c$.)
Find a sequence of row operations carrying $\left(\begin{array}{ccc} b_1+c_1 & b_2+c_2 & b_3+c_3 \\ c_1+a_1 & c_2+a_2 & c_3+a_3 \\ a_1+b_1 & a_2+b_2 & a_3+b_3 \end{array}\right)$ to $\left(\begin{array}{ccc} a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \\ c_1 & c_2 & c_3 \end{array}\right)$.
A system has augmented matrix $A$ and coefficient matrix $C$. Decide each (prove or give a counterexample):
a. More than one solution $\Rightarrow$ $A$ has a row of zeros. b. $A$ has a row of zeros $\Rightarrow$ more than one solution. c. No solution $\Rightarrow$ REF of $C$ has a zero row. d. REF of $C$ has a zero row $\Rightarrow$ no solution. e. No system is inconsistent for every choice of constants. f. Consistent for some constants $\Rightarrow$ consistent for every choice.
Now assume $A$ has 3 rows and 5 columns: g. consistent $\Rightarrow$ more than one solution. h. rank $A \le 3$. i. rank $A = 3 \Rightarrow$ consistent. j. rank $C = 3 \Rightarrow$ consistent.