When does a spanning set have no waste? Linear independence, unique representations, bases, and the single number that measures the size of a subspace — its dimension
§1A Story Before We Begin
In Lecture 14 you learned that a handful of vectors can span an entire subspace. But Example 5 there showed something unsettling: different spanning sets can describe the same subspace. Some of them might carry a passenger — a vector that adds nothing, because it is already a combination of the others.
Think of a set of directions for navigating a city. "Go east, go north, go northeast" — the third instruction is redundant, because northeast is just east-plus-north. You could delete it and still reach everywhere. A good set of directions has no waste: every instruction adds a genuinely new direction. That "no waste" property is called linear independence, and it is the whole subject of today.
🎯
Why we chase independence
A spanning set tells you a subspace is reachable. An independent spanning set — a basis — tells you it is reachable in exactly one way. Unique coordinates. No ambiguity. This is why GPS uses exactly three satellites' worth of independent directions, why colour screens use exactly three independent primaries, and why data scientists hunt for the smallest independent set of features that still captures their data. Independence is the mathematics of "no redundancy," and dimension is the number you get when you count what is left.
💡
The payoff at the end
By the end of this lecture you will be able to attach a single number — the dimension — to any subspace, capturing exactly how many independent directions it contains. A line is 1-dimensional, a plane is 2-dimensional, and this finally makes those words precise.
§2What Is Linear Independence?
We start from the goal: we want spanning sets where every vector has exactly one representation as a linear combination. It turns out this is captured by a clean condition on the zero vector.
Linearly independent set
A set of vectors $\{\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_k\}$ in $\mathbb{R}^n$ is linearly independent (or just independent) if the only way to build the zero vector as a combination $$t_1\mathbf{x}_1 + t_2\mathbf{x}_2 + \cdots + t_k\mathbf{x}_k = \mathbf{0}$$ is with every coefficient zero: $t_1 = t_2 = \cdots = t_k = 0$.
In plain words: no vector in an independent set can be built from the others. If you could — say $\mathbf{x}_3 = 2\mathbf{x}_1 - \mathbf{x}_2$ — then $2\mathbf{x}_1 - \mathbf{x}_2 - \mathbf{x}_3 = \mathbf{0}$ would be a way to reach zero without all-zero coefficients, and the set would fail the test. Independence means each vector points in a genuinely new direction.
🎛 Independent or Dependent? — Enter Your Own Vectors
Space
Number of vectors
2max 5
v1
v2
✓ Independent
The only linear combination that reaches 0 is the trivial one — every coefficient must be zero.
rank = 2 · 2 vectors · they span a 2-dimensional subspace of ℝ².
§3Why Independence Matters: Unique Representation
Here is the theorem that justifies the whole idea. Independence is exactly the condition that makes coordinates unambiguous.
Theorem 5.2.1 — unique representation
If $\{\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_k\}$ is an independent set in $\mathbb{R}^n$, then every vector in $\operatorname{span}\{\mathbf{x}_1, \ldots, \mathbf{x}_k\}$ has exactly one representation as a linear combination of the $\mathbf{x}_i$.
🧠
What this theorem is really saying
Take any vector $\mathbf{v}$ in the span. There is one and only one list of coefficients $(t_1, \ldots, t_k)$ with $\mathbf{v} = t_1\mathbf{x}_1 + \cdots + t_k\mathbf{x}_k$. Those coefficients are the coordinates of $\mathbf{v}$ in this system. Without independence, the same $\mathbf{v}$ could have many different coordinate lists — like a GPS that gives two different addresses for one house. Independence is what makes coordinates well-defined. That is why we care about it.
§4The Independence Test
It helps to restate independence in compact language. Say a linear combination vanishes if it equals the zero vector, and call it trivial if every coefficient is zero. Then:
Independence, restated
A set of vectors is independent if and only if the only linear combination that vanishes is the trivial one.
Independence Test — the procedure
To check that $\{\mathbf{x}_1, \ldots, \mathbf{x}_k\}$ is independent:
$\textbf{1.}$ Set a general linear combination equal to zero: $t_1\mathbf{x}_1 + t_2\mathbf{x}_2 + \cdots + t_k\mathbf{x}_k = \mathbf{0}$.
$\textbf{2.}$ Solve the resulting homogeneous system and show every $t_i = 0$.
If you can force all coefficients to zero, the set is independent. If some nontrivial solution exists (coefficients not all zero), the set is dependent.
🔧
This is just a homogeneous system
Step 1 always turns into a homogeneous system $A\mathbf{t} = \mathbf{0}$, where the columns of $A$ are your vectors. From Lecture 1 you know: this has only the trivial solution exactly when there are no free variables — that is, when the rank equals the number of vectors. So "independent" and "the vectors form full-rank columns" are the same statement.
§5Worked Examples
★ 1Example 5.2.1 — testing independence directly
Determine whether $\{(1, 0, -2, 5), (2, 1, 0, -1), (1, 1, 2, 1)\}$ is independent in $\mathbb{R}^4$.
★ 2Example 5.2.3 — independence survives an invertible mix
If $\{\mathbf{x}, \mathbf{y}\}$ is independent, show that $\{2\mathbf{x} + 3\mathbf{y},\; \mathbf{x} - 5\mathbf{y}\}$ is also independent.
Example 3Example 5.2.4 — the zero vector poisons independence
Show that the zero vector in $\mathbb{R}^n$ can never belong to an independent set.
Example 4Example 5.2.5 — a single vector
For $\mathbf{x}$ in $\mathbb{R}^n$, show that $\{\mathbf{x}\}$ is independent if and only if $\mathbf{x} \neq \mathbf{0}$.
§6Linear Dependence
Linearly dependent set
A set of vectors in $\mathbb{R}^n$ is linearly dependent (or just dependent) if it is not independent — equivalently, if some nontrivial linear combination vanishes. That means at least one vector can be written in terms of the others; it is a redundant passenger.
⚖️
The two words, side by side
$\textbf{Independent:}$ the only vanishing combination is trivial (all coefficients zero). No redundancy. $\;\;\textbf{Dependent:}$ some nontrivial combination vanishes (some coefficient nonzero). At least one vector is redundant. Every set is one or the other.
§7Two Powerful Theorems
These connect independence and spanning to things you already know how to compute: matrix equations and invertibility.
Theorem 5.2.2 — columns of a matrix
Let $A$ be an $m \times n$ matrix with columns $\{\mathbf{c}_1, \mathbf{c}_2, \ldots, \mathbf{c}_n\}$.
$\textbf{1.}$ The columns are independent in $\mathbb{R}^m$ if and only if $A\mathbf{x} = \mathbf{0}$ (for $\mathbf{x}$ in $\mathbb{R}^n$) forces $\mathbf{x} = \mathbf{0}$.
$\textbf{2.}$ The columns span $\mathbb{R}^m$ if and only if $A\mathbf{x} = \mathbf{b}$ has a solution for every $\mathbf{b}$ in $\mathbb{R}^m$.
🧠
What Theorem 5.2.2 means
It translates the two big ideas into system-solving. Independence = "the homogeneous system has only the zero solution." Spanning = "every right-hand side is achievable." You already know how to test both by row reduction — so you already know how to test independence and spanning. No new computation, just new vocabulary.
Theorem 5.2.3 — five faces of an invertible matrix
For an $n \times n$ (square) matrix $A$, the following are equivalent — each one implies all the others:
$\textbf{1.}$ $A$ is invertible.
$\textbf{2.}$ The columns of $A$ are linearly independent.
$\textbf{3.}$ The columns of $A$ span $\mathbb{R}^n$.
$\textbf{4.}$ The rows of $A$ are linearly independent.
$\textbf{5.}$ The rows of $A$ span the set of all $1 \times n$ rows.
🔑
Why Theorem 5.2.3 is a gift
For a square matrix, "independent," "spanning," and "invertible" all collapse into one thing. And from Lecture 10, invertible means $\det A \neq 0$. So to test whether $n$ vectors in $\mathbb{R}^n$ are independent, you do not need to solve a system at all — just stack them into a square matrix and check the determinant. Non-zero determinant $\Rightarrow$ independent. This is the fastest test you have.
§8The Determinant Shortcut in Action
★ 5Example 5.2.9 — independence via determinant
Show that $S = \{(2, -2, 5), (-3, 1, 1), (2, 7, -4)\}$ is independent in $\mathbb{R}^3$, using Theorem 5.2.3.
§9Part B — Dimension: Measuring the Size of a Subspace
We casually say a line is "one-dimensional" and a plane is "two-dimensional." But what does that number actually count? Now we can say it precisely: dimension is the number of independent directions a subspace contains — equivalently, the size of any basis for it.
🌐
Why dimension is everywhere
Dimension is one of the most important numbers in mathematics and science. A robot arm's degrees of freedom is the dimension of its space of motions. The number of independent features in a dataset is a dimension — and "dimensionality reduction" (the heart of modern machine learning) is literally the art of finding a smaller-dimensional subspace that still holds your data. Physicists argue about whether spacetime is $4$-dimensional or $11$-dimensional. In every case, the question "what is the dimension?" means "how many independent directions do I really have?"
🎲
A surprising fact
Here is something remarkable that we are about to prove: every basis of a given subspace has the same number of vectors. You cannot span a plane with a basis of $2$ vectors and also with a basis of $3$ — the plane "knows" it is $2$-dimensional, and no honest basis can disagree. This is not obvious, and it is what makes "dimension" a well-defined number at all.
§10The Fundamental Theorem
Theorem 5.2.4 — Fundamental Theorem
Let $U$ be a subspace of $\mathbb{R}^n$. If $U$ is spanned by $m$ vectors, and if $U$ contains a set of $k$ linearly independent vectors, then $$k \leq m.$$
🧠
What the Fundamental Theorem is saying
In plain words: you can never have more independent vectors than you have spanning vectors. If $m$ vectors are enough to build the whole subspace, then no independent set inside it can be bigger than $m$. Intuitively, independent vectors each demand their own "room," and a spanning set of size $m$ only provides $m$ rooms. Try to fit $k > m$ independent vectors and at least one must be a combination of the others — contradicting independence. This one inequality is the engine behind everything that follows, including the fact that all bases have the same size.
§11Basis: A Spanning Set With No Waste
Now we name the ideal spanning set — one that spans, but has no redundant passengers.
Basis (Definition 5.4)
If $U$ is a subspace of $\mathbb{R}^n$, a set $\{\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_m\}$ of vectors in $U$ is a basis of $U$ if it satisfies both:
$\textbf{1.}$ $\{\mathbf{x}_1, \ldots, \mathbf{x}_m\}$ is linearly independent (no waste), and
$\textbf{2.}$ $U = \operatorname{span}\{\mathbf{x}_1, \ldots, \mathbf{x}_m\}$ (it builds all of $U$).
🎯
Basis = the two ideas together
A basis is the perfect middle ground. Spanning alone might have redundancy (too many vectors). Independence alone might not reach everything (too few). A basis is exactly enough: independent, so nothing is wasted; spanning, so nothing is missing. It is the most efficient description of a subspace.
Theorem 5.2.5 — Invariance Theorem
If $\{\mathbf{x}_1, \ldots, \mathbf{x}_m\}$ and $\{\mathbf{y}_1, \ldots, \mathbf{y}_k\}$ are both bases of a subspace $U$, then $m = k$.
🧠
What the Invariance Theorem means (and why it works)
Every basis of $U$ has the same size. This follows from the Fundamental Theorem applied twice. The first basis spans (with $m$ vectors) and the second is independent (with $k$ vectors), so $k \leq m$. But the second also spans (with $k$) and the first is independent (with $m$), so $m \leq k$. Together $m \leq k$ and $k \leq m$ force $m = k$. Because the count never changes, it is safe to define a number from it — the dimension.
§12Dimension Defined
Dimension (Definition 5.5)
If $U$ is a subspace of $\mathbb{R}^n$ and $\{\mathbf{x}_1, \ldots, \mathbf{x}_m\}$ is any basis of $U$, the number $m$ of vectors in that basis is the dimension of $U$, written $\dim U = m$.
This definition only makes sense because of the Invariance Theorem — since every basis has the same size, it does not matter which basis you pick; you always get the same number. A line has dimension $1$, a plane has dimension $2$, the zero subspace $\{\mathbf{0}\}$ has dimension $0$ (its basis is the empty set), and $\mathbb{R}^n$ itself has dimension $n$.
🎛 Dimension = Number of Basis Vectors
dim 1 — a line
One basis vector. Every point is a scalar multiple of it. One degree of freedom.
Add one independent direction, gain one dimension. The dimension counts the basis vectors — no more, no less.
§13Finding a Basis and Dimension
★ 6Example 5.2.11 — subspace, basis, and dimension
Let $U = \{ (r, s, r) : r, s \in \mathbb{R} \}$. Show $U$ is a subspace of $\mathbb{R}^3$, find a basis, and compute $\dim U$.
Example 7Extra practice — a subspace given by an equation
Find a basis and dimension of $W = \{ (x, y, z) : x + y + z = 0 \}$ in $\mathbb{R}^3$.
★ 8Extra practice — a span that secretly collapses
Find the dimension of $V = \operatorname{span}\{(1, -1, 2, 0),\, (2, 3, 0, 3),\, (1, 9, -6, 6)\}$ in $\mathbb{R}^4$.
🛠
The general recipe for basis and dimension
(1) Write the subspace as a span by solving out its free parameters or constraints. (2) Check whether those spanning vectors are independent. (3) If yes, they are a basis and the dimension is how many there are. (4) If no, discard the redundant ones until what remains is independent — that trimmed set is a basis. The dimension is the size of the trimmed set.
§14Exercises
The true/false set (5.2.7) is worth real attention — several claims are subtly false, and two are "technically true" traps. Work them before reading the answers.
Exercise 5.2.7True or false? (with counterexamples) [FULLY SOLVED]
For each claim, decide if it is always true; if false, give a counterexample.
Same method: expand, group by the original vectors, and use their independence to get a system in the new coefficients. The "loop" structure is the key — a cycle of differences or sums often cancels. Results to confirm: $\textbf{(b)}$ is independent (the coefficient matrix has determinant $2 \neq 0$). $\textbf{(c)}$ and $\textbf{(d)}$ are both dependent — each has a cyclic cancellation (for (c), the four differences sum to $\mathbf{0}$; for (d), take alternating signs $+,-,+,-$).
Exercise 5.2.3Basis and dimension of span{(1,−1,2,0),(2,3,0,3),(1,9,−6,6)} [SOLVED — see Example 8]
✅
Already worked
This is exactly Example 8 above. The third vector equals $-3$(first)$+ 2$(second), so it is redundant. A basis is $\{(1,-1,2,0),\, (2,3,0,3)\}$ and $\dim = 2$.
Exercise 5.2.4Basis and dimension of four subspaces of ℝ⁴ [ONE SOLVED, REST HINTED]
🧭
Hints for (b), (c), (d)
Same recipe: rewrite each general element as a combination of constant vectors, one per free parameter, then check those vectors for independence. Answers to confirm: $\textbf{(b)}$ $(a+b, a-b, b, a) = a(1,1,0,1) + b(1,-1,1,0)$, so $\dim = 2$. $\textbf{(c)}$ has three free parameters $a,b,c$ giving three independent vectors, so $\dim = 3$. $\textbf{(d)}$ also works out to $\dim = 3$ (three parameters, and the resulting vectors turn out independent — verify by checking no one is a combination of the others).
Exercise 5.2.6Which sets are a basis? (use Theorem 5.2.3) [ONE SOLVED, REST HINTED]
🧭
Hints for (b)–(e)
Stack each set as a square matrix and take the determinant — nonzero means basis. First check the count: you need exactly $n$ vectors to be a basis of $\mathbb{R}^n$ (three for $\mathbb{R}^3$, four for $\mathbb{R}^4$). Results to confirm: $\textbf{(b)}$ det $= -2 \neq 0$, basis. $\textbf{(c)}$ det $= 0$, not a basis. $\textbf{(d)}$ det $= 12 \neq 0$, basis. $\textbf{(e)}$ det $= 0$, not a basis.
Exercise 5.2.9–5.2.12Proof problems [HINTS — these are worth attempting]
🧭
Hints
$\textbf{5.2.9:}$ given independent $\{\mathbf{x},\mathbf{y},\mathbf{z}\}$ in $\mathbb{R}^4$, show adding some standard vector $\mathbf{e}_k$ makes a basis of $\mathbb{R}^4$. Since $\dim \mathbb{R}^4 = 4$, you need one more independent vector; argue that not all four $\mathbf{e}_k$ can lie in $\operatorname{span}\{\mathbf{x},\mathbf{y},\mathbf{z}\}$ (that span is only $3$-dimensional), so some $\mathbf{e}_k$ is outside it — adding that one keeps independence and reaches dimension $4$. $\;\;\textbf{5.2.10:}$ a subset of an independent set is independent — this is exactly the reasoning in 5.2.7(b), generalized. $\;\;\textbf{5.2.11:}$ if the columns $\mathbf{b}_i$ are independent and each $A\mathbf{x}_i = \mathbf{b}_i$, show the $\mathbf{x}_i$ are independent — set $\sum t_i\mathbf{x}_i = \mathbf{0}$, apply $A$, and use independence of the $\mathbf{b}_i$. $\;\;\textbf{5.2.12:}$ show $\{\mathbf{x}_1, \mathbf{x}_1+\mathbf{x}_2, \ldots, \mathbf{x}_1+\cdots+\mathbf{x}_k\}$ is independent — set a combination to zero and peel off coefficients from the last term backward.
Summary of key points
A set is independent if the only vanishing combination $t_1\mathbf{x}_1 + \cdots + t_k\mathbf{x}_k = \mathbf{0}$ is the trivial one (all $t_i = 0$).
Independence $\Leftrightarrow$ every vector in the span has a unique representation (Theorem 5.2.1).
$\textbf{Test:}$ set the combination to $\mathbf{0}$, solve the homogeneous system, show all coefficients are forced to zero.
$\textbf{Dependent}$ = not independent = some nontrivial combination vanishes = a vector is redundant.
For $n$ vectors in $\mathbb{R}^n$: independent $\Leftrightarrow$ spanning $\Leftrightarrow$ invertible $\Leftrightarrow \det \neq 0$ (Theorem 5.2.3). Use the determinant as a shortcut.
$\textbf{Fundamental Theorem:}$ independent count $\leq$ spanning count. You can't have more independent vectors than spanning ones.
A basis is independent and spanning — exactly enough, no waste, nothing missing.
$\textbf{Invariance:}$ every basis of a subspace has the same size — so dimension $\dim U$ is well-defined as that common size.
$\textbf{Recipe:}$ write the subspace as a span, trim to an independent set — that is a basis, and its size is the dimension.
Looking ahead
Independence removes waste; a basis is the perfect spanning set; dimension counts what remains.
With dimension in hand, a whole toolkit opens up. Next we connect these ideas to matrices directly through rank — the dimension of the column space — and see how the dimensions of a matrix's null space and image are locked together by the elegant Rank–Nullity Theorem. The single number you learned to compute today turns out to govern the entire behaviour of linear systems.