Geometry Arrives: Dot Product, Distance & Gram–Schmidt

§1Motivation — Why Geometry, Why Now?

Every lecture since Week 4 has been purely algebraic: a vector was just an object you could add and scale, obeying eight axioms. That abstraction was powerful — it let matrices and polynomials become "vectors" too — but it threw away something $\mathbb{R}^n$ has that a general vector space doesn't: geometry. In $\mathbb{R}^2$ and $\mathbb{R}^3$, vectors have length, and pairs of vectors have an angle between them. Today we bring that geometry back, formalize it for any $\mathbb{R}^n$, and then use it to build the single most useful construction in applied linear algebra: a way to manufacture a perpendicular basis out of any basis at all.

The plan: first, revisit the dot product, length, and distance from Nicholson §5.3 — tools you have used informally since high school, now placed on rigorous footing in $\mathbb{R}^n$. Then, in §8.1, we ask a sharper question: given any basis of a subspace, can we always replace it with an equally good basis whose vectors are mutually perpendicular? The answer is yes, and the algorithm that does it — the Gram–Schmidt process — is today's main event.

§2The Dot Product

Notice immediately that $\mathbf{x}\cdot\mathbf{y}$ is a single number (a scalar), not a vector — this is the first genuinely new kind of operation we've met: it takes two vectors in and produces a number out.

§3An Observation: Dot Product as Matrix Product

This is not a new definition — it is the same number, seen through the matrix-multiplication rule from Week 2. Recall that multiplying a $1\times n$ row by an $n\times1$ column produces a $1\times1$ matrix, whose single entry is exactly "multiply matching entries and add them up." That is precisely the dot product formula.

§4The Length of a Vector

In $\mathbb{R}^2$, the Pythagorean theorem gives the length of $(x_1,x_2)$ as $\sqrt{x_1^2+x_2^2}$. The dot product lets us extend this to any $\mathbb{R}^n$ at once, since $\mathbf{x}\cdot\mathbf{x} = x_1^2+x_2^2+\cdots+x_n^2$ is exactly the sum of squares under that root.

§5Theorem 5.3.1 — The Algebra of Dot Products

§6Expanding ‖x + y‖²

§7Orthogonal to a Spanning Set Forces x = 0

§8Measuring the Gap Between Two Vectors

Length tells us the size of a single vector. A closely related question: how "far apart" are two vectors $\mathbf{x}$ and $\mathbf{y}$? The natural answer, exactly as in $\mathbb{R}^2$ and $\mathbb{R}^3$, is to measure the length of the vector connecting them.

§9Theorem 5.3.3 — Properties of Distance

§10Orthogonality

We now name the single most useful relationship two vectors can have with each other.

§11Motivation — Why Do We Want Orthonormal Sets?

§12The Standard Basis Is Orthonormal

§13Normalizing an Orthogonal Set

Rescaling each vector to length $1$ cannot disturb orthogonality — by Theorem 5.3.1 rule 3, $(a\mathbf{x}_i)\cdot(b\mathbf{x}_j) = ab(\mathbf{x}_i\cdot\mathbf{x}_j)$, and if $\mathbf{x}_i\cdot\mathbf{x}_j=0$, this stays $0$ no matter what $a,b$ are. So "make orthogonal" and "make unit length" are independent jobs — normalizing only ever handles the second one.

§14A Worked Orthonormal Set in R⁴

§15Theorem 5.3.5 — Orthogonal Sets Are Independent

§16§8.1 — Orthogonal Complements and Projections

We now switch chapters, but not topics — everything above was preparation for this. Recall from Lecture 17 the tool that let us grow an independent set into a basis:

§17Lemma 8.1.1 — The Orthogonal Lemma

§18Theorem 8.1.1

§19The Gram–Schmidt Process — Why It's Worth Learning

Theorem 8.1.1 is an existence statement: an orthogonal basis exists. It doesn't tell you how to build one from a basis you already have in hand. The Gram–Schmidt process closes that gap completely: give it any basis at all, however skewed and non-perpendicular, and it hands back an orthogonal basis for the exact same subspace — with a fully explicit, step-by-step recipe.

§20The Gram–Schmidt Process — How It Works

The idea is simply to apply the Orthogonal Lemma over and over, once for each vector of a starting basis $\{\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_k\}$ of a subspace $U$, in order.

1. 1Keep the first vector as-is: $\mathbf{f}_1 = \mathbf{x}_1$. (A single nonzero vector is trivially an orthogonal set — there's nothing to compare it to yet.)
2. 2Strip $\mathbf{x}_2$'s overlap with $\mathbf{f}_1$: $\;\mathbf{f}_2 = \mathbf{x}_2 - \dfrac{\mathbf{x}_2\cdot\mathbf{f}_1}{\|\mathbf{f}_1\|^2}\mathbf{f}_1$. By the Orthogonal Lemma, $\mathbf{f}_2\cdot\mathbf{f}_1=0$ automatically.
3. 3Strip $\mathbf{x}_3$'s overlap with both $\mathbf{f}_1$ and $\mathbf{f}_2$: $\;\mathbf{f}_3 = \mathbf{x}_3 - \dfrac{\mathbf{x}_3\cdot\mathbf{f}_1}{\|\mathbf{f}_1\|^2}\mathbf{f}_1 - \dfrac{\mathbf{x}_3\cdot\mathbf{f}_2}{\|\mathbf{f}_2\|^2}\mathbf{f}_2$.
4. 4Continue the same pattern. At step $i$, subtract off the projection of $\mathbf{x}_i$ onto every $\mathbf{f}_j$ built so far ($j<i$):

   $$\mathbf{f}_i = \mathbf{x}_i - \sum_{j=1}^{i-1} \frac{\mathbf{x}_i\cdot\mathbf{f}_j}{\|\mathbf{f}_j\|^2}\mathbf{f}_j.$$

5. 5Stop after $k$ steps. Since $\{\mathbf{x}_1,\ldots,\mathbf{x}_k\}$ was independent, each $\mathbf{x}_i$ is genuinely outside $\operatorname{span}\{\mathbf{f}_1,\ldots,\mathbf{f}_{i-1}\}$ (it's part of an independent set!), so by the Orthogonal Lemma every $\mathbf{f}_i$ comes out nonzero, and $\{\mathbf{f}_1,\ldots,\mathbf{f}_k\}$ ends up an orthogonal basis of $U$.
6. 6Optional final step: normalize, dividing each $\mathbf{f}_i$ by $\|\mathbf{f}_i\|$, to get a fully orthonormal basis (Definition 5.9).

§21Full Worked Example

§22Two More — Quicker — Examples

§23Looking Ahead

§24Exercises

Four problems to practice on your own, in the same spirit as today's examples. Hints only — try each one properly before reading further.