Polynomial Interpolation & Linear Recurrences
Two applications: fitting an exact polynomial through data points (via determinants), and turning a recurrence into a matrix so diagonalization hands you a closed formula — Fibonacci included
§1Quick Recall
Today is a lecture of two applications. Each leans on one big tool you already own.
• From Lecture 10: a square matrix is invertible exactly when its determinant is not zero. A system $M\mathbf{x} = \mathbf{b}$ then has one unique solution. We use this for the first topic, interpolation.
• From Lecture 12: if a matrix $A$ has enough eigenvectors it is diagonalizable, and then $A^k = PD^kP^{-1}$ is easy to compute. We use this for the second topic, recurrences.
§2Part A — Polynomial Interpolation: The Idea
Very often two quantities are related, but you do not know the formula linking them. You only have measurements: a few input values $x_1, x_2, \ldots, x_n$ and their matching outputs $y_1, y_2, \ldots, y_n$. The question is simple:
The most convenient curve to use is a polynomial, because polynomials are easy to write down, easy to evaluate, and easy to work with. A polynomial that passes through all the given points is called an interpolating polynomial for the data. Let us see it in action before stating anything general.
§3Example 3.2.10 — Guessing a Tree's Age
A forester wants to estimate the age of a tree from the diameter of its trunk. She measures three trees:
| Diameter (cm) | 5 | 10 | 15 |
| Age (years) | 3 | 5 | 6 |
So the data points are $(5, 3)$, $(10, 5)$, $(15, 6)$. Estimate the age of a tree whose trunk diameter is $12$ cm.
§4The General Setup
The tree example shows the whole pattern. Suppose you have $n$ data points $(x_1, y_1), \ldots, (x_n, y_n)$ with all the $x_i$ distinct (no two measurements at the same input). Look for a polynomial of degree $n-1$:
$$p(x) = r_0 + r_1 x + r_2 x^2 + \cdots + r_{n-1}x^{n-1}.$$
It has $n$ unknown coefficients $r_0, \ldots, r_{n-1}$. Demanding $p(x_i) = y_i$ for each point gives $n$ linear equations:
$$\begin{aligned} r_0 + r_1 x_1 + \cdots + r_{n-1}x_1^{n-1} &= y_1 \\ r_0 + r_1 x_2 + \cdots + r_{n-1}x_2^{n-1} &= y_2 \\ &\;\;\vdots \\ r_0 + r_1 x_n + \cdots + r_{n-1}x_n^{n-1} &= y_n \end{aligned}$$
In matrix form $V\mathbf{r} = \mathbf{y}$, where the coefficient matrix $V$ collects the powers of the inputs:
$$\underbrace{\begin{bmatrix} 1 & x_1 & x_1^2 & \cdots & x_1^{n-1} \\ 1 & x_2 & x_2^2 & \cdots & x_2^{n-1} \\ \vdots & \vdots & \vdots & & \vdots \\ 1 & x_n & x_n^2 & \cdots & x_n^{n-1} \end{bmatrix}}_{V}\begin{bmatrix} r_0 \\ r_1 \\ \vdots \\ r_{n-1} \end{bmatrix} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}.$$
Given $n$ data pairs $(x_1, y_1), \ldots, (x_n, y_n)$ with the $x_i$ all distinct, there is exactly one polynomial $p(x)$ of degree at most $n-1$ with $p(x_i) = y_i$ for every $i$.
§5Why It Always Works: The Vandermonde Determinant
Theorem 3.2.6 claims a unique solution always exists. From Lecture 10 we know a square system $V\mathbf{r} = \mathbf{y}$ has a unique solution exactly when $\det V \neq 0$. So the whole theorem rests on one fact: the determinant of that special powers-matrix is never zero (as long as the inputs are distinct).
The matrix $V$ built from powers of $x_1, \ldots, x_n$ is called a Vandermonde matrix. Its determinant has a beautiful closed form — it is the product of all the differences of the inputs:
$$\det V = \prod_{i > j}(x_i - x_j).$$
$\textbf{Why this settles everything.}$ Each factor $(x_i - x_j)$ is the gap between two of the inputs. If all the inputs are distinct, every gap is non-zero, so the whole product is non-zero, so $\det V \neq 0$. That is precisely the condition for a unique solution. The polynomial exists and is one-of-a-kind. $\;\blacksquare$
§6Worked Example & Explorer
Find the interpolating polynomial of degree 3 for the data $(0, 1)$, $(1, 2)$, $(2, 5)$, $(3, 10)$, and estimate $y$ at $x = 1.5$.
Try it below. Switch between 2, 3, and 4 of the points and watch the polynomial change degree to thread exactly through them. Notice that at 4 points, the curve is the same parabola as at 3 — because this data is quadratic.
§7Part B — Linear Recurrences: The Idea
A recursive sequence is one where each new term is built from earlier terms by a fixed rule. You know the first few values, and a formula tells you how to get the next from the ones before it.
A sequence $x_0, x_1, x_2, \ldots$ satisfies a linear recurrence of length $m$ if each term is a fixed combination of the $m$ terms before it. For length 2: $$x_{k+2} = a\,x_{k+1} + b\,x_k \quad \text{for all } k \geq 0,$$ once the starting values $x_0$ and $x_1$ are given.
Computing terms one by one is easy but slow — to get $x_{100}$ you must first find all $99$ earlier terms. The goal is a closed formula: a direct expression for $x_k$ as a function of $k$, so you can jump straight to $x_{100}$. Diagonalization delivers exactly that. But first, where do these sequences even come from?
§8Example 3.4.1 — Counting Parking Arrangements
A row has $k$ parking spaces. A car takes $1$ space, a truck takes $2$. Let $x_k$ be the number of different ways to fill the row exactly. Find the first few values, and a rule connecting them.
§9The Matrix Trick: Turning a Recurrence into Aⱽₖ
A recurrence links numbers. Diagonalization works on matrices. The bridge is a simple, clever repackaging: instead of tracking one number at a time, track a vector holding two consecutive terms.
$\textbf{Step 1 — stack two terms into a vector.}$ Define
$$\mathbf{v}_k = \begin{bmatrix} x_k \\ x_{k+1} \end{bmatrix}.$$
$\textbf{Step 2 — find the matrix that advances it one step.}$ We want a matrix $A$ with $\mathbf{v}_{k+1} = A\mathbf{v}_k$. The top of $\mathbf{v}_{k+1}$ is $x_{k+1}$ (already in $\mathbf{v}_k$), and the bottom is $x_{k+2} = a x_{k+1} + b x_k$ (the recurrence). So
$$\mathbf{v}_{k+1} = \begin{bmatrix} x_{k+1} \\ x_{k+2} \end{bmatrix} = \begin{bmatrix} 0\cdot x_k + 1\cdot x_{k+1} \\ b\, x_k + a\, x_{k+1} \end{bmatrix} = \underbrace{\begin{bmatrix} 0 & 1 \\ b & a \end{bmatrix}}_{A}\begin{bmatrix} x_k \\ x_{k+1} \end{bmatrix} = A\mathbf{v}_k.$$
$\textbf{Step 3 — recognise a dynamical system.}$ This is exactly the setup from Lecture 12: $\mathbf{v}_k = A^k\mathbf{v}_0$. And if $A$ is diagonalizable with eigenvalues $\lambda_1, \lambda_2$ and eigenvectors $\mathbf{x}_1, \mathbf{x}_2$, we get the closed formula
$$\mathbf{v}_k = b_1\lambda_1^k\,\mathbf{x}_1 + b_2\lambda_2^k\,\mathbf{x}_2, \qquad \text{where } \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = P^{-1}\mathbf{v}_0.$$
Reading off the top entry of $\mathbf{v}_k$ gives $x_k$ as a direct formula in $k$. No more stepping through every term.
$\textbf{1.}$ Build the matrix $A = \begin{bmatrix} 0 & 1 \\ b & a \end{bmatrix}$ from $x_{k+2} = a x_{k+1} + b x_k$.
$\textbf{2.}$ Find eigenvalues (roots of $\lambda^2 - a\lambda - b = 0$) and eigenvectors.
$\textbf{3.}$ Compute $\mathbf{b} = P^{-1}\mathbf{v}_0$ with $\mathbf{v}_0 = \begin{bmatrix} x_0 \\ x_1 \end{bmatrix}$.
$\textbf{4.}$ Read $x_k$ from the top entry of $b_1\lambda_1^k\mathbf{x}_1 + b_2\lambda_2^k\mathbf{x}_2$.
§10Worked Example 3.4.2
Solve the recurrence $x_{k+2} = x_{k+1} + 6x_k$ for two cases: (i) $x_0 = 1, x_1 = 3$, and (ii) $x_0 = 1, x_1 = 1$.
§11The Payoff: Fibonacci and the Binet Formula
Return to the parking sequence $1, 1, 2, 3, 5, 8, 13, 21, \ldots$ with $x_{k+2} = x_{k+1} + x_k$. These are the Fibonacci numbers, first written down in 1202 by Leonardo of Pisa (nicknamed Fibonacci). They appear in the spirals of sunflowers, pinecones, and nautilus shells. Let us get their exact formula.
Solve $x_{k+2} = x_{k+1} + x_k$ with $x_0 = 1$, $x_1 = 1$.
§12A Warning: Not Every Recurrence Is Tame
Diagonalization cracks linear recurrences. But a tiny change to the rule can produce a sequence that no one on Earth understands. Here is the most famous unsolved problem in this corner of mathematics.
Define a sequence by a rule that depends on whether the current term is even or odd:
$$x_{k+1} = \begin{cases} \tfrac12 x_k & \text{if } x_k \text{ is even} \\ 3x_k + 1 & \text{if } x_k \text{ is odd.} \end{cases}$$
Starting from $x_0 = 7$, the sequence goes
$$7,\, 22,\, 11,\, 34,\, 17,\, 52,\, 26,\, 13,\, 40,\, 20,\, 10,\, 5,\, 16,\, 8,\, 4,\, 2,\, 1,\, \ldots$$
and then cycles $1, 4, 2, 1, 4, 2, \ldots$ forever. Try $x_0 = 1$: it immediately falls into the same $1, 4, 2$ loop.
The lesson: the clean closed-form world of linear recurrences is a special, lucky case. Diagonalization is powerful precisely because linear structure is exactly what it needs — and exactly what the Collatz rule lacks.
§13Interactive: Recurrence Explorer
Pick a recurrence and watch its terms grow. The bar chart shows $|x_k|$; the panel shows the eigenvalues, which are the growth rates. Notice how the larger eigenvalue sets how fast the bars climb.
- Interpolation: $n$ data points with distinct inputs $\Rightarrow$ a unique polynomial of degree $\leq n-1$ through them all (Theorem 3.2.6).
- Find it by solving $V\mathbf{r} = \mathbf{y}$, where $V$ is the Vandermonde matrix of input powers.
- It works because $\det V = \prod_{i>j}(x_i - x_j) \neq 0$ whenever the inputs are distinct.
- Interpolation is reliable near the data, unreliable far from it.
- Recurrences: a length-2 recurrence $x_{k+2} = ax_{k+1} + bx_k$ becomes $\mathbf{v}_{k+1} = A\mathbf{v}_k$ with $A = \begin{bmatrix} 0 & 1 \\ b & a \end{bmatrix}$ and $\mathbf{v}_k = \begin{bmatrix} x_k \\ x_{k+1} \end{bmatrix}$.
- Diagonalize $A$: $x_k$ is the top entry of $b_1\lambda_1^k\mathbf{x}_1 + b_2\lambda_2^k\mathbf{x}_2$, with $\mathbf{b} = P^{-1}\mathbf{v}_0$.
- Companion-matrix eigenvector shortcut: for eigenvalue $\lambda$, use $\begin{bmatrix} 1 \\ \lambda \end{bmatrix}$.
- Fibonacci $\to$ Binet formula; the golden ratio $\varphi \approx 1.618$ is the dominant eigenvalue and long-term growth rate.
- Non-linear recurrences (Collatz) fall outside this method — and can be genuinely unsolved.
§14Exercises
Two are solved in full below. The rest are the same type — hints are given, but work them yourself. That is where the learning happens.
$\textbf{(b)}\; x_{k+2} = 2x_k - x_{k+1}$, $x_0 = 1, x_1 = 2$.
$\textbf{(c)}\; x_{k+2} = x_{k+1} + 2x_k$, $x_0 = 0, x_1 = 1$.
$\textbf{(d)}\; x_{k+2} = 6x_k - x_{k+1}$, $x_0 = 1, x_1 = 1$.
This is length 3 — each term uses the three before it. The method is the same idea scaled up: stack three terms into a vector.
In the parking problem, now also allow buses, which take 3 spaces. Let $x_k$ count the ways to fill $k$ spaces with cars, trucks, and buses.
A man climbs $k$ stairs, taking $1$ or $2$ at a time. Let $s_k$ be the number of ways. Find $s_k$.
$\textbf{3.4.5:}$ How many "words" of $k$ letters from $\{a, b\}$ have no two $a$'s next to each other? $\quad\textbf{3.4.6:}$ How many sequences of $k$ coin flips have no two heads in a row?
Two applications, one lesson: rewrite a hard problem as a matrix, and linear algebra hands you the answer.
Interpolation turned "fit a curve" into "solve $V\mathbf{r} = \mathbf{y}$." Recurrences turned "list every term" into "diagonalize $A$." Both are the same move — find the right matrix, and the structure does the work. Next we leave the concrete world of $\mathbb{R}^n$ and ask the deeper question these tools have been hinting at all along: what is a vector space, really? That abstraction is where the rest of the course lives.