This lecture introduces a new proof technique called proof by induction. It works particularly well for theorems that have the form $\forall n\in\mathbb{N}, P(n)$ for some predicate $P$. It turns out to be an incredibly robust and widely applicable method of proof within computer science, especially as we move towards algorithms. It is also structurally more complex than any of the prior proofs we've seen, and is notoriously one of the most difficult topics in discrete math.
Before we get into the details of induction, I want to start with a story.
I am the CEO and founder of a company. I want to make sure that, even when I retire, the company's vision never drifts too far from mine. I implement the following rule: "Any CEO of the company must select a successor with the same name as them." 1000000 years later, and many many CEOs later, what is the name of the CEO?
Obviously, the CEO in the future has the same name that I do. In fact, every CEO has the same name! This should hopefully be obvious, but the logical underpinnings are not necessarily as obvious. Consider the predicate $P(n)$ for $n \in \mathbb{N}$ to mean "The $n$th CEO is named Jonathan." Then, what we've done is said "If $P(1)$ is true, and $P(i) \to P(i+1)$ for all $i$, then $\forall n P(n)$ must be true.
Now, consider a direct proof approach. How might I have proven that $P(n)$ is true for all $n$ if I couldn't rely on this structure? It's not clear that, if I just looked at what I know about CEO number $100000$, I would know enough to tell you their name.
This story reveals to us a new proof method. If I want to prove that $P(n)$ is true for all $n$, but I don't actually know enough about any individual $n$, I may instead be able to utilize the fact that some structure is retained between numbers - that $P(i) \to P(i+1)$ for all $i$.
Here is the abstract idea behind proof by induction.
Let $P(n)$ be a predicate with domain $n \in \mathbb N$. If
This is called the principle of mathematical induction. In terms of predicate logic, we can state it as
$$(P(0) \wedge (\forall k, P(k)\rightarrow P(k+1)))\rightarrow (\forall n, P(n)),$$where both quantifiers are over $\mathbb{N}$. The predicate $P$ is often called the inductive hypothesis.
How and when do we make use of this? Mathematical induction is a property of the natural numbers. What this means is that if there is some property that we want to prove about all natural numbers, all we need to do is prove that the property holds for the number 0 and that for any other natural number $k$, if it's true for $k$, then we can show that it's true for the next natural number, $k+1$.
More formally, to prove that for all $n \in \mathbb N$ that $P(n)$ holds,
So why does this even work? Basically, what we're really proving is a very large, infinite chain of implications,
\begin{align*} P(0)&, \\ P(0) &\rightarrow P(1), \\ P(1) &\rightarrow P(2), \\ P(2) &\rightarrow P(3), \\ & \vdots \end{align*}The usual analogy for this is a chain of dominoes: we set up our argument so that any one domino is guaranteed to knock over the next domino ($P(k) \rightarrow P(k+1)$ for any $k$). Once we've set this up, all we need to do is knock over the first domino (showing $P(0)$) to set off the chain reaction.
This leads us to what a proof by induction actually looks like.
Here is a famous example where induction is useful.
If $m,n$ are integers with $n\geq m$ and $x_1,\ldots,x_n$ are integers, we use the following notation:
$$\sum_{i = m}^n x_i = x_m + x_{m+1} + \cdots + x_{n-1} + x_n.$$For every $n \in \mathbb N$,
$$\sum_{i=0}^n i = \frac{n(n+1)}2.$$If you've been taught this formula before, then you've probably been told the story of Gauss solving this problem when he was 7. However, his teachers, in assigning $n = 100$, gave him an easy out since he only needed to consider one instance of the problem, so his solution does not use induction. We don't have that luxury, since we want to prove that this formula holds for all $n$.
We will prove this by induction on $n$. The inductive hypothesis, $P(n)$, is the theorem statement (i.e. $P(n)$ asserts that $\sum_{i=0}^n i = n(n+1)/2.$).
Base case. $n = 0$. Then $$\sum_{i = 0}^0 i = 0 \quad \text{and} \quad \frac{0 \cdot (0+1)}2 = 0,$$ so clearly $\sum_{i=0}^n i = \frac{n(n+1)}2$ holds for $n=0$.
Inductive Hypothesis. Let $k \in \mathbb N$ be arbitrary and assume that $\sum_{i=0}^k i = \frac{k(k+1)}2$.
Inductive step. We will show that $\sum_{i=0}^{k+1} i = \frac{k+1(k+2)}2$. We have \begin{align*} \sum_{i=0}^{k+1} i &= \sum_{i=0}^k i + (k+1) \\ &= \frac{k(k+1)}2 + (k+1) & \text{by ind. hyp.} \\ &= (k+1) \cdot \left(\frac k 2 + 1 \right) \\ &= (k+1) \cdot \left(\frac k 2 + \frac 2 2 \right) \\ &= \frac{(k+1)(k+2)}2 \end{align*} Thus, we have shown that $\sum_{i=0}^n i = \frac{n(n+1)}2$ holds for $n=k+1$. Therefore, $\sum_{i=0}^n i = \frac{n(n+1)}2$ holds for all $n \in \mathbb N$.
This method works remarkably well for proving formulas like this. Once you've done a few of them, new problems will mostly reduce to finding the right algebraic manipulation. Here's another.
For all $n \in \mathbb N$ and $x \geq 0$, $(1+x)^n \geq 1+nx$.
We will show this by induction on $n$.
Base case. Let $n = 0$. Then $(1+x)^0 = 1 \geq 1 = 1 + 0 \cdot x$.
Inductive hypothesis. Let $k \in \mathbb N$ be arbitrary and assume that $(1+x)^k \geq 1+kx$.
Inductive step. Consider $n = k+1$.
\begin{align*} (1+x)^{k+1} &= (1+x)^k (1+x) \\ &\geq (1+kx) (1+x) & \text{ind. hyp.} \\ &= 1 + kx + x + kx^2 \\ &= 1 + (k+1)x + kx^2 \\ &\geq 1+(k+1)x & kx^2 \geq 0 \end{align*}which completes the induction.
Note this result had two variables, $x$ and $n$. This is a great example of why what you're applying induction to needs to be carefully considered and clearly stated.
There is room for creativity in using induction. Here is another famous application.
There are $n$ people, and each of them know a different piece of gossip. They all want to share their gossip with one another, but they're all busy, so only two of them can meet at a time. When two people meet, they share all the gossip they've heard so far.
Define the Gossip Number $G(n)$ to be the number of meetings necessary for everyone to hear every piece of gossip.
Say there are four people, labeled $A, B, C, D$. First, $A$ meets $B$ and $C$ meets $D$, so the situation is as follows:
Then, person $A$ meets person $C$ to trade stories.
Finally, person $B$ meets person $D$ to spill tea as well.
For all $n \geq 4$, $G(n) \leq 2n - 4$.
We will show this by induction on $n$.
Base case. Note that because we only need to show this for $n \geq 4$, our base case here is actually $4$. By the example earlier, $G(4) = 4$, and $2n-4 = 2(4)-4 = 4$, so $G(4)$ satisfies the inequality.
Inductive Hypothesis. Assume that, for some fixed $n$, $G(n) \leq 2n-4$.
Inductive Step. We aim to show that $G(n+1) \leq 2(n+1)-4$. We know something about the gossip number for any $n$ people. So, let's split our $n+1$ people into a "core" group of $n$ people and one extra person. Denote by $p_1, p_2, ..., p_n, p_{n+1}$ the $n+1$ people in the group. Then say that the core group is $p_1, ..., p_n$. By the Inductive Hypothesis, we know that it takes less than $2n-4$ meetings for everyone in the core group to spread each others' gossip, so consider the following process:
Through this process, we can see that everyone learns all the gossip. Furthermore, it takes $2$ meetings between $p_1$ and $p_{n+1}$, as well as at most $2n-4$ meetings between the core group (due to our IH), which adds up to a total of $2 + (2n-4) = 2(n+1) - 4$, so $G(n+1) \leq 2(n+1)-4$.
As a result, we have shown that if $G(n) \leq 2n-4$ then $G(n+1) \leq 2(n+1)-4$.
Now let's see what can go wrong with induction. Consider the following statement.
All horses are the same color.
This is obviously false, so the following proof must be wrong. Try to spot the error.
We will that for all sets $H$ of horses, the horses in $H$ are all the same color. We do this by induction on $n = |H|$, the number of horses under consideration.
Base case. If $|H|=1$, then there is only one horse. A single horse is the same color as itself, so we are done.
Inductive step. Assume that any set of $n$ horses is the same color. Let $H$ be a set of $n+1$ horses, and consider two distinct horses in $H$. By the inductive assumption, if we remove either horse, then the remaining $n$ horses will be the same color. Thus, the first horse is the same color as the other horses, and similarly so is the second horse. Therefore all horses are the same color, completing the inductive proof.
The base case is fine, so the error is in the inductive step. Does this argument work for all $n \geq 2$? (Hint: It doesn't work in exactly one case).
Sometimes, ordinary induction will prove inconvenient for accomplishing a given task. Conveniently, probably the best example of this is the Fundamental Theorem of Arithmetic, which we want to prove anyway. Here's the statement of the theorem:
Every integer $n \gt 1$ can be written as a product of primes. This product is unique up to reordering.
We'll define them precisely below, but the primes are what you might have encountered before; They are the numbers $2,3,5,7,11,\ldots$ that cannot be factored in any nontrivial way.
First, let's think about the problem, leaving the "uniquely" aside for the moment. We want to show that every natural number $n$ has a prime factorization. So assuming that we set everything up properly, we make our hypothesis, that if $n$ has a prime factorization, then $n+1$ has a prime factorization. So we consider $n+1$. If it's prime, then we're good. If it isn't, then we observe that $n+1 = ab$ for some natural numbers $a$ and $b$. What we want is that we can apply our inductive hypothesis to $a$ and $b$. However, our inductive hypothesis doesn't quite say what we want: it only says that $n$ has a prime factorization, not $a$ or $b$!
Strong induction is a reformulation of induction that gives us a "stronger" hypothesis to say these kinds of things.
Let $P(n)$ be a predicate with domain $\mathbb N$. If
are both true, then the following is true:
This is called the principle of strong induction.
What is the difference between ordinary mathematical induction and strong induction? Or, in other words, what makes strong induction strong? The hypothesis that we make in part 2 is stronger. Recall that ordinary induction was stated as a chain of implications $P(k) \rightarrow P(k+1)$. Strong induction is a chain of implications of the form $$P(0) \wedge P(1) \wedge P(2) \wedge \cdots \wedge P(k) \rightarrow P(k+1).$$ Then what we're really proving is all of the following statements, \begin{align*} &P(0), \\ P(0) \rightarrow &P(1), \\ P(0) \wedge P(1) \rightarrow &P(2), \\ P(0) \wedge P(1) \wedge P(2) \rightarrow &P(3), \\ & \vdots \end{align*}
It's important to note that strong induction is not stronger in the sense that it allows us to prove things that we wouldn't be able to with mathematical induction. This is why I described strong induction as an alternate form of mathematical induction. It is possible to use mathematical induction to prove anything we would prove with strong induction, but this involves slightly altering the statement that you want to prove to incorporate your stronger assumptions.
A proof by strong induction looks similar to a proof by mathematical induction.
There is room for variation in how you carry out the induction, and your proofs should be careful about the details. For instance, induction still makes sense if we start from 1 instead of 0 (we just won't conclude that $P(0)$ is true, but we'll know that $P(n)$ is true for all $n\geq 1$.) Starting from a larger number is also fine, as long as we adjust the conclusion.
Another variation of induction will sometimes prove $P(0)$ and $P(1)$ in the base case instead of just $P(0)$. When this is necessary depends on what is needed to "get the induction going". Proving $P(1)$ might not depend on $P(0)$, but $P(2)$ might need both of them. We will see an instance where this is natural, with Fibonacci numbers.
Finally, you may opt to change the notation in the inductive step without affecting your proof. Instead of showing that $P(0)\wedge \cdots \wedge P(k-1)$ implies $P(k)$, you can show $P(0)\wedge \cdots \wedge P(k)$ implies $P(k+1)$. Just be careful that you're actually proving step 2 of the induction template. It's also worth checking that your notation does not inadvertently use something undefined like $P(-1)$, which can happen if you try to use case $P(k-2)$ when $k=1$.
For the remainder of the lecture, we are going to develop some more number theory and apply strong induction to prove some famous theorems.
The study of primes is a central topic of number theory and has fascinated people for thousands of years.
An integer $p$ greater than 1 is called prime if the only positive divisors of $p$ are 1 and $p$. Otherwise, $p$ is called composite.
Primes are obviously relatively prime to any positive number less than them, since they don't have any divisors except 1 and themselves. However, non-prime numbers (called composite numbers) can be relatively prime, even to each other. The numbers 10 and 21 are not prime, since $10 = 2 \cdot 5$ and $21 = 3 \cdot 7$. However, they are relatively prime, since the divisors of $10$ are $\{\pm 1, \pm 2, \pm 5, \pm 10\}$ and the divisors of $21$ are $\{\pm 1, \pm 3, \pm 7, \pm 21\}$.
Simple questions about primes immediately get into deep mathematical territory. However we can establish several important results in this lecture. Here's one that we'll prove next lecture:
There exist infinitely many prime numbers.
This is interesting, in that it might not be intuitively obvious that arbitrarily large prime numbers exist; It seems like once we accumulate several primes, then all large numbers will be divisible by one of those primes and hence composite. We defer the proof for now.
An even stronger theorem is the following, which is called the Prime Number Theorem.
Let $\pi(x)$ be the number of primes at most $x$. Then
$$ \lim_{x\rightarrow\infty} \frac{\pi(x)}{x/\ln(x)} = 1. $$Here $\ln(x)$ is the natural logarithm of $x$. This striking theorem not only says there are infinitely many primes, but also that, amongst the $x$ numbers $\{1,\ldots,x\}$, approximately a $1/\ln(x)$-fraction of them are prime! (And in the limit the fraction is exactly $1/\ln(x)$, which provides an example of the Euler's number $e$ from calculus showing up in number theory.) This means there are a lot or primes. A concrete analysis shows, for example, that a random $1000$-digit number has about a $1/7000$ chance of being prime. Combined with the fact that we can efficiently recognize prime numbers (even large ones, but this is a story for another day), this means that finding a $1000$-digit prime is a simple matter with a computer, which is important in cryptography.
The Prime Number Theorem is a hard theorem that was famously proved using complex analysis (i.e. calculus with imaginary numbers). We definitely won't be proving it in this course.
Here's an even harder question, which is open. Define twin primes to be numbers $p, p+2$ that are both prime. For example, $3,5$ are twin primes, as are $137,139$.
Open Question. Do there exist infinitely many twin primes?
It's a very old question that remains unsolved. However there has been some spectacular progress on this question in the last decade. This started with the following breakthrough result in 2013:
There exist infinitely many pairs of primes $p,q$ satisfying $|p-q| < 10^7$.
The $10^7$ number is far cry from 2, but at least it's finite, and this was the first theorem showing there infinitely many primes within some constant distance of each other. The number $10^7$ was quickly improved to $246$, which remains the record today. You can read more about these events here.
Let's prove Theorem 3.8, The Fundamental Theorem of Arithmetic.
We will first prove that for $n \gt 1$, $n$ can be written as a product of primes. (That is, we'll put off proving the uniqueness up to reordering for now.) We use strong induction on $n$.
Base case. $n = 2$. Clearly, 2 is a product of primes because 2 itself is a prime number.
Inductive Hypothesis. Assume that for $2 \leq m \lt k$, $m$ can be written as a product of primes.
Inductive step. We want to show that $k$ can be written as a product of primes. There are two cases to consider.
This completes the proof that any $n$ can be written as a product of primes. The proof of uniqueness, below, is optional.
If $p$ is prime, and $a,b$ are integers such that $p \mid a \cdot b$, then $p \mid a$ or $p \mid b$.
Without loss of generality, suppose $p \not \mid a$. Then $\gcd(p,a) = 1$ and there exist integers $x$ and $y$ such that $xa + yp = 1$. Then multiplying both sides by $b$, we have $$b = xab + ypb.$$ Since $p \mid ab$, let $n$ be an integer such that $ab = pn$. Then we have $$b = xab + ypb = xpn + ypb = p \cdot (xn+yb).$$ Therefore, by definition of divisibility, $p \mid b$.
Since $3\mid 10\cdot 15 = 150$, the theorem says that $3$ must divide $10$ or $15$ and indeed $3\mid 15$. This isn't true if $p$ is not prime. For instance, $6\mid 10\cdot 15$ but $6$ does not divide either $10$ or $15$ (it gets the factor of $2$ from $10$ and the factor of $3$ from $15$).
We will prove this by (non-strong) induction on $k$, the number of primes.
Base case. In this case, $p \mid p_1$, so $p$ is a positive divisor of $p_1$. Since $p_1$ is prime, its only positive divisors are $1$ and $p_1$. But $p\neq 1$, so it must be the case that $p = p_1$.
Inductive hypothesis.Suppose our lemma is true for $k$.
Inductive step. Consider $p \mid p_1 \cdots p_k \cdot p_{k+1}$. Let $a = p_1 \cdots p_k$ and $p_{k+1} = b$. Since $p \mid ab$, by Theorem 7.7, $p \mid a$ or $p \mid b$. If $p \mid a$, then $p = p_i$ for some $i$ with $1 \leq i \leq k$ by our inductive hypothesis and our result holds. If $p \mid b$, then an argument similar to the base case shows that $p = p_{k+1}$, and our result holds. This completes the induction.
With this corollary in tow, we can finally prove that factorization is unique.
Suppose that $$n = p_1p_2\dots p_s = q_1q_2 \dots q_t$$ are two ways of writing $n$ as a product of primes. We must show that $q_1, q_2, \ldots, q_t$ is a reordering of $p_1, p_2, \ldots, p_s$. We'll prove this by strong induction on $n$ again.
Base case. Our base case is the claim with $n=2$. This is easy to establish (any other product of primes will be larger than $2$).
Inductive step. Now we suppose the theorem is true for all $1,\ldots,n$ and show it is true for $n+1$. Suppose that $$ n+1 = p_1p_2\dots p_s = q_1q_2 \dots q_t. $$ Then $p_1$ divides $q_1 q_2 \dots q_t$. By Corollary 7.10, $p_1 = q_j$ for some $j$. Therefore $$ p_2\dots p_s = q_1\cdots q_{j-1} q_{j+1} \dots q_t. $$ Call this value $m$. Then $1 \leq m \leq n$, so the strong inductive hypothesis applies, we have that $q_1\cdots q_{j-1} q_{j+1} \dots q_t$ is a reordering of $p_2\dots p_s$. Therefore $p_1\dots p_s$ is a reordering of $q_1\dots q_t$.
Rosen Chapter 5.1 and Velleman Chapter 6 both have a plethora of great examples.