Lecture 4 - Undecidability
Still in preparation. Please check the references at the end of the document.
In the previous lecture, we have learned about and proved the Recursion Theorem, which in essence states that any computable function can be computed by a Turing machine which can obtain its own description and then compute with it. Today, we will use the Recursion Theorem to prove that certain problems are undecidable, meaning that there is no algorithm that can solve them.
Undecidability
As we have seen in lecture 2, a language $L$ is decidable if there exists a Turing machine $M$ that halts on all inputs and accepts all strings in $L$ and rejects all strings not in $L$.
A language $L$ is undecidable if there is no Turing machine that decides $L$.
Let us now state our first undecidability result.
Theorem 1: The language $$A_{TM} = \{ \langle M \rangle x \mid M \text{ is a Turing machine that accepts input } x \}$$ is undecidable.
Proof: Suppose, for the sake of contradiction, that $A_{TM}$ is decidable. Then there exists a Turing machine $T$ that decides $A_{TM}$. Now, consider the following Turing machine $M$:
On input $x$:
Obtain its own description $\langle M \rangle$ using the recursion theorem.
Run $T$ on input $\langle M \rangle x$.
If $T$ accepts, then reject. If $T$ rejects, then accept.
By construction, $M$ accepts $x$ if and only if $T$ rejects $\langle M \rangle x$. This contradicts the assumption that $T$ decides $A_{TM}$.
Therefore, $A_{TM}$ is undecidable.
The above theorem can also be proved using a diagonalization argument, which we now show.
Alternative Proof of Theorem 1: Suppose, for the sake of contradiction, that $A_{TM}$ is decidable. Then there exists a Turing machine $T$ that decides $A_{TM}$. Since the set of all Turing machines is countable, we can list them as $M_1, M_2, M_3, \dots$ Let $D$ be the following Turing machine:
On input $\langle M \rangle$:
Run $T$ on input $\langle M \rangle$.
If $T$ accepts, then reject. If $T$ rejects, then accept.
On all other inputs (i.e. which are not encodings of any Turing machine), reject.
Since $D$ is a Turing machine, it must correspond to some $M_k$ in our list. Now, consider the input of $A_{TM}$ given by $\langle M_k \rangle \langle M_k \rangle$. Then, by definition of $M_k$, we have that $M_k$ accepts $\langle M_k \rangle$ if and only if $T$ rejects $\langle M_k \rangle \langle M_k \rangle$. This contradicts the assumption that $T$ decides $A_{TM}$.
The two proofs are quite similar in spirit, but the first one is quite shorter.
Also, one of the reasons which makes the language $A_{TM}$ undecidable is that this language tries to predict the behavior of any Turing machine on any input. As we will now see, this is a fundamental limitation of Turing machines. We state this as a principle:
Turing machines cannot predict what other Turing machines will do without simulating them.
Let us now see this principle at work in other undecidability results.
Theorem 2: The language $$\texttt{HALT}_{TM} = \{ \langle M \rangle x \mid M \text{ is a Turing machine that halts on input } x \}$$ is undecidable.
Proof: Suppose, for the sake of contradiction, that $\texttt{HALT}_{TM}$ is decidable. Then there exists a Turing machine $T$ that decides $\texttt{HALT}_{TM}$.
Now, consider the following Turing machine $M$:
On input $x$:
Obtain its own description $\langle M \rangle$ using the recursion theorem.
Run $T$ on input $\langle M \rangle \langle M \rangle$.
If $T$ accepts, then accept. If $T$ rejects, then go on an infinite loop.
By construction, $M$ halts on input $x$ if and only if $T$ accepts $\langle M \rangle \langle M \rangle$. This contradicts the assumption that $T$ decides $\texttt{HALT}_{TM}$.
Another undecidability result is the following:
Theorem 3: The language $$\texttt{Empty}_{TM} = \{ \langle M \rangle \mid L_M = \emptyset \}$$ is undecidable.
Proof: Suppose, for the sake of contradiction, that $\texttt{Empty}_{TM}$ is decidable. Then there exists a Turing machine $T$ that decides $\texttt{Empty}_{TM}$.
Now, consider the following Turing machine $M$:
On input $x$:
Obtain its own description $\langle M \rangle$ using the recursion theorem.
Run $T$ on input $\langle M \rangle$.
If $T$ accepts, then accept. If $T$ rejects, then reject.
By construction, $L_M = \emptyset$ if and only if $T$ rejects $\langle M \rangle$ (note that $L_M = \{0,1\}^*$ if $T$ accepts $\langle M \rangle$). This contradicts the assumption that $T$ decides $\texttt{Empty}_{TM}$.
After seeing such results, one may wonder if we can prove a more general result along the lines of the principle we stated earlier. The answer is yes, and given by Rice’s Theorem.
Theorem 4 (Rice’s Theorem): Let $P$ be a subset of all languages such that
- There is a Turing machine $M$ such that $L_M \in P$.
- There is a Turing machine $N$ such that $L_N \notin P$.
Then the language
$$L_P = \{ \langle M \rangle \mid L_M \in P \}$$ is undecidable.
Proof: Suppose, for the sake of contradiction, that $L_P$ is decidable. Then there exists a Turing machine $T$ that decides $L_P$.
Now, consider the following Turing machine $R$:
On input $x$:
- Obtain its own description $\langle R \rangle$ using the recursion theorem.
- Run $T$ on input $\langle R \rangle$.
- If $T$ accepts, then simulate $N$ on $x$. If $T$ rejects, then simulate $M_1$ on $x$.
By the above construction, $L_R = L_N$ if $T$ accepts $\langle R \rangle$ and $L_R = L_M$ if $T$ rejects $\langle R \rangle$. This contradicts the assumption that $T$ decides $L_P$.
Acknowledgements & References
This lecture was based on these resources:
- [S13, Chapters 4 & 5]
- Lecture notes by Prof. Eric Blais.