Space Complexity

Still in preparation. Please check the references at the end of the document.

“Space: the final frontier.” - Captain Kirk

Today we will begin the study of another important resource in computation: the space complexity of algorithms. The space complexity of a given problem is the optimal amount of memory used by any algorithm that solves the given problem.

Space Complexity

Before we define the classes of problems based on space complexity, we need to define what we mean by the space used by a Turing machine.

Note that it is important to distinguish in the case of space complexity between the space used by the work tape and the space used by the input tape, as we will want to study machines which use an amount of work space which may be smaller than the input size. In this case, we consider the input tape to be read-only, and the work tape to be used for computation.

This is a slight distinction between space and time, as a machine which runs in time sublinear in the input size cannot even read the entire input, and thus cannot really solve any interesting problems.

To study space complexity, we need to define the space cost of a given Turing machine.

Definition 1: The (worst-case) space cost of a Turing machine $M$ is the function $s_M : \mathbb{N} \to \mathbb{N}$ defined as $$ s_M(n) = \max_{x \in \{0, 1\}^n} \{ \text{work tape space used by } M \text{ on input } x \}. $$

For every function $s : \mathbb{N} \to \mathbb{N}$, we define the class $\mathsf{SPACE}(s)$ as the set of languages that can be decided by a deterministic Turing machine $M$ with space cost $s_M = O(s)$.

Analogously to the study of time, we can define the classes $\mathsf{NSPACE}(s)$, $\mathsf{BPSPACE}(s)$, and along with these the space counterparts of $\mathsf{P}$, $\mathsf{NP}$, $\mathsf{BPP}$, and $\mathsf{EXP}$.

For every function $s : \mathbb{N} \to \mathbb{N}$, we define the class $\mathsf{NSPACE}(s)$ as the set of languages that can be verified by a Turing machine $M$ with space cost $s_M = O(s)$, where the certificate is given in another tape (and the space used by the certificate is not counted towards the space cost of the machine - we assume that the certificate is in a read-only tape, the “certificate tape”).

For every function $s : \mathbb{N} \to \mathbb{N}$, we define the class $\mathsf{BPSPACE}(s)$ as the set of languages that can be decided by a probabilistic Turing machine $M$ with space cost $s_M = O(s)$. Again, we consider the random tape to be read-only, and distinct from the work tape.

With the above definitions, we can define the following classes:

Note that $\mathsf{P} \subseteq \mathsf{PSPACE}$, as any poly-time TM can only use a polynomial amount of space. However, $\mathsf{PSPACE}$ contains problems which are not known to be in $\mathsf{P}$, as we now investigate.

Proposition 1: $\mathsf{NP} \subseteq \mathsf{PSPACE}$.

Proof: It is enough to show that $\mathsf{3SAT} \in \mathsf{PSPACE}$. Given a 3CNF formula $\Phi(x_1, \dots, x_n)$, let $M$ be the TM which brute-force tests every possible assignment to the variables of $\Phi$, and returns “yes” iff there is a satisfying assignment. The space cost of $M$ is $O(n)$, as it only needs to store the current assignment and the current clause being tested.

The above proof can be generalized to show that $\mathsf{PH} \subseteq \mathsf{PSPACE}$. Thus, we obtain:

Proposition 2: $\mathsf{PH} \subseteq \mathsf{PSPACE}$.

As it turns out, $\mathsf{PSPACE}$ contains problems which are not known to be in $\mathsf{PH}$, as we now investigate.

Proposition 3: The language TQBF is in $\mathsf{PSPACE}$, where TQBF stands for “Totally Quantified Boolean Formula,” and is defined as follows: $$ \text{TQBF} = \{ \Phi \mid Q_1 x_1 \dots Q_n x_n \Phi(x_1, \dots, x_n) = 1 \}, $$ where $Q_i \in \{ \forall, \exists \}$, and $\Phi$ is a quantifier-free Boolean formula.

The proof of Proposition 3 is quite similar to the proof of Proposition 1, as the TM which decides TQBF simply has to brute-force all possible assignments to the variables, keeping track of the quantifier being tested.

Acknowledgements & References

This lecture was based on these resources:

• Lecture notes by Prof. Eric Blais.
• [S13, Chapter 8]
• [AB09, Chapter 4]