Lecture 11 - Lower Bounds I
In the previous lectures, we have seen how to efficiently reduce the depth of algebraic circuits (computing forms of polynomial degree) up to depth $3$. Thus, to obtain any superpolynomial upper bounds for explicit families of forms, it suffices to prove strong enough lower bounds for homogeneous depth-$4$ circuits, or for general depth-$3$ circuits.
In this lecture, we will begin by showing exponential lower bounds agains homogeneous depth-$3$ circuits.
Homogeneous Depth-$3$ Circuits
Theorem 1 (Nisan-Wigderson 1980s):
Lower Bounds for General Circuits
We will now see how to prove lower bounds for general circuits. The only known lower bound for general circuits is the superlinear lower bound obtained by Baur and Strassen in 1983.
Theorem 2 (Baur-Strassen 1983): any circuit computing the polynomial $f(x_1, \ldots, x_n) = x_1^{d+1} + \ldots + x_n^{d+1}$ must have size $\Omega(n \log d)$.
Proof: Given a circuit $C$ of size $s$ computing $f$, where each gate has fanin $2$, each gate of $C$ computes a quadratic polynomial of its inputs. We can formalize this as follows: assign to each (non-input) gate $u$ of $C$ a variable $y_u$, and for each gate $u$ with children $v, w$, define the quadratic constraints: $$ Q_u = \begin{cases} y_u - (y_v + y_w) & \text{if $u$ is an addition gate}, \\ y_u - (y_v y_w) & \text{if $u$ is a multiplication gate}. \end{cases} $$
Thus, for any set of gates $S$, the system of equations $$ {Q_u = 0 \mid u \in C} \cup {y_v = 1 \mid v \in S} $$ has a solution if and only if the gates in $S$ of the circuit $C$ evaluate to $1$ for some evaluation on the inputs $x_1, \ldots, x_n$.
In particular, if we have a circuit which contains a subset of gates $S$ such that the polynomials $Q_u$ (when seen as polynomials in the $x$ variables) have a finite number of common zeroes, we can apply Bezout’s theorem to obtain an upper bound on the number of common zeroes of the polynomials $Q_u$. This will allow us to give a lower bound on the size of the circuit $C$.
Given our choice of $f(x_1, \ldots, x_n) = x_1^{d+1} + \ldots + x_n^{d+1}$, we may not know how a given circuit $C$ computes $f$. However, by the Baur-Strassen theorem from lecture 5 (Theorem 4, Lecture 5), we have that given $C$ of size $s$ computing $f$, there is a circuit $C’$ of size $\leq 5 \cdot s$ computing $f$ and all of its partial derivatives $\partial_i f$, for $i = 1, \ldots, n$.
Thus, with our choice of $f$, we have a circuit $C’$ of size $\leq 5 \cdot s$ computing $f$ and all of its partial derivatives $\partial_i f = x_i^d$. Let $S$ be the set of gates of $C’$ computing the derivatives $\partial_i f$. Thus, by the above discussion and Bezout’s theorem, we have that the system of equations $$ {Q_u = 0 \mid u \in C’} \cup {y_v = 1 \mid v \in S} $$ (note that the number of variables is $\leq 5s$, as we count all the $y_u$ variables, which include the input variables) has a solution iff $x_i^d = 1$ for all $i = 1, \ldots, n$.
As the above set of equations can have only finitely many common zeroes, we have that the number of common zeroes is at most $$ \prod_{u \in C’} \deg(Q_u) \leq \prod_{u \in C’} 2 = 2^{|C’|} \leq 2^{5s}. $$ However, the number of common zeroes is at least $d^n$, as any choice of $x_i$ such that $x_i^d = 1$ for all $i$ is a common zero of the system of equations (the roots of unity). Hence, we obtain $$ 2^{5s} \geq d^n \Rightarrow s \geq \frac{n \log(d)}{5} \Rightarrow s = \Omega(n \log d). $$
To make this lecture self-contained we state Bezout’s Theorem, a fundamental result in algebraic geometry, which we used in the proof of the lower bound by Baur and Strassen.
Theorem 3 (Bezout’s Theorem): Let $g_1, \ldots, g_m \in \mathbb{F}[x_1, \ldots, x_n]$ be polynomials of degree $d_1, \ldots, d_m$, respectively. Moreover, let $V(g_1, \ldots, g_m)$ denote the set of common zeroes of $g_1, \ldots, g_m$. If $V(g_1, \ldots, g_m)$ is finite, then the number of common zeroes (counted with multiplicities) is at most $\prod_{i=1}^m d_i$.