Lecture 9 - Depth Reduction II

In the previous lecture, we saw Brent’s formula balancing technique, as well as Hyafil’s circuit balancing technique, which were used to reduce the depth of a circuit. While Brent’s technique reduced the depth of a formula while preserving the size of the formula (up to a constant factor), Hyafil’s technique reduced the depth of a circuit while incurring a quasi-polynomial blow-up in the size of the circuit.

In this lecture, we will see a different technique for depth reduction, which is due to Valiant, Skyum, Berkowitz, and Rackoff. They show that any arithmetic circuit of size $s$ computing a polynomial of degree $d$ can be converted into an arithmetic circuit of depth $O ((\log s + \log d) \log d)$ and size $poly (s, d)$ that computes the same polynomial.

Before we state the theorem, recall that Hyafil’s technique seemed a bit coarse, as it didn’t take much into account the structure of the circuit. In particular, the technique didn’t take much into account the relations between intermediate computations in the circuit (the depth reduction had a “large gap” in degree).

We have seen in Baur-Strassen how to take derivatives with respect to a gate in a circuit. Let us recall that definition.

Definition 1 (Partial derivative with respect to a gate): Let $Φ (\vec{x})$ be an algebraic circuit and $u, v$ be two gates in $Φ$ , computing polynomials $f_{u}, f_{v}$ respectively. Let $Φ_{u = y}$ denote the circuit obtained by deleting the incoming edges to $u$ and replacing the output of $u$ by the variable $y$ . Let $f_{v, u} (\vec{x}, y)$ be the polynomial computed by $v$ in the circuit $Φ_{u = y}$ . Now, define the partial derivative of $v$ with respect to $u$ as the polynomial $\partial_{u} f_{v} := (\partial_{y} f_{v, u}) |_{y = f_{u}} .$

Now, we have the following properties of such gate derivatives, which are simply derived from the basic rules of differentiation.

Proposition 1 (properties of gate derivatives): Let $Φ$ be a homogeneous algebraic circuit computing a polynomial $f$ , and let $u, v$ be two gates in $Φ$ . Then, the following properties hold:

Either $\partial_{u} f_{v} = 0$ or $\deg (\partial_{u} f_{v}) = \deg (f_{v}) - \deg (f_{u})$ .
If $v$ is a sum gate, with children $v_{1}, v_{2}$ , then $\partial_{u} f_{v} = \partial_{u} f_{v_{1}} + \partial_{u} f_{v_{2}}$ .
If $v$ is a product gate, with children $v_{1}, v_{2}$ , such that $\deg (f_{v_{1}}) \geq \deg (f_{v_{2}})$ . If $\deg (f_{u}) > \deg (f_{v}) / 2$ , then $\partial_{u} f_{v} = f_{v_{2}} \cdot \partial_{u} f_{v_{1}}$ .

Now that we have the notion of gate derivatives, we can refine our depth reduction. We will now define the notion of a frontier of a circuit.

Definition 2 (Frontier of a circuit): For an integer $r \geq 0$ , the $r$ -th frontier of a circuit $Φ$ , denoted by $F_{r} (Φ)$ , is the set of all multiplication gates in $Φ$ that compute a polynomial of degree larger than $r$ , and whose children compute polynomials of degree less than or equal to $r$ . That is $F_{r} (Φ) := {v \in Φ ∣ f_{v} = f_{v_{1}} \cdot f_{v_{2}}, \deg (f_{v}) > r, \deg (f_{v_{1}}), \deg (f_{v_{2}}) \leq r} .$

With the above definition, we can get a refined decomposition of the circuit into layers, where each layer is a frontier of the circuit.

The following proposition shows the usefulness of the frontiers.

Proposition 2 (Frontier decomposition): Let $Φ$ be a homogeneous algebraic circuit and $r > 0$ be an integer. Let $u, v$ be two gates in $Φ$ such that $\deg (f_{u}) \leq r < \deg (f_{v}) < 2 \deg (f_{u})$ . Then, we have $f_{v} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v} and \partial_{u} f_{v} = \sum_{w \in F_{r} (Φ)} \partial_{u} f_{w} \cdot \partial_{w} f_{v} .$

Proof: Let’s prove the first equality by induction on the length of the longest path from $F_{r} (Φ)$ to $v$ . Let $v_{1}, v_{2}$ be the children of $v$ .

Base case: $v \in F_{r} (Φ)$ .

In this case, for any $w \in F_{r} (Φ)$ different from $v$ , we have $\partial_{w} f_{v} = 0$ , as $w$ is not in $Φ_{v}$ . Since $\partial_{v} f_{v} = 1$ , we have $\sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v} = f_{v} \cdot \partial_{v} f_{v} = f_{v} .$

Inductive step: we have two cases to consider.

Case 1: $v$ is a sum gate.

In this case, we have $f_{v} = f_{v_{1}} + f_{v_{2}}$ , as well as $\deg (f_{v}) = \deg (f_{v_{1}}) = \deg (f_{v_{2}})$ . Thus, by induction we have $f_{v_{1}} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v_{1}} and f_{v_{2}} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v_{2}} .$ Hence, we have $f_{v} = f_{v_{1}} + f_{v_{2}} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot (\partial_{w} f_{v_{1}} + \partial_{w} f_{v_{2}}) = \sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v} .$ where we have used Proposition 1 Part 2 in the last equality.

Case 2: $v$ is a product gate.

Assume w.l.o.g. that $\deg (f_{v_{1}}) \geq \deg (f_{v_{2}})$ . Since $v \notin F_{r} (Φ)$ , we have $r < \deg (f_{v_{1}}) < 2 r$ . By induction, we have $f_{v_{1}} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v_{1}} .$ For all $w \in F_{r} (Φ)$ , we have $\deg (f_{v}) < 2 r < 2 \cdot \deg (f_{w})$ . Thus, by Proposition 1 Part 3, we have $\partial_{w} f_{v} = f_{v_{2}} \cdot \partial_{w} f_{v_{1}} .$ Hence, we have $f_{v} = f_{v_{1}} \cdot f_{v_{2}} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot f_{v_{2}} \cdot \partial_{w} f_{v_{1}} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v} .$

The proof of the second equality is similar to the first one.

The point of the above proposition is to show that we can use the frontier to compute all the gates with degree between $r$ and $2 r$ in the circuit. We are now ready to state and prove the Valiant-Skyum-Berkowitz-Rackoff theorem.

Valiant-Skyum-Berkowitz-Rackoff (VSBR) Theorem

Theorem 1 (Valiant-Skyum-Berkowitz-Rackoff): For any arithmetic circuit $Φ$ of size $s$ computing a polynomial $f$ of degree $d$ , there is an arithmetic circuit $Ψ$ of depth $O (\log (s) \log (d))$ and size $poly (s, d)$ that computes $f$ .

The theorem above will follow if we prove the following version:

Theorem (variant of theorem 1): For any homogeneous arithmetic circuit $Φ$ of size $s$ computing a polynomial $f$ of degree $d$ , there is a homogeneous arithmetic circuit $Ψ$ computing $f$ having the following properties:

$Ψ$ has alternating layers of addition and multiplication gates.
Each multiplication gate $v$ computes a product of at most $5$ forms, each with degree at most $\deg (v) / 2$ .
Sum gates have arbitrary fan-in.
The size of $Ψ$ is $poly (s, d)$ .

In particular, the above properties imply that the depth of $Ψ$ is $O (\log d)$ .

Proof: Note that we can assume w.l.o.g. that $s \geq n$ , as a circuit of size $< n$ cannot use all the variables. We will construct the circuit $Ψ$ iteratively, where in iteration $k$ we will do the following:

compute all the forms $f_{v}$ from $Φ$ such that $2^{k - 1} < \deg (f_{v}) \leq 2^{k}$ .
compute all the forms $\partial_{u} f_{v}$ for all $u, v$ such that $2^{k - 1} < \deg (f_{v}) - \deg (f_{u}) \leq 2^{k}$ , and $\deg (f_{v}) < 2 \deg (f_{u})$ .

Base case: In iteration $k = 0$ , we compute all the forms $f_{v}$ such that $\deg (f_{v}) \leq 2^{0} = 1$ . As these are just linear forms in the input variables, we can compute each of them by an addition gate with fanin $\leq s$ and depth $1$ . Also, for every two gates $u, v$ such that $\deg (f_{v}) - \deg (f_{u}) \leq 1$ , Proposition 1 Part 1 implies that $\partial_{u} f_{v} = 0$ or $\deg (\partial_{u} f_{v}) = \deg (f_{v}) - \deg (f_{u}) \leq 1$ . Hence, we can compute all the forms $\partial_{u} f_{v}$ by addition gates with fanin $\leq s$ and depth $1$ .

Inductive step: Assume that we have completed iteration $k$ . Let us compute iteration $k + 1$ . Let $v$ be a gate in $Φ$ such that $2^{k} < \deg (f_{v}) \leq 2^{k + 1}$ and let the frontier parameter be $r = ⌊ \deg (f_{v}) / 2 ⌋$ . By Proposition 2, we have $f_{v} = \sum_{w \in F_{r} (Φ)} f_{w} \cdot \partial_{w} f_{v} = \sum_{w \in F_{r} (Φ)} f_{w_{1}} \cdot f_{w_{2}} \cdot \partial_{w} f_{v} .$ where $f_{w} = f_{w_{1}} \cdot f_{w_{2}}$ (since $w \in F_{r} (Φ)$ it must be a multiplication gate). As $w \in F_{r} (Φ)$ , we have $r < \deg (f_{w}) \leq 2 r$ and $\deg (f_{w_{1}}), \deg (f_{w_{2}}) \leq r$ . Thus, by the inductive hypothesis, we have already computed $f_{w_{1}}, f_{w_{2}}$ . Similarly, we have that $\deg (\partial_{w} f_{v}) \leq \deg (f_{v}) - \deg (f_{w}) < r$ . Thus, we have already computed $\partial_{w} f_{v}$ . Hence, we can compute $f_{v}$ by an addition gate with fanin $\leq 2 \cdot s$ whose children are product gates of $3$ forms each, each of degree at most $r$ .

Now, let us compute the derivatives $\partial_{u} f_{v}$ such that $2^{k} < \deg (f_{v}) - \deg (f_{u}) \leq 2^{k + 1}$ and $\deg (f_{v}) < 2 \deg (f_{u})$ . Let the frontier parameter be $r = ⌊ \frac{\deg (f_{v}) + \deg (f_{u})}{2} ⌋$ . By Proposition 2, we have $\partial_{u} f_{v} = \sum_{w \in F_{r} (Φ)} \partial_{u} f_{w} \cdot \partial_{w} f_{v} .$ As $w \in F_{r} (Φ)$ and we can assume $\partial_{w} f_{v} \neq 0$ , we have $r < \deg (f_{w}) \leq min \deg (f_{v}), 2 r$ and $r \geq \deg (f_{w_{1}}) \geq \deg (f_{w_{2}})$ . Moreover, we have $2 \deg (f_{u}) > \deg (f_{v}) \geq r$ . Hence, by Proposition 1 Part 3, we have $\partial_{u} f_{w} = f_{w_{2}} \cdot \partial_{u} f_{w_{1}}$ . By Proposition 1 Part 1, we have $\deg (\partial_{w} f_{v}) = \deg (f_{v}) - \deg (f_{w}) \leq ⌊ \frac{\deg (f_{v}) - \deg (f_{u})}{2} ⌋ \leq 2^{k}$ . Similarly, we have $\deg (\partial_{u} f_{w_{1}}) \leq \deg (f_{w_{1}}) - \deg (f_{u}) \leq ⌊ \frac{\deg (f_{v}) - \deg (f_{u})}{2} ⌋ \leq 2^{k}$ . Thus, by the inductive hypothesis, we have already computed $\partial_{w} f_{v}$ and $\partial_{u} f_{w_{1}}$ .

The only problem now is that we need to compute $f_{w_{2}}$ , and by the above we can only upper bound its degree by $\deg (f_{v}) - \deg (f_{u})$ . In this case, we simply expand $f_{w_{2}}$ as in Proposition 2, and we have $\partial_{u} f_{v} = \sum_{w \in F_{r} (Φ)} f_{w_{2}} \cdot \partial_{u} f_{w_{1}} \cdot \partial_{w} f_{v} = \sum_{w \in F_{r} (Φ)} (\sum_{z \in F_{m} (Φ)} f_{z_{1}} \cdot f_{z_{2}} \cdot \partial_{z} f_{w_{2}}) \cdot \partial_{u} f_{w_{1}} \cdot \partial_{w} f_{v} .$ where $m = ⌊ \frac{\deg (f_{v}) - \deg (f_{u})}{2} ⌋ \leq 2^{k}$ . Thus, by the inductive hypothesis, we have also computed $f_{z_{1}}, f_{z_{2}}$ and $\partial_{z} f_{w_{2}}$ .

Now, all the terms in the multiplication have degree at most $m$ , and we have at most $5$ terms in each product. This computation can be done by an addition gate with fanin $\leq 4 s^{2}$ whose inputs are product gates of $5$ forms each, each of degree at most $m$ .

This concludes the theorem.

References

The material of this lecture is based on the following sources:

Chapter 2 of [SY].
Chapter 5 of [R]

Last updated on Jun 9, 2024

Edit this page