Splay Trees
In the previous lecture, we learned the basics of amortized analysis and applied it to the binary counter data structure. In this lecture, we will learn about a data structure called the splay tree, which is a self-adjusting binary search tree. We will see that the splay tree has the same worst-case time complexity as AVL trees, but it has a better amortized time complexity.
Binary search trees are extremely useful data structures, pervasive in computer science. The worst-case time complexity of binary search trees is $\Theta(\text{height of tree})$ for all three operations (insert, delete, search). Thus, to achieve better performance, we need to ensure that the height of the tree is small. To this end, there are different implementations such as AVL trees and red-black trees, see for instance [CLRS Chapter 13] for more details. Both of the above solutions to balancing the height of the tree are quite involved, require a lot of bookkeeping, and are difficult to analyze.
In this lecture, we will learn about a simpler solution to balancing the height of the tree, called the splay tree. Splay trees are much easier to implement, and they do not require any bookkeeping!
We begin by stating the main theorem which will be proved in this lecture:
Theorem [Sleator & Tarjan 1985]: Splay trees have $\Theta(\log n)$ amortized time complexity (per operation) for all three operations (insert, delete, search). The worst case running time per operation is $\Theta(n)$.
The main idea of splay trees is to adjust the tree after each operation, hence the name self-adjusting trees. Every time we search some key $k$ in the tree, we imagine this will be a “popular query” and we will move the node with key $k$ to the root of the tree. Moving a node to the root of the tree is called a splay operation.
Naive idea: perform single rotations to move the node with key $k$ to the root of the tree.
The above strategy is not good enough, since it can lead to a linear height tree. (We will see this in the exercises.) How do we fix this? By adding different types of rotations, called zig-zig and zig-zag rotations. We have these more general rotations in addition to the single rotations, which we denote by zig.
Notation:
- $n$ is the number of keys (we denote the keys by $1, \ldots, n$)
- $m$ is the number of operations. That is $$ m = \text{# searches} + \text{# insertions} + \text{# deletions} $$
- SEARCH($k$) is the operation of searching for key $k$ in the tree
- INSERT($k$) is the operation of inserting key $k$ into the tree
- DELETE($k$) is the operation of deleting key $k$ from the tree
- given key $k$, let $k.p$ be the parent of $k$ in the tree
Splay Operations
See lecture slides for pictures of zig, zig-zig and zig-zag rotations.
Once we have defined the splay rotations, we can now define the splay operation:
Splay($k$):
- Input: key $k$
- Output: the binary tree with key $k$ at the root of the tree
- Repeat until $k$ is the root of the tree:
- If $k$ is a child of the root, then perform a zig rotation
- If $k$ is a left (right) child of $k.p$ and $k.p$ is a left (right) child of $k.p.p$, then perform a zig-zig rotation
- If $k$ is a left (right) child of $k.p$ and $k.p$ is a right (left) child of $k.p.p$, then perform a zig-zag rotation
See example in slides.
Splay Tree Algorithm
Now that we have defined the splay operation, we can define the splay tree algorithm:
- Input: set of elements ${1, \ldots, n}$
- Output: at each step, a binary search tree data structure and the answer to the query being asked
- SEARCH($k$): after searching for $k$, if $k$ is in the tree, perform Splay($k$). If $k$ is not in the tree, then perform Splay($k’$), where $k’$ is the last key that was compared to $k$ when doing the search.
- INSERT($k$): insert $k$ into the tree, then perform Splay($k$)
- DELETE($k$): delete $k$ from the tree, then perform Splay($k.p$).
Analysis of Splay Trees
For the analysis we will use the potential method.
Potential method: Let $\Phi$ be a function that maps a state of the data structure to a real number. In this case, denoting the cost of the $i^{th}$ operation by $c_i$, the charge $\gamma_i$ at the $i^{th}$ operation is: $$ \gamma_i = c_i + \Phi(T_i) - \Phi(T_{i-1}) $$
The amortized cost of $m$ operations is: $$ \sum_{i=1}^m \gamma_i = \Phi(T_m) - \Phi(T_0) + \sum_{i=1}^m c_i $$ where $T_i$ is our splay tree after the $i^{th}$ operation.
So we need to devise a valid potential function.
Let $\delta(k) :=$ number of descendants of $k$ in the tree (including $k$). Let $rank(k) := \log \delta(k)$.
Our potential function will be: $$ \Phi(T) := \sum_{k \in T} rank(k) $$
examples:
- Max potential is $n \log n$ (when the tree is a completely unbalanced binary tree)
- Min potential is $n$ (when the tree is a completely balanced binary tree)
Let us now see how the potential changes after each basic rotation, where we rotate the node $k$. If we let $rank(k)$ be the current rank of $k$ and $rank’(k)$ be the new rank of $k$ after the rotation, then we have the following:
Lemma 1 (Potential change from SPLAY subroutines): The charge $\gamma$ of a basic rotation of node $k$ is bounded by: $$ \gamma \leq \begin{cases} 3 \cdot (rank’(k) - rank(k)), \ \text{ for zig-zig, zig-zag} \ 3 \cdot (rank’(k) - rank(k)) + 1, \ \text{ for zig} \end{cases} $$
Proof: Let $T’$ be the tree after the rotation.
We begin by analyzing the zig rotation. Let $k$ be the node we are rotating, $b = k.p$ be the parent of $k$. Then, $rank’(k) = rank(b)$. The charge in this case is given by: $$ \gamma = cost + \Phi(T’) - \Phi(T) = 1 + rank’(k) + rank’(b) - rank(k) - rank(b) $$ $$ = 1 + rank’(b) - rank(k) \leq 1 + rank’(k) - rank(k) \leq 1 + 3(rank’(k) - rank(k))$$ where $rank’(b) \leq rank’(k)$ since $b$ is a child of $k$ in $T’$.
Now, let us analyze the zig-zig rotation. Let $k$ be the node we are rotating, $b = k.p$ be the parent of $k$, and $a = b.p$ be the parent of $b$. Then, $rank’(k) = rank(a)$, $rank’(b) \leq rank’(k)$ and $rank(k) \leq rank(b) \leq rank(a)$. Moreover, $\delta’(k) \geq \delta’(a) + \delta(k)$. The charge in this case is given by: $$ \gamma = (\text{cost of rotations}) + \Phi(T’) - \Phi(T) $$ $$= 2 + rank’(a) + rank’(b) + rank’(k) - rank(a) - rank(b) - rank(k) $$ $$= 2 + rank’(a) + rank’(b) - rank(b) - rank(k) \leq 2 + rank’(a) + rank’(k) - 2 \cdot rank(k) $$ Now, since $\delta’(k) \geq \delta’(a) + \delta(k)$, by concavity of $\log$, we have: $$\log\left( \dfrac{\delta’(a)}{\delta’(k)} \right) + \log\left( \dfrac{\delta(k)}{\delta’(k)} \right) \leq -2 \ \Rightarrow \ \log \delta’(a) + \log \delta(k) \leq 2 \log \delta’(k) - 2.$$ Equivalently, we have $rank’(a) \leq 2 \cdot rank’(k) - rank(k) - 2$.
Thus, we have: $\gamma \leq 2 + rank’(a) + rank’(k) - 2 \cdot rank(k) \leq 3 \cdot (rank’(k) - rank(k))$.
The proof for the zig-zag rotation is similar to the above proof for the zig-zig rotation.
Lemma 2 (Potential change from Splay($k$) operation): Let $T$ be our current tree, with root $t$ and $k$ be a node in $T$. The charge $\Gamma$ of the Splay($k$) operation is bounded by: $$ \Gamma \leq 3 \cdot (rank(t) - rank(k)) + 1 \leq 3 \cdot rank(t) + 1 = O(\log n) $$
Proof: Note that $\Gamma$ is the sum of the charges of the basic rotations performed during the Splay($k$) operation. Let $\gamma_i$ be the charge of the $i^{th}$ basic rotation, and $rank_i(k)$ be the rank of $k$ after the $i^{th}$ basic rotation. Then, we have: $rank_0(k) = rank(k)$, $rank_\ell(k) = rank(t)$, where $\ell$ is the number of basic rotations performed during the Splay($k$) operation.
Thus, $$ \Gamma = \sum_{i=1}^\ell \gamma_i \leq 1 + \sum_{i=1}^\ell 3 \cdot (rank_i(k) - rank_{i-1}(k)) = 1 + 3 \cdot (rank(t) - rank(k)) $$ where we used Lemma 1 to get the fist inequality.
Now we are ready to provide the amortized cost of the sequence of $m$ operations. For each operation, we have: $$(\text{charge per operation}) = (\text{charge of SPLAY}) + (\text{cost of operation}) + $$ $$+ (\text{potential change NOT from SPLAY})$$
By Lemma 2, we know that the charge of SPLAY is bounded by $O(\log n)$. We also know that the cost of each operation is upper bounded by the charge of the SPLAY operation (as we are traversing down the tree). So, the only thing left to do is to track the potential change NOT from SPLAY.
- SEARCH: only splay changes the potential
- DELETE: removing a node only decreases the potential
- INSERT: adding a node $k$ increases the potential of all of its ancestors post insertion (and pre-splaying, since we are not counting the effect of splaying)
So we need to handle case 3.
Let $k := k_0 \mapsto k_1 \mapsto k_2 \mapsto \cdots \mapsto k_|ell$ where $k_i$ is the $i^{th}$ ancestor of $k$ and $k_\ell$ is the root of the tree.
If we denote by $\delta’(a)$ the number of descendants of $a$ post insertion, then we have $\delta’(k) = 1$ (since before splying $k$ is a leaf of the tree) and $\delta’(k_i) = \delta(k_i) + 1$ for $1 \leq i \leq \ell$.
Hence, the change in potential is:
$$
\sum_{i=1}^\ell \log \dfrac{\delta’(k_i)}{\delta(k_i)} =
\sum_{i=1}^\ell \log \left( \dfrac{\delta(k_i) + 1}{\delta(k_i)} \right) \leq $$
$$ \leq \sum_{i=1}^n \log\left( \dfrac{i + 1}{i} \right)
= \log (n+1) = O(\log n) $$
Thus, the amortized cost of each operation is $O(\log n)$, since we upper bounded each of the 3 quantities (charge of SPLAY, cost of operation, potential change NOT from SPLAY) by $O(\log n)$.
To conclude, we need to show that our potential function is valid. This is easy to see, as the initial potential is zero (for the empty tree), and the potential is always non-negative (since it is a sum of logarithms of integers).