Chapter_13_Red-Black_Trees/problems.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13-1 Persistent dynamic sets\n",
    "\n",
    "> During the course of an algorithm, we sometimes find that we need to maintain past versions of a dynamic set as it is updated. We call such a set __*persistent*__. One way to implement a persistent set is to copy the entire set whenever it is modified, but this approach can slow down a program and also consume much space. Sometimes, we can do much better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. For a general persistent binary search tree, identify the nodes that we need to change to insert a key $k$ or delete a node $y$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Insert: the number of nodes in the simple path plus 1.\n",
    "\n",
    "Delete: the ancestors of $y$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*b*__. Write a procedure PERSISTENT-TREE-INSERT that, given a persistent tree $T$ and a key $k$ to insert, returns a new persistent tree $T'$ that is the result of inserting $k$ into $T$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TreeNode:\n",
    "    def __init__(self, key, left=None, right=None):\n",
    "        self.key = key\n",
    "        self.left = left\n",
    "        self.right = right\n",
    "\n",
    "\n",
    "def insert(root, x):\n",
    "    if root is None:\n",
    "        return TreeNode(x)\n",
    "    new_root = TreeNode(root.key)\n",
    "    if root.key <= x:\n",
    "        new_root.left = root.left\n",
    "        new_root.right = insert(root.right, x)\n",
    "    else:\n",
    "        new_root.left = insert(root.left, x)\n",
    "        new_root.right = root.right\n",
    "    return new_root"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. If the height of the persistent binary search tree $T$ is $h$, what are the time and space requirements of your implementation of PERSISTENT-TREE-INSERT?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\Theta(h)$ and $\\Theta(h)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. Suppose that we had included the parent attribute in each node. In this case, PERSISTENT-TREE-INSERT would need to perform additional copying. Prove that PERSISTENT-TREE-INSERT would then require $\\Omega(n)$ time and space, where $n$ is the number of nodes in the tree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$T(n)=2T(n/2)+\\Theta(1)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*e*__. Show how to use red-black trees to guarantee that the worst-case running time and space are $O(\\lg n)$ per insertion or deletion."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on Exercise 13.3-6."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13-2 Join operation on red-black trees\n",
    "\n",
    "> The __*join*__ operation takes two dynamic sets $S_1$ and $S_2$ and an element $x$ such that for any $x_1 \\in S_1$ and $x_2 \\in S_2$, we have $x_1.key \\le x.key \\le x_2.key$. It returns a set $S = S_1 \\cup \\{x\\} \\cup S_2$. In this problem, we investigate how to implement the join operation on red-black trees."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. Given a red-black tree $T$, let us store its black-height as the new attribute $T.bh$. Argue that RB-INSERT and RB-DELETE can maintain the $bh$ attribute without requiring extra storage in the nodes of the tree and without increasing the asymptotic running times. Show that while descending through $T$, we can determine the black-height of each node we visit in $O(1)$ time per node visited."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize: $bh = 0$.\n",
    "\n",
    "RB-INSERT: if in the last step the root is red, we increase $bh$ by 1.\n",
    "\n",
    "RB-DELETE: if $x$ is root, we decrease $bh$ by 1.\n",
    "\n",
    "Each node: in the simple path, decrease $bh$ by 1 each time we find a black node."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We wish to implement the operation RB-JOIN$(T_1, x, T_2)$, which destroys $T_1$ and $T_2$ and returns a red-black tree $T=T_1 \\cup \\{x\\} \\cup T_2$. Let $n$ be the total number of nodes in $T_1$ and $T_2$.\n",
    "\n",
    "> __*b*__. Assume that $T_1.bh \\ge T_2.bh$. Describe an $O(\\lg n)$-time algorithm that finds a black node $y$ in $T_1$ with the largest key from among those nodes whose black-height is $T2.bh$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Move to the right child if the node has a right child, otherwise move to the left child. If the node is black, we decease $bh$ by 1. Repeat the step until $bh = T2.bh$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. Let $T_y$ be the subtree rooted at $y$. Describe how $T_y \\cup \\{ x \\} \\cup T_2$ can replace $T_y$ in $O(1)$ time without destroying the binary-search-tree property."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$x$'s parent is $T_y$'s parent, $x$'s left child is $T_y$ and its right child is $T_2$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. What color should we make $x$ so that red-black properties 1, 3, and 5 are maintained? Describe how to enforce properties 2 and 4 in $O(\\lg n)$ time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Red. RB-INSERT-FIXUP(T, x)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*e*__. Argue that no generality is lost by making the assumption in part (b). Describe the symmetric situation that arises when $T_1.bh \\le T_2.bh$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Symmetric."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*f*__. Argue that the running time of RB-JOIN is $O(\\lg n)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$O(1) + O(\\lg n) = O(\\lg n)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13-3 AVL trees\n",
    "\n",
    "> An __*AVL tree*__ is a binary search tree that is __*height balanced*__: for each node $x$, the heights of the left and right subtrees of $x$ differ by at most 1. To implement an AVL tree, we maintain an extra attribute in each node: $x.h$ is the height of node $x$. As for any other binary search tree $T$, we assume that $T.root$ points to the root node."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. Prove that an AVL tree with $n$ nodes has height $O(\\lg n)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$T(h) = T(h-1) + T(h-2)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*b*__. To insert into an AVL tree, we first place a node into the appropriate place in binary search tree order. Afterward, the tree might no longer be height balanced. Specifically, the heights of the left and right children of some node might differ by 2. Describe a procedure BALANCE$(x)$, which takes a subtree rooted at $x$ whose left and right children are height balanced and have heights that differ by at most 2, i.e., $|x.right.h - x.left.hj|\\le 2$, and alters the subtree rooted at $x$ to be height balanced."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See __*c*__."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. Using part (b), describe a recursive procedure AVL-INSERT$(x, z)$ that takes a node $x$ within an AVL tree and a newly created node $z$ (whose key has already been filled in), and adds $z$ to the subtree rooted at $x$, maintaining the property that $x$ is the root of an AVL tree. As in TREE-INSERT from Section 12.3, assume that $z.key$ has already been filled in and that $z.left = NIL$ and $z.right = NIL$; also assume that $z.h = 0$. Thus, to insert the node $z$ into the AVL tree $T$, we call AVL-INSERT$(T.root, z)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "class AVLTreeNode:\n",
    "    def __init__(self, key, left=None, right=None):\n",
    "        self.key = key\n",
    "        self.h = 0\n",
    "        self.p = None\n",
    "        self.left = left\n",
    "        self.right = right\n",
    "        if self.left is not None:\n",
    "            self.left.p = self\n",
    "        if self.right is not None:\n",
    "            self.right.p = self\n",
    "\n",
    "\n",
    "class AVL:\n",
    "    def __init__(self):\n",
    "        self.root = None\n",
    "\n",
    "    def left_rotate(self, x):\n",
    "        y = x.right\n",
    "        x.right = y.left\n",
    "        if y.left is not None:\n",
    "            y.left.p = x\n",
    "        y.p = x.p\n",
    "        if x.p is None:\n",
    "            self.root = y\n",
    "        elif x == x.p.left:\n",
    "            x.p.left = y\n",
    "        else:\n",
    "            x.p.right = y\n",
    "        y.left = x\n",
    "        x.p = y\n",
    "\n",
    "    def right_rotate(self, x):\n",
    "        y = x.left\n",
    "        x.left = y.right\n",
    "        if y.right is not None:\n",
    "            y.right.p = x\n",
    "        y.p = x.p\n",
    "        if x.p is None:\n",
    "            self.root = y\n",
    "        elif x == x.p.left:\n",
    "            x.p.left = y\n",
    "        else:\n",
    "            x.p.right = y\n",
    "        y.right = x\n",
    "        x.p = y\n",
    "\n",
    "    def get_height(self, node):\n",
    "        if node is None:\n",
    "            return -1\n",
    "        return node.h\n",
    "\n",
    "    def update_height(self, node):\n",
    "        if node is None:\n",
    "            return\n",
    "        node.h = max(self.get_height(node.left), self.get_height(node.right))+1\n",
    "\n",
    "    def balance_factor(self, node):\n",
    "        return self.get_height(node.left) - self.get_height(node.right)\n",
    "\n",
    "    def avl_insert(self, x):\n",
    "        self.root = self.avl_insert_rec(self.root, x)\n",
    "\n",
    "    def avl_insert_rec(self, root, x):\n",
    "        if root is None:\n",
    "            return AVLTreeNode(x)\n",
    "        if root.key > x:\n",
    "            root.left = self.avl_insert_rec(root.left, x)\n",
    "            root.left.p = root\n",
    "        else:\n",
    "            root.right = self.avl_insert_rec(root.right, x)\n",
    "            root.right.p = root\n",
    "        if self.balance_factor(root) == 2:\n",
    "            if self.balance_factor(root.left) == -1:\n",
    "                self.left_rotate(root.left)\n",
    "            self.right_rotate(root)\n",
    "            root = root.p\n",
    "            self.update_height(root.left)\n",
    "            self.update_height(root.right)\n",
    "            self.update_height(root)\n",
    "        elif self.balance_factor(root) == -2:\n",
    "            if self.balance_factor(root.right) == 1:\n",
    "                self.right_rotate(root.right)\n",
    "            self.left_rotate(root)\n",
    "            root = root.p\n",
    "            self.update_height(root.left)\n",
    "            self.update_height(root.right)\n",
    "            self.update_height(root)\n",
    "        else:\n",
    "            self.update_height(root)\n",
    "        return root"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. Show that AVL-INSERT, run on an n-node AVL tree, takes $O(\\lg n)$ time and performs $O(1)$ rotations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$O(\\lg n)$: the length of path from root to the inserted node.\n",
    "\n",
    "$O(1)$: the height will decrease by 1 after the rotation, therefore the ancestors will not be affected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13-4 Treaps\n",
    "\n",
    "> If we insert a set of $n$ items into a binary search tree, the resulting tree may be horribly unbalanced, leading to long search times. As we saw in Section 12.4, however, randomly built binary search trees tend to be balanced. Therefore, one strategy that, on average, builds a balanced tree for a fixed set of items would be to randomly permute the items and then insert them in that order into the tree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. Show that given a set of nodes $x_1, x_2, \\dots, x_n$, with associated keys and priorities, all distinct, the treap associated with these nodes is unique."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The root is the node with smallest priority, the root divides the sets into two subsets based on the key. In each subset, the node with smallest priority is selected as the root, thus we can uniquely determine a treap with a specific input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*b*__. Show that the expected height of a treap is $\\Theta(\\lg n)$, and hence the expected time to search for a value in the treap is $\\Theta(\\lg n)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Same as randomly built BST."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. Explain how TREAP-INSERT works. Explain the idea in English and give pseudocode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TreapNode:\n",
    "    def __init__(self, key, left=None, right=None):\n",
    "        self.key = key\n",
    "        self.priority = random.random()\n",
    "        self.p = None\n",
    "        self.left = left\n",
    "        self.right = right\n",
    "        if self.left is not None:\n",
    "            self.left.p = self\n",
    "        if self.right is not None:\n",
    "            self.right.p = self\n",
    "\n",
    "\n",
    "class Treap:\n",
    "    def __init__(self):\n",
    "        self.root = None\n",
    "\n",
    "    def left_rotate(self, x):\n",
    "        y = x.right\n",
    "        x.right = y.left\n",
    "        if y.left is not None:\n",
    "            y.left.p = x\n",
    "        y.p = x.p\n",
    "        if x.p is None:\n",
    "            self.root = y\n",
    "        elif x == x.p.left:\n",
    "            x.p.left = y\n",
    "        else:\n",
    "            x.p.right = y\n",
    "        y.left = x\n",
    "        x.p = y\n",
    "\n",
    "    def right_rotate(self, x):\n",
    "        y = x.left\n",
    "        x.left = y.right\n",
    "        if y.right is not None:\n",
    "            y.right.p = x\n",
    "        y.p = x.p\n",
    "        if x.p is None:\n",
    "            self.root = y\n",
    "        elif x == x.p.left:\n",
    "            x.p.left = y\n",
    "        else:\n",
    "            x.p.right = y\n",
    "        y.right = x\n",
    "        x.p = y\n",
    "\n",
    "    def insert(self, x):\n",
    "        self.root = self.insert_rec(self.root, x)\n",
    "\n",
    "    def insert_rec(self, root, x):\n",
    "        if root is None:\n",
    "            return TreapNode(x)\n",
    "        if root.key > x:\n",
    "            root.left = self.insert_rec(root.left, x)\n",
    "            root.left.p = root\n",
    "            if root.left.priority < root.priority:\n",
    "                self.right_rotate(root)\n",
    "                root = root.p\n",
    "        else:\n",
    "            root.right = self.insert_rec(root.right, x)\n",
    "            root.right.p = root\n",
    "            if root.right.priority < root.priority:\n",
    "                self.left_rotate(root)\n",
    "                root = root.p\n",
    "        return root"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. Show that the expected running time of TREAP-INSERT is $\\Theta(\\lg n)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rotation is $\\Theta(1)$, at most $h$ rotations, therefore the expected running time is $\\Theta(\\lg n)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*e*__. Consider the treap $T$ immediately after TREAP-INSERT has inserted node $x$. Let $C$ be the length of the right spine of the left subtree of $x$. Let $D$ be the length of the left spine of the right subtree of $x$. Prove that the total number of rotations that were performed during the insertion of $x$ is equal to $C + D$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Left rotation increase $C$ by 1, right rotation increase $D$ by 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*f*__. Show that $X_{ik} = 1$ if and only if $y.priority > x.priority$, $y.key < x.key$, and, for every $z$ such that $y.key < z.key < x.key$, we have $y.priority < z.priority$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first two are obvious.\n",
    "\n",
    "The min-heap property will not hold if $y.priority > z.priority$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*g*__. Show that\n",
    "\n",
    "> $$\\begin{array}{rll}\n",
    "\\text{Pr}\\{X_{ik}=1\\} &=&\n",
    "\\displaystyle \\frac{(k-i-1)!}{(k-i+1)!} \\\\\n",
    "&=& \\displaystyle \\frac{1}{(k-i+1)(k-i)} \\\\\n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Total number of permutations: $(k-i+1)!$\n",
    "\n",
    "Permutations satisfy the condition: $(k-i-1)!$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*h*__. Show that\n",
    "\n",
    "> $$\\begin{array}{rll}\n",
    "\\text{E}[C] &=&\n",
    "\\displaystyle \\sum_{j=1}^{k-1} \\frac{1}{j(j+1)} \\\\\n",
    "&=& \\displaystyle 1 - \\frac{1}{k} \\\\\n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{array}{rll}\n",
    "\\text{E}[C] \n",
    "&=& \\displaystyle \\sum_{j=1}^{k-1} \\frac{1}{(k-i+1)(k-i)} \\\\\n",
    "&=& \\displaystyle \\sum_{j=1}^{k-1} \\left ( \\frac{1}{k-i} - \\frac{1}{k-i+1} \\right ) \\\\\n",
    "&=& \\displaystyle 1 - \\frac{1}{k} \n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*i*__. Use a symmetry argument to show that\n",
    "\n",
    "> $\\displaystyle \\text{E}[D] = 1 - \\frac{1}{n-k+1}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{array}{rll}\n",
    "\\text{E}[D] \n",
    "&=& \\displaystyle \\sum_{j=1}^{n-k} \\frac{1}{(k-i+1)(k-i)} \\\\\n",
    "&=& \\displaystyle 1 - \\frac{1}{n-k+1}\n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*j*__. Conclude that the expected number of rotations performed when inserting a node into a treap is less than 2."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\text{E}[C] + \\text{E}[D] \\le 2$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}