\documentclass[11pt]{article}
\usepackage{handout}
\usepackage{amsmath,amssymb,amsthm,epsfig}
\begin{document}
\input{courseMacros}

\newtheorem{Theorem}{Theorem}
\newtheorem{Corollary}{Corollary}
\newtheorem{Lemma}{Lemma}
\newtheorem{Claim}{Claim}
\theoremstyle{definition}
\newtheorem{Definition}{Definition}
\newtheorem{problem}{Problem}
\newenvironment{Proof}{\noindent{\bf Proof: }}{\qed}
%\theoremstyle{remark}
\newtheorem{Remark}{Remark}

\thispagestyle{empty}
\thispagestyle{empty}
\setlength{\parindent}{0pt}
\setlength{\parskip}{1.8ex}

\lecturehead{CS G399}{Algorithmic Power Tools I}{Fall 2007}{4 December
2007}{Max Bandazian}

\newcommand{\p}{\mathcal{P}}
\newcommand{\U}{\mathcal{U}}
\newcommand{\coll}{\mathcal{S}}
\newcommand{\xpy}{x + \lambda y}
\newcommand{\Alg}{\mathcal{A}}
\newcommand{\opt}{\mbox{OPT}}
\newcommand{\eps}{\varepsilon}

\noindent {\bf Lecture Outline:}
\begin{itemize}
\item
Recursion Theory
\item
Randomness
\item
Volume Estimation
\end{itemize}

In this lecture, we have an overview of the Kleene hierarchy, followed
by a discussion of the power of randomized algorithms. We look at the
connections between the Kleene hierarchy and randomized classes such as
BPP.  We then introduce volume estimation.

\section{Recursion Theory}

\begin{figure}[h]	
\begin{picture}(300,100)
\put(0,10){\line(1,0){300}}
\put(100,10){\line(0,1){80}}
\put(200,10){\line(0,1){80}}
\put(125,15){Recursive}
\put(0,50){Uncomputable}
\put(0,0){Computable}
\put(100,20){\line(4,1){100}}
\put(200,20){\line(-4,1){100}}
\put(100,45){\line(4,1){100}}
\put(200,45){\line(-4,1){100}}
\put(75,45){$\Sigma_1(\exists)$}
\put(65,70){$\Sigma_2(\exists\forall)$}
\put(205,45){$\Pi_1(\forall)$}
\put(205,70){$\Pi_2(\forall\exists)$}
\put(150,80){$\vdots$}
\end{picture}
\label{fig:kleene}
\caption{Kleene Hierarchy}
\end{figure}

Recursion theory is an important branch of Computer Science, although
it is not very active currently. It categorizes problems by computability;
see the Kleene hierarchy.  Problems in this hierarchy can be characterized
by logical formulae, with a move up and left in the diagram corresponding
to adding an existential quantifier, and moving up and right corresponding
to adding a universal quantifier.

Problems in $\Sigma_1$ can be accepted by a Turing machine, but not rejected;
similarly, problems in $\Pi_1$ can be rejected by a Turing machine, but 
not accepted.  Given an oracle to decide some level in the hierarchy, some
Turing machine can accept or reject the next level up, depending on which
quantifier is added.

\begin{figure}
\begin{picture}(300,100)
\put(100,10){\line(0,1){80}}
\put(200,10){\line(0,1){80}}
\put(145,15){P}
\put(100,20){\line(4,1){100}}
\put(200,20){\line(-4,1){100}}
\put(100,45){\line(4,1){100}}
\put(200,45){\line(-4,1){100}}
\put(75,45){$\Sigma_1^P$}
\put(75,70){$\Sigma_2^P$}
\put(205,45){$\Pi_1^P$}
\put(205,70){$\Pi_2^P$}
\put(150,80){$\vdots$}
\end{picture}
\caption{Time/space hierarchy}
\label{fig:timespace}
\end{figure}

The recursive problems are also divided into a hierarchy (fig. \ref{fig:timespace}).
This hierarchy divides the computable problems by the amount of
resources that a machine requires to decide them.  Here, $\Sigma_1^P$
is the class NP, and $\Pi_1^P$ is the class co-NP. There are
polynomially many levels to this hierarchy.  Since everything in the
hierarchy can be written as a quantified boolean formula, it is
reducible to QBF; thus, it is closed by the class PSPACE.

The next time complexity class is $\mathrm{EXPTIME}$.  We know that
$\mathrm{P} \subset \mathrm{EXPTIME}$, and 
$\mathrm{P} \subseteq \mathrm{PSPACE} \subseteq \mathrm{EXPTIME}$,
but it is not known whether any two of these classes are equal.

\section{Randomness}
Randomness is a resource that allows us to do interesting things.
For instance, consider the graph nonisomorphism problem. This is
in co-NP, so suppose we have access to a god. Even if the oracle
is untrustworthy (may give the wrong answer), we can get a
reliable answer using randomness.  If we have graphs $G_0$ and
$G_1$, and the god says that they are nonisomorphic, then we
randomly select one, permute it, and ask the god to identify it.
If the graphs are nonisomorphic, the god can tell which it is
by checking for isomorphism with $G_0$ and $G_1$. If the god
lied, then there is no way to tell, and the response has only a
50\% chance of being correct.  We can then repeat this
experiment as many times as necessary, until a desired
probability of being correct is reached.

\subsection{BPP}
BPP is the class of problems solvable given polynomial time
and random coins.  More precisely, for a language $L$ in BPP,
there is some randomized machine that decides $L$ in polynomial
time, with at most a $1/3$ chance of deciding wrong. Note that
any probability less than $.5$ of deciding wrong allows
an arbitrarily small error probability to be reached by iterating,
and using whichever answer was given the most times.


Related to BPP is the class RP, where machines always answer
correctly when the correct answer is ``yes'', but have some
probability of answering wrong when the correct answer is ``no''.
co-RP is the opposite of this situation; the answer is always
right in the ``no'' case, and may be wrong for the ``yes'' case.


ZPP is the class of problems that a machine can decide correctly
in expected polynomial time.  $\mathrm{ZPP} = \mathrm{RP} \cap \mathrm{co-RP}$, by bisimulation.
We also know that $P \subseteq \mathrm{ZPP} \subseteq \mathrm{BPP}$, but nobody knows
any more than that.  It does seem, though, that there is something
weird going on in these classes.


For instance, primeness testing used to be the canonical problem
known to be in BPP but not known to be in P, until recently
a polynomial time algorithm for it was found.  A problem in BPP
that is still not known to be in P is determining whether the
determinant of a matrix with both constant and variable entries
is 0. Since in this case the determinant can have exponentially
many terms, it is not nown to be in P.  However, we know the
maximum degree of the determinant polynomial by the size of the
matrix, so we can randomly try to guess the roots.


E is the class of problems that are accepted in $2^{O(n)}$ time,
and SUBEXP is the class accepted in $2^{O(n^\epsilon)}$. Surprisingly,
if E is different from SUBEXP, then P = BPP.  If $\mathrm{P} \neq \mathrm{BPP}$, then
the entire polynomial hierarchy collapses.


A toy example of speeding up computation with randomness is matrix
product verification. Suppose we have matrices $A$ and $B$, and 
we want to verify $AB = C$. Performing the multiplication is
roughly $O(n^3)$.  However, we can choose a random vector $r$,
and check whether $(AB-C)r = 0$. By algebra, this is $A(Br) = Cr$.
Since the matrix-vector multiplications can be done in quadratic
time, the verification becomes $O(n^2)$.


\section{Volume Estimation}
Toda's theorem says that $PH \subseteq P^{\sharp P}$.  $\sharp P$
is equivalent to counting, which is equivalent to volume estimation
(i.e., volume estimation is very hard).


The idea is to generate a polytope whose volume is equal to the
number of things we are counting.  This is difficult to do 
deterministically, but easy with randomness (This is a big deal).


Our model is n-dimensional space, and the goal is to estimate
the volume of a convex object.  We
requires as input the dimension n and $r1 \leq r2$ so that
the object contains $B({r_1})$ and is contained by $B({r_2})$.
($B_{r_1} \subseteq k \subseteq B_{r_2}$).
Alternatively, we can require that the object contains the
origin, and the unit ball contains the object.
($0^n \in k \subseteq B(1)$)

To estimate the volume, we select $m$ points at random from
$B(r_2)$. For each point $p_1 \dots p_m$, we check whether it is inside the
object.  We claim, then, that
 $\frac{\mathtt{Vol}(\mathtt{Conv}(\langle p_i\rangle))}
 {\mathtt{Vol}(B(1))} \leq m / (2 - \epsilon)^n$.

We can create an approximation of the polytope itself as follows.
For each point $p_i$, let $B_i$ be the ball with the line segment
from the origin to $p_i$ as a diameter.(see figure \ref{fig:ball}
Then we use $\bigcup_i{B_i}$
as an approximation; we claim that $\mathtt{conv}(p_i) \subseteq
\bigcup_i{B_i}$.


\begin{figure}[h]	
\center
\begin{picture}(150,150)
\put(100,100){\circle*{4}}

\put(100,100){\line(0,1){20,1}}
\put(100,110){\circle{20}}
\put(100,120){\circle*{2}}
\put(100,128){$p_1$}
\put(75,105){$B_1$}

\put(100,100){\line(1,0){40,1}}
\put(120,100){\circle{40}}
\put(140,100){\circle*{2}}
\put(142,100){$p_2$}
\put(120,110){$B_2$}

\put(100,100){\line(2,-3){12,1}}
\put(107,91){\circle{24}}
\put(113,80){\circle*{2}}
\put(110,73){$p_3$}
\put(80,90){$B_3$}
\end{picture}
\label{fig:ball}
\caption{Ball construction}
\end{figure}

We demonstrate this claim as follows. Consider a point
in the convex hull. If $B_i$ contains it, then
the angle from pole to pole is $\geq 90 \deg$. For
points outside a ball, this angle is acute.  We need to
show that $\forall{x} \notin \bigcup{B_i}$ and 
$\forall{v} \in \mathtt{conv}(p_i)$, the line
$\overline{0v}$ subtends $< 90 \deg$ at $x$. This is true
because:
$\forall i, \overline{0p_i}$ subtends $< 90 \deg$
at $x$. Therefore $\forall{i}, (p_i - x) \dot (0 - x) > 0$,
so $\langle v - x, 0 - x\rangle > 0$.

\end{document}