\documentclass[11pt]{article}
\usepackage{handout}
\usepackage{amsmath,amssymb,amsthm,epsfig}
\usepackage[version=3]{mhchem}
\begin{document}
\input{courseMacros}

\newtheorem{Theorem}{Theorem}
\newtheorem{Corollary}{Corollary}
\newtheorem{Lemma}{Lemma}
\newtheorem{Claim}{Claim}
\theoremstyle{definition}
\newtheorem{Definition}{Definition}
\newtheorem{problem}{Problem}
\newenvironment{Proof}{\noindent{\bf Proof: }}{\qed}
%\theoremstyle{remark}
\newtheorem{Remark}{Remark}

\thispagestyle{empty} \thispagestyle{empty}
\setlength{\parindent}{0pt} \setlength{\parskip}{1.8ex}

\head{CS G399}{Algorithmic Power Tools I}{Fall 2007}{22 November
2007}{Tao Jin}


\newcommand{\p}{\mathcal{P}}
\newcommand{\U}{\mathcal{U}}
\newcommand{\coll}{\mathcal{S}}
\newcommand{\xpy}{x + \lambda y}
\newcommand{\Alg}{\mathcal{A}}
\newcommand{\opt}{\mbox{OPT}}
\newcommand{\eps}{\varepsilon}

\noindent {\bf Lecture Outline:}
\begin{itemize}
\item
Review of Sparsest-Cut problem
\item
L1 Embedding
\item
Frechet's Embedding and Bourgain's Theorem

\begin{itemize}
\item
Construction of Frechet's Embedding
\item
Proof of Bourgain's Theorem
\end{itemize}
\end{itemize}

In this lecture we will first review the sparsest cut problem and its relation
with the metric embeddings. Then we will prove the Bourgain's Theorem with the 
construction of the Frechet's embedding, which guarantee us a $O(log n)$ approximation
factor to the optimal solution to the sparsest-cut problem.

\section{Sparsest-Cut Review}

In the last lecture we've defined the sparsest-cut problem. Recall that:

{\bf Sparsest-Cut Problem:}

Given a graph $G = (V,E)$ and $w: E \rightarrow Z^+$. Determine cut
$(S,\bar{S})$ such that $\frac{E(S,\bar{S})}{|S||\bar{S}|}$ is
minimized where $E(S,\bar{S})$ is number of edges crossing $S$ and
$\bar{S}$.


For any cut $(S,\bar{S})$, we defined the distance between two vertices in the graph as follow,
\[
d_S(i,j)  = \left\{ {\begin{array}{*{20}c}
   1  \\
   0  \\
\end{array}} \right.\begin{array}{*{20}c}
   {(i,j)\ is\ cross\ cut\ edge}  \\
   {otherwise}  \\
\end{array}
\]

We claim that any cut induces a so-called cut metric $d_S$. It is clear to see that
\begin{eqnarray}
Sparsest Cut &=& \min_S \frac{E(S,\bar{S})}{|S||\bar{S}|}\nonumber\\
             &=& \min_{d_S} \frac {\sum_{i,j \in E} d_S(i,j)}{\sum_{i,j}
             d_S(i,j)} \nonumber
\end{eqnarray}

We claim that any cut induces a so-called cut metric $d_S$. Also, we have proved that we could relax any metric cut to $l_1$ metric. Then we have,
\begin{eqnarray}
Sparsest Cut &=& \min_S \frac{E(S,\bar{S})}{|S||\bar{S}|}\nonumber\\
             &=& \min_{d_S} \frac {\sum_{i,j \in E} d_S(i,j)}{\sum_{i,j}
             d_S(i,j)} \nonumber\\
             &=& \min_{d_{l_1}} \frac {\sum_{i,j \in E} d_{l_1}(i,j)}{\sum_{i,j}
             d_{l_1}(i,j)}
\end{eqnarray}

Suppose there is a metric $d$ which embeds in $l_1$ with distortion $D$. According to the definition of distortion, 
\[
\frac {d(i,j)}{D} \nonumber \le d_{l_1}(i,j) \le d(i,j)
\]
$d(i,j) \ge d_{l_1}(i,j)$

$\sum_{i,j \in E} d(i,j) \ge \sum_{i,j \in E} d_{l_1}(i,j)$

$d(i,j) \le Dd(i,j)$

$\sum_{i,j} d(i,j) \le \sum_{i,j} d_{l_1}(i,j)$

So for any metric d
\[
\frac{\sum_{i,j \in E} d(i,j)}{\sum_{i,j} d(i,j)}\nonumber \ge 
\frac{1}{D}\nonumber \frac {\sum_{i,j \in E} d_{l_1}(i,j)}{\sum_{i,j} d_{l_1}(i,j)} \nonumber
\]

Since we have proved the equality (1), it is easy to see that for any metric $d$ which embeds in $l_1$ with distortion $D$, the optimal solution to minimize $\frac{\sum_{i,j \in E} d(i,j)}{\sum_{i,j} d(i,j)}\nonumber$ has approximation factor of D to the optimal solution to the sparsest-cut problem.




\section{L1 Embedding}

The aim in Sparsest Cut is to break the graph into two pieces such
that the pieces are large but the number edges crossing the cut is
small. In other words, we want small number edges connecting two
large pieces.

{\bf Problem:}

Given a graph $G = (V,E)$ and $w: E \rightarrow Z^+$. Determine cut
$(S,\bar{S})$ such that $\frac{E(S,\bar{S})}{|S||\bar{S}|}$ is
minimized where $E(S,\bar{S})$ is number edges crossing $S$ and
$\bar{S}$.

{\bf Motivation:}

Sparsest-Cut Problem is much better motivated as compared to Max-Cut
Problem. One motivation for Sparsest-Cut is Balanced-Cut (where we
have two pieces with small number of edges crossing and each piece
is constant fraction of $n$). Example: (1/3,2/3) balanced cuts,
$(\alpha,(1-\alpha))$ balanced cuts.

Balanced Cuts are widely studied because it is very fundamental to
Divide and Conquer paradigm. It is because if we can break our
problem instance into balanced pieces, then the algorithm would be
better in terms of running time and quality. From efficiency
standpoint if both the pieces are some constant fraction of total
size, it is guaranteed that we are making progress. From quality of
algorithm standpoint, if the  instance is broken evenly, then when
the solutions of each pieces are stitched together, the loss of
stitching the constant fraction pieces is not too much. As Divide
and Conquer is a basic paradigm in algorithm design, people care
about Balanced-Cut. It can be shown that if we can get a
approximation for Sparsest-Cut, then we can get a
pseudo-approximation to Balanced-Cut.

Another motivation for Sparsest-Cut is counting substructures in
graph. For this random walks on graphs is required. And to do random
walks on graph, it is important to show that we get good counts if
the random walks converge to a stationary distribution. Intuition is
to randomly walk on a graph for a long time, and after sometime get
lost in the graph (i.e. after getting lost it is equally likely to
be present anywhere in the graph). And getting lost is necessary
requirement for getting substructures (i.e. this means that you
count substructures in different parts of the graphs). Sparsest-Cut
is a motivation for random walks.

\subsection{Algorithm}

We discussed Leighton-Rao(LR) algorithm couple of weeks back which
is a combinatorial algorithm for uniform case. We will present
$O(log n)$ due to Linial-London-Rabinovich (LLR) and
Auman-Rabani(AR) which trivially handles non-uniform case.

As per LLR, we take $\frac{E(S,\bar{S})}{|S||\bar{S}|}$ as
optimization over a $metric$.

\begin{Definition}
\label{def:metric} If $S$ is a set of points and $R^{\ge 0}$ is a
set of positive reals then {\bf metric} is a function $d: (S * S)
\rightarrow R^{\ge 0}$ satisfying:

i. $Strict Metrics: $ $d(S_i,S_j) =  0 \Leftrightarrow S_i =S_j$

ii. $Symmetry: $ $d(S_i,S_j) = d(S_j,S_i)$

iii. $Triangle Inequality: $ $d(S_i,S_j) + d(S_i,S_j) \ge
d(S_i,S_k)$
\end{Definition}

\begin{Definition}
\label{def:norm} Suppose we have d-dimensional vector $\vec{v_i}$
i.e. $\vec{v_i} \in R^d$, then {\bf norm} (represented by $\| \|$)
can be defined as a function $ \| \|: R^d \rightarrow R^{\ge 0}$. We
can think of a norm as the length of the vector. It satisfies the
following:

i. There is only vector whose norm is zero and that is zero vector
i.e. $\|\vec{v}\| = 0 \Leftrightarrow \vec{v}=\vec{0}$

ii. Linearity of a norm under scalar multiplication (if we scale a
vector by some amount, its norm also scales by the same amount) i.e.
$ \|\lambda\vec{v}\| = |\lambda|\|\vec{v}\|$

iii. Triangle Inequality i.e. $ \|\vec{u} + \vec{v}\| \le
\|\vec{u}\| + \|\vec{v}\|$
\end{Definition}

{\bf Examples of norm: }
\begin{itemize}
\item
$L_1 = \sum |v_i|$
\item
$L_2 = \sqrt{\sum |v_i|^2}$
\item
$L_n = \sqrt[n]{\sum |v_i|^n}$ where $n$ is a real number $\ge$ 1
\item
$L_{\inf} = \max_i |v_i|$
\end{itemize}

Consider $R^d$. Take any centrally symmetric convex ($CSC$) body.
Centrally Symmetric means if $\vec{u} \in body$ then $-\vec{u} \in
body$. $Convex$ means if $\vec{u}, \vec{v} \in body$ then $\alpha
\vec{u} + (1-\alpha)\vec{v} \in body$.

Now it is to be noted that every norm corresponds one-to-one to
$CSC$. In other words, $CSC$ induces a norm. To reduce a $CSC$ to a
norm, for any vector, we will compute the $\lambda$ by which, the
vector need to be scaled to bring it to the boundary of the ball.
Since $CSC$ is convex and centrally symmetric, there is exactly one
point at which the vector can be scaled to the boundary. Then,
$\lambda$ is the norm of the vector. This norm satisfies $linearity$
trivially. The $triangle$ $inequality$ comes from the $convexity$ of
CSC. The following are $CSC$ corresponding to some norms.

\begin{figure}[htb!]
  \begin{center}
  \includegraphics[height=60mm]{csc.png}
  \end{center}
\end{figure}


Norms induce metrics. Suppose we have bunch of vectors in $R^d$,
then pick any $CSC$ body, which induces a norm. And define metric as
$d(\vec{u},\vec{v}) = \|\vec{u}-\vec{v}\|$.

$l_1$ is used to denote family of metrics. In particular $l_1^d$
denotes that there are $n$ points belonging to $d$-dimensional space
and the distance between them is induced by the $L_1$ norm.

\begin{Definition}
\label{def:embedding} We take a metric $S$, with distance function
$d$ on it and embed into another metric $T$ with distance function
$d'$. $Embedding$ $(E)$ can be formally defined as

\[ E:(S,d) \ce{<=>[\text{$D$}][\text{$D$}]}  (T,d')
\]

If $S_1$ and $S_2$ are the two points, then ratio of $d(S_1,S_2)$
and $d'(E(S_1),E(S_2))$ is captured by $distortion$
$(D_{(S_1,S_2)})$ for the edge $(S_1, S_2)$. $Distortion$ $(D)$ for
the entire $embedding$ is
$\max_{(i,j)}(dilation,\frac{1}{contraction})$. The notion of
$Average Distortion$ can be defined as $\frac{\sum_{i,j}
D_{(i,j)}}{number of pairs}$.
\end{Definition}

Consider the following graph. Take a subset S.  Distance between any
pair of vertices on the same side is 0, else if they are on opposite
side then 1 (irrespective of if there exists an edge between them).
We claim that any $cut$ induces a $metric$ called $cut$ $metric$
$d_S$ (note that distance defined satisfies non negativity, strict
metrics, symmetry and triangular inequality).

\begin{figure}[htb!]
  \begin{center}
  \includegraphics[height=60mm]{cut_metric.png}
  \end{center}
\end{figure}

Recall, we said $Sparsest Cut$ as optimization over a $metric$.
\begin{eqnarray}
Sparsest Cut &=& \min_S \frac{E(S,\bar{S})}{|S||\bar{S}|}\nonumber\\
             &=& \min_{d_S} \frac {\sum_{i,j \in E} d_S(i,j)}{\sum_{i,j}
             d_S(i,j)} \nonumber\\
             &=& \min_{d_{l_1}} \frac {\sum_{i,j \in E} d_{l_1}(i,j)}{\sum_{i,j}
             d_{l_1}(i,j)}
\end{eqnarray}

Note the equality (1). It says, we can relax from $cut$ $metrics$ to
$l_1$ $metrics$. It can be observed that any $cut$ $metric$ is an
$l_1$ $metric$.

One claim is, if we take at any $l_1$ metric, it is equal to the
combination of $cut$ $metrics$ for some collection of cuts. In other
words, $l_1$ is $the$ cut cone. So, if we take any metric induced by
$manhattan$ $distance$ $(L_1)$, then that can be represented by sum
of $cut$ $metrics$. Also, the reverse direction is true i.e. any
point that is represented as some combination of cuts, from that we
can get a $l_1$ embedding. (Note that the reverse direction is not
required for our analysis).

\[
d_{l_1} = \sum_S(\lambda_S d_S)
\]

This claim doesn't imply equality (1) as an $l_1$ metric we have is
sum of $cut$ $metrics$ (not one particular cut). We can break the
equality (1) as follows:

\[
\min_{d_{l_1}} \frac {\sum_{i,j \in E} d_{l_1}(i,j)}{\sum_{i,j}
d_{l_1}(i,j)} = \min_{S_1 \ldots S_k} \frac {\sum_{i,j \in E}
\sum_{S_t} \lambda_{S_t}d_{S_t}(i,j)}{\sum_{i,j} \sum_{S_t}
\lambda_{S_t}d_{S_t}(i,j)}
\]

Using $\frac{\sum_i a_i}{\sum_i b_i} \ge min_i \frac{a_i}{b_i}$ and
the above equality (just switch the order of summation and move
$\lambda$ outside), we can argue that the equality (1) holds.

Now,we know that we can optimize over $l_1$. If we remove $\lambda$
from both numerator and denominator, the ratio remains the same.
Hence, we can write LP for sparsest cut as follows:

\[
 \min_{d_{l_1}} \sum_{i,j \in E} d_{l_1}(i,j)
\]
\[
 \mbox{such that } \sum_{i,j} d_{l_1}(i,j) =1
\]

Now, from $l_1$, we can relax it to any metric. After relaxation the
value of objective function will reduce. But we can add additional
constraint (triangle inequality) to represent it as LP.

\[
 \min \sum_{i,j \in E} d_{l}(i,j)
\]
\[
 \mbox{such that } \sum_{i,j} d_{l}(i,j) =1
\]
 $\forall i, j, k d_l(i,j)+d_l(j,k) \ge d_l(i,k)$

We will solve this LP, get a metric and use the following theorem by
Bourgain.

{\bf Bourgain:}

Any metric \ce{->[\text{$O(log n)$}]} $l_1^{O(log^2n)}$ i.e. any
metric embeds in $l_1$ with $O(log n)$ distortion.

{\bf Roadmap to Solution:}

We wanted to minimize over some cuts. Then we said that minimizing
over cuts is same as minimizing over $l_1$. Then we relaxed it and
minimize over any metric. By Bourgain,  any metric embeds in $l_1$
with $O(log n)$ distortion. From $l_1$, we can induce the cut ($l_1$
and cut cone are same).

The roadmap for Arora, Rao, Vazirani is similar. We want to minimize
over some cuts. Then minimizing over cuts is same as minimizing over
$l_1$. Then we relaxed it and minimize over $l_2^2$ metric (this
metric is capturing SDP). By ARV, $l_2^2$ embeds in $l_1$ with
$O(\sqrt{log n)}$ average distortion. From $l_1$, we can induce the
cut.

\end{document}
