College of Computer & Information Science Fall 2007 Northeastern University 22 November 2007 CS G399: Algorithmic Power Tools I Scribe: Tao Jin

Lecture Outline:

• Review of Sparsest-Cut problem

• Frechet’s-Embedding

• Bourgain’s Theorem

– Construction of Bourgain’s Embedding – Proof of Bourgain’s Theorem

In this lecture we will first review the sparsest cut problem and its relation with metric embeddings. Then we will show that for any metric space, there is an isometric Frechet’s embedding. Last, we will prove the Bourgain’s Theorem with the construction of Frechet’s Embedding, which guarantee us a O(log n) approximation factor to the optimal solution of the sparsest-cut problem.

1 Sparsest-Cut Review

In the last lecture we’ve defined the sparsest-cut problem. Recall that:

Sparsest-Cut Problem:

Given a graph G = (V, E) and w : E → Z^{+}. Determine cut (S, S̄) such that \frac{E(S,S̄)}{|S||S̄|} is minimized where E(S, S̄) is number of edges crossing S and S̄.

For any cut (S, S̄), we defined the distance between two vertices in the graph as follow,

d_{S}(i, j) = \{ \begin{matrix}1 \\ 0 \end{matrix} \begin{matrix}(i, j) is cross cut edge \\ otherwise \end{matrix}

We claim that any cut induces a so-called cut metric d_{S}. Also, we have proved that we could relax any metric cut to l_{1} metric. Then we have,

SparsestCut = min_{S} \frac{E(S, S̄)}{|S||S̄|}
= min_{d_{S}} \frac{∑_{i,j∈E} d_{S}(i, j)}{∑_{i,j} d_{S}(i, j)}
= min_{d_{l_{1}}} \frac{∑_{i,j∈E} d_{l_{1}}(i, j)}{∑_{i,j} d_{l_{1}}(i, j)}  (1)

Suppose there is a metric d which embeds in l_{1} with distortion D. According to the definition of distortion,

\frac{d(i, j)}{D} ≤ d_{l_{1}}(i, j) ≤ d(i, j)

Then we have,

d(i, j) ≥ d_{l_{1}}(i, j) ⇒ ∑_{i,j∈E} d(i, j) ≥ ∑_{i,j∈E} d_{l_{1}}(i, j)  (2)
d(i, j) ≤ D ∗ d(i, j) ⇒ ∑_{i,j} d(i, j) ≤ D ∗ ∑_{i,j} d_{l_{1}}(i, j)  (3)

So by inequalities 2 and 3, for any metric d

\frac{∑_{i,j∈E} d(i, j)}{∑_{i,j} d(i, j)} ≥ \frac{1}{D} \frac{∑_{i,j∈E} d_{l_{1}}(i, j)}{∑_{i,j} d_{l_{1}}(i, j)}

Since we have proved the equality (1), it is easy to see that for any metric d which embeds in l_{1} with distortion D, the optimal solution to minimize \frac{∑_{i,j∈E} d(i,j)}{∑_{i,j} d(i,j)} has approximation factor of D to the optimal solution of the sparsest-cut problem.

By Bourgain’s Theorem, we can choose d to have dist(d) = O(log n), thus we could obtain a O(log n)-approximate solution by solving the linear program below,

min ∑_{i,j∈E} d_{l}(i, j)

such that ∑_{i,j} d_{l}(i, j) = 1
∀i, j, kd_{l}(i, j) + d_{l}(j, k) ≥ d_{l}(i, k)

2 Frechet’s-Embedding

Frechet’s Theorem: Let(V,d) be an arbitrary n-point metric space. Then there is an isometric embedding Φ:V → l_{∞}^{n}.

Proof: Let’s consider the following Frechet’s embedding. Given a metric (V, d), |V | = n. Define Φ : v_{i} →< d(v_{i}, v_{1}), d(v_{i}, v_{2}), ..., d(v_{i}, v_{n}) >.

By triangle inequality,

∥Φ(v_{i}) − Φ(v_{j})∥_{∞} = max_{t} |d(v_{i}, v_{t}) − d(v_{j}, v_{t})|
≤ d(v_{i}, v_{j})

so the embedding is nonexpanding. On the other hand,

for all u ∈ V:

∥Φ(v_{i}) − Φ(v_{j})∥_{∞} ≥ |Φ_{j}(v_{i}) − Φ_{j}(v_{j})| = d(v_{i}, v_{j})

where Φ_{j}(v_{i}) denotes the j-th coordinate of v in the new metric space. By the above two inequalities, it is easy to see that the embedding is isometric.

Generally, it is rare to find isometric embeddings between two spaces of interst, so we relax and allow the embedding to alter the distances. By Bourgain’s Theorem, any metric embeds in l_{1} with O(log n) distortion. In the rest of the lecture, we will prove Bourgain’s Theorem with Frechet’s embedding, which is pretty similar as what we discussed above.

3 Bourgain’s Theorem

Theorem 1. Let (V, d) be a metric space, and let n denote |V |. There exists an embedding ϕ from (V, d) into l_{1}^{h}, where h = O(log n), such that the distortion is O(log n). Moreover, ϕ can be computed in polynomial time by a randomized algorithm.

3.1 Construction of Bourgain’s Embedding

Given a metric (V, d) on n points, we will randomly pick k sets of nodes: S_{1}, ..., S_{k} ⊆ V and define the embedding as

Φ : v_{i} →< d(v_{i}, S_{1}), d(v_{i}, S_{2}), ..., d(v_{i}, S_{k}) >,

where d(v_{i}, S) = min_{s∈S} d(v_{i}, s). Let k = O(log^{2} n). For simplicity, we assume that n is power of two.

How to pick the subsets:

pick log n random sets of size 1 pick log n random sets of size 2 pick log n random sets of size 4 ... pick log n random sets of size n/2

for each j = 0, 1, 2, ..., log n − 1, we randomly and independently choose log n subsets of V, each of

cardinality n/2^{j}, i.e. each of the ( \begin{matrix}n \\ n/2^{j} \end{matrix} ) substes of V of cardinality n/2^{j} is equally likely to be

chosen.

3.2 Proof of Bourgain’s Theorem

Proof: Based on the embedding construction above, we claim that

Ω(log n) ∗ d(v_{i}, v_{j}) ≤ ∥ϕ(v_{i}), ϕ(v_{j})∥_{1} ≤ k ∗ d(v_{i}, v_{j})  (4)

Lemma 1. For any S ⊆ V, |d(v_{i}, S) − d(v_{j}, S)| ≤ d(v_{i}, v_{j}) .

As shown in Figure 1, for any S ⊆ V, assume d(v_{i}, S) = d(v_{i}, v_{m}), d(v_{j}, S) = d(v_{j}, v_{n}), where n and m could be the same,

|d(v_{i}, S) − d(v_{j}, S)| = |d(v_{i}, v_{m}) − d(v_{j}, v_{n})|
≤ |d(v_{i}, v_{m}) − d(v_{j}, v_{m})|  (5)
≤ d(v_{i}, v_{j})  (6)

It is clear to see that the inequality 5 holds due to the definition of the d(v, S), and the inequality 6 holds due to the triangle inequality.

Figure 1: |d(v_{i}, S) − d(v_{j}, S)| ≤ d(v_{i}, v_{j})

Upper Bound:

∥ϕ(v_{i}), ϕ(v_{j})∥_{1} = ∑_{t=1.. log^{2} n} |d(v_{i}, S_{t}) − d(v_{j}, S_{t})|
≤ log^{2} n ∗ d(v_{i}, v_{j})  (7)

Since we have proved the Lemma 1, it is clear to see that the inequality 7 holds. Then we have proved the upper bound for our claim of inequality 4.

Lower Bound:

Figure 2: d(u, A) = d(v, A)

First we consider a simple example shown in Figure 2. Let A_{ij} denotes the jth set selected with cardinality 2^{i−1}. As shown in Figure 2, if the distance between u and set A_{ij} is the same as the one between v and A_{ij}, that means |d(u, A_{ij}) − d(v, A_{ij})| = 0, which doesn’t contributes to new metric distance ∥ϕ(u), ϕ(v)∥_{1}. In order to fine the lower bound, we are more interested in the case that how the selected sets could contribute to ∥ϕ(u), ϕ(v)∥_{1}.

Definition 1. B(x, r) = {y ∈ V |d(x, y) ≤ r}. The ball includes all nodes the distance between which and x is no greater than r.

Definition 2. for fixed u and v, R_{t} = min{r: |B(u, r)| ≥ 2^{t}&|B(v, r)| ≥ 2^{t}}. Further, R_{t} is defined if and only if R_{t} ≤ d(u, v)/2, i.e. we don’t consider the case that B(u, r) and B(v, r) intersects, but they could touch.

Now we consider the case shown in Figure 3. The set A of cardinality 2^{i} intersects the ball B(u, R_{i−1}) and B(v, R_{i}), then it is easy to see that d(u, A) ≤ R_{i−1} and d(v, A) ≥ R_{i}. Thus, |d(u, A) − d(v, A)| ≥ R_{i} − R_{i−1}.

Figure 3: good ball example

Claim 1. When we select set A of cardinality 2^{i−1}, for any i, the probability that we chose a set that intersects B(u, R_{i−1}) and disjoints from B(v, R_{i}) is constant.

Proof: By Definition 2, |B(u, R_{i−1})| ≤ 2^{i} and |B(v, R_{i})| ≥ 2^{i}. So the probability of choosing a set A which intersects |B(u, R_{i−1})| but misses |B(v, R_{i})| is

P r[A ∩ B(u, R_{i−1}) ≠ ∅&A ∩ B(v, R_{i}) = ∅] = P r[A ∩ B(u, R_{i−1}) ≠ ∅] ∗ P r[A ∩ B(v, R_{i}) = ∅]
≥ (1 − (1 − \frac{1}{2^{i}})^{2^{i−1}}) ∗ (1 − \frac{1}{2^{i}})^{2^{i}}
≥ (1 − (1 − \frac{1}{2^{i}})^{2^{i}/2}) ∗ (1 − \frac{1}{2^{i}})^{2^{i}}
≥ (1 − e^{−1/2}) ∗ \frac{1}{4}

Therefore, for a set of subsets A_{ij} of cardinality 2^{i−1},

E[ ∑_{j=1.. log n} |d(u, A_{ij}) − d(v, A_{ij})|] ≥ log nΩ(1)(R_{i} − R_{i−1})
≥ Ω(log n)(R_{i} − R_{i−1})

E[∥ϕ(u), ϕ(v)∥_{1}] ≥ E[ ∑_{R_{i} is defined} ∑_{j=1.. log n} |d(u, A_{ij}) − d(v, A_{ij})|]
≥ ∑_{R_{i} is defined} Ω(log n)(R_{i} − R_{i} − 1)  (8)
≥ Ω(log n) ∗ (R_{m} − R_{0})  (9)
≥ Ω(log n)d(u, v)  (10)

It is easy to see that the inequality 8 is of telescope style, and the inequality 9 is the result of reducing 8. By the definition of R_{m}, we know that the inequality 10 holds for certain.

Now we have proved Ω(log n) ∗ d(v_{i}, v_{j}) ≤ ∥ϕ(v_{i}), ϕ(v_{j})∥_{1} ≤ k ∗ d(v_{i}, v_{j}). Then according to the Linearity of a norm under scalar multiplication, i.e.∥λv⃗∥ = |λ|∥v⃗∥, we have

\frac{d(v_{i}, v_{j})}{Ω(log n)} ≤ ∥ϕ′(v_{i}), ϕ′(v_{j})∥_{1} ≤ d(v_{i}, v_{j}),

where ϕ′(v_{i}) = ϕ(v_{i})/log^{2}n. Thus, this embedding has distortion of O(logn) on O(log^{2}n) dimension.