\documentclass[11pt]{article}
\usepackage{handout}
\usepackage{amsmath,amssymb,amsthm,epsfig}
\begin{document}
\input{courseMacros}

\newtheorem{Theorem}{Theorem}
\newtheorem{Corollary}{Corollary}
\newtheorem{Lemma}{Lemma}
\newtheorem{Claim}{Claim}
\theoremstyle{definition}
\newtheorem{Definition}{Definition}
\newtheorem{problem}{Problem}
\newenvironment{Proof}{\noindent{\bf Proof: }}{\qed}
%\theoremstyle{remark}
\newtheorem{Remark}{Remark}

\thispagestyle{empty}
\thispagestyle{empty}
\setlength{\parindent}{0pt}
\setlength{\parskip}{1.8ex}

\newcommand{\Alg}{\mbox{ALG}}
\newcommand{\Opt}{\mbox{OPT}}

\lecturehead{CS G399}{Algorithmic Power Tools I}{Fall 2007}{11
September 2007}{Eric Robinson}


%\newcommand{\p}{\mathcal{P}}
%\newcommand{\U}{\mathcal{U}}
%\newcommand{\coll}{\mathcal{S}}
%\newcommand{\xpy}{x + \lambda y}
%\newcommand{\Alg}{\mathcal{A}}
%\newcommand{\opt}{\mbox{OPT}}
%\newcommand{\eps}{\varepsilon}

\noindent {\bf Lecture Outline:}
\begin{itemize}
\item
Set Cover: Alternate Analysis
\item
Linear Programming: The Set Cover Problem
\item
Linear Programming: An Overview
\item
Linear Programming: The Forms
\item
Linear Programming: Vertices
\end{itemize}

In this lecture, an alternate analysis of the approximation to the set
cover problem is provided.  The set cover problem is then used to
demonstrate an application in linear programming.  An overview of
linear programming is then provided and the three forms for linear
programs are shown.  Finally, a vertex is defined and it is positted
that all linear programs must either have no vertices, have an
infinite solution, or have a solution on a vertex.

\section{Set Cover}

Set cover was described in the previous class.

\subsection{Alternate Analysis}

The previous analysis of the set cover problem focussed on the
relative cost for each set picked.  Another approach is provided here
which focuses instead on the relative cost for each element covered.
First, recall the goal of the set cover problem.  The input to the
problem is,
\begin{itemize}
\item A set $U$ of $n$ elements, and
\item A set $S$ of subsets of $U$.
\item A cost function for the subsets, $c: S \longrightarrow Q^+$
\end{itemize}
The goal is to pick the least costly set of subsets, $S'$, such that $\bigcup(s \in S') = U$, or the set of subsets covers all elements in $U$.

The greedy algorithm for this is simple, it says, at each step, to
pick the set with the smallest ratio of cost to newly covered
elements, or, where $U'$ is the set of elements covered currently by
the solution, find the set:
\[
\max_{s \in S} \frac{c(s)}{|s - U'|}
\]

The previous proof worked by iterating over the cost of sets chosen in
the greedy solution, here a different approach is shown iterating over
the cost of new elements covered as they are chosen.  The cost of an
element is the normalized cost of the first set to cover that element,
or, where $s'$ is the first set to cover an element $u$, and elements
$U'$ have been seen in the current solution:
\[
p(u)  = \frac{c(s')}{|s' - U'|}
\]
From this, the total cost of the greedy solution can be found to be:
\[
\sum_{u \in U} p(u)
\]

For simplicity's sake, the assumption that the elements are covered in
order will be made.  This is a safe assumption as renumbering the
elements in $U$ does not affect the solution.  With this, we can
evaluate the cost of $e_i$ in the greedy solution:
\begin{eqnarray*}
p(e_i) & \le & \frac{\Opt}{n-i+1} \\
       & =   & \Opt \times \frac{1}{n-i+1} \\
\end{eqnarray*}
This must be true because an optimal solution covers these with at
most a cost of $\Opt$, because of averaging over the remaining costs,
there exists a set that covers thes these with at most
$\frac{\Opt}{n-i+1}$.  From this, now the entire cost of the greedy
solution can be bounded:
\begin{eqnarray*}
\Alg  & =    & \sum_{u \in U} p(u) \\
     & \le  & \sum_{u \in U} \Opt \times \frac{1}{n-i+1} \\
     & =    & \Opt \times \sum_{u \in U} \frac{1}{n-i+1} \\
     & =    & \Opt \times H_n \approx \Opt \times ln(n) \\
\end{eqnarray*}
As can be seen, the bound of $H_n$ is found as it was with the
previous analysis.

\section{Linear Programming}

Here linear programming is introduced.  First, an example of the set
cover problem written as a linear program is provided.  After this,
linear programming itself is discussed, the different linear
programming forms are provided, and the definition of a vertex is
given.  Two excellent reference sources (among many others) are Michel
Goemans's lecture notes on linear programming~\cite{goemans:lp}, and
Howard Karloff's text on linear programming~\cite{karloff:lp}.

\subsection{The Set Cover Problem}

Here the set cover problem will be put into a linear programming
framework.  First, let's look at the result, our goal is to minimize
the cost of each set.  We will allow the variable x to represent the
inclusion of each set, where:
\[
x_s = \left\{ \begin{array}{ll} 1 & \mbox{if set is selected} \\
                                0 & \mbox{otherwise} \end{array} \right.
\]

From this, the linear programming set cover problem can be formulated.
Note that each element $u$ should be in some chosen subset, or $u \in
\bigcup_{s \in S} x_s = 1 \forall e \in U$:

\begin{center}
\begin{tabular}{lll}
min   & $\sum_{s \in S} x_s \times c(s)$        & \\
s.t.  & $\sum_{s \in S} (x_s \times s_u) \ge 1$ & $\forall u \in U$ \\
      & $x_s \in \{ 0, 1 \}$                    & $\forall s \in S$ \\
\end{tabular}
\end{center}

An important note here is that if $x_s \in \{ 0, 1 \}$ could be
replaced with $x_s \ge 0$ then the problem would become solvable in
polynomial time.  Linear programming is a problem in $P$, the same
cannot be said for integer linear programming, which is known to be
NP-complete.

In addition, one can envision taking an approximation by solving the
linear programming problem, then approximating an integer solution by
rounding the result in an intelligent way.  Consider the following
example:
\begin{eqnarray*}
U & = & \{ A, B, C \} \\ S & = & \{ \{ A, B \}, \{ B, C \}, \{ A, C \}
\} \\ c(s \in S) & = & 1 \\
\end{eqnarray*}
The optimal integer solution here is to choose any two of the sets,
for a total cost of $2$.  However, a non-integer solution could pick
half of each set, leading to each value being covered half-ways by two
sets.  The total cost of this solution is only $1.5$.  The fractional
solution is always a bound on the optimal solution for integer linear
programming.  If the gap between the fractional solution and an
algorithm can be determined, this is an adequate proof of the upper
bound of the gap between the integer solution and the algorihtm as
well.

\subsection{Overview}

Integer linear programming is an NPC problem, whereas linear
programming is in P.  In addition, the non-integer solution is always
guarenteed to be as good, if not better, than the integer solution.
If an integer solution is desired, it is often possible to find a
non-integer solution, and then through smart rounding approximate what
an integer solution may be.

Below lists the three main linear programming algorithms listed historically:
\begin{description}
\item[Simplex] - Invented by George Dantzig in 1947 to solve linear
programming problems, this technique was fast for most practical
applications.  However, it is non-polynomial in worst case scenarios,
and later examples were provided where simplex failed to perform
efficiently.
\item[Ellipsoid] - Introduced by Naum Z. Shor, Arkady Nemirovsky, and
David B. Yudin in 1972, and shown to be polynomial by Leonid
Khachiyan, this technique, while polynomial, is in practice typically
much slower than the simplex algorithm and was therefor rarely used.
It was useful for showing that general linear programming was in P
however.
\item[Interior Point] - Introduced and developed by Narendra Karmarkar
in 1984, this mehtod has the advantage of being polynomial like the
ellipsoid algorithm, but also fast in practice, like the simplex
algorithm.  For this reason it was used in practice for a long time.
Now, however, hybrids and other variations are used in many cases.
\end{description}

\subsection{The Forms}
There are three main forms that linear programs may take.  Those forms
are listed below, a short description of each is provided, and a
practical example of each is also given.  The forms are in order
according to restrictiveness.  Fortunately, any linear program (a
linear program in its general form), can be converted into the most
restrictive form (the slack form) easily.

\subsubsection{General Form}

All forms of linear inequalities are allowed in the general form.  
\begin{center}
\begin{tabular}{lll}
min   & $\sum_{i=1}^n c_i \times x_i$             & \\
s.t.  & $\sum_{i=1}^n a_{ij} \times x_i > b_j$    & $\forall j$ \\
      & $\sum_{i=1}^n a_{ij} \times x_i = b_j$    & $\forall j$ \\
      & $x_i \ge 0$                               & $\forall i$ \\
      & $x_i \not = 0$                            & $\forall i$ \\
\end{tabular}
\end{center}

Note that upper bound constraints (with a $\le$ sign) are not
required, as they can be transformed into lower bound constraints
simply by multiplying both sides of the constraint by $-1$.

\subsubsection{Standard Form}
The standard form does not allow for equality comparisons and requires
that all variables be nonnegative.
\begin{center}
\begin{tabular}{lll}
min   & $\sum_{i=1}^n c_i \times x_i$             & \\
s.t.  & $\sum_{i=1}^n a_{ij} \times x_i \ge b_j$    & $\forall j$ \\
      & $x_i \ge 0$                               & $\forall i$ \\
\end{tabular}
\end{center}

To get rid of equality comparisons, simply ensure that $\ge$ and $\le$
both hold, which will require negating the $\le$ comparison to make it
a $\ge$, as described in the general form.

\subsubsection{Slack Form}
The slack form only allows for equality comparisons for the
summations, and requires that all variables be nonnegative.  An
example is shown below.
\begin{center}
\begin{tabular}{lll}
min   & $\sum_{i=1}^n c_i \times x_i$             & \\
s.t.  & $\sum_{i=1}^n a_{ij} \times x_i = b_j$    & $\forall j$ \\
      & $x_i \ge 0$                               & $\forall i$ \\
\end{tabular}
\end{center}

To obtain an equality comparison, an extra variable can be introduced
(such variables are referred to as {\em slack variables}, hence the
name slack form).  For example
\begin{eqnarray*}
2x_1+3x_2  & \ge & 5
\end{eqnarray*}
is the same as
\begin{eqnarray*}
2x_1+3x_2 - t & = & 5 \\
t & \ge & 0 \\
\end{eqnarray*}

\subsection{Vertices}
A linear system always forms a polytope such that, for the linear system:
\begin{itemize}
\item The solution is unbounded (infinite)
\item The solution is infeasible (no polytope exists)
\item An optimal solution occurs on a vertex of the polytope
\end{itemize}

A linear system always produces a convex polytope.  A convex polytope
$P$, is a polytope where:
\begin{itemize}
\item A line between 2 points in the polytope remains in the polytope
\item If $x, y \in P$, $\alpha x + (1-\alpha)y \in P$ for $\alpha \in [ 0, 1 ]$
\end{itemize}

A vertex, $x$, of a convex polytope $P$, is a ``corner point'' in the
polytope and can be defined by any one of these three equivalent
statements, which we will prove in the next lecture.
\begin{itemize}
\item $\neg \exists y \not = 0 \mbox{ s.t. } x + y, x - y \in P$
\item $\neg \exists y, z \in P \mbox{ s.t. } x = \alpha y + (1-\alpha)z, \alpha \in ( 0, 1 )$
\item $\exists n$ linear inequalities of $P$ that are tight at $x$
\end{itemize} 

\bibliographystyle{alpha}
\bibliography{/home/rraj/tex/bibtex/refs}

\end{document}
