Efficient Algorithms Using The Multiplicative Weights Update Method (thesis)

Abstract:

Algorithms based on convex optimization, especially linear and semidefinite programming, are ubiquitous in Computer Science. While there are polynomial time algorithms known to solve such problems, quite often the running time of these algorithms is very high. Designing simpler and more efficient algorithms is important for practical impact.

In this thesis, we explore applications of the Multiplicative Weights method in the design of efficient algorithms for various optimization problems. This method, which was repeatedly discovered in quite diverse fields, is an algorithmic technique which maintains a distribution on a certain set of interest, and updates it iteratively by multiplying the probability mass of elements by suitably chosen factors based on feedback obtained by running another algorithm on the distribution.

We present a single meta-algorithm which unifies all known applications of this method in a common framework. Next, we generalize the method to the setting of symmetric matrices rather than real numbers. We derive the following applications of the resulting Matrix Multiplicative Weights algorithm:

1. The first truly general, combinatorial, primal-dual method for algorithms for semidefinite programming. Using these techniques, we obtain significantly faster algorithms for obtaining O(sqrt(log n)) approximations to various graph partitioning problems, such as Sparsest Cut, Balanced Separator in both directed and undirected weighted graphs, and constraint satisfaction problems such as Min UnCut and Min 2CNF Deletion.

2. An O*(n^3) time derandomization of the Alon-Roichman construction of expanders using Cayley graphs. The algorithm yields a set of O(log n) elements which generates an expanding Cayley graph in any group of n elements.

3. An O*(n^3) time deterministic O(log n) approximation algorithm for the quantum hypergraph covering problem.

4. An alternative proof of a result of Aaronson that the gamma-fat-shattering dimension of quantum states on n qubits is O(n/gamma^2).

Using our framework for the classical Multiplicative Weights Update method, we derive the following algorithmic applications:

1. Fast algorithms for approximately solving several families of semidefinite programs which beat interior point methods. Our algorithms rely on eigenvector computations, which are very efficient in practice compared to the Cholesky decompositions needed by interior point methods. We also give a matrix sparsification algorithm to speed up the eigenvector computation using the Lanczos iteration.

2. O(sqrt(log n)) approximation to the Sparsest Cut and the Balanced Separator problems in undirected weighted graphs in O*(n^2) time by embedding expander flows in the graph. This improves upon the previous O*(n^{4.5}) time algorithm of Arora, Rao, and Vazirani, which was based on semidefinite programming.