COS 226 Lecture 23: Linear programming %ps /lecture 23 def "surrounded by notational and terminological thickets" "both of these defenses are lovingly cultivated by a coterie of stern acolytes who have devoted themselves to the field..." 2 Numerical Recipes (Press, et. al) ----- Linear programming Mathematical formulation of optimization problems next (last?) step towards UNIVERSAL PROBLEM-SOLVING MODEL for combinatorial optimization problems Like mincost flow, LP is important for two primary reasons it is a GENERAL PROBLEM-SOLVING MODEL solves (through reduction) numerous practical problems more general than mincost flow it is TRACTABLE and PRACTICAL we know fast algorithms that solve LP problems more complicated than mincost-flow algorithms 1 SIMPLEX method generalizes network simplex algorithm ----- Reduction 1 SOLUTION (algorithm designer's goal) an EFFICIENT algorithm that can ALWAYS find the answer Ex: network simplex algorithms solve mincost flow 1 REDUCTION (algorithm designer's main tool) solve a problem by converting it to another Ex: Bipartite matching reduces to maxflow Reduction, in practice, may provide quick solution is better than brute force is better than investing years of research to understand details of a problem gives insight into difficulty Vast array of tractable problems reduce to LP Simplex algorithm solves practical instances efficiently ----- Linear programming Maximize OBJECTIVE FUNCTION subject to CONSTRAINTS Ex: maximize x+y subject to the constraints -x + y <= 5 x + 4y <= 45 2x + y <= 27 3x - 4y <= 24 x, y >= 0 Vast number of applications 1 "LINEAR": no x^2, xy, etc. 1 "PROGRAMMING": formulate the equations (reduce given problem to linear programming) /lines lines 2 add def ----- LP example: DIET problem Given table of food nutrients and costs . bacon eggs cereal milk MDR . protein 3 2 0 0 1 . vitamin A 5 1 1 0 3 . iron 2 5 0 1 4 . $ 2 1 1 2 Find the least expensive breakfast that provides the minimum daily requirements 2 LP formulation minimize 2b+e+c+2m subject to the constraints 3b + 2e >= 1 5b + e + c >= 3 2b + 5e + m >= 4 b, e, c, m >= 0 /lines lines 1 add def ----- LP example: mincost flow reduction maximize c3 subject to the constraints e01 <= 2 e02 <= 3 e13 <= 3 e14 <= 1 e23 <= 1 e24 <= 1 e35 <= 2 e45 <= 3 e50 = e01 + e02 e01 = e13 + e14 e02 = e23 + e24 e35 = e13 + e23 e45 = e14 + e24 e50 = e35 + e45 c1 = 3*e01 + e02 + e13 + e14 c2 = 4*e23 + 2*e24 + 2*e35 + e45 c3 = 9e50 - c1 - c2 all >= 0 %% 0 %ps 2 2 scale 180 -80 translate %include figs/01intro/ps/ds.ps %include figs/21flow/ps/LPmincost.ps %%% /lines lines 3 sub def ----- Geometric interpretation (two variables) Inequalities: halfplanes intersecting halfplanes: convex polygon maximize x+y subject to the constraints -x + y <= 5 x + 4y <= 45 2x + y <= 27 3x - 4y <= 24 x, y >= 0 %% 0 %ps 1.3 1.3 scale 220 0 translate %include figs/19flow/ps/lp2dex.ps %%% 1 THM Objective function is maximized at a simplex vertex 1 INFEASIBLE: add x > 13 1 REDUNDANT: add x < 13 1 UNBOUNDED: delete 2nd and 3rd inequalities ----- Geometric interpretation (three variables) Inequalities: halfspaces intersecting halfspaces: convex polytope %% 9 %ps 1.5 1.5 scale 0 0 translate %include figs/19flow/ps/lp3dex.ps %%% Higher dimensions: multifacted convex polytope (SIMPLEX) N inequalities in k vars could have (k-1)^N simplex vertices /lines lines 2 add def ----- Standard form of LPs maximize x+y+z subject to the constraints -x + y <= 5 x + 4y <= 45 2x + y <= 27 3x - 4y <= 24 z <= 4 x, y, z >= 0 Add SLACK variables to convert inequalities to equations constrain all variables to be nonnegative . -x + y + a = 5 . x + 4y + b = 45 . 2x + y + c = 27 . 3x - 4y + d = 24 . z + e = 4 M SIMULTANEOUS EQUATIONS with excess variables /lines lines 3 add def ----- pivoting choose M of the variables as a BASIS solve by setting nonbasis variables to 0 corresponds to a simplex vertex -- . -1 -1 -1 0 0 0 0 0 0 . -1 1 0 1 0 0 0 0 5 . 1 4 0 0 1 0 0 0 45 . 2 1 0 0 0 1 0 0 27 . 3 -4 0 0 0 0 1 0 24 . 0 0 1 0 0 0 0 1 4 --- PIVOT STEP: add one variable to the basis, drop another add appropriate multiple of row to every other row -- . 0 -7 -3 0 0 0 1 0 24 . 0 -1 0 1 0 0 1 0 39 . 0 16 0 0 1 0 -1 0 111 . 0 8 0 0 0 1 -2 0 33 . 1 -4 0 0 0 0 1 0 24 . 0 0 1 0 0 0 0 1 4 --- corresponds to moving to an adjacent simplex vertex /lines lines 5 sub def ----- Simplex algorithm 1 THM: Any LP problem is either unbounded or the objective function is maximized at a simplex vertex. The simplex algorithm finds such a vertex. Generic simplex algorithm start at some simplex vertex increase objective function by pivoting to move to an adjacent vertex Issues which adjacent vertex? avoid degenerate pivots (new basis, same vertex) avoid cycles (get stuck on a set of vertices)? Code not much more complicated than Gaussian elimination [admittedly, many details omitted!] ----- Approaches to linear programming 2 SIMPLEX algorithm classical approach (1940s) generalizes network simplex method for mincost flow generalizes Gaussian elimination exponential in worst case fast in practice 2 Ellipsoid methods new (1980s) algorithmic approach geometric divide-and-conquer polynomial time (!) 2 Interior-point methods make poly-time methods faster than simplex in practice ----- NP-completeness (quick review) 1 P is the class of decision problems that are easy to solve solvable on a deterministic machine in polynomial time 1 NP is the class of problems where solutions are easy to check solvable on a nondeterministic machine in polynomial time 3 An NP-hard problem is one that every problem in NP easily reduces to 3 NP-complete problems are both in NP and NP-hard The main question: Is P = NP? %ps 11 1 60 410 redbox no Universal Problem-Solving Model exists unless P = NP Current practice for NP-hard problems: assume that "trying all possibilities" is the best we can do hope that real problems are not worst-case instances work on easier versions ----- Perspective LP is near the deep waters of NP-hardness LP is in P (known for less than 20 years) INTEGER programming is NP-complete Cross-currents among fields of research 1960s Operations research 1970s Design and analysis of algorithms 1980s Geometric algorithms Specialized algorithms may provide elegant optimal solution for all instances worth pursuing General models (like mincost flow and LP) may solve specific practical instances quickly worth pursuing 3 Universal Problem-Solving Model?