Академический Документы
Профессиональный Документы
Культура Документы
Linear Programming
Linear programming Simplex method LP duality
What is it?
n
Quintessential tool for optimal allocation of scarce resources, among a number of competing activities. (e.g., see ORF 307) Powerful and general problem-solving method. shortest path, max flow, min cost flow, generalized flow, multicommodity flow, MST, matching, 2-person zero sum games
Why significant?
n
Fast commercial solvers: CPLEX, OSL. Powerful modeling languages: AMPL, GAMS. Ranked among most important scientific advances of 20th century. Also a general tool for attacking NP-hard optimization problems. Dominates world of industry. ex: Delta claims saving $100 million per year using LP
Reference: The Allocation of Resources by Linear Programming, Scientific American, by Bob Bland.
Princeton University COS 226 Algorithms and Data Structures Spring 2004 Kevin Wayne
http://www.Princeton.EDU/~cos226
Applications
Agriculture. Diet problem. Computer science. Compiler register allocation, data mining. Electrical engineering. VLSI design, optimal clocking. Energy. Blending petroleum products. Economics. Equilibrium theory, two-person zero-sum games. Environment. Water quality management. Finance. Portfolio optimization. Logistics. Supply-chain management, Berlin airlift. Management. Hotel yield management. Marketing. Direct mail advertising. Manufacturing. Production line balancing, cutting stock. Medicine. Radioactive seed placement in cancer treatment. Operations research. Airline crew assignment, vehicle routing. Physics. Ground states of 3-D Ising spin glasses. Plasma physics. Optimal stellarator design. Telecommunication. Network design, Internet routing. Sports. Scheduling ACC basketball, handicapping horse races.
Production limited by scarce resources: corn, hops, barley malt. Recipes for ale and beer require different proportions of resources.
Beverage Ale Beer Quantity Corn (pounds) 5 15 480 Hops (ounces) 4 4 160 Malt (pounds) 35 20 1190 Profit ($) 13 23
Devote all resources to ale: 34 barrels of ale 7.5 barrels of ale, 29.5 barrels of beer 12 barrels of ale, 28 barrels of beer
Brewery Problem
Hops 4A + 4B 160 Ale s. t. Beer Profit Corn Hops Malt (0, 32) 160 0
(26, 14)
Beer
(0, 0)
5
Ale
(34, 0)
6
Extreme points
Pr of it
(26, 14)
Beer
(0, 0)
Beer
(0, 0)
7
Ale
(34, 0)
Ale
(34, 0)
8
Standard Form LP
"Standard form" LP.
n
Input: real numbers cj, bi, aij . Output: real numbers xj. n = # nonnegative variables, m = # constraints. Maximize linear objective function subject to linear inequalities.
n
(P) max s. t.
j =1
aij x j
j =1 n
cj x j
= bi 1 i m
(P) max c T x s. t.
Ax = b x 0
Standard form.
n
xj 0 1 j n
Linear. No x2, xy, arccos(x), etc. Programming. Planning (term predates computer programming).
s. t.
4A +
5A + 15B + SC 4B
= 480 + SH , SH = , SM 160 0
10
35A + 20B
A
9
, SC
+ SM = 1190
Geometry
Geometry.
n
Geometry
Extreme point property. If there exists an optimal solution to (P), then there exists one that is an extreme point.
n
Inequalities : halfplanes (2D), hyperplanes. Bounded feasible region: convex polygon (2D), (convex) polytope.
Convex: if a and b are feasible solutions, then so is (a +b) / 2. Extreme point: feasible solution x that can't be written as (a + b) / 2 for any two distinct feasible solutions a and b.
extreme point a a b b
Convex
Not convex
11
12
Simplex Algorithm
Simplex algorithm. (George Dantzig, 1947)
n
Developed shortly after WWII in response to logistical problems. Used for 1948 Berlin airlift.
never decrease objective function
Generic algorithm.
n
Solve m equations in m unknowns. If unique and feasible solution BFS. BFS corresponds to extreme point! Simplex only considers BFS.
{B, SH, SM } (0, 32) Basis {A, B, SM } (12, 28)
Start at some extreme point. Pivot from one extreme point to a neighboring one. Repeat until optimal.
Beer
35A + 20B
, SC
13
Ale
Z =
=
0 160 0
= 480 + SM , SM = 1190
35A + 20B
, SC
Z =
=
0 160 0
= 480 + SM , SM = 1190
35A + 20B
, SC
Each unit increase in B increases objective value by $23. Pivoting on column 1 also OK.
A A A A
, B
A + B +
23 15 1 15 4 15 4 3
SC SC SC SC +
,
Z = - 736
= 32 32 550 0 =
SH SH
SC
+ SM , SM
Preserves feasibility by ensuring RHS 0. Minimum ratio rule: min { 480/15, 160/4, 1190/20 }.
17
16
A A A A
, B
A + B +
23 15 1 15 4 15 4 3
SC SC SC SC +
,
Z = - 736
= 32 32 550 0 =
SH SH
SC
+ SM , SM
Any feasible solution satisfies system of equations in tableaux. in particular: Z = 800 SC 2 SH Thus, optimal objective value Z* 800 since SC, SH 0. Current BFS has value 800 optimal.
B + A A , B
,
1 10 1 10 25 6
SC
SC
SC
SC
SC
- 2 SH + + ,
1 8 3 8 85 8
Z = - 800
= = = 28 12 110 0
SH
SH + SM
, SM
SH
SH
max Z subject to -
B + A A , B
18
1 10 1 10 25 6
SC
SC
SC
SC
SC
- 2 SH + + ,
1 8 3 8 85 8
Z = - 800
= = = 28 12 110 0
SH
SH + SM
, SM
SH
SH
19
No polynomial pivot rule known. Most pivot rules known to be exponential in worst-case.
A* = 12 B* = 28 OPT = 800
Issues. Which neighboring extreme point? Degeneracy. New basis, same extreme point.
n
Cycling. Get stuck by cycling through different bases that all correspond to same extreme point.
n
C, H, M = unit price for corn, hops, malt. Brewer won't agree to sell resources if 5C + 4H + 35M < 13.
(D) min 480C s. t. 5C 15C + 160H + 1190M + + , 4H + 4H + 35M 13 20M 23
Doesn't occur in the wild. Bland's least index rule finite # of pivots.
C
20
C* = 1 H* = 2 M* = 0 OPT = 800
21
LP Duality
Primal and dual LPs. Given real numbers aij, bi, cj, find real numbers xi, yj that optimize (P) and (D).
n
Q. How much should brewer be willing to pay (marginal price) for additional supplies of scarce resources? A. corn $1, hops $2, malt $0. Q. New product "light beer" is proposed. It requires 2 corn, 5 hops, 24 malt. How much profit must be obtained from light beer to justify diverting resources from production of beer and ale? A. Breakeven: 2 ($1) + 5 ($2) + 24 (0$) = $12 / barrel.
(P) max s. t.
j =1
aij x j bi 1 i m
xj 0 1 j n
j =1 n
cj x j
(D) min s. t.
i =1
aij yi c j 1 j n
yi 0 1 i m
i =1 m
bi yi
n
Duality Theorem (Gale-Kuhn-Tucker 1951, Dantzig-von Neumann 1947). If (P) and (D) have feasible solutions, then max = min.
n
Simplex solves primal and dual simultaneously. Top row of final simplex tableaux provides optimal dual solution!
22
23
History
1939. Production, planning. (Kantorovich, USSR)
n
History
1939. Production, planning. (Kantorovich) 1947. Simplex algorithm. (Dantzig)
USSR
USA
Kantorovich awarded 1975 Nobel prize in Economics for contributions to the theory of optimum allocation of resources. Staple in MBA curriculum. Used by most large companies and other profit maximizers.
25 26
History
1939. Production, planning. (Kantorovich) 1947. Simplex algorithm. (Dantzig) 1950. Applications in many fields.
n
History
1939. 1947. 1950. 1979.
n n
Military logistics. Operations research. Control theory. Filter design. Structural optimization.
Production, planning. (Kantorovich) Simplex algorithm. (Dantzig) Applications in many fields. Ellipsoid algorithm. (Khachian)
Geometric divide-and-conquer. Solvable in polynomial time: O(n4 L) bit operations. n = # variables L = # bits in input Theoretical tour de force, not remotely practical.
27
28
History
1939. 1947. 1950. 1979. 1984.
n n
History
1939. 1947. 1950. 1979. 1984. 1990.
n n
Production, planning. (Kantorovich) Simplex algorithm. (Dantzig) Applications in many fields. Ellipsoid algorithm. (Khachian) Projective scaling algorithm. (Karmarkar)
Production, planning. (Kantorovich) Simplex algorithm. (Dantzig) Applications in many fields. Ellipsoid algorithm. (Khachian) Projective scaling algorithm. (Karmarkar) Interior point methods.
29
30
History
1939. 1947. 1950. 1979. 1984. 1990.
n n
History
1939. 1947. 1950. 1979. 1984. 1990. Production, planning. (Kantorovich) Simplex algorithm. (Dantzig) Applications in many fields. Ellipsoid algorithm. (Khachian) Projective scaling algorithm. (Karmarkar) Interior point methods.
Production, planning. (Kantorovich) Simplex algorithm. (Dantzig) Applications in many fields. Ellipsoid algorithm. (Khachian) Projective scaling algorithm. (Karmarkar) Interior point methods.
Interior point faster when polyhedron smooth like disco ball. Simplex faster when polyhedron spiky like quartz crystal.
Current research.
n
Approximation algorithms. Software for large scale optimization. Interior point variants.
31
32
Perspective
LP is near the deep waters of NP-completeness.
n
Shortest path. Maximum flow. Min cost flow. Generalized multicommodity flow. Linear programming. Semidefinite programming. ... TSP (or any NP-complete problem)
intractable (conjectured) tractable
An unsuspecting MBA student transitions from tractable LP to intractable ILP in a single mouse click.
33 34