Lecture 9 Nonlinear Programming Models

Topics Convex sets and convex programming First-order optimality conditions Examples Problem classes

General NLP
Minimize f(x) s.t. gi(x) (, , =) bi, i = 1,,m x = (x1,,xn)T is the n-dimensional vector of decision variables f (x) is the objective function gi(x) are the constraint functions bi are fixed known constants

Convex Sets
Definition: A set S n is convex if every point on the line segment connecting any two points x1, x2 S is also in S. Mathematically, this is equivalent to x0 = x1 + (1)x2 S for all such 0 1.

x1

x2

x1

x1 x2 x2

(Nonconvex) Feasible Region

S = {(x1, x2) : (0.5x1 0.6)x2 1 2(x1)2 + 3(x2)2 27; x1, x2 0}

x1

Convex Sets and Optimization

Let S = { x n : gi(x) bi, i = 1,,m } Fact: If gi(x) is a convex function for each i = 1,,m then S is a convex set. Convex Programming Theorem: Let x n and let f (x) be a convex function defined over a convex constraint set S. If a finite solution exists to the problem Minimize { f (x) : x S } then all local optima are global optima. If f (x) is strictly convex, the optimum is unique.

Convex Programming
Min f (x1,,xn) s.t. gi(x1,,xn) bi i = 1,,m x1 0,,xn 0 is a convex program if f is convex and each gi is convex. Max f (x1,,xn) s.t. gi(x1,,xn) bi i = 1,,m x1 0,,xn 0 is a convex program if f is concave and each gi is convex.

Linearly Constrained Convex Function with Unique Global Maximum

x2

Maximize f (x) = (x1 2)2 + (x2 2)2 subject to 3x1 2x2 6 x1 + x2 x1 + x2 2x1 3x2 3 7 4

5 4 3 2 1

x1

(Nonconvex) Optimization Problem

First-Order Optimality Conditions

Minimize { f (x) : gi(x) bi, i = 1,,m } Lagrangian: L(x, ) = f (x) + i (gi (x) bi )
m i=1

Optimality conditions Stationarity: L(x, ) = f (x) + igi (x) = 0 Complementarity: igi(x) = 0, i = 1,,m Feasibility: gi(x) bi, i = 1,,m Nonnegativity: i 0, i = 1,,m
i=1 m

Importance of Convex Programs

Commercial optimization software cannot guarantee that a solution is globally optimal to a nonconvex program. NLP algorithms try to find a point where the gradient of the Lagrangian function is zero a stationary point and complementary slackness holds. Given we want L(x,) = f(x) + (g(x) b) L(x,) = f(x) + g(x) = 0 (g(x) b) = 0 ( g(x) b 0, 0
For a convex program, all local solutions are global optima.

Example: Cylinder Design

We want to build a cylinder (with a top and a bottom) of maximum volume such that its surface area is no more than s units. Max V(r,h) = r2h s.t. 2r2 + 2rh = s r 0, h 0
There are a number of ways to approach this problem. One way is to solve the surface area constraint for h and substitute the result into the objective function.
r

Solution by Substitution
s 2r 2 h= 2r Volume = V = r2 [ s 2r 2 rs r 3 ] = 2 2r

s 1/2 dV s 1/2 s = 0 r=( h = 2r r = 2( ) ) dr 6 6 s 3/2 V = r 2h = 2 ( ) 6 s 1/2 ) 6 s 1/2 ) 6

r=(

h = 2(

Is this a global optimal solution?

Test for Convexity

rs r 3 2 dV(r) s 3r 2 dr = 2 d2V(r ) dr 2 = 6r

V(r ) =

d2V 0 for all r 0 dr 2 Thus V(r ) is concave on r 0 so the solution is a global maximum.

Direct Current Network

10 20

I2 100v I1 I6 I3 10

I4 I5 20

I7

Problem: Determine the current flows I1, I2,,I7 so that the total content is minimized Content: G(I) =

v(i)di for I 0 and G(I) = I v(i)di for I < 0

Solution Approach
Electrical Engineering: Use Kirchoffs laws to find currents when power source is given. Operations Research: Optimize performance measure in network taking flow balance into account. Linear resistor: Voltage, v(I ) = IR Content function, G(I ) = I R/2 Battery: Voltage, v(I ) = E Content function, G(I ) = EI
2

Network Flow Model

Network diagram:
2 5 I2
2

10 I 4

-100 I1

5 I3

10 I5

0 I6

0 I7

Minimize Z = 100I1 + 5I22 + 5I32 + 10I42 + 10I52 subject to I1 I2 = 0, I2 I3 I4 = 0, I5 I6 = 0, I5 + I7 = 0, I3 + I6 I7 = 0, I1 I6 = 0 Solution: I1 = I2 = 50/9, I3 = 40/9, I4 = I5 = 10/9, I6 = 50/9, I7 = 10/9

NLP Problem Classes

Constrained vs. unconstrained Convex programming problem Quadratic programming problem
f (x) = a + cTx + xTQx, Q 0

Separable programming problem

f (x) = j=1,n fj(xj)

Geometric programming problem

g(x) = t=1,T ctPt(x), Pt(x) = (x1at1) . . . (xnatn), xj > 0

Equality constrained problems

What You Should Know About Nonlinear Programming

How to identify a convex program. How to write out the first-order optimality conditions. The difference between a local and global solution. How to classify problems.

What You Should Know About Nonlinear Programming

How to identify a convex program. How to write out the first-order optimality conditions. The difference between a local and global solution. How to classify problems.

10