The Pontryagins Maximum Principle An alternative technique to dynamic programming, which is particularly well suited for serial structures,

makes use of the Maximum Principle. The maximum principle yields the same solutions as dynamic programming, but it avoids the problem of dimensionality. The application of the maximum principle to the optimization of discrete automatic control systems was first announced by Rozonoer in 1959. Later, Chang developed the basic algorithm of the discrete version of the principle with heuristic arguments based on the principle of optimality (Chang, S.L; 1961). Furthermore, Katz recognized these arguments and demonstrated that the principle was at least a necessary condition for the best control action of quite arbitrary nonlinear systems (Katz, S.; 1962a). He also applied the method to the optimization of a cascade of stirred tank reactors (Katz, S; 1962b). Fan and Wang (1964) showed that some classes of multistage chemical processes with internal extrema and a special form of the cascade performance equations can be treated in a unified manner by a direct use of Katzs algorithm. In order to illustrate the application of maximum principle, lets consider the optimization of the serial structure as shown in Figure 2.1 and represented by the transformation Equation (2.1) ( ) n = 1, 2 ,3

which relates the output from stage n to the feed (i.e., output from the preceding stage) and to the control or operating variables un operating on the nth stage, with initial state or feed x0 specified as v0. Lets consider further that in addition to the equality constraints as

represented by Equation (2.1), the operative or control inequality constraints is bounded between un* and un* , i.e., ( The objective function for the entire cascade is represented as in Equation (2.4) ( ) [ ( ) ( )] )

The Pontryagins maximum principle asserts that we can optimize the scalar function I using a Hamiltonian function H over one period. In order to optimize a scalar function I(...)subject to the equality constraint as defined by Equation (2.1), we form the Langrangian ( )

If we rearrange Equation (2.1) and substituting it and Equation (2.4) into (2.11), we have * ( ) , ( )-+ ( ) ( ) ( )

where

is the Langrangian multiplier associated with the equality constraints (Equation is associated with the feed condition to stage 1. On applying the Kuhn-

2.1), and

Tucker stationary conditions for an optimum, we obtain , ( ) ( )-

, (

)-

for unconstrained un, and the quantity Hn ( ) ( ) ( )

taking a maximum at the upper and lower bounds as in Equation (2. 10). The quantity Hn can be loosely interpreted as the Langragian for the nth stage. Therefore in compact form, our result can be restated in the following theorem: in order for un, n = 1, 2, . . , N to be the optimal control variable for the N stage problem defined by Equations (2.1), (2.4) and (2.10), un must satisfy the conditions

for unconstrained un ; at constrained values, un has to maximize Hn. The Langragian multipliers, also called adjoint variables, , are defined by

The maximum principle can also be illustrated in a more simply manner by considering the set of simultaneous differential equations ( ) ( )

with specified initial and final conditions ( ) ( ) ( )

whereby we are required to choose, from all the admissible controls u = u(t), 0 t T, which transfer the system from the state x0 to the state x1, the one which the time of transition T is a minimum. The problem of finding the best control in this sense, and the trajectory corresponding to it, is called the time-optimal problem. In order to state Pontryagins maximum principle we introduce the n-dimensional adjoint vector ( ) where

and the Hamiltonian function

Equations (2.18) and (2.20) can now be rewritten in the form

( and (

Pontryagins maximum principle now states: Let

( )

be an admissible

control which transfer the system from the state x0 to the state x1, and let x(t) be the corresponding trajectory, so that x(0) = x0 and x(T) = x1. In order that u(t) be time-

optimal for the problem (Equation 2.18), it is necessary that there exist a non-zero (a constant), continuous vector function (t) corresponding to u(t) and x(t) such that: 1. For all t, maximum at the point i.e., , ( ) ( ) ( )the function ( ) , ( ) ( )( ) ( ( ) ( ) ) of the variable u attains its

2. At the terminal time T the relation , ( ) ( )is satisfied. It can be shown that, if (t), x(t) and u(t) satisfy Equations (2.22) and (2.23) and ( )

condition 1 above, the time function M[ (t), x(t)] is constant, so that the inequality (Equation 2.25) is true for all t,

2.7

Algorithm for Solving the Set of Differential Equations generated from the Maximum Principle

Equations (2.1), (2.16) and (2.17) represent a set of differential equations in which can be solved by any of this method: 2.7.1 Control Vector Iteration If un were known, Equations 2.1 and 2.17 are uncoupled and can be solved sequentially. The following computational algorithm is considered. i. ii. Guess un, n = 1, 2, . . , N Solve Equation (2.1) forward from x0 to produce xn, n = 1, 2, . . . , N

iii. Solve Equation (2.17) backward from

to produce

iv.

Correct the guess of un by ( ) ( )

v.

Go back to step 2 and iterate until convergence is obtained.

The validity of the correction (Equation 2.26) can be seen by considering the first order variation in the objective (Equation 2.4) under the constraint (Equation 2.1) ( , ( Or { } ( ) ) ) ( ( ) ) ( )

Now if Equation (2.1) is satisfied,

, and if Equation (2.17) is satisfied,

thus at each iteration in our algorithm ( )

is valid to a first-order approximation. 2.7.2 Boundary condition Iteration If Equation (2.16) could be solved to produce the explicit expression ( ) ( )

then this could be substituted into Equations (2.1) and (2.17) to create coupled equations in . Since the boundary conditions on these differential equations are split (i.e., are given), an iterative procedure is required to solve them: i. ii. Guess (or ) , ,

Solve Equations (2.1) and (2.17) backward (forward) to yield n = 0, 1, . . ., N

iii.

Compare ( ).

) computed with

) given and correct guess of

iv.

Iterate until the boundary conditions are satisfied.

There is couple of techniques for solving the two point boundary value problem. Graphical methods are suitable for simple problems, while the shooting methods are used in solving complex systems. 2.7.3 Quasi-linearization. The two-point boundary value problem resulting from the use Equations (2.1), (2.16) and (2.17) can also be solved by the method of quasi-linearization. The algorithm is as outlined: i. ii. Guess Linearize Equations (2.1) and (2.17) about this guess and solve the resulting linear two-point boundary conditions iii. Re-linearize about this new approximate solution and iterate until convergence is obtained.

The boundary condition iteration and quasi-linearization approaches have several drawbacks. In situations where the problems are complex, it may be difficult or impossible to invert Equation (2.16) to produce Equation (2.30), therefore making the approach ineffective. Secondly, one must have a perfect convergence in order to have a useful answer. This problem occurs because at each iteration an optimal control is produced - for the wrong problem. For boundary condition iteration, the boundary conditions are wrong and for quasi-linearization the system equations are in error. If one were interested in the optimal controls for a wide variety of feed conditions, x0, then the boundary condition iteration approach could be useful. The control vector iteration algorithm has the advantage that Equation (2.30) is not necessary, and the right problem (but with a suboptimal result) is solved at each iteration. This latter property allows one to stop the computation at any iteration and have a suboptimal but useful solution. The maximum principle is applied in optimal time problems, such as in binary batch distillation (Coward, I; 1967) and cross-current extraction (Zahradnik and Archer, 1963). Lee (1963) demonstrated that the design and operation of certain chemical processes can be optimized by the combination of maximum principle and the dynamic programming techniques. He stated, the limitations of the maximum principle (i.e., the two-point boundary problem) and the dynamic programming (i.e., the curse of dimensionality) are eliminated by combining these techniques.