Numerical Methods in Chemical Engineering

5 Solving systems of non-linear equations

5 Solving systems of non-linear equations .......................................................1 5.1 5.2 5.3 5.4 Overview ..................................................................................................2 Passing functions......................................................................................3 1D Newtons Method (something you did at school) ...............................4 Newton's method in more than one dimension........................................6 Solving the system of two coupled equations...................................9 Solution trajectories ........................................................................11 The reduced Newton step method...................................................13 Problems with Newton's Method ....................................................14 The element balance equations .......................................................17 The equilibrium equations...............................................................17 Thermodynamic data.......................................................................18 Solving the equations .....................................................................19 Full solution to the equilibrium problem in Matlab........................21

5.4.1 5.4.2 5.4.3 5.4.4 5.5 5.5.1 5.5.2 5.5.3 5.5.4 5.5.5 5.6 5.7

A chemical engineering example - equilibrium....................................16

Convergence of the Newton-method .....................................................23 Calculating the Jacobian Matrix ............................................................24 Finite differences.............................................................................24 Broydens Method ............................................................................25 Taking advantage of sparsity ..........................................................26

5.7.1 5.7.2 5.7.3 5.8 5.9 5.10

Trust region method (Used by fsolve) ...................................................27 Summary ................................................................................................28 Amendments .......................................................................................29 Updated version of Newton1D ....................................................29 Updated version of Newton .........................................................30

5.10.1 5.10.2

-1-

Numerical Methods in Chemical Engineering 5.1 Overview

Now we will look at solving systems of non-linear equations. We will look at one method in particular, Newton's method. We are going to write our own routine for solving non-linear equations using Newton's method. Matlab has its own routines for solving systems of non-linear equations (e.g. fsolve, which is loosely based on Newton's method). You are encouraged to use these Matlab routines as they are likely to be more robust than anything we can write in a lecture! Additional reading for this lecture: Chapter 1 of "Numerical Methods for Chemical Engineers with Matlab Applications" by Constantinides and Mostoufi Look up the routines fsolve and fzero in the Matlab help file

-2-

Numerical Methods in Chemical Engineering 5.2 Passing functions In Matlab, function names can be passed as arguments to functions. This means we can write routines which act on functions. E.g. if we want to integrate under the curve y = f(x), we could write a function which, given x would return a value of y. We could then pass this function to another routine which does the numerical integration. The mechanism Matlab provide for this is a function handle (other programming languages would call this a pointer). E.g. to solve x2-2x= 0 numerically, we write a function which returns the value of x2-2x. function [residual] = Myfunc(x) residual = x^2 2*x return We then call a solving routine (e.g. fzero - look it up in the help file) Answer = fzero(@MyFunc,-5) @MyFunc is a handle to the function MyFunc(x), which means that it can be used as an alias for MyFunc(x) e.g. if we do the following FunctionHandle = @MyFunc The variable FunctionHandle can then be used as an alias for MyFunc, i.e. y = FunctionHandle(x) is the same as y = MyFunc(x) Another command you can use is y = feval(FunctionHandle,x) This latter command has the advantage of also be able to take a text string (i.e. the name of the function) in place of a function handle and evaluate it. The ability to store functions in a variable means that it is possible to pass functions as arguments to other functions.

-3-

Numerical Methods in Chemical Engineering 5.3 1D Newtons Method (something you did at school) Lets solve x2 - 2x = 0 using Newton's Method. In Newton's method, we set f(x) = x2-2x take an initial guess (x0) for the solution and calculate f(x0). We then extrapolate (assuming f(x) is linear) to a new value of x which will make f(x) = 0. i.e.

df f new = f o + ( xnew xo ) dx xo

or

( xnew xo ) = x =

fo

df dx xo

The procedure is then repeated until the solution is reached. i.e.

20 y = f(x)

15

10 y 5 0

x3 x

x1

x0 4 5

-5 -1

2 x

Figure 1. The solution to x2 - 2x = 0 by Newton's method

A Matlab function to perform this iterative procedure might look like:

-4-

Numerical Methods in Chemical Engineering

function [solution] = Newton1D(MyFunc,Gradient,Guess,tol) % solves the non-linear vector equation F(x)=0 % set using Newton Raphson iteration % % F(x) % F'(X) % % % % % % % INPUTS: Myfunc Gradient Guess Tol OUTPUTS: solution

= Handle to the function which calculates = Handle to the function which calculates = Initial Guess = Tolerance

= A solution to the set of equations

Description Solves a 1 D non-linear equation by Newtons method

x = Guess; %set the error 2*tol to make sure the loop runs at least once error = 2*tol while error > tol

%calculate the function iteration F = feval(MyFunc,x); %calculate the Gradient G = feval(Gradient,x); %calculate the update dx = (-F)/G; %update the x value x = x+dx; %calculate the error F = feval(MyFunc,x); error = (abs(F)); end %while loop solution = x; return

values

at

the

current

-5-

Numerical Methods in Chemical Engineering

5.4 Newton's method in more than one dimension

Lets look at an example: f1 = x13 + x22 = 0 f2 = x12 F(x) = 0 Where x is the vector of unknowns and F is a vector of function values. e.g. for this example,
3 x1 f1 x 1 + x 2 2 F = = 2 3 x 2 f 2 x1 x 2

(5-1) (5-2)

x23 = 0

We must first arrange our equations in the form:

If we start with a guessed value, xi we can find a better guess by extrapolation. Near our guessed value, the function can be expanded
f 1 f 1 f 1 x dx1 + x dx 2 + x dx3 ... 2 3 f 1 ( x i + d x) f 1 ( x i ) 1 i i f 2 dx + f 2 dx + f 2 dx ... 2 3 f 2 ( x + d x) = f 2 ( x ) + x 1 x x3 1 2 M M M Note that these derivatives are evaluated at the x = xi f1 f1 ( x i + d x) f1 ( x i ) x1 i i f 2 f 2 ( x + d x) = f 2 ( x ) + x M 1 M M f1 x2 f 2 x2 M f1 x3 L. dx1 f 2 dx2 L x3 dx3 M M xi

(5-3)

(5-4)

-6-

Numerical Methods in Chemical Engineering

f 1 ( x i + d x) f 1 ( x i ) i i f 2 ( x + d x) = f 2 ( x ) + J dx M M

or F ( x + d x) = F ( x ) + J dx
i i

(5-5)

Now we want our new vector xi+1 = xi + dx to be a better approximation to the solution, so we set F(xi+1) = 0 in Eq 5-5. Then, F ( x ) = J dx
i

(5-6)

where J is called the Jacobian matrix.

The procedure for Newton's method is: 1) calculate the function values at the guessed value of xi =[x1, x2, x3]T 2) calculate the Jacobian matrix using the current guess for the solution i 3) solve the linear system F ( x ) = J dx for the values of dx 4) update the guessed value xi+1 = xi + dx This procedure should be repeated, using the updated value of xi as the guess, until the values of F(x) are sufficiently close to zero. F(x) is a vector of
residual errors. For sufficiently close to zero we could use

Max{f1(x), f2(x) , f3(x) , f4(x) , f5(x)... } < Tolerance or the 2 norm of the vector F(x),
||F(x)|| = f i 2 i
1/ 2

(5-7)

A function which will perform this procedure is given overleaf. Notice how similar it is to the one dimensional code, and that a set of linear equations must be solved!

-7-

Numerical Methods in Chemical Engineering

function [solution] = Newton(MyFunc,Jacobian,Guess,tol) % solves the non-linear vector equation F(x)=0 % set using Newton Raphson iteration % INPUTS; % Myfunc=Handle to the function which returns the vector F(x) % Jacobian=Handle to the function which returns the Jacobian Matrix % Guess = Initial Guess (a vector) % tol = Tolerance % % OUTPUTS % solution = The solution to F(x) = 0

x = Guess; %set the error 2*tol to make sure the loop runs at least once error = 2*tol while error > tol %calculate the function iteration F = feval(MyFunc,x); values at the current

%calculate the jacobian matrix J = feval(Jacobian,x); %calculate the update (solve the linear system) dx = J\(-F); %update the x value x = x+dx; %calculate the error F = feval(MyFunc,x); error = max(abs(F)); end %while loop solution = x; return

-8-

Numerical Methods in Chemical Engineering 5.4.1 Solving the system of two coupled equations The equations we want to solve are: f1 = x13 + x22 = 0 f2 = x12 The Jacobian is
f1 x J = 1 f 2 x1 3 x12 = 2 x1 f1 x2 f 2 x2 xi 2 x2 3 x22 xi

(5-1) (5-2)

x23 = 0

All we need to do to solve our equations is to write two functions: (i) that returns the vector of function values (function y = Func(x)) and (ii) that returns the Jacobian matrix (function J = Jacobian(x)).

-9-

Numerical Methods in Chemical Engineering

function main %main function which call the Newton solver %call the solver solution = Newton(@Func,@Jac,[1;1],1e-6)

return

function y = Func(x) %the function which returns the values of F(x) y(1) = x(1).^3 + y(2) = x(1).^2 y= y'; return function J = Jac(x) %The function that returns the Jacobian matrix J(1,1) J(1,2) J(2,1) J(2,2) return = = = = 3*x(1)^2; 2*x(2); 2*x(1); -3*x(2)^2; x(2).^2 ; x(2).^3;

- 10 -

Numerical Methods in Chemical Engineering 5.4.2 Solution trajectories From our definition of the norm (5-7) ||F(x)|| 0 So at a solution we must have the smallest possible value of ||F(x)||, and the solution is a global minimum in ||F(x)||. For a given set of equations F(x) = 0, we can plot the values of ||F(x)|| at all values of x (however, anything more complicated than 2D is a bit difficult to interpret). Returning to our example, f1 = x13 + x22 = 0 f2 = x12 x23 = 0
||[f1,f2] T|| 5 4 3 2 1 0 2 1 0 x2 -1 -1.5 -1 -0.5 x1 0 0.5 1

(5-1) (5-2)

A plot of the norm of [f1 , f2]T is given below

Figure 2. ||F(x)||

There are solutions located at [0,0]T and [-1,1]T

- 11 -

Numerical Methods in Chemical Engineering

2 1 0 -1 -2 x2 -3 -4 -5 -6 -7 -3.5

Solution at [0,0]T

-3

-2.5

-2

-1.5

-1 x1

-0.5

0.5

1.5

Figure 3. ||F(x)|| and the trajectory taken by the Newton method

Figure 3 shows the trajectory followed when the solver is started from [1,0.4]T. The trajectory the solution follows can be very erratic, especially when the solver is stated a long way from the solution. In fact, Figure 3 shows that the error (||F(x)||) actually increases on the first iteration.

- 12 -

Numerical Methods in Chemical Engineering 5.4.3 The reduced Newton step method We don't have to take a full Newton step, instead we can search along the direction of the Newton step, for a point where the error is less than the error at the starting point. An algorithm to do this is simply:

Calculate Newton step, x

i=0 Work out ||F(x + 0.5 i x)||

is ||F(x + 0.5 i x)|| < ||F(x )|| Yes New x = x + 0.5 i x

No

i = i+1

- 13 -

Numerical Methods in Chemical Engineering

1.5

Trajectory taken by Newton method

1

0.5 x2

Trajectory taken if we search back along the line of a full Newton step

-0.5

Solution

-1 -1.5

-1

-0.5 x1

0.5

Figure 4. ||F(x)|| and the trajectory taken by the Newton method (with line searching to reduce the step)

Much more robust! 5.4.4 Problems with Newton's Method Newton's method will fail, when the Jacobian is singular, but we are not at the solution. When the Jacobian is singular, we cannot find a solution to F ( x i ) = J ( x i +1 x i ) so we cannot find our update vector x. This is analogous to what happens with the 1 dimensional Newton method, and a maximum or minimum in the function (i.e. zero gradient) is encountered.

- 14 -

Numerical Methods in Chemical Engineering For the reduced step Newton algorithm, a Newton step is calculated and then a search is performed along the direction of the Newton step to find a step which will reduce the error. When the update step is almost perpendicular to the direction of the gradient of the 2 Norm, F (x) (i.e. parallel to the contours of ||F(x)||), the value of ||F(x)|| (which we are trying to make as small as possible) doesn't change much along the line we are searching. The solver will then take very small steps.

- 15 -

Numerical Methods in Chemical Engineering

5.5 A chemical engineering example - equilibrium

Consider the reaction of methane, with water at high temperatures CH4 + 2H2O

CO2 + 3H2

Steam reforming of methane has been proposed as one way to produce clean Hydrogen for the 'Hydrogen economy'. In any reactor containing CO2 and H2, the shift reaction will also occur. CO2 + H2

CO + H2O

So we have a reactor as follows CH4 CO CO2 H2 H2O

H2O CH4

and we want to know, what the composition of the outlet stream is, if the system is allowed to reach equilibrium. To solve this problem we can use the method of minimum reactions. There are 5 species of interest (i.e. 5 unknown flows leaving the reactor we need to solve for). The amounts of H, C and O entering the system must be equal to the amounts leaving; this generates three elemental balance equations. constant. Also, there are two reactions (shift and reformation), these will yield two equations, each involving an equilibrium

- 16 -

Numerical Methods in Chemical Engineering 5.5.1 The element balance equations

0 = E j ij N i
i

(5-8)

where Ej and Ni are the number of moles of element j entering and species i leaving the system the system per unit time; i the number of atoms of j in one molecule of species i. e.g. NCH4 + 4 NCH4 NCO + 2NCO2 NCO + NCO2 = EC + 2 NH2 + 2 NH2O = EH + NH2O = EO (5-9) (5-10) (5-11)

This can be represented by a stiochiometric matrix equation

N CH 4 H 2O N CO 0 N CO2 2 N H2 1 N H O 2 IN N CH 4 H 2O N CO 0 N CO2 2 N H2 1 N H O 2 OUT

EC E = C H H EO O

CH 4 1 4 0 CH 4

CO CO2 1 1 0 1 0 2

H2 0 2 0 H2 0 2 0

CO CO2 1 0 1 1 0 2

C H O

1 4 0

5.5.2 The equilibrium equations For the steam reforming reaction

PH 2 PCO2 o o P P = P yH2 = o 2 PH 2O PCH 4 P y H 2O o o P P
3

Kp

( ) (y ) P = ( ) (y ) P
3 CO2 2 o CH 4

1
i i

(N ) (N ) (N ) (N )
3 H2 CO2 2 H 2O CH 4

- 17 -

Numerical Methods in Chemical Engineering Taking natural logs

Ln K p

( )
1

P = Ln o P Ni i

+ 3Ln N H 2 + Ln N CO2 2 Ln N H 2O Ln N CH 4

(5-12)

For the water-gas shift

PH 2O PCO o o P P y H 2O ( y CO ) N H 2O (N CO ) = = = y H 2 y CO2 N H 2 N CO2 PCO2 PH 2 o o P P

Kp

( ) ( )(

( ) (

) )(

Taking natural logs

Ln K p
2

( ) = Ln(N ) + Ln(N
H 2O

CO

) Ln(N H

) Ln(N )
CO2

(5-13)

5.5.3 Thermodynamic data The values of Kp as a function of temperature given below

30 25 20 Ln(Kp) 15 10 5 0 -5 600 y = -6.095E-06x 2 + 1.692E-02x - 1.116E+01 R2 = 9.970E-01 y = 2.616E-05x 2 - 7.491E-02x + 6.251E+01 R2 = 9.979E-01 Ln(Kp1) Ln(Kp2)

800

1000 Temperature (K)

1200

1400

Figure 5, Ln (Kp) as a function of temperature

- 18 -

Numerical Methods in Chemical Engineering A function I have written to calculate these values is:
function [LnKp1,LnKp2] = LnKp(T) LnKp1 = 2.616e-5*T^2 - 7.491e-2*T + 6.251e1; LnKp2 = -6.095e-6*T^2 + 1.692e-2*T - 1.116e1; return

5.5.4 Solving the equations The equation set we have to solve is NCH4 + NCO + NCO2 4 NCH4 NCO + 2 NCO2
P Ln o P Ni i

- EC = 0 + 2 NH2 + 2 NH2O - EH = 0 + NH2O - EO = 0

f1 (5-9) f2(5-10) f3(5-11) f4(5-12) f5(5-13)

Ln(N H 2O ) + Ln( N CO ) Ln(N H 2 ) Ln(N CO2 ) Ln(Kp 2 ) = 0

Ln(Kp1 ) + 3Ln(N H 2 ) + Ln(N CO2 ) 2 Ln(N H 2O ) Ln(N CH 4 ) = 0

We need to write a function which returns the left hand side of these equations
function [residual] = f(N) global Stoichiometry global E global T global P_Po %the element balance equations residual(1:3) = Stoichiometry*N - E %the equilibrium constants [LnKp1,LnKp2] = LnKp(T) %equilibrium equations residual(4)= -LnKp1 + log(P_Po/sum(N))+3*log(N(4)) +... log(N(3))-2*log(N(5))-log(N(1)) residual(5)=- LnKp2 + log(N(5))+log(N(2))- log(N(3))log(N(4)) return

- 19 -

Numerical Methods in Chemical Engineering We also need the Jacobian: The first 3 rows of the Jacobian are just the stoichiometry matrix. The row for equation (5-12), i.e. the 4th row is made up of the following terms.

f f 1 1 3 1 f 4 f 1 1 1 = , 4 = 0 , 4 = , 4 = N 1 N 1 N i N 2 N i N 3 N 3 N i N 4 N 4 N i
i i

f 4 2 1 = N 5 N 5 N i
i

The 5th row of the Jacobian is made up of

f 5 f f 5 1 1 f 5 1 f 5 1 = 0, , 5 = , , = = = N 2 N 2 N 3 N 3 N 4 N 4 N 5 N 5 N1

function [Jac] = jac(N) global Stoichiometry Jac(1:3,:) = Stoichiometry; %row 4 (for Jac(4,1) = Jac(4,2) = Jac(4,3) = Jac(4,4) = Jac(4,5) = %row 5 (for Jac(5,1) = Jac(5,2) = Jac(5,3) = Jac(5,4) = Jac(5,5) = the steam -1/N(1) 0 1/N(3) 3/N(4) -2/N(5) reforming reaction) 1/sum(N); 1/sum(N); 1/sum(N); 1/sum(N); 1/sum(N);

the water-gas shift) 0; +1/N(2); -1/N(3); -1/N(4); +1/N(5);

return

- 20 -

Numerical Methods in Chemical Engineering 5.5.5 Full solution to the equilibrium problem in Matlab
%MAIN FUNCTION %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function main global T global E global Stoichiometry global P_Po %set the inlet flows [CH4;CO;CO2;H2;H2O] Nin = [10;0;0;0;1]; %set the temperature (K) T = 900; %set the pressure (Po = 1 bar) P_Po = 1; %set the stiociometry matrix Stoichiometry = [ 1, 1, 1, 0, 0;... 4, 0, 0, 2, 2;... 0, 1, 2, 0, 1]; %Calculate the flows of elements entering the system E = Stoichiometry*Nin; %solve the system of equations [N] = NewtonReduced(@f,@jac,[1,1,1,1,1]',100*eps); %print the solution disp(N) return %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %Function that computes the Jacobian %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function [Jac] = jac(N) global Stoichiometry Jac(1:3,:) = Stoichiometry; %row 4 (for Jac(4,1) = Jac(4,2) = Jac(4,3) = Jac(4,4) = Jac(4,5) = the steam -1/N(1) 0 1/N(3) 3/N(4) -2/N(5) reforming reaction) 1/sum(N); 1/sum(N); 1/sum(N); 1/sum(N); 1/sum(N);

- 21 -

Numerical Methods in Chemical Engineering

%row 5 (for Jac(5,1) = Jac(5,2) = Jac(5,3) = Jac(5,4) = Jac(5,5) = return %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %Function that computes equilibrium constants %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function [LnKp1,LnKp2] = LnKp(T) LnKp1 = 2.616e-5*T^2 - 7.491e-2*T + 6.251e1; LnKp2 = -6.095e-6*T^2 + 1.692e-2*T - 1.116e1; return %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %Function that computes the function values equations to be solved) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function [residual] = f(N) global Stoichiometry global E global T global P_Po %the element balance equations residual(1:3) = Stoichiometry*N - E; %the equilibrium constants [LnKp1,LnKp2] = LnKp(T); %equilibrium equations residual(4) = -LnKp1 + log(P_Po/sum(N))+3*log(N(4)) + log(N(3))-2*log(N(5))-log(N(1)); residual(5) = - LnKp2 + log(N(5))+log(N(2))- log(N(3))log(N(4)); residual = residual'; return the water-gas shift) 0; +1/N(2); -1/N(3); -1/N(4); +1/N(5);

(set

of

- 22 -

Numerical Methods in Chemical Engineering

5.6 Convergence of the Newton-method

The error for a given iteration can be taken as ei = xi - x* where x* is the true solution. Now assuming we are close to the solution, we can expand the function F about x*, and use this expansion in place of F
F ( x) = F * + J * ( x x*) + O[( x x*)2 ] F ( x) = J * ( x x*) + O[( x x*)2 ]

(5-14) (5-15) (5-16)

F ( x) = J * (e) + O[e ]

Substituting this into our update formula

F (x ) = J (x
i i +1

x )
i

(5-6)
i +1

gives
J * (e ) O[(e ) 2 ] = J ( x
i i

x )
i

(5-17) (5-18)

J * (e ) O[(e ) 2 ] = J (e
i i

i +1

e )
i

If we are sufficiently close to the solution, then J J*, and

O[(e ) 2 ] = J * (e )
i i +1

(5-19)

Thus, the convergence is quadratic (this of-course only applies when the Jacobian is sensible). If the Jacobian is singular, or nearly singular near the solution, the error can be very large. [In fact, when you have a singular Jacobian at the root itself, convergence becomes linear.]

- 23 -

Numerical Methods in Chemical Engineering

5.7 Calculating the Jacobian Matrix

The methods presented so far assume that an analytical form of the Jacobian was available. When we have a large system of equations and the functions are 'expensive' to evaluate, calculating the Jacobian can become inefficient. Efficient calculation of the Jacobian then becomes important. 5.7.1 Finite differences The Jacobian is just a matrix of derivatives: f 1 x 1 f J = 2 x1 M f 1 x 2 f 2 x 2 M f 1 x3 L. f 2 L x3 M

Each of these derivatives can be calculated using finite differences

f (x + ) fi (x ) f i = i x j

(5-20)

The value of cannot be made too small, because computers are only able to store numbers to a finite precision. The working precision of the computer can be found with the Matlab command eps (which is equal to 2.2204e-016 on my computer). A common value of is eps . Calculating the Jacobian in this way is less efficient than if an analytical form of the Jacobian was supplied. The Matlab Routine 'fsolve' will use finite differences to calculate the Jacobian, if you don't supply a function (or if the Jacobian is constant, a matrix) to do it.

- 24 -

Numerical Methods in Chemical Engineering 5.7.2 Broydens Method It is not necessary to calculate the exact value of the Jacobian at each iteration. Broyden's method provides a way of updating an estimate of the Jacobian. Generally, the residual function values at the ith and i+1th iteration are linked by
F
i +1

( x) F ( x) + J i ( x
i

i +1

x)
i

(5-21)

So if we replace the exact Jacobian by an estimate of the Jacobian at iteration i+1, Bi+1, we desire this approximate Jacobian to obey F ( x) F ( x) + B i +1 ( x x )
i i i +1 i +1

(5-22)

If we use Bi (the estimate of the Jacobian at iteration i), with a Newton step to get our updated x, xi+1 F (x ) = Bi (x
i i +1

x ) = B i ( x )
i i

(5-23)

Using (5-22) in (5-23) Bi (x

i +1

x ) F ( x) + B i +1 ( x x )
i i i +1 i

i +1

i +1

B i x F ( x) B i +1 x
i

Post multiply both sides by x

i i i T i +1

( ) B x ( x ) F ( x)( x ) B x ( x ) B x F ( x)( x ) B x
i T i T i +1 i

i T

i 2

i +1

i T

i +1

i 2

i +1

B +
i

i +1

( x) x x
i 2

( )

i T

(5-24)

So on the i+1th iteration we can calculate an updated estimate of the Jacobian, from information calculated in the previous iteration, and the value of F(xi+1), which we would have to calculate anyway. Using this type of update results in a Quasi-Newton scheme, which has linear convergence.
- 25 -

Numerical Methods in Chemical Engineering

5.7.3 Taking advantage of sparsity Often, the set of equations we will be solving will be sparse. The Jacobian will contain mainly zeros. If we know that an element of the Jacobian is zero then we don't have to calculate it. Also, a set of linear equations is solved at each iteration, if these linear equations are sparse then it should be possible to use a efficient solution method (recall handout 3).There are two ways to exploit sparsity when solving non-linear equations in Matlab: 1. Use sparse matrices. If you write a function to calculate the Jacobian, make sure it returns a sparse matrix. Efficient methods can then be used to solve the resulting sets of linear equations. Note that the Matlab \ command will detect sparse matrices and try to use the most efficient method it can find. If you are using fsolve, it will detect that the Jacobian is sparse and act accordingly. 2. If you are using fsolve, but you don't want to calculate the Jacobian analytically, you can instead supply a Jacobian pattern. The Jacobian pattern is a sparse matrix with ones where the Jacobian is non-zero. This means that fsolve does not have to calculate every element of the Jacobian. It also means that it will create a sparse Jacobian.

- 26 -

Numerical Methods in Chemical Engineering

5.8 Trust region method (Used by fsolve)

The update equation for x is

F ( x ) = J ( x )
i i

(5-6)

The solution to this set of linear equations, will minimise the value of
J ( x ) + F ( x )
i i 2

(5-25)

So, rather than solving the set of linear equations, we can minimise (5-25), with respect to x. A long way from the solution, the update can be erratic, so we constrain to ||x || to be within a trust region. We then solve the constrained optimisation problem minimise q = J ( x i ) + F ( x i )
2

with respect to x, subject to the constraint that ||x || < some value If the Newton step is within the trust region, the update will be a full Newton step (identical to if we has solved (5-6), by e.g. elimination). If however, the full Newton step is outside the trust region, then a minimisation routine is free to vary both the direction and length of x to minimise the cost function q. We will get as close as possible to a solution of (5-6), without stepping too far, because the length of x is constrained. This overcomes problems with the reduced step method, in that both the direction and the length of the update can be varied to keep the step within the trust region. A discussion of how you set the trust region at each iteration is beyond the scope of these lectures.

- 27 -

Numerical Methods in Chemical Engineering

5.9 Summary

The solution of non-linear equations using a computer will generally require some sort of iterative procedure. In this handout the Newton method for solving systems of non-linear equations was presented. This method is one of the most widely used schemes for solving non-linear equations. Matlab also contains routines which can be used to solve systems of non-linear equation; the most useful being fsolve. When solving a set of non-linear equations, at some point in the procedure the solution to a set of linear equations is often required. Thus, efficient solution of non-linear equations requires the ability to solve a set of linear equations. In previous lectures we have seen that if a set of linear equations is sparse, it can be solved efficiently. Thus, when the Jacobian of a non-linear system is a sparse matrix, any non-linear iterative routine should be written to take advantage of this.

- 28 -

Numerical Methods in Chemical Engineering

5.10 Amendments

The Newton and Newton 1D routines given earlier evaluate the functions twice for each iteration. This is inefficient, and can be avoided by changing where in the routine the error is evaluated. Updated routines are given below. 5.10.1 Updated version of Newton1D
function [solution] = Newton1D(MyFunc,Gradient,Guess,tol) % solves the non-linear vector equation F(x)=0 % set using Newton Raphson iteration % INPUTS: % Myfunc = Handle to the function which calculates F(x) % Gradient = Handle to the function which calculates F'(X) % Guess = Initial Guess % Tol = Tolerance % % OUTPUTS: % solution = A solution to the set of equations % Description % Solves a 1 D non-linear equation by Newtons method x = Guess; %set the error 2*tol to make sure the loop runs at least once error = 2*tol while error > tol %calculate the function iteration F = feval(MyFunc,x); %calculate the error error = (abs(F)); %calculate the Gradient G = feval(Gradient,x); %calculate the update dx = (-F)/G; %update the x value x = x+dx; end %while loop solution = x; return values at the current

- 29 -

Numerical Methods in Chemical Engineering

5.10.2 Updated version of Newton

function [solution] = Newton(MyFunc,Jacobian,Guess,tol) % solves the non-linear vector equation F(x)=0 % set using Newton Raphson iteration % INPUTS; % Myfunc=Handle to the function which returns the vector F(x) % Jacobian=Handle to the function which returns the Jacobian Matrix % Guess = Initial Guess (a vector) % tol = Tolerance % % OUTPUTS % solution = The solution to F(x) = 0

x = Guess; %set the error 2*tol to make sure the loop runs at least once error = 2*tol while error > tol %calculate the function iteration F = feval(MyFunc,x); error = max(abs(F)); values at the current

%calculate the jacobian matrix J = feval(Jacobian,x); %calculate the update (solve the linear system) dx = J\(-F); %update the x value x = x+dx; end %while loop solution = x; return

- 30 -