Вы находитесь на странице: 1из 8

Chapter

8

Iterative

Methods

1

Basic

Iterative

Methods

In

this

section, we

Iterative

methods

and

go

through

a

study

iterative

start

out

with

ﬁxed

procedure

methods

an

initial

for

solving

a

linear

approximation

x

system

to

the

Ax =

b.

solution

to

obtain

a

better

approximation, x .

The

 same procedure is then repeated on x to obtain an improved approximation, x ; and so on. The iterations terminate when a desired accuracy has been achieved. Iterative methods are most useful in solving large sparse systems. Such sys- tems occur, for example, in the solution of boundary value problems for partial diﬀerential equations. The number of ﬂops necessary to solve an n × n using iterative methods is proportional to n , whereas, the amount necessary using Gaussian elimination is proportional to n . Thus for large values of n iterative methods provide the only practical way of solving the system. Furthermore, the amount of memory required for a sparse coeﬃcient matrix A is proportional to n, whereas, Gaussian elimination and the other direct methods studied in earlier chapters usually tend to ﬁll in the zeros of A and hence require an amount of storage proportional to n . This can present quite a problem when n is very large, say n ≥ 20000.

we

The

can

iterative

multiply

methods

A times

we

will

a

vector

describe

in

R

.

only

require

that

in

If

A is

sparse,

this

each

iteration

can

usually

be

accomplished

of

A need

be

solving

Ax =

solve

Ax =

b

in

a

systematic

way

so

that

only

a

small proportion

of

the

entry

accessed.

b

,

one

The

must

one

of

start

all over

again

iterative

from

the

methods

is

that

after

beginning

in

order

to

.

 Matrix Splittings Given a system Ax = b, we write the coeﬃcient matrix A in the form A = C − M, where C is a nonsingular matrix which is in some form that is easily invertible ( e. g. , diagonal or triangular) . The representation A = C − M is

referred

to

as

a

matrix

splitting.

Cx

The

system

=

Mx +

b

1

can

then

be

rewritten

in

the

form

x

=

C

Mx +

C

b

 If we set B = C M = I − C A and c = C b then (1 ) x = Bx + c To solve the system, we start out with an initial guess x , which vector in R . We then set x = Bx + c x = Bx + c and in general

Let

x be

a

solution

of

the

x

=

Bx

+

linear

system.

If

c

·

denotes

some

 may be any vector norm

on

R

x

and

x

the

0

corresponding

matrix

as

k

.

Indeed,

norm

of

B

is

less

than

1

,

we

claim

that

x

x

x

x

=

=

(Bx

(Bx

+

+

c)

c)

( Bx +

( Bx +

c)

c)

=

=

B( x

B( x

x)

x)

=

B

( x

x)

and

(2)

and

so

on.

hence

In

general,

x

x =

B

( x

x)

x

x

=

B

B

B

( x

x)

x

x

x

x

Thus, if B

<

1 , then

x

x

0

as

k

.

 The foregoing result simplest to use the ·

holds

for

or

the

any

·

matrix

A so

C

be

that

easily

invertible

and

that

norm

on

R

,

.

C

Essentially,

be

a

good

I

C

A

=

B

<

1

although

then,

we

in

practice

require

that

it

is

the

enough

approximation

to

This

last

modulus.

condition

implies

Deﬁnition.

The

constant

Let

λ

ρ( B)

,

is

.

.

.

, λ

called

 that all the eigenvalues of B are less be the eigenvalues of B and let ρ(B) = the spectral radius of B.

than

1

in

max | λ | .

2

Theorem

Bx

+

c

8. 1 . 1 .

for

i

=

suﬃcient

condition

Let

0, 1

,

for

x

.

.

.

.

x

be

an

arbitrary

vector

If

x

is

the

x is

that

solution

ρ(B) <

1

to

.

in

R

( 1 ),

and

then

deﬁne

x

=

a

necessary

and

 Proof. We will prove the theorem only in independent eigenvectors. The case where B

the

case

where

B

has

n

linearly

is

not

diagonalizable

is

beyond

the

 scope of this book. If x , we can write x and it follows from (2) that

,

x

are

n

linearly

independent

x =

α x +

·

·

·

+

α x

eigenvectors

of

B,

x

x

=

=

B

(α x +

α λ x +

·

·

·

·

·

·

+

α x )

+

α λ

x

Thus

x

x

0

 if and only if | λ | < 1 for i = 1 , . , n. Thus x → x if and only if ρ( B) < 1 . The simplest choice of C is to let C be a diagonal matrix whose diagonal elements are the diagonal elements of A. The iteration scheme with this choice of C is called Jacobi iteration.
 Jacobi Iteration Let C = and M = and set B = C M and c =  0 − a       − a B =      a a . 0         . .   − a − a  a a

a

0

.

.

.

0

0

a

0

C

0

a

.

.

.

a

a

0

a

b. Thus

·

·

·

·

·

·

a

a

a

a

·

·

·

0

3

·

.

·

.

·

.

·

.

·

.

·

.

a

0

a

a

0

and

c

=

b

a

b

a

b

.

.

.

a

At

(3)

the

( i

+

1 ) st

iteration, the

x

=

1

a

vector

x

is

calculated

a x

+

b

j

=

by

1

,

.

.

.

, n

The

vector

x

is

used

in

calculating

x .

Consequently,

these

two

vectors

must

be

stored

separately.

the

If the

diagonal elements

entries

of

B

should

all

of A are

be

small

much

and

We

say

that

A is

diagonally

dominant

if

larger

than

the

oﬀ-diagonal elements,

the

Jacobi

iteration

should

converge.

| a

|

>

j

=

1

| a

|

for

i

=

1

,

.

.

. , n

If

A is

diagonally

dominant, the

matrix

property

Thus

| b

|

=

B

| a |

| a

|

<

1

=

max

B

of the

for

Jacobi iteration

i

=

1

,

.

.

.

, n

| b |

<

1

It

follows,

converge

to

then,

that

if

the

solution

A

is

diagonally

of Ax =

b.

dominant,

the

Jacobi

 will have the iteration will

An

alternative

to

the

Jacobi iteration

is

to

part

of

A ( i. e. ,

approximation

expect

that

C

c

=

a

if

i

j

and

c

=

to

A than

the

diagonal matrix

is

a

better

approximation

to

take

C

to

be

the

lower

triangular

 0 if i < j) . Since C is a better in the Jacobi iteration, we would

A , and

hopefully

B

will have

a

smaller

norm.

The

iteration

scheme

with

this

choice

of C

is

called

Gauss—Seidel

iteration.

It

usually

converges

faster

than

Jacobi iteration.

Gauss—Seidel

Let

L

Iteration

=

a

0

a

.

.

.

a

a

0

0

a

·

·

·

·

·

·

·

·

·

a

0

0

0

0

0

0

0

4

and

Set

C

=

have

D

We

can

solve

coordinate

of

L

D

and

=

a

0

.

.

.

0

U

=

M

=

 

U.

0

a

0

0 a

·

·

·

0 0

.

.

.

0 0

0 0

Let

x

·

·

·

·

·

·

·

·

·

·

be

a

0

0

a

·

a

·

an

0

0

a

a

a

0

 

 

 

 

arbitrary

nonzero

(D

Cx

L) x

Dx

this

last

x

is

equation

given

by

for

x

=

1

a

=

=

=

Mx

+

b

Ux

+

Lx

b

+

Ux

+

b

x

one

coordinate

a

x

+ b

at

 vector in R . We a time. The ﬁrst

The

and

second

the

last

coordinate

of

x

n

2

coordinates

can

be

of x

.

solved

for

in

terms

of the

ﬁrst

coordinate

x

=

1

a

a

x

a

x

+

b

In

(4)

general,

x

=

1

a

a x

a x

+

b

 It is interesting to compare ( 3) and (4). The diﬀerence between the Gauss—Seidel iterations is that in the latter case, one is using the

Jacobi and

coordinates

of x as soon

as

they

are

calculated

rather

than

in

the

next

iteration.

The

program

for

the

Gauss—Seidel iteration

the

Jacobi

iteration.

The

vectors

x

is

actually

and

x

simpler

than

the

are

both

stored

program

for

in

the

same

vector, x.

coordinate

As

of

a

coordinate

x .

of

x

 is calculated, it replaces the corresponding 5

Theorem

converges

8. 1 . 2.

If

A is

to

a

solution

of

diagonally

Ax =

b.

dominant,

then

the

Gauss—Seidel

iteration

Proof.

For

α

=

j

=

1

| a

,

.

| ,

Since

A is

diagonally

, n,

let

β

=

| a

| ,

dominant, it

follows

and

that

M

and

We

Let

| a

|

>

α

+

β

 consequently M < 1 for j = 1 , M = will show that

.

, n.

Thus

max M

<

1

B

=

max

x

Bx

x

M

<

1

x be

a

nonzero

vector

in

R

and

let

y

=

Bx.

Choose

y

=

max | y | =

| y

|

=

k

β

( | a

|

so

that

α )

 It follows from the deﬁnition of B that y = Bx = ( D − L) Ux and hence y = D ( Ly + Ux) Comparing the kth coordinates of each side, we see that

y

=

1

a

a

y

a

x

 and hence (5) y = | y | ≤ | a 1 | ( α y + β x ) It follows from ( 5) that Bx y M M x = x ≤ ≤ Thus Bx ≤ M 1 B = max x x < and hence the iteration will converge to the solution of Ax = 6

b.

Exercises

1

.

Let

Use

(1

,

1

2. Let

A =

Jacobi

)

. ]

A =

1 0

2

1

1 0

iteration

1

0

0

1

1

0

to

1

1

1

,

b

=

compute

1

1

x

,

b

=

1

2

and

3

2

1

, and

,

x .

[The

and

x

exact

x

=

0

0

solution

=

1

0

0