Вы находитесь на странице: 1из 8

Chapter

8

Iterative

Methods

1

Basic

Iterative

Methods

In

this

section, we

Iterative

methods

and

go

through

a

study

iterative

start

out

with

fixed

procedure

methods

an

initial

for

solving

a

linear

approximation

x

system

to

the

Ax =

b.

solution

to

obtain

a

better

approximation, x .

The

same

procedure

is

then

repeated

on

x

to

obtain

an

improved

approximation,

x ;

and

so

on.

The

iterations

terminate

when

a

desired

accuracy

has

been

achieved.

 

Iterative

methods

are

most

useful in

solving

large

sparse

systems.

Such

sys-

tems

occur, for

example, in

the

solution

of boundary

value

problems

for

partial

differential

equations.

The

number

of

flops

necessary

to

solve

an

n

× n

using

iterative

methods

is

proportional

to

n

,

whereas,

the

amount

necessary

using

Gaussian

elimination

is

proportional to

n .

Thus

for

large

values

of n

iterative

methods

provide

the

only

practical way

of solving

the

system.

Furthermore, the

amount

of memory

required

for

a

sparse

coefficient

matrix

A is

proportional to

n, whereas, Gaussian

elimination

and

the

other

direct methods

studied

in

earlier

chapters

usually

tend

to

fill

in

the

zeros

of

A and

hence

require

an

amount

of

storage

proportional

to

n .

This

can

present

quite

a

problem

when

n

is

very

large, say

n

20000.

we

The

can

iterative

multiply

methods

A times

we

will

a

vector

describe

in

R

.

only

require

that

in

If

A is

sparse,

this

each

iteration

can

usually

be

accomplished

of

A need

be

solving

Ax =

solve

Ax =

b

in

a

systematic

way

so

that

only

a

small proportion

of

the

entry

accessed.

b

,

one

The

must

one

disadvantage

of

start

all over

again

iterative

from

the

methods

is

that

after

beginning

in

order

to

.

Matrix

Splittings

 

Given

a

system

Ax =

b,

we

write

the

coefficient

matrix

A in

the

form

A =

C

M, where

C

is

a

nonsingular

matrix

which

is

in

some

form

that

is

easily

invertible

( e. g. ,

diagonal

or

triangular) .

The

representation

A

=

C

M

is

referred

to

as

a

matrix

splitting.

Cx

The

system

=

Mx +

b

1

can

then

be

rewritten

in

the

form

x

=

C

Mx +

C

b

If we

set

 
 

B

=

C

M

=

I

C

A

and

c

=

C

b

then

 

(1 )

x =

Bx +

c

To

solve

the

system, we

start

out

with

an

initial guess x , which

vector

in

R

.

We

then

set

 
 

x

=

Bx

+

c

x

=

Bx

+

c

and

in

general

 

Let

x be

a

solution

of

the

x

=

Bx

+

linear

system.

If

c

·

denotes

some

may

be

any

vector

norm

on

R

x

and

x

the

0

corresponding

matrix

as

k

.

Indeed,

norm

of

B

is

less

than

1

,

we

claim

that

x

x

x

x

=

=

(Bx

(Bx

+

+

c)

c)

( Bx +

( Bx +

c)

c)

=

=

B( x

B( x

x)

x)

=

B

( x

x)

and

(2)

and

so

on.

hence

In

general,

x

x =

B

( x

x)

x

x

=

B

B

B

( x

x)

x

x

x

x

Thus, if B

<

1 , then

x

x

0

as

k

.

The

foregoing

result

simplest

to

use

the

·

holds

for

or

the

any

·

matrix

A so

C

be

that

easily

invertible

and

that

norm

on

R

,

.

C

Essentially,

be

a

good

I

C

A

=

B

<

1

although

then,

we

in

practice

require

that

it

is

the

enough

approximation

to

This

last

modulus.

condition

implies

Definition.

The

constant

Let

λ

ρ( B)

,

is

.

.

.

, λ

called

that

all

the

eigenvalues

of

B

are

less

be

the

eigenvalues

of

B

and

let

ρ(B)

=

the

spectral

radius

of B.

 

than

1

in

max | λ | .

2

Theorem

Bx

+

c

8. 1 . 1 .

for

i

=

sufficient

condition

Let

0, 1

,

for

x

.

.

.

.

x

be

an

arbitrary

vector

If

x

is

the

x is

that

solution

ρ(B) <

1

to

.

in

R

( 1 ),

and

then

define

x

=

a

necessary

and

Proof.

We

will

prove

the

theorem

only

in

independent

eigenvectors.

The

case

where

B

the

case

where

B

has

n

linearly

is

not

diagonalizable

is

beyond

the

scope

of

this

book.

If

x ,

we

can

write

 
 

x

and

it

follows

from

(2)

that

,

x

are

n

linearly

independent

x =

α x +

·

·

·

+

α x

eigenvectors

of

B,

x

x

=

=

B

(α x +

α λ x +

·

·

·

·

·

·

+

α x )

+

α λ

x

Thus

x

x

0

if

and

only

if

| λ

|

<

1

for

i

=

1

,

.

, n.

Thus

x

x if

and

only

if

ρ( B)

<

1 .
1
.

The

simplest

choice

of

C

is

to

let

C

be

a

diagonal

matrix

whose

diagonal

elements

are

the

diagonal elements

of A.

The

iteration

scheme

with

this

choice

of C

is

called

Jacobi

iteration.

 

Jacobi

Iteration

 

Let

 

C

=

and

 

M

=

and

set

B

=

C

M

and

c

=

 

0

a

a

 
 

B

=

a

a

.

0

 

.

.

a

a

a

a

a

0

.

.

.

0

0

a

0

C

0

a

.

.

.

a

a

0

a

b. Thus

·

·

·

·

·

·

a

a

a

a

·

·

·

0

3

·

.

·

.

·

.

·

.

·

.

·

.

a

0

a

a

0

and

c

=

b

a

b

a

b

.

.

.

a

At

(3)

the

( i

+

1 ) st

iteration, the

x

=

1

a

vector

x

is

calculated

a x

+

b

j

=

by

1

,

.

.

.

, n

The

vector

x

is

used

in

calculating

x .

Consequently,

these

two

vectors

must

be

stored

separately.

the

If the

diagonal elements

entries

of

B

should

all

of A are

be

small

much

and

We

say

that

A is

diagonally

dominant

if

larger

than

the

off-diagonal elements,

the

Jacobi

iteration

should

converge.

| a

|

>

j

=

1

| a

|

for

i

=

1

,

.

.

. , n

If

A is

diagonally

dominant, the

matrix

property

Thus

| b

|

=

B

| a |

| a

|

<

1

=

max

B

of the

for

Jacobi iteration

i

=

1

,

.

.

.

, n

| b |

<

1

It

follows,

converge

to

then,

that

if

the

solution

A

is

diagonally

of Ax =

b.

dominant,

the

Jacobi

will have

the

iteration

will

An

alternative

to

the

Jacobi iteration

is

to

part

of

A ( i. e. ,

approximation

expect

that

C

c

=

a

if

i

j

and

c

=

to

A than

the

diagonal matrix

is

a

better

approximation

to

take

C

to

be

the

lower

triangular

0

if

i

<

j) .

Since

C

is

a

better

in

the

Jacobi iteration, we

would

A , and

hopefully

B

will have

a

smaller

norm.

The

iteration

scheme

with

this

choice

of C

is

called

Gauss—Seidel

iteration.

It

usually

converges

faster

than

Jacobi iteration.

Gauss—Seidel

Let

L

Iteration

=

a

0

a

.

.

.

a

a

0

0

a

·

·

·

·

·

·

·

·

·

a

0

0

0

0

0

0

0

4

and

Set

C

=

have

D

We

can

solve

coordinate

of

L

D

and

=

a

0

.

.

.

0

U

=

M

=

 

U.

0

a

0

0 a

·

·

·

0 0

.

.

.

0 0

0 0

Let

x

·

·

·

·

·

·

·

·

·

·

be

a

0

0

a

·

a

·

an

0

0

a

a

a

0

 

 

 

 

arbitrary

nonzero

(D

Cx

L) x

Dx

this

last

x

is

equation

given

by

for

x

=

1


a

=

=

=

Mx

+

b

Ux

+

Lx

b

+

Ux

+

b

x

one

coordinate

a

x

+ b

at

vector

in

R

.

We

a

time.

The

first

The

and

second

the

last

coordinate

of

x

n

2

coordinates

can

be

of x

.

solved

for

in

terms

of the

first

coordinate

x

=

1

a

a

x

a

x

+

b

In

(4)

general,

x

=

1

a

a x

a x

+

b

It

is

interesting

to

compare

( 3)

and

(4).

The

difference

between

the

Gauss—Seidel

iterations

is

that

in

the

latter

case,

one

is

using

the

Jacobi and

coordinates

of x as soon

as

they

are

calculated

rather

than

in

the

next

iteration.

The

program

for

the

Gauss—Seidel iteration

the

Jacobi

iteration.

The

vectors

x

is

actually

and

x

simpler

than

the

are

both

stored

program

for

in

the

same

vector, x.

coordinate

As

of

a

coordinate

x .

of

x

is

calculated, it

replaces

the

corresponding

5

Theorem

converges

8. 1 . 2.

If

A is

to

a

solution

of

diagonally

Ax =

b.

dominant,

then

the

Gauss—Seidel

iteration

Proof.

For

α

=

j

=

1

| a

,

.

| ,

Since

A is

diagonally

, n,

let

β

=

| a

| ,

dominant, it

follows

and

that

M

and

We

Let

| a

|

>

α

+

β

consequently

M

<

1

for

j

=

1

,

 

M

=

will show

that

.

, n.

Thus

max M

<

1

B

=

max


x

Bx

x

M

<

1

x be

a

nonzero

vector

in

R

and

let

y

=

Bx.

Choose

y

=

max | y | =

| y

|

=

k

β

( | a

|

so

that

α )

It

follows

from

the

definition

of B

that

 

y

=

Bx =

( D

L) Ux

 

and

hence

 

y

=

D

( Ly

+

Ux)

 

Comparing

the

kth

coordinates

of each

side, we

see

that

y

=

1

a

a

y

a

x

and

hence

(5)

y

=

| y

|

| a

1

| ( α

y

+

β x

)

It

follows

from

( 5)

that

 

Bx

 

y

 

M

 

M

 

x

=

x

Thus

 

Bx

M

1

 

B

=

max


x

x

<

and

hence

the

iteration

will converge

to

the

solution

of Ax =

 

6

b.

x < and hence the iteration will converge to the solution of A x =  

Exercises

1

.

Let

Use

(1

,

1

2. Let

A =

Jacobi

)

. ]

A =

1 0

2

1

1 0

iteration

1

0

0

1

1

0

to

1

1

1

,

b

=

compute

1

1

x

,

b

=

1

2

and

3

2

1

, and

,

x .

[The

and

x

exact

x

=

0

0

solution

=

1

0

0