3

Regularization for Ill-Posed Problems

Chapter 2 showed a class of inverse problems that can be ill-posed, leading

to unstable solutions. Therefore, the issue is how to stabilize the solution and
the cost of stabilization in terms of efficiency and accuracy. To stabilize the
solution at the lowest possible cost is the task of regularization. This chapter
presents a number of often used methods of regularization, including

• Tikhonov regularization (stabilization)

• Regularization by singular value decomposition (null space removal)
• Iterative regularization methods (discrepancy principle)
• Regularization by discretization or projection (operator smoothing)
• Regularization by filtering (noise removal)

The first four methods are standard and described well by Engl et al. (2000);
practical techniques for implementation of the first two regularization meth-
ods are well presented by Santamarina and Fratta (1998). This chapter is
written in reference to these works and offers detailed discussion and exam-
ination on the regularization by discretization or projection and regulariza-
tion by filtering.
Filtering is a common practice to remove noise in measurement data in
experiments for all purposes. Because all types of ill-posedness are often
triggered by the presence of noise, the removal of the noise is naturally the
most effective and practical method to stabilize the solution or mitigate the
ill-posedness and is effective for all types of ill-posedness. Therefore, this
book treats it as a regularization method. Examples will be presented to
demonstrate the effectiveness of the regularization by filtering.
For Type I and Type II ill-posed problems, the root of the ill-posedness is
insufficient or insensitive information used in the inversion (see Table 2.2).
Therefore, the regularization should try to make use of additional informa-
tion to supplement the information. The Tikhonov regularization provides
a way to make use of the information to stabilize the solution.
For Type III ill-posedness, the instability is triggered by noise or error in
the measurement data and a priori information of the noise must be used to
restore stability. The property of the noise consists of two important factors:

© 2003 by CRC Press LLC

 noise level and the frequency (in time domain) or wavenumber (in spatial
domain) of the noise. Any regularization method makes use of the noise
level or the frequency (or wavenumber) of the noise or both. The regular-
ization by discretization (projection) makes use of the frequency (or wave-
number) of the noise and iterative regularization methods make use of the
noise level. The Tikhonov regularization, which is regarded as the most
popular method for ill-posed problems, uses the frequency and the level of
the noise. Regularization by filtering requires the use of the frequency of the
noise to design the filter properly.

3.1 Tikhonov Regularization

3.1.1 Regularizing the Norm of the Solution
A simple solution to the ill-posed problem is to use the so-called damped
least-squares (DLS) method that is one of the methods of Tikhonov regular-
ization. The process of deriving this Tikhonov regularization method is as
follows:
First define the following functional Π in the form of

{ } {Y }
T
Π = Y m − SX m
− SX + αX T X (3.1)

The first term in this function is the L2 norm used in deriving the LS method
for over-posed problems (see Section 2.6.4). The second term is the
Pythagorean length used for deriving minimum length method for under-
posed problems (see Section 2.6.2). The nonnegative α is the regularization
parameter called the damping factor that is used to penalize the Pythagorean
length. The second term prevents the solution having too large length, and
thus controls the stability of the solution. Seeking the minimum error with
the penalty on the solution length requires

∂Π
=0 (3.2)
∂X

which leads to

∂Π
∂X
{ }
= −2ST Y m − SX + 2αX = 0 (3.3)

or

ST Y m = [ST S + αI]X (3.4)

© 2003 by CRC Press LLC

 Note that matrix [ST S + αI] is surely symmetric, and will be invertible due
to the presence of αI . This clearly shows that the effect of the damping
factor α is to increase the positive definiteness of the matrix and improve
the condition of the system matrix. If a very large damping factor is used in
comparison with the diagonal term of matrix ST S , matrix [ST S + αI] will
be very well conditioned, and the condition number will approach 1. The
solution will be very stable at this extreme.
The estimated X can then be found using Equation 3.4 as

X e = [ST S + αI] ST Y m
−1
(3.5)

The definition of the generalized inverse matrix for the damped even- or
over-posed inverse problem becomes

S − g = [ST S + αI] ST
−1
(3.6)

and the output reproducibility matrix is

R O = SS − g = S[ST S + αI] ST
−1
(3.7)

which, in general, may not be an identity matrix. Therefore, the DLS solution
will generally not be output reproducible.
On the other hand, the input reproducibility matrix is

R I = S − g S = [ST S + αI] ST S
−1
(3.8)

which also will not be the identity matrix, unless α vanishes. This implies
that the DLS solution is not input reproducible and does not give the true
estimation of the input of the system. The accuracy of the estimation depends
on the damping factor used.
Note that a large damping factor is preferred when trying to improve the
condition of the equation system of the inverse problem. In improving the
accuracy of the estimation, however, a small damping factor must be used;
therefore, the analyst needs to compromise. The guideline should be to use
the smallest damping factor that is just enough to prevent the ill conditioning
of the matrix [ST S + αI] . Therefore, the damping factor used is often very
small. The so-called L-curve method (Hansen, 1992) has been used for deter-
mining α .
Note also that the presence of a small α in matrix [ST S + αI] effectively
increases these smallest eigenvalues of the matrix. Therefore, it reduces the
largest eigenvalues of the matrix [ST S + αI]−1 in Equation 3.5 that give the

© 2003 by CRC Press LLC

 solution of the estimation. Therefore, α effectively acts to attenuate the high
frequencies in the solution and has earned the name of damping factor.

3.1.2 Regularization Using Regularization Matrix

Next, the so-called regularized least-squares (RLS) method is formulated. The
process to derive the RLS method is as follows:
First define the functional Π in the form of

{ } {Y }
− SX + α[RX] [RX]
T
Π = Y m − SX m T
(3.9)

The first term in this function is the L2 norm of the error between the
prediction of the forward model and the measurements, which is the same
as that used in deriving LS method for over-posed problems. The second
term is the regularization term where R is the regularization matrix, and α
is the regularization factor. The regularization matrix contains the a priori
information about the unknown X, and the regularization factor controls the
degree of the regularization. The formation of the regularization matrix will
be detailed in the next section.
In seeking for the minimization condition of the regularized function,
Equation 3.9 leads to

∂Π
∂X
{ }
= −2ST Y m − SX + 2αR T RX = 0 (3.10)

which gives

ST Y m = [ST S + αR T R]X (3.11)

It is clear that matrix [ST S + αR T R] is surely symmetric and usually will

be invertible due to the presence of αR T R . The addition of the matrix αR T R
increases the positive definiteness of the matrix and hence improves the
condition of the system matrix. It can be easily seen that, if the regularization
matrix is an identity matrix, the RLS method becomes the DLS method, and
if α = 0, the RLS method becomes the LS method.
From Equation 3.11, the estimated X can then be found as

X e = [ST S + αR T R] ST Y m
−1
(3.12)

and the definition of the generalized inverse matrix for the damped even-
or over-posed inverse problem becomes

© 2003 by CRC Press LLC

 S − g = [ST S + αR T R] ST
−1
(3.13)

The output reproducibility matrix is

R O = SS − g = S[ST S + αR T R] ST
−1
(3.14)

which, in general, may not be an identity matrix. Therefore, the DLS solution
will generally not be output reproducible.
On the other hand, the input reproducibility matrix is

R I = S − g S = [ST S + αR T R] ST S
−1
(3.15)

which also will not be the identity matrix, unless α vanishes. This implies
that the DLS solution is not input reproducible and does not give the true
estimation of the input of the system. The accuracy of the estimation depends
on the regularization factor used. The guideline for selecting the regulariza-
tion factor should be to use the smallest regularization factor that is just
enough to prevent the ill conditioning of the matrix [ST S + αR T R] ; therefore,
the damping factor used is often very small. The so-called L-curve method
(Hansen, 1992) may be used for determining α .

3.1.3 Determination of the Regularization Matrix

Using Equation 3.9 means basically trying to regularize RX to the degree
controlled by α . Therefore, it is necessary to decide what in the unknowns
should be regularized, for which information about X is needed. Often the
information of neighboring components of X is used. The finite difference
formulations are therefore often used to create matrix R (Press et al., 1989;
Santamarina and Fratta, 1998).

• Case 1 — if X should be more or less constant, the gradient there

should then be regularized. In this case the ith row ri in the regular-
ization matrix R should be

[
ri = 0
0 −
1
i −1
1
i
0
0
] (3.16)

so that

ri X = xi − xi −1 (3.17)

© 2003 by CRC Press LLC

 This implies that the difference between the two neighboring com-
ponents will be regularized.
• Case 2 — if the gradient of X should be more or less constant, the
first derivative should then be regularized. In this case, ith row ri in
the regularization matrix R should be

[
ri = 0
0 1
i −1
−
2
i
1
i +1
0
0
] (3.18)

so that

ri X = xi +1 − 2 xi + xi −1 (3.19)

This implies that the second derivative will be regularized.

• Case 3 — if the second derivative of X should be more or less
constant, the second derivative should then be regularized. In this
case, ith row ri in the regularization matrix R should be

ri = 0
[
0 1
i−2
−
3
i −1
3
i
−
1
i +1
0
0
] (3.20)

so that

ri X = xi −2 − 3xi −1 + 3xi − xi +1 (3.21)

This implies that the third derivative will be regularized.

• Case 4 — the same method can be extended to form regularization
matrices of two-dimensional cases of regular grids. For example, to
regularize the Laplacian

∂2x ∂2x
+ (3.22)
∂ξ 2 ∂η2

In this case, ith row ri in the regularization matrix should be

ri = 0
[
1
1
i −1
−
4
i
1
i +1

1
0
]
ξ →0 1 0↓ η (3.23)
1 −4 1
0 1 0

© 2003 by CRC Press LLC

 3.1.4 Tikhonov Regularization for Complex Systems
The Tikhonov regularization method can also be implemented for complex
systems that cannot be formulated explicitly in a matrix form by adding the
regularization functional terms into the object function of error. Optimization
methods are then used as usual to minimize the modified object function to
obtain the inverse solution. Tikhonov regularization is the most effective and
powerful regularization method for many inverse problems arising from
complex systems. The choice of the regularization parameter α , however,
is not always straightforward.

3.2 Regularization by SVD

The method of regularization by singular value decomposition (SVD) is a
very straightforward method applicable when no additional information on
the solution is available. It is a mathematic means to obtain a stable solution;
therefore, there is no guarantee on the quality of the solution. It is used when
the SVD method described in Section 2.7 is used to solve the inverse prob-
lems. The solution is then given by Equation 2.119.

r
ΨiT Y
X = S− g Y = ∑
i =1
λi
Φi (3.24)

where λi (i = 1, 2,…, r = Rank(S)) are the nonzero eigenvalues of matrix SST

or STS. In actual practice of solving ill-posed inverse problems, there is an
issue on how to define “nonzero” because a number of eigenvalues can be
very close to zero but they are not. It is clearly seen from Equation 3.24 that
the inclusion of small value of λi will magnify the measurement (output)
error in Y, which is the source of the ill-posedness of the problem. Therefore,
choosing a sufficiently large cut-off eigenvalue can effectively obtain the
stabilized inverse solution. However, it is often difficult to make the decision
on cutting off the frequencies, which should be related somehow to the
information on the noise. Engineering experience and judgment and trial
and error are needed to make a proper decision. Computation of the output
and input reproducibility matrices can reveal the reproduce property and
hence help to make a decision.
In summary, Table 3.1 gives several different representations of S − g for
different cases. In this table, [S Y] is an expanded matrix by adding Y as
the last column of S , and R is a regularization matrix.

© 2003 by CRC Press LLC

 TABLE 3.1
Formulation and Solution for Ill-Posed Inverse Problems with Regularization
Type Criterion Formulation
Under-posed (Rank[S] = Minimum length solution [S]–g = ST(SST)–1
Rank(SY)
Over-posed Least-squares solution S–g = (STS)–1ST
Mixed-posed Damped least-squares solution S–g = (STS + αI)–1ST
Information about the solution Regularized least-squares S–g = (STS + αRTR)–1ST
is available solution
General Singular value decomposition S–g = VΛ–1UT (S = UΛV–1)
Functional of error Discrepancy principle Minimize (Π(X) + αδ)
Note: For well-posed problems, α = 0.

3.3 Iterative Regularization Method

The iterative regularization method can be used when iterative methods (see,
for example, Santamarina and Fratta, 1998; Engl et al., 2000) are used to solve
inverse problems by minimizing the functional of errors. The method is
based on the so-called discrepancy principle and is applicable to systems
with or without explicit matrix form. This method works in obtaining a stable
solution when the information on the level of the measurement noise is
available.
The goal of solving an inverse problem is to solve

SX = Y (3.25)

for X that is a vector of inputs, and Y is the measured outputs. A functional

can then be defined:

Π (X) = SX − Y (3.26)

where defines a norm. Y is measured and hence will be noisy; however,

it is possible to have

Y − Ym ≤ δ (3.27)

where δ represents the magnitude or the level of the noise or discrepancy. The
functional of error given in Equation 3.26 can be rewritten as

Π (X) = SX − Y m (3.28)

© 2003 by CRC Press LLC

 Because the measurement has the error of δ, in seeking for the minimum
of the preceding function, there is no point to looking for an X that gives the
Π (X) in the order of δ. Therefore, when an iterative method is used to
minimize Π (X) , it is necessary to stop iterating any further as soon as Π (X)
reaches αδ, where α > 1 (e.g., α = 1.1). Seeking any Π (X) smaller than αδ
is physically meaningless.
Therefore, the discrepancy principle states that when an iterative method is
used to minimize the function of error, the iteration process should stop when

Π (X) = αδ (3.29)

This is the stopping criteria for the iteration process that prevents triggering
the ill-posedness of the problem leading to an unstable solution.
The iterative regularization method is, in principle, also applicable to
inverse problems that are formulated in the form of functional of errors, as
shown in Table 3.1.
Iterative method using the sensitivity matrix-based method will be
addressed in detail in Section 14.3. A total solution of engineering problems
will be received.

3.4 Regularization by Discretization (Projection)

For the detailed theoretical background of this regularization, readers are
referred to the book by Engl et al. (2000). This section demonstrates how
regularization by discretization or projection works with a simple problem
of the force estimation of a clamped bar.

3.4.1 Exact Solution of the Problem

Consider a clamped bar of length L and cross-sectional area A, subjected to
a body force b(x) continuously acting along the bar, as illustrated in Figure
3.1. Suppose that the material of the bar is homogeneous and the cross section

E, A,L

O
x
b (x)

FIGURE 3.1
A clamped straight uniform bar subjected to a distributed body force b(x). The bar is made of
a homogeneous material with the elastic Young’s modulus of E and constant cross section of
A; the length of the bar is L.

© 2003 by CRC Press LLC

 A is constant. The governing differential equation for this static problem is
given as:

d 2u
EA + b( x) = 0 (3.30)
dx 2

where E is the elastic Young’s modulus, and u denotes the displacement.

The boundary conditions are:

du
u( x = 0) = 0; EA =0 (3.31)
dx x = L

In this study, the body force b(x) is assumed to be one-quarter sine function
along the bar as

 π 
( )
b( x) = sin ω f x = sin x
 2L 
0≤x≤L (3.32)

Also, L = 0.4 , EA = 1.0 are given for obtaining the numerical results.
The exact solution of Equation 3.30, considering the boundary conditions
of Equation 3.31, can be easily obtained as

( )
1
u a (x) = sin ω f x 0≤x≤L (3.33)
ω 2f

Correspondingly, the member force Q can be derived from Equation 3.33 as

( )
du 1
Q a ( x) = −EA =− cos ω f x 0≤x≤L (3.34)
dx ωf

3.4.2 Revealing the Ill-Posedness

In order to reveal the ill-posedness clearly and for further comparison with
the result after the regularization, the process described in Section 2.3.2 is
repeated here. Consider the inverse problem of estimating the force of the
bar. In this case, the distribution of the displacements of the bar is known
somehow, as well as the boundary conditions in Equation 3.31, the cross
section, and the elastic modulus. Assume now that the measurement of the
displacements contains a noise of low level but high frequency as

u m = u a + e sin(ω noise x) 0 ≤ x ≤ L (3.35)

© 2003 by CRC Press LLC

 where ω noise = 30π , e = 2 / umax * 0.01 and umax is the maximum value of
the displacement in the bar. Note that the magnitude of the noise is constant
in 0 ð x ð L.
The simulated measurement given in Equation 3.35, as well as the exact
displacement given in Equation 3.33, is plotted in Figure 3.2. This figure
shows that the measured displacement (dashed line) is very accurate com-
pared to the actual value (solid line). This simulated measurement with very
low noise level can be used to estimate the force.
Substituting Equation 3.35 into Equation 3.34 obtains the estimated inter-
nal force analytically

( )
1
Qae ( x) = − cos ω f x − eω noise cos(ω noise x) 0 ≤ x ≤ L (3.36)
ωf

Qa ( x )

or

Qae ( x) = Q a ( x) − e ω noise cos(ω noise x) 0 ≤ x ≤ L (3.37)


factor of magnification

Clearly, the inverse solution for the estimation of the internal force is mag-
nified drastically by the frequency ω noise of the noise of the measurement.
This magnification is shown in Figure 3.3 with the comparison between the

0.07
actual displacement
0.06 noise-contaminated displacement

0.05

0.04
u
0.03

0.02

0.01

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
x

FIGURE 3.2
Comparison of the actual displacements and the simulated one by adding a noise of low level
(1%) but high frequency (ωnoise = 30π) for the clamped bar.

© 2003 by CRC Press LLC

 0.05
inverse solution with noise-free displacement
inverse solution with noise-contaminated displacement
0

-0.05

-0.1
Q
-0.15

-0.2

-0.25

-0.3
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

x
FIGURE 3.3
Inverse solution of the internal force using the simulated measurement with 1% oscillatory
noise-contaminated in displacement data.

exact internal force and the estimated results obtained with the noise-con-
taminated displacement.
The estimated external body force could also be obtained in the following
close form


(
bae ( x) = sin ω f x + e) ω noise
 factor of
2

sin(ω noise x) 0 ≤ x ≤ L (3.38)
magnification
b( x )

The inverse body force is even magnified by ω noise

2
times as specified in
Equation 3.38. This inverse problem has the Type III ill-posedness, as was
observed in Section 2.3.

3.4.3 Numerical Method of Discretization for Inverse Problem

As illustrated in Section 3.4.2, the inverse problem for force estimation of
the bar could be analytically solved using the known displacements; the
Type III ill-posed problem has been also revealed in this example. However,
most engineering problems cannot be solved in a close form formulation.
They are normally solved using a method of discretization, such as the
finite element method, mesh-free method, finite difference method, etc. The
same inverse problem studied in the previous subsection is now solved
using the FEM.

© 2003 by CRC Press LLC

 3.4.3.1 Finite Element Solution
The finite element equation can be written as

ku = f (3.39)

where k is the stiffness matrix, f is the nodal force vector, and u is the nodal
displacement vector for the element. The detailed procedure that leads to
Equation 3.39 can be found in any textbook for FEM, such as the one by Liu
and Quek (2003).
Assume the bar is divided into N elements with a typical element defined
in Ω i = ( xi , xi +1 ) , whose nodes are at x = xi and x = xi +1 . For the choice of
linear shape functions,

x i +1 − x x − xi
ψ i1 (x) = ; ψ i 2 (x) = x i ≤ x ≤ x i +1 (3.40)
x i +1 − x i x i +1 − x i

yields

 xi + 1

k=
EA  1 −1
=


∫xi
f ( x)ψ 1 ( x)dx 

l −1 1 
, f xi + 1 (3.41)

 ∫xi
f ( x)ψ 2 ( x)dx 


where l = xi +1 − xi .
The assembled equations for all elements in the whole bar can be
obtained as

KU = F (3.42)

The matrices K, U, and F are obtained by assembling k, u, and f from the

connectivity of the elements. The displacements on the node of the bar can
be obtained by solving Equation 3.42 after the imposition of the displacement
boundary condition.
The comparison of the FEM results and those from the exact solution given
in Equation 3.33 has been carried out. The results are plotted in Figure 3.4,
in which the FEM results are obtained using 24 elements of equal length. It
is shown that the FEM results are very accurate.

3.4.3.2 Inverse Force Estimation

Now the nodal force F with the known displacements U can be “inversely”
obtained using Equation 3.42. From the nodal force F, the internal force Q(x)
can be estimated using

© 2003 by CRC Press LLC

 0.07
Exact solution
0.06 FEM solution

0.05

0.04
u
0.03

0.02

0.01

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
x

FIGURE 3.4
The finite element solution with 24 elements for the clamped bar as illustrated in Figure 3.1.

Q fe ( x1 = 0) = Ffe ( x1 = 0)
(3.43)
( ) ( )
Q fe x j = Q fe x j −1 + Ffe x j ( ) j = 2, … , N

where the subscript f stands for the value estimated from the FEM solution.
Using the FEM model, the continuous domain of the problem is projected
into a discretized space. The so-called regularization by discretization or
projection is expected to mitigate the ill-posedness of the inverse problems
to a certain degree and to reduce the factor of magnification of error. The
next section reveals the effect of the discretization regularization via the
simple force estimation of the bar from the displacements.

3.4.4 Definition of the Errors

To investigate the accuracy of the inversely estimated results of the internal
force using the FEM model, as well as to compare them with the directly
analytical solution, the following error norms between the estimated and the
exact solutions are defined

max Qia − Qaie

εa = , for all i
ε
(3.44)
max Qia − Q fie
εt = , for all i
ε

© 2003 by CRC Press LLC

 max Qia − Q fia
εm = , for all i
ε
(3.44 continued)
max Q − Q
a
fi
e
fi
εf = , for all i
ε

where

Qia — the estimated internal force at the ith node obtained using the
analytical Equation 3.34 without noise in the displacement
Qaie — the estimated internal force at the ith node obtained using the
analytical Equation 3.36 with noise in the displacement
a
Q fi — the estimated internal force at the ith node obtained using the
FEM model without noise in the displacement
Q fie — the estimated internal force at the ith node obtained using the
FEM model with noise in the displacement
ε — the reference error defined by

ε = eω noise (3.45)

which is the maximum error of the estimated internal force in the bar
obtained using Equation 3.36.
Using Equation 3.34 and Equation 3.36, the error of the estimated internal
force-based analytical formula using the noisy displacement becomes

εa = 1 (3.46)

For convenience, εa is termed the analytical inverse error. All the other errors
are in relation to εa:

εf stands for the maximum error of the estimated force in the bar using
the FEM model and noisy displacement (short for FEM inverse
error).
ε m stands for the maximum error from the FEM model (short for FEM
model error).
ε t stands for the total maximum error of the estimated force and com-
bines the FEM model error and the FEM inverse error. Thus

εt = εm + ε f (3.47)

© 2003 by CRC Press LLC

 1.4

1.2

Error 0.8

0.6
Best number of εa
divison of elements
0.4 εm
εt
0.2 εf

0
0 20 40 60 80 100 120 140
Number of element

5x10-4

u noise 0

-5
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

FIGURE 3.5
Errors involved in the inverse analysis of the internal force in a clamped bar. εa: analytical
inverse error; εf : FEM inverse error; εm: FEM model error; εt = εm + εf: total error. The frequency
of the noise in the displacement is ωnoise = 60π; that is the factor of error magnification. The FEM
discretization can always reduce the inverse error compared to the analytical inverse error εa.
This error reduction effect is termed regularization by discretization or projection and is more
effective when the number of division of the bar is small; the effect reduces with the increase
of the number of the elements. The FEM mode error, however, increases with the decrease of
the number of the elements. The best division of element should correspond to the minimum
total error point. In a special case the FEM inverse error drops to zero when the element number
is 24 due to the special element division that picks up no noise.

3.4.5 Property of Projection Regularization

Figure 3.5 shows the errors involved in the inverse analysis of the internal
force in a clamped bar, for different numbers of element divisions. The
frequency of the noise is ω noise = 60π .

• The FEM discretization can always reduce the inverse error com-
pared to the analytical inverse error ε a . This error reduction effect
is termed regularization by discretization or projection.
• Regularization by discretization is more effective when the number
of division of the bar is small, and the effect reduces with the increase
of the number of the elements.

© 2003 by CRC Press LLC

 0.1
True solution
0.05 Analytical estimation
FEM estimation with 20 elements
0 FEM estimation with 100 elements

-0.05

-0.1
Q
-0.15

-0.2

-0.25

-0.3

-0.35
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
x

FIGURE 3.6
Estimated force using FEM with different numbers of the elements. The frequency of contam-
inated noise is ωnoise = 60π. Clearly, using a fewer number of elements leads to better solution .
The estimated results with too many elements will approach the analytical estimation. This
figure shows that the projection regularization only works well at coarse discretization.

• The FEM mode error, however, increases with the decrease of the
number of the elements.
• The best division of elements should correspond to the minimum
total error point. Therefore, in the context of inverse analysis, it is
not always true that the use of more elements leads to a more
accurate solution.

In a special case the FEM inverse error drops to zero when the element
number is 24 because of the special element division that picks up no noise.
Generally, this will not happen.
Figure 3.6 gives the estimated force from the noisy displacement using the
FEM with different numbers of the elements. In this estimation, the fre-
quency of the contaminated noise is ω noise = 60π . Two divisions of 20 even
elements and 100 even elements are used in the analysis. It can be clearly
found from this figure that using fewer elements (20) obtains a better inverse
solution than using more elements (100).

3.4.6 Selecting the Best Mesh Density

The FEM inverse error increases with the density of discretization. In con-
trast, the FEM model error will decrease when more elements are used in
the analysis. Thus the question now is how to select the best division in an
inverse analysis to achieve least total error.
Figure 3.5 shows that the number of elements should be at the deep point
(marked with a dot) of the total error curve, where the total error is minimum.

© 2003 by CRC Press LLC

 At this point, the total error is around one fifth of the analytical error ε a .
For this problem of force estimation, the best density of discretization can
be determined easily because the force is given. However, for practical engi-
neering problems, this will be very difficult because the forces are not known
and all the curves cannot be drawn. The best density of discretization can
only be found if some information about the force is known in terms of
rough distribution and the frequency (wavenumber). This is often possible,
as shown in examples in later chapters.

3.5 Regularization by Filtering

Filtering is the most effective and straightforward method of regularization.
Because all the ill-posedness is triggered by the presence of the noise,
removal of noise naturally works best. In practice, the data should always
be filtered before they are used for the inverse analysis. The basic idea is to
remove noise whose frequency is higher than that of the true outputs
(effects). The filtering can be carried out simply by selecting a proper filter
from the filter bank (see Strong and Nguyen, 1996) or directly using the
signal processing toolbox provided in the commercial software MATLAB.
The following gives a method of the smoothing with moving average method
to treat the noisy data for inverse analysis.
High-frequency noise can be reduced by running a moving average. The
ith smoothed value of the signal is computed from the original noisy signal
u as an average of neighboring values around i. A kernel h of m-values (m
is an odd number) contains the weights to be applied to neighboring ele-
ments. The sum of all weights in a smoothing kernel equals 1. The filtered
data uF can be written as

k =( m−1)/2

uiF = ∑
k =− ( m−1)/2
hp ui + k (i = 1, 2, … , N ) (3.48)

where h is the kernel with m parameters.

1 2 4 6 4 2 1
A kernel, h =  , and m = 7 are
 20 20 20 20 20 20 20 
selected for the study. Two kinds of assumed noises will be considered in
the following application of the force estimation: high-frequency sine noise
and the Gaussian noise along the bar.

3.5.1 Example I: High-Frequency Sine Noise

Considering the force estimation of the clamped bar as studied in Section
3.4.1, the noise is given in Equation 3.35. Figure 3.7 shows the estimation of

© 2003 by CRC Press LLC

 0.05

-0.05

-0.1

Q -0.15

-0.2

-0.25
With filtering
Without filtering
-0.3 Analytical estimation

-0.35
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

FIGURE 3.7
Comparison of estimated force with and without filtering for the displacement containing a
sine function noise as specified by Equation 3.35. The FEM solution is obtained using 20
elements.

the force using FEM with 20 elements. In this figure, the estimated results
with the filtered displacement are plotted in a dashed line, the estimated
results with the noise-contaminated displacements are plotted in a long
dashed-dot line, and the true results are plotted in a solid line as the reference
of the comparison of the estimated results. This figure shows that the esti-
mated results with filtered displacements are very close to the exact ones
and that the oscillation of the estimated results has been mitigated with the
filtered displacement.

3.5.2 Example II: Gaussian Noise

Gaussian noise (see Chapter 2) is directly added to the computer-generated
displacements to simulate the noise contamination. The Gaussian noise is
generated from a Gaussian distribution with a zero mean and standard
deviation of b = ( 2 / 2)umax * 0.01, as shown in Figure 3.8. Then the noise-
contaminated displacement, obtained by adding the Gaussian noise to the
displacement obtained using FEM with 100 elements, is used to simulate the
measurement. Figure 3.8 shows that the Gaussian noise is random in nature
and much more complex in distribution than the sine function noise along
the bar.
Applying the filter on the noisy displacement, inverse analysis is then
performed. Figure 3.9 gives the estimated force results together with those
obtained using the unfiltered noisy displacements. The estimated results
with the filtered displacements (dashed line) are much more accurate and
very close to the exact results.
Designing a good filter requires a good understanding of the noise nature
of the measurement data, as well as some information on the frequency (or
wavenumber) of the solution. If the possible frequency of the noise is known

© 2003 by CRC Press LLC

 -3
x 10
1.5

Gaussian noise
0.5

-0.5

-1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

FIGURE 3.8
Gaussian noise generated with a zero mean and standard deviation of b = ( 2 / 2)umax * 0.01
for simulating the measured displacement of the clamped bar.

with filter
true solution
without filter

FIGURE 3.9
Comparison of estimated force of the bar with and without filtering for the displacement
containing the Gaussian noise. The FEM solution is obtained using 100 elements.

© 2003 by CRC Press LLC

 from the experimental setup, as well as the fact that the solution will not
contain such a frequency, a filter can always be designed to filter out the
noise before the data are used for the inverse analysis.

3.6 Remarks

Remark 3.1 — a set of regularization methods for ill-posed inverse prob-

lems has been presented. The Tikhonov regularization is considered
the most popular regularization method for ill-posed problems.
Remark 3.2 — projection regularization works well for coarse mesh used
in the discrete methods. In contrast, the model error will decrease
when the density of discretization increases. The best density of
discretization should lead to least total error.
Remark 3.3 — practical engineering inverse problems are often nonlin-
ear and of high dimension, so it is difficult to develop a “magic”
regularization method that works for all the ill-posed problems.
Performing a proper regularization requires a good understanding
of the nature of the ill-posedness of the inverse problem, and pre-
vention is always better than cure. Some ways of prevention that
cannot be over emphasized include:
• The reduction of the numbers of parameters to be inversely de-
termined in an inverse procedure is always the first effort to
make. The search bounds of these parameters should be as nar-
row as possible.
• To reduce the Type I ill-posedness, the number of the knowns
(measurements) should be at least more than the number of
unknowns to be inversely determined, leading to at least an even-
posed problem. Efforts should be made to make the problem
over-posed.
• To reduce the Type II ill-posedness, ensure a high sensitivity of
the effects to the parameters (including the inputs) to be inversely
identified. The parameters should be influential to the effects and
as independent as possible.
• When a forward solver of discretization is used, coarse mesh is
preferred as long as it gives slightly more accurate results com-
pared to the experiments. This can reduce the Type III ill-pos-
edness.
• Filter the measurement data before they are fed to the inverse
analyses. This can very effectively reduce the ill-posedness.

© 2003 by CRC Press LLC

 • Use discrepancy principle as the stopping criterion to avoid trig-
gering the ill-posedness.
• As the last resort, use the Tikhonov regularization to restore the
stability of the inverse solution.

© 2003 by CRC Press LLC