`

Curve Fittings, B-Splines

& Approximations
Contents:

Page

51

o Curve Fitting & Regression: Introduction

o Least Square Regression Technique
o Fitting Linear Equations
o Fitting Transcendental Equations
o Fitting Polynomial Equations
o Multiple Linear Equations
o Spline Interpolation: Introduction
o Cubic B-Splines
o Approximation of Functions
o Assignment 4

Prepared By
Er. Shree Krishna Khadka

CURVE FITTING & REGRESSION: INTRODUCTION

Previously, we have discussed the methods of curve fitting for the data points of
well defined functions. In this topic, we will discuss on methods of curve fitting for
experimental data. In many applications it often becomes necessary to establish a
mathematical relationship between experimental values. The mathematical
equation can be used to predict values of the dependent variables. For example:
we would like to know the maintenance cost of equipment as a function of age or
mileage.
The process of establishing such a relationship in the form of mathematical model
is known as regression analysis or curve fitting. In this topic, we shall discuss a
technique known as least square regression to fit the data under the following
situation.
a) Relation is linear
b) Relation is transcendental
c) Relation is polynomial

LEAST SQUARE REGRESSION

Let us consider the mathematical equation for a straight line: y = f(x) = a + bx, to
describe the data. We know that a is the intercept of the line and b is its slope.
f(x)

f(x) = a + bx
ai

(Xi, yi)

a
x
Fig: Least Square Regression

Consider a point (xi, yi) as shown in figure. The vertical distance of this point from
the line f(x) = a + bx is the error: ai. Then ai = yi f(xi) = yi a bxi .. (i)
The best approach that could be tried for fitting a best line through the data is to
minimize the sum of squares of errors.
ai 2 = (yi a bxi )2

Page

The technique of minimizing the sum of square of errors is known as least square
regression.

52

i.e.

Prepared By
Er. Shree Krishna Khadka

FITTING LINEAR EQUATIONS

Let, the sum of square of individual errors be as:
Q=

n
2
i=1 a i

n
i=1(yi

a bxi )2 (i)

In this method of least square, we choose a and b such that Q is minimum. Since,
Q depends on a and b, a necessary condition for Q to be minimum is given by:
Q

=0

&

a bxi ) = 0

&

=0

Then:
Q
a

i.e.

= 2

n
i=1(yi

yi = a + b

xi . (ii)

&

= 2xi

xi yi = a

n
i=1(yi

a bxi ) = 0

xi + b

xi 2 (iii)

Equations, (ii) and (iii) are called normal equations, solving for a and b, we get:

b=

xi yi xi yi
n x i 2 ( x i )2

. (iv)

&

a=

yi
n

xi
n

= y bx (v)

Example
Fit a straight line to the following set of data.
x 1 2 3 4 5
y 3 4 5 6 8
Solution:
The various summations are calculated as below:

1
4
9
16
25
xi 2 = 55

3
8
15
24
40
xi yi = 90

Here, n = 5
On calculation using above formula, we get: b = 1.2 and a = 1.6
Therefore, the required linear equation is: y = a + bx = 1.6 + 1.2x
Prepared By
Er. Shree Krishna Khadka

53

yi
3
4
5
6
8
yi = 26

Page

xi
1
2
3
4
5
xi = 15

FITTING TRANSCENDENTAL EQUATIONS

The relationship between dependent and independent variables is not always
linear. The non-linear relationship may exist in the form of transcendental
equation. For example, the familiar equation of population growth is given by:
= (i)
Where, P0 is the initial population, K is the rate of growth and t is time.
Another example is the non-linear model in which relation is established between
pressure and volume.
= (ii)
Where, P is pressure and v is volume. If we observe the values of P for the
various values of v, we can then determine the parameter a and b.
The problem of finding the values of a and b can be solved by using the
algorithm given for linear equation. Let us rewrite the equation using the
conversion of variable as x and y. So, the equation becomes:
= (iii)
Taking log on both sides, we get:
ln y = ln a + bln(x)

(iv)

Now, comparing equation (iv) with standard linear equation, y = a + bx, we found:
y ln y , b b, a ln a &
() . (v)
Then, obviously we will have the following results:
ln x i ln (y i ) ln (x i ) ln (y i )
n {ln (x i )}2 { ln (x i )}2

ln a = R =

i.e.

ln (y i )
n

a = eR = exp

. (vi) &

ln (x i )
n

ln(yi )
ln(x )
b n i
n

. (vi)

Prepared By
Er. Shree Krishna Khadka

54

Page

b=

Example:
Given the data table:
x 1 2 3 4
5
y 0.5 2 4.5 8 12.5
Fit a power function model of the form y = axb.
Solution:
xi
1
2
3
4
5
= 15

yi
0.5
2
4.5
8
12.5
= 27.5

ln(xi)
0
0.6931
1.0986
1.3863
1.6094
= 4.7869

{
( )} ln(xi).ln(yi)
0
0
0.4804
0.4804
1.2069
1.6524
1.9218
2.8827
2.5901
4.0649
= 6.1992
= 9.0804

ln(yi)
-0.6931
0.6931
1.5641
2.0794
2.5257
= 6.1092

Now:

b=

ln(a) =

ln x i ln (y i ) ln (x i ) ln (y i )
n {ln (x i )}2 { ln (x i )}2
ln (y i )
n

ln (x i )
n

27.5
5

=b=

59.0804 4.78696.1092

2.2832

56.19924.7869 2
15
5

= 2.2832

= 0.9641

i.e. a = e-0.9641 = 0.381

Finally, the power form equation is: y = 0.381.x2.2832
Home Work
The temperature of a metal strip was measured at various time intervals
during heating and the values are given in the table below:
Time t(min)
12 2
3
4
o
Temperature T( C) 70 83 100 124

Prepared By
Er. Shree Krishna Khadka

Page

Solution:
Hint: As T = b.et/4 +a (i), we can write this equation in the form: y = b.f(x) +a .
(ii), where, y = T and f(x) = et/4. Now, use linear formula and adjust the values at
final. Ans: 37.56et/4+21.31

55

If the relation between the time t and temperature T is of the form: T =

b.et/4 + a, estimate the temperature at t = 6 minute.

FITTING A POLYNOMIAL EQUATION

When a given series of data does not appear to satisfy a linear equation, we can try
a suitable polynomial as a regression to fit the data. The least square technique can
be used to fit the data to a polynomial
Consider the polynomial of degree (m-1) as below:
y = a1 + a2 x + a3 x 2 + a4 x 3 + . +am x m1 (i)
If the data contains n sets of x and y, then the sum of squares of the error is
given by:
Q = ni=1{yi f xi }2 . (ii)
Hence; f(x) is a polynomial and contain coefficients: a1, a2,.,am. We have to
estimate all m coefficients. As before, we have the following m equations that can
be solved for these coefficients.
i.e.

Q
a 1

= 0;

a 2

= 0; . a = 0;
m

Consider a general term:

Q
a j

= 2

n
i=1{yi

f xi }

f(x i )
a j

=0;

Where:

f(x i )
a j

= xi j1

Thus we have:
n
i=1{yi

Or

n
i=1

f xi } xi j1 = 0 ;

yi . xi j1 xi j1 . f xi

= 0;

Where: j = 1, 2, . m
. (iii)

Now, Substituting for f(xi):

n
j1
(a1
i=1 xi

+ a2 x + a3 x 2 + a4 x 3 + . +am x m1 ) =

n
j1
i=1 yi . xi

yi
= xi yi
..
..
xi 2m2 = a3

Prepared By
Er. Shree Krishna Khadka

xi m1 yi

Page

a1 n + a2 xi + a3 xi 2 + + am xi m1 =
a1 xi + a2 xi 2 + a3 xi 3 + + am xi m1

..

..
m1
m
m+1
a1 x i
+ a2 xi + a3 xi
+ + am

56

So, there are m equations: (j=1, 2, 3, . ,n) and each summation is for i =1 to i =n.

#Note: For a second order polynomial, we will have the following simultaneous
equations:
a1 n + a2 xi + a3 xi 2 = yi
a1 xi + a2 xi 2 + a3 xi 3 = xi yi
a1 xi 2 + a2 xi 3 + a3 xi 4 = xi 2 yi
Example:
Fit a second order polynomial to the data in the table below:
x 1 2 3 4
y 6 11 18 27
Solution:
Below is the tabulation the values as second order polynomial components.
x
1
2
3
4
= 10

y
6
11
18
27
= 62

x2
1
4
9
16
= 30

x3
1
8
27
64
= 100

x4
1
16
81
256
= 354

xy
6
22
54
108
= 190

x 2y
6
44
162
432
= 644

So, we have the following simultaneous equations:

4a1 + 10a2 + 30a3 = 62
(i)
10a1 + 30a2 + 100a3 = 190 (ii)
30a1 + 100a2 + 226a3 = 644 (iii)
On calculation: a1 = 3, a2 = 2 and a3 = 1
Hence, the least square quadratic polynomial is: y = 3 + 2x + x2.
Home Work:
From the given data table, fit a second order polynomial.
x 1 2.1 3.2 4
y 2 2.5 3.0 4

Prepared By
Er. Shree Krishna Khadka

Page

The polynomial is: y = 2.074 -0.205x + 0.168x2

57

Equations:
4a1 + 10.3a2 + 31.65a3 = 11.5
(i)
10.3a1 + 31.65a2 + 107.029a3 = 32.85
(ii)
31.65a1 + 107.029a2 + 381.3057a3 = 107.745 (iii)

MULTIPLE LINEAR REGRESSIONS:

There are number of situations, where the dependent variable is a function of two
or more variables. For example, the salary of a sales person may be expressed as: y
= 500 + 5x1 + 8x2, where x1 and x2 are the number of unit sold of product 1 and 2
respectively. We shall discuss an approach to fit the experimental data, where the
variable under consideration is a linear function of two independent variables.
Let us consider a two variables linear function as follows:
y = f x, z = a1 + a2 xi + a3 zi (i)
Then, the sum of square of errors is given by:
Q=

n
i=1{yi

f x, z }2 =

n
i=1{yi

a1 + a2 xi + a3 zi }2 (ii)

Differentiation equation (ii) w. r. t. a1, a2, a3 and equating them to zero, we will
get the condition for minimum error.
Q
a 1
Q
a 2
Q
a 3

= 2 {yi a1 + a2 xi + a3 zi } = 0
= 2 {yi a1 + a2 xi + a3 zi xi } = 0
= 2 {yi a1 + a2 xi + a3 zi zi } = 0

With these conditions, we will get the following simultaneous equation:

a1 n + a2 xi + a3 zi = yi
a1 xi + a2 xi 2 + a3 zi xi =
a1 zi + a2 xi zi + a3 zi 2 =

xi yi
yi zi

Example:
Given the table of data:
x 1 2 3
4
z 0 1 2
3
y 12 18 24 30

Constructing the table as a component of multiple linear regression equation.

Prepared By
Er. Shree Krishna Khadka

Page

58

Solution:

x
1
2
3
4
= 10

z
0
1
2
3
= 62

y
12
18
24
30
= 84

x2
1
4
9
16
= 30

z2
0
1
4
9
= 14

z.x
0
2
6
12
= 20

x.y
12
36
72
120
= 240

y.z
0
18
48
90
= 156

So, three simultaneous equations will be:

4a1 + 10a2 + 6a3 = 84 (i)
10a1 + 30a2 + 20a3 = 240 (ii)
6a1 + 20a2 + 14a3 = 156 (iii)
On solving we can get the values of unknowns.
Home Work
Given the data points
x 5 4 3 2 1
z 3 -2 -1 4 0
y 15 8 -1 26 8
Fit the multiple linear regressions.
Solution:

Page

59

Ans: y = 10 2x + 5z

Prepared By
Er. Shree Krishna Khadka

SPLINE INTERPOLATION:
Data fitting by means of polynomials has been considered in the previous exercise.
From the viewpoint of cubic splines, the resulting approximation called the cubic
spline approximation, suffers from the disadvantage of being a global
approximation, which means that a change in one point affects the entire
approximation curve. So, a method based on the basis splines, which posses a local
character, viz; a change in one point introduces a change only in the immediate
neighbourhood of that point, called B-spline method that finds important
application in computer graphics and smoothing of data.
Cubic B-Splines:
It resembles the ordinary cubic spline in which a separate cubic is derived for each
interval. Specially, a cubic B-spline (or a B-spline of order-4) denoted by B4i(x), is a
cubic spline with knots Ki-4, Ki-3, Ki-2, Ki-1 and Ki, which is zero everywhere except in
the range Ki-4 < x < Ki. In such case, B4i(x) is said to have a support [Ki-1, Ki]. It may
be noted that a B-spline need not necessarily pass through any or all of the data
points.
Cubic splines are popular because of their ability to interpolate data with smooth
curves. If we consider the construction of a cubic spline function which would
interpolate the points (x0, f0), (x1, f1) (xn, fn). The cubic spline S(x) consists of (n1) cubic corresponding to (n-1) sub intervals. If we denote such cubic by Si(x),
then:
S(x) = Si(x); for I = 1, 2, n.
These cubic must satisfy the following conditions:

a) S(x) must interpolate the function at all the points: x0 , x1, . xn; i.e. for each
i', S(xi) = fi.
b) The function values must be equal at all the interior knots, i.e. Si(xi) = Si+1(xi)
c) The first derivative at the interior knots must be equal; Si(x) = Si+1(x)
d) The second derivative at the interior knots must be equal; Si(xi) = Si+1(xi)
e) The second derivative at the end points are zero; S(x0) = S(xn) = 0

gi(x) = ai(x-xi)3 + bi(x-xi)2 + ci(x-xi) + di

. (i)

Page

Then:

60

Let us consider (n+1) sample points with n number of different section as shown
in the figure below. Suppose, gi(x) represents a cubic polynomial between [xi, f(xi)]
and [xi+1, f(xi+1)].

Prepared By
Er. Shree Krishna Khadka

g(xi)

F(x)

g(xi+1)

xi-2

xi-1

xi

xi+1

x
x

Fig: Spline Interpolation

1.) Since, function must pass through all the points; i.e; gi(xi) = di = f(xi) .(i)
2.) Differentiating equation (i) w. r. t. x we get:

gi(x) = 3ai(x-xi)2 + 2bi(x-xi) + Ci (ii)

gi(x) = 6ai(x-xi) + 2bi (iii)
at x = xi , equation (iii) becomes: gi(x) = 2bi = Si say;

i.e. bi = Si/2 (iv)

Similarly:
gi(xi+1) = Si+1 = 6ai(xi+1 - xi) + 2bi = 6aihi + 2bi
Where; xi+1 - xi = hi
Then; ai = (Si+1 Si)/6hi . (v)
3.) Again, gi(xi+1) = f(xi+1);
But, f(xi+1) = aihi3 + bihi2 + Cihi + di = aihi3 + bihi2 + Cihi + f(xi)
i.e. Ci = [{f(xi+1) f(xi)}/h] - aihi2 - bihi .(vi)
4.) Also from equation (ii) we have: gi-1(x) = 3ai-1(x-xi-1)2 + 2bi-1(x-xi-1) + Ci-1 (vii)

And we also have the condition that, at x = xi, gi(xi) = gi-1(xi), then:
Ci = 3ai-1(hi-1)2 + 2bi-1(hi-1) + Ci-1 . (viii)
5.) Finally on manipulating all these values, we get a final equation as:
hi1 Si1 + 2 hi1 + hi S + hi Si+1 = 6

f i+1 f i
hi

f i f i1
h i1

6.) Expressing above equation in matrix form:

0.
f x1 , x2 f x0 , x1
S1
h2
S2 =6 f x2 , x3 f x1 , x2
S
2 h2 + h3 3
f x3 , x4 f x2 , x3

Where, f x1 , x2 , f x0 , x1 . are first divided difference values calculated from

the tabulated data and h0, h1 are the successive intervals between two data

Prepared By
Er. Shree Krishna Khadka

61

h1
2(h1 + h2 )
h2

Page

2(h0 + h1 )
h1
0

points that may or may not be equal. For natural spline, we have: S0 =Sn = 0
(Second order derivative at end points is zero)
Example:
Fit the data of table with a natural cubic spline curve and evaluate the spline
values, g(0.66) and g(1.75), with h0 = 1, h1 = 0.5 and h2 = 0.75
x 0
1
1.5
2.25
f(x) 2 4.4366 6.7134 13.9130
(Hint: The relation is described as: f(x) = 2ex x2)
Solution:
1.) From the given table, we have:
h0 = x1 x0 = 1 0 = 1
h1 = x2 x1 = 1.5 1 = 0.5
h2 = x3 x2 = 2.25 1.5 = 0.75
2.) For natural spline, S0 = 0 and S3 = 0
3.) Now, Finding Si = S1 and S2. Generating Matrix Model as follow:
2(h0 + h1 )
h1
f x1 , x2 f x0 , x1
S1
=6
h1
2(h1 + h2 ) S2
f x2 , x3 f x1 , x2
i.e.

2(1 + 0.5)
0.5

0.5
S1
12.702
=6
2(0.5 + 0.75) S2
30.2748

i.e. 3S1 + 0.5S2 = 12.702 (i) and 0.5S1 + 2.5S2 = 30.2742 (ii)
On solving: S1 = 2.292 and S2 = 11.65
4.) Now, finding ai, bi, ci and di
1
3.12
1.146
3.200
4.4366

2
-2.598
5.825
6.680
6.7134

5.) Finally, g(0.66) = a0(x-x0)3 + b0(x-x0)2 + c0(x-x0) + d0 = 3.466

And, g(1.75) = a2(x-x2)3 + b2(x-x2)2 + c2(x-x2) + d0 = 8.70
Prepared By
Er. Shree Krishna Khadka

62

0
0.381
0
2.0556
2

Page

i
ai = (Si+1 Si)/6hi
bi = Si/2
Ci = [{f(xi+1) f(xi)}/h] - aihi2 - bihi
di = f(xi)

Home Work:
Given the data points:
i
0 1 2
xi 4 9 16
f(xi) 2 3 4
Estimate the value of f(7)
Solution:
h0 = 5 and h1 = 7
S0 = 0 = S2, but S1 = -0.01428
a0 = -0.000476, a1 = 0.00034
b0 = 0, b1 = -0.00714
c0 = 0.2119, c1 = 0.6259
d0 = 2, d1 = 3

Finally, f(7) = g0(x) = 2.6228

APPROXIMATION OF FUNCTIONS:
The problem of approximating a function is a central problem in numerical
analysis due to its importance in the development of software for digital
computers. Functions evaluation through interpolation techniques over stored
table of values has been found to be quite costlier, when compared to the use of
efficient function approximations.
If f1, f2 fn be the values of the given function and g1, g2, gn be the corresponding
values of the approximating function; then the error vector e, where the
components of e is given by: ei = fi gi ..(i).
The approximation may be chosen in a number of ways. For example, we may find

Page

63

the approximation such that the quantity 1 2 + 1 2 + 1 2 + is minimum. This

leads us to the least square approximation, which we have already studied. On the
other hand, we may choose the approximation such that the maximum component
of e is minimized. This leads us to the Celebrated Chebyshev Polynomials, which
have found important application in the approximation of function in digital
computers.

Prepared By
Er. Shree Krishna Khadka

Assignment 4
Full Marks: 25
Pass Marks: 15

1. The table below gives the temperatures T(OC) and lengths(mm) of a

heated rod. If l = a0 + a1T, find the values of a0 and a1 using linear least
squares. Find the length of the rod when T = 72.5oC.
T
l

40
600.5

50
600.6

60
600.8

70
600.9

[5]

80
601.0

2. The temperature of a metal strip was measured at various time intervals

during heating and the values are given in the table below.
Time(t min)
Temp(T OC)

1
70

2
83

3
100

4
124

If the relation between the time t and temperature T is of the form: T =

bet/4 + a. Estimate the temperature at t = 6 minute.
3. Fit a second order polynomial to the data given in the table below.
x
f(x)

1
2

2.1
2.5

3.2
3.0

[5]
[5]

4
4

Find the polynomial value at 1.5 and 3.8.

4. Given the table of values
x
z
y

5
3
15

4
-2
8

3
-1
-1

2
4
26

1
0
8

Use multiple linear regression formula to fit the data of the form:
y = a1 + a2(x) +a3(z) & Find f(5.3) and f(3.7).

[5]

5. Fit a natural cubic B-Spline, S to the data given below in the table.

[5]

x
f(x)=x3-1

1.2
-2

2.8
-1

3.5
0

4.1
1

5.9
2

Page

64

Find the functional value at 1.5 and 6

Prepared By
Er. Shree Krishna Khadka