Method of Greville

The Moore-Penrose generalized matrix inverse

Samuel Pasquier
David Rousselie
October 15, 2003
i
4 ALGORITHMOF GREVILLE
1 Denition of the matrix inverse
The inverse of a square matrix A, sometimes called a reciprocal matrix, is a matrix A
1
such
that :
AA
1
= I (1)
2 Denition of the generalized matrix inverse
The generalized inverse of a matrix A, is a matrix A

such that :
AA

A = A (2)
There are many solutions for A

3 Denition of the Moore-Penrose generalized matrix inverse

The Moore-Penrose generalized matrix inverse of a matrix A, is a matrix A
+
such that :
AA
+
A = A
A
+
AA
+
= A
+
(A
+
A)
t
= A
+
A
(AA
+
)
t
= AA
+
(3)
4 Algorithm of Greville
The Greville Algorithm build row by row the matrix A
+
using the columns of the matrix A.
4.1 Notations
a
m
column m of A
A
m
the matrix build with the m rst columns of A.
A
+
m
A
+
at stage m (the m rst lines).
Method of Greville 1 -
5 DEMONSTRATION OF THE ALGORITHM
4.2 Recursive denition of the algorithm
Initial conditions :
A
1
= a
1
(4)
A
+
1
=
_
a
t
1
(a
t
1
a
1
)
1
if a
1
= 0
0
t
else
(5)
At the stage k we have :
Let :
p
k
=
_

_
IA
k1
A
+
k1
a
k
(IA
k1
A
+
k1
a
k
)
2
if (I A
k1
A
+
k1
)a
k
= 0
(A
+
k1
)
t
A
+
k1
a
k
(1+(A
+
k1
a
k

2
)
else
(6)
So we have :
A
+
k
=
_
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
(7)
5 Demonstration of the algorithm
Creation of the A
+
k
matrix from A
+
k1
We suppose we have made the A
+
k1
matrix so we have :
A
k1
A
+
k1
A
k1
= A
k1
A
+
k1
A
k1
A
+
k1
= A
+
k1
A
k1
A
+
k1
= (A
k1
A
+
k1
)
t
A
+
k1
A
k1
= (A
+
k1
A
k1
)
t
We want A
+
k
with A
k
A
+
k
A
k
= A
k
and A
k
=
_
A
k1
a
k
_
. To have a matrix multiplication
coherent, we must add one line to A
+
k
from A
+
k1
. So A
+
k
=
_
B
k1
b
k
_
with B
k1
R
(k1)n
and
b
k
R
1n
.
A
k
A
+
k
A
k
=
_
A
k1
a
k
_
_
B
k1
b
k
_
_
A
k1
a
k
_
=
_
A
k1
a
k
_
_
B
k1
A
k1
B
k1
a
k
b
k
A
k1
b
k
a
k
_
=
_
A
k1
B
k1
A
k1
+a
k
b
k
A
k1
A
k1
B
k1
a
k
+a
k
b
k
a
k
_
=
_
A
k1
a
k
_
(8)
Method of Greville 2 -
5 DEMONSTRATION OF THE ALGORITHM
Then we have a system of two equations to nd B
k1
and b
k
:
A
k1
B
k1
A
k1
+a
k
b
k
A
k1
= A
k1
A
k1
B
k1
a
k
+a
k
b
k
a
k
= a
k
(9)
We suppose (this only reduce solution possibilities but do not alter possible solution) :
A
k1
B
k1
a
k
= 0
a
k
b
k
A
k1
= 0
(10)
So we have :
a
k
b
k
a
k
= a
k
A
k1
B
k1
A
k1
= A
k1
(11)
A solution for a
k
b
k
a
k
= a
k
could be bk =
a
t
k
a
t
k
a
k
but for the second equation a
k
a
t
k

a
k

2
A
k1
= 0.
We can choose b
k
=
a
t
k

a
k

2
(X Y ) with X and Y R
nn
.
Thus
a
k
a
t
k
_
_
a
k
_
_
2
XA
k1
a
k
a
t
k
_
_
a
k
_
_
2
Y A
k1
= 0 (12)
We search b
k
= 0 thus X = Y and we know that A
k1
A
+
k1
A
k1
= A
k1
thus X = I and
Y = A
k1
A
+
k1
could be a solution to second equation.
But now :
a
k
a
t
k
_
_
a
k
_
_
2
(I A
k1
A
+
k1
)a
k
= a
k
because
a
t
k
_
_
a
k
_
_
2
(I A
k1
A
+
k1
)a
k
= 1
(13)
We can now replace
_
_
a
k
_
_
2
by a
t
k
(I A
k1
A
+
k1
)a
k
to satisfy this equation and now we can
verify easily that this is also a solution for a
k
b
k
A
k1
= 0.
Thus
bk =
a
t
k
(I A
k1
A
+
k1
)
a
t
k
(I A
k1
A
+
k1
)a
k
=
a
t
k
(I A
k1
A
+
k1
)
_
_
(I A
k1
A
+
k1
)a
k
_
_
(14)
We also want :
A
k1
B
k1
A
k1
= A
k1
(15)
B
k1
= A
+
k1
is a solution to this equation but A
k1
B
k1
a
k
= 0.
Now we suppose B
k1
= A
+
k1
(X Y ) with X and Y R
nn
.
Now we have
A
k1
A
+
k1
Xa
k
A
k1
A
+
k1
Y a
k
= 0 with X = Y because we do not want B
k1
= 0 (16)
We know that a
k
b
k
a
k
= a
k
, thus X = I and Y = a
k
b
k
could be a solution.
We have
A
k1
A
+
k1
a
k
A
k1
A
+
k1
a
k
b
k
a
k
= A
k1
A
+
k1
a
k
A
k1
A
+
k1
a
k
= 0
(17)
Method of Greville 3 -
5 DEMONSTRATION OF THE ALGORITHM
We replace this new solution in the other equation :
A
k1
B
k1
A
k1
= A
k1
A
+
k1
(I a
k
b
k
)A
k1
= A
k1
A
+
k1
A
k1
A
k1
A
+
k1
a
k
b
k
A
k1
but A
k1
A
+
k1
A
k1
= A
k1
and a
k
b
k
A
k1
= 0
thus A
k1
B
k1
A
k1
= A
k1
(18)
Thus we have :
A
+
k
=
_
A
+
k
(I a
k
b
k
)
b
k
_
with b
k
=
a
t
k
(I A
k1
A
+
k1
)
_
_
(I A
k1
A
+
k1
)a
k
_
_
(19)
Initial conditions :
A
1
= a
1
(20)
A
+
1
= a
t
1
1
(a
t
1
a
1
)
(21)
We want to verify the conditions at stage k = 1
We want to prove that A
1
A
+
1
A
1
= A
1
:
A
1
A
+
1
A
1
= A
1
_
a
t
1
a
t
1
a
1
_
A
1
=
1
a
t
1
a
1
(A
1
a
t
1
A
1
)
=
1
a
t
1
a
1
(a
1
a
t
1
A
1
)
A
1
A
+
1
A
1
= A
1
(22)
If A
1
= 0 then A
+
1
= 0
t
, thus we can quickly nd that A
1
A
+
1
A
1
= A
1
.
We want to prove that A
+
1
A
1
A
+
1
= A
+
1
:
A
+
1
A
1
A
+
1
=
A
t
1
a
t
1
a
1
A
1
A
t
1
a
t
1
a
1
=
A
t
1
a
t
1
a
1
a
1
a
t
1
a
t
1
a
1
=
A
t
1
a
t
1
a
1
A
+
1
A
1
A
+
1
= A
+
1
(23)
If A
1
= 0 then A
+
1
= 0
t
, thus we can quickly nd that A
+
1
A
1
A
+
1
= A
+
1
Method of Greville 4 -
5 DEMONSTRATION OF THE ALGORITHM
We want to prove that (A
+
1
A
1
)
t
= (A
+
1
A
1
) :
(A
+
1
A
1
)
t
= A
t
1
(A
+
1
)
t
= A
t
1
(
a
t
1
a
t
1
a
1
)
t
=
1
a
t
1
a
1
(a
t
1
a
1
)
= 1
(24)
(A
+
1
A
1
) = a
t
1
1
a
t
1
a
1
a
1
=
1
a
t
1
a
1
a
t
1
a
1
= 1
= (A
+
1
A
1
)
t
(25)
If A
1
= 0 then A
+
1
= 0
t
, thus we can quickly nd that (A
+
1
A
1
)
t
= (A
+
1
A
1
).
We want to prove that (A
1
A
+
1
)
t
= A
1
A
+
1
:
(A
1
A
+
1
)
t
= (A
1
A
t
1
1
a
t
1
a
1
)
t
=
1
a
t
1
a
1
A
1
A
t
1
(26)
A
1
A
+
= A
1
A
t
1
1
a
t
1
a
1
= (A
1
A
+
)
t
(27)
If A
1
= 0 then A
+
1
= 0
t
, thus we can quickly nd that (A
1
A
+
1
)
t
= A
1
A
+
1
The conditions are veried at stage 1, suppose that they are veried at a stage (k 1) with
k > 1.
Method of Greville 5 -
5 DEMONSTRATION OF THE ALGORITHM
Suppose that we have :
A
k1
A
+
k1
A
k1
= A
k1
A
+
k1
A
k1
A
+
k1
= A
+
k1
A
k1
A
+
k1
= (A
k1
A
+
k1
)
t
A
+
k1
A
k1
= (A
+
k1
A
k1
)
t
We prove that A
k
A
+
k
= (A
k
A
+
k
)
t
:
A
k
A
+
k
=
_
A
k1
a
k
_
_
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
= A
k1
(A
+
k1
(I a
k
p
t
k
)) +a
k
p
t
k
= A
k1
(A
+
k1
A
+
k1
a
k
p
t
k
)) +a
k
p
t
k
= A
k1
A
+
k1
A
k1
A
+
k1
a
k
p
t
k
+a
k
p
t
k
but we have p
k
=
(I A
k1
A
+
k1
)a
k
_
_
(I A
k1
A
+
k1
)a
k
_
_
2
A
k
A
+
k
=
A
k1
A
+
k1
+
1
_
_
(I A
k1
A
+
k1
)a
k
_
_
2
(A
k1
A
+
k1
a
k
((I A
k1
A
+
k1
)a
k
)
t
+a
k
((I A
k1
A
+
k1
)a
k
)
t
)
=
A
k1
A
+
k1
+
1
_
_
(I A
k1
A
+
k1
)a
k
_
_
2
(A
k1
A
+
k1
a
k
a
t
k
A
k1
A
+
k1
a
k
a
t
k
A
k1
A
+
k1
+a
k
a
t
k
a
k
a
t
k
A
k1
A
+
k1
)
thus :
(A
k
A
+
k
)
t
=
A
k1
A
+
k1
+
1
_
_
(I A
k1
A
+
k1
)a
k
_
_
2
(a
k
a
t
k
A
k1
A
+
k1
A
k1
A
+
k1
a
k
a
t
k
A
k1
A
+
k1
+a
k
a
t
k
A
k1
A
+
k1
a
k
a
t
k
)
because :
(A
k1
A
+
k1
)
t
= A
k1
A
+
k1
(a
k
a
t
k
)
t
= a
k
a
t
k
thus :
(A
k
A
+
k
)
t
= A
k
A
+
k
(28)
Method of Greville 6 -
5 DEMONSTRATION OF THE ALGORITHM
For the particular case : p
k
=
(A
+
k1
)
t
A
+
k1
a
k
(1+(A
+
k1
a
k

2
)
A
k
A
+
k
=
_
A
k1
a
k
_
_
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
= A
k1
(A
+
k1
(I a
k
p
t
k
)) +a
k
p
t
k
= A
k1
(A
+
k1
A
+
k1
a
k
p
t
k
)) +a
k
p
t
k
= A
k1
A
+
k1
A
k1
A
+
k1
a
k
p
t
k
+a
k
p
t
k
but we have A
k1
A
+
k1
a
k
= a
k
because (I A
k1
A
+
k1
)a
k
= 0
= A
k1
A
+
k1
a
k
p
t
k
+a
k
p
t
k
= A
k1
A
+
k1
(29)
We have A
k1
A
+
k1
symetric thus A
k
A
+
k
is symetric.
Method of Greville 7 -
5 DEMONSTRATION OF THE ALGORITHM
We want to prove that A
+
k
A
k
= (A
+
k
A
k
)
t
:
A
+
k
A
k
=
_
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
_
A
k1
a
k
_
=
_
A
+
k1
(I a
k
p
t
k
)A
k1
A
+
k1
(I a
k
p
t
k
)a
k
p
t
k
A
k1
p
t
k
a
k
_
but we have :
p
t
k
A
k1
=
1
_
_
(I A
k1
A
+
k1
)a
k
_
_
2
((I A
k1
A
+
k1
)a
k
)
t
A
k1
=
1
_
_
(I A
k1
A
+
k1
)a
k
_
_
2
(a
t
k
A
k1
a
t
k
A
k1
A
+
k1
A
k1
)
=
1
_
_
(I A
k1
A
+
k1
)a
k
_
_
2
(a
t
k
A
k1
a
t
k
A
k1
)
= 0
and :
p
t
k
a
k
=
a
t
k
a
k
a
t
k
A
k1
A
+
k1
a
k
(a
t
k
a
t
k
A
k1
A
+
k1
)(a
k
A
k1
A
+
k1
a
k
)
=
a
t
k
a
k
a
t
k
A
k1
A
+
k1
a
k
a
t
k
a
k
a
t
k
A
k1
A
+
k1
a
k
a
t
k
A
k1
A
+
k1
a
k
+a
t
k
A
k1
A
+
k1
A
k1
A
+
k1
a
k
=
a
t
k
a
k
a
t
k
A
k1
A
+
k1
a
k
a
t
k
a
k
a
t
k
A
k1
A
+
k1
a
k
a
t
k
A
k1
A
+
k1
a
k
+a
t
k
A
k1
A
+
k1
a
k
=
a
t
k
a
k
a
t
k
A
k1
A
+
k1
a
k
a
t
k
a
k
a
t
k
A
k1
A
+
k1
a
k
= 1
thus :
A
+
k
A
k
=
_
A
+
k1
A
k1
A
+
k1
a
k
p
t
k
A
k1
A
+
k1
a
k
A
+
k1
a
k
p
t
k
a
k
0 1
_
=
_
A
+
k1
A
k1
0
0 1
_
but we haveA
+
k1
A
k1
symetric, thus
_
A
+
k1
A
k1
0
0 1
_
is symetric.
(30)
For the particular case : p
k
=
(A
+
k1
)
t
A
+
k1
a
k
(1+(A
+
k1
a
k

2
)
p
t
k
a
k
=
a
t
k
(A
+
k1
)
t
A
+
k1
a
k
1 +
_
_
A
+
k1
a
k
_
_
2
=
_
_
A
+
k1
a
k
_
_
2
1 +
_
_
A
+
k1
a
k
_
_
2
(31)
Method of Greville 8 -
5 DEMONSTRATION OF THE ALGORITHM
p
t
k
A
k1
=
a
t
k
(A
+
k1
)
t
A
+
k1
A
k1
1 +
_
_
A
+
k1
a
k
_
_
2
=
(A
+
k1
A
k1
A
+
k1
a
k
)
t
1 +
_
_
A
+
k1
a
k
_
_
2
=
(A
+
k1
a
k
)
t
1 +
_
_
A
+
k1
a
k
_
_
2
(32)
A
+
k1
(I a
k
p
t
k
)a
k
= A
+
k1
a
k
A
+
k1
a
k
p
t
k
a
k
= A
+
k1
a
k
(1
_
_
A
+
k1
a
k
_
_
2
1 +
_
_
A
+
k1
a
k
_
_
2
)
= A
+
k1
a
k
(
1 +
_
_
A
+
k1
a
k
_
_
2

_
_
A
+
k1
a
k
_
_
2
1 +
_
_
A
+
k1
a
k
_
_
2
)
=
A
+
k1
a
k
1 +
_
_
A
+
k1
a
k
_
_
2
= (p
t
k
a
k
)
t
(33)
A
+
k1
(I a
k
p
t
k
)A
k1
= A
+
k1
A
k1
A
+
k1
a
k
p
t
k
A
k1
= A
+
k1
A
k1

(A
+
k1
a
k
)(A
+
k1
a
k
)
t
1 +
_
_
A
+
k1
a
k
_
_
2
(34)
but (A
+
k1
a
k
)(A
+
k1
a
k
)
t
is a symetric matrix thud A
+
k1
(I a
k
p
t
k
)A
k1
is symetric, thus A
+
k
A
k
is
symetric.
We want to prove that A
k
A
+
k
A
k
= A
k
:
A
k
A
+
k
A
k
=
_
A
k1
a
k
_
_
A
+
k1
A
k1
0
0 1
_
=
_
A
k1
A
+
k1
A
k1
a
k
_
=
_
A
k1
a
k
_
= A
k
(35)
Method of Greville 9 -
5 DEMONSTRATION OF THE ALGORITHM
For the particular case : p
k
=
(A
+
k1
)
t
A
+
k1
a
k
(1+(A
+
k1
a
k

2
)
A
k
A
+
k
A
k
= A
k1
A
+
k1
_
A
k1
a
k
_
=
_
A
k1
A
+
k1
A
k1
A
k1
A
+
k1
a
k
_
but we have A
k1
A
+
k1
a
k
= a
k
thus :
=
_
A
k1
a
k
_
= A
k
(36)
We want to prove that A
+
k
A
k
A
+
k
= A
+
k
:
A
+
k
A
k
A
+
k
=
_
A
+
k1
A
k1
0
0 1
__
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
=
_
A
+
k1
A
k1
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
=
_
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
= A
+
k
(37)
For the particular case : p
k
=
(A
+
k1
)
t
A
+
k1
a
k
(1+(A
+
k1
a
k

2
)
A
+
k
A
k
A
+
k
=
_
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
A
k1
A
+
k1
=
_
A
+
k1
(I a
k
p
t
k
)A
k1
A
+
k1
p
t
k
A
k1
A
+
k1
_
=
_
A
+
k1
A
k1
A
+
k1
A
+
k1
a
k
p
t
k
A
k1
A
+
k1
p
t
k
A
k1
A
+
k1
_
=
_
_
_
_
A
+
k1

A
+
k1
a
k
a
t
k
(A
+
k1
)
t
A
+
k1
A
k1
A
+
k1
1+

A
+
k1
a
k

2
a
t
k
(A
+
k1
)
t
A
+
k1
A
k1
A
+
k1
1+

A
+
k1
a
k

2
_
_
_
_
=
_
_
_
_
A
+
k1

A
+
k1
a
k
a
t
k
(A
+
k1
)
t
A
+
k1
1+

A
+
k1
a
k

2
a
t
k
(A
+
k1
)
t
A
+
k1
1+

A
+
k1
a
k

2
_
_
_
_
=
_
A
+
k1
A
+
k1
a
k
p
t
k
p
t
k
_
=
_
A
+
k1
(I a
k
p
t
k
)
p
t
k
_
= A
+
k
(38)
Method of Greville 10 -
5 DEMONSTRATION OF THE ALGORITHM
The matrix A
+
k
satises the four property at the stage k, thus it is a Moore-Penrose Matrix.
Thus Greville algorithm builds the Moore-Penrose generalized matrix inverse.
Method of Greville 11 -