Greek Civilization: Up Your Anus

Researh Paper: Anal Research Association of

America
Ling Ming Tyrese Schultz

13/100/2

29.1. This five part problem asks you to put together a Matlab program that
finds all the eigenvalues of a real symmetric matrix, using only elementary
building blocks. It is not necessary to achieve optimal constant factors by
exploiting symmetry or zero structure optimally. It is possible to solve the
whole problem by a program about fifty lines long.
(a) Write a function T = tridiag(A) that reduces a real symmetric m x m
matrix to tridiagonal form by orthogonal similarity transformations. Your
program should use only elementary Matlab operations not the built-in
function hess, for example. Your output matrix T should be symmetric and
tridiagonal up to rounding errors. Apply your program to A = hilb(4).
A function that reduces a square real symmetric matrix to tridiagonal form is
given as follows:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Function input: Real symmetric square matrix A
%
%Function output: Tridiagonal matrix T found by
%
%householder transformations.
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [T] = tridiag(A)
%Get input matrix size:
n = length(A);
%Start iteration process from 2 to n-1:
for k = 2:n-1
vec = zeros(n,1);
vec(k:n) = A(k:n,k-1);
vec = vec/norm(vec,2);
if vec(k) < 0
vec = -vec;
end
vec(k) = vec(k) + 1;
alpha = -vec(k);
V = A*vec/alpha;
beta = V'*vec/(2*alpha);
V = V + beta*vec;
A = A + vec*V' + V*vec';
A(k-1,k+1:n) = zeros(1,n-k);
end
%Assign modified A as output matrix T:
T = A;
end

Assuming A is given as hilb(4), then corresponding tridiagonalization is as

follows:

(b) Write a function Tnew = qralg(T) that runs the unshifted QR algorithm on
a real tridiagonal matrix T. For the QR factorization at each step, use
programs [W,R] = house(A) and Q = formQ(W) of Exercise 10.2 if available,
or Matlabs command qr, or, for greater efficiency, a new code based on
Givens rorations or 2 x 2 Householder reflections rather than m x m
operations. Again, you may wish to enforce symmetry and tridiagonality at
each step. Your program should stop and return the current tridiagonal
matrix T as Tnew when the m, m-1 element satisfies |t m,m-1| < 10-12. Again,
apply your program to A = hilb(4).
A function that performs an unshifted QR algorithm on symmetric tridiagonal
matrix T is as follows:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Function input: Tridiagonal symmetric matrix T
%
%Function output: Current tridiagonal symmetric matrix %
%after QR algorithm.
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [Tnew] = qralg(T)
m = length(T);
while abs(T(m,m-1)) >= 10^(-12)
[Q,R] = qr(T);
T = R * Q;
end
Tnew = T;
end

Again, if we reduce A = hilb(4) into a tridiagonal form using tridiag in part(a)

and then used such tridiagonal matrix as an input to qralg above, what we
get is as follows:

Notice that when we run eig(A), the diagonal entries of Tnew actually
matches the values in eig(A).

(c) Write a driver program which (i) calls tridiag, (ii) calls qralg to get one
eigenvalue, (iii) calls qralg with a smaller matrix to get another eigenvalue,
and so on until all of the eigenvalues of A are determined. Set things up so
that the values of |tm,m-1| at every QR iteration are stored in a vector and so
that at the end, your program generates a semilogy plot of these values as a
function of the number of QR factorizations. Run your program for A =
hilb(4). The output should be a set of eigenvalues and a sawtooth plot.
Since I dont want to completely modify qralg, I made a new function called
qralg_ray to implement the QR algorithm with Rayleigh shifts:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Function Input: Symmetric tridiagonal matrix A
%
%Function Output: Eigenvalue lambda of A
%
%
Subdiagonal absolute values vec
%
%
Modified A using rayleight shifts %
%
newA
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [lambda,vec,newA] = qralg_ray(A)
m = length(A);
vec = [];
vecdex = 1;
while abs(A(m,m-1)) >= 10^(-12)
vec(vecdex) = abs(A(m,m-1));
[Q,R] = qr(A - (A(m,m) * eye(m,m)));
A = (R * Q) + (A(m,m) * eye(m,m));
vecdex = vecdex + 1;
end
lambda = A(m,m);
vec = vec;
newA = A;
end

Now, using the above function I implemented a the algorithm with deflation
as described above on an input of H = hilb(4) using the code as follows:
clear
clc
%QR Algorithm with Rayleigh shifts and deflation implementation:
%Initialize Hilbert Matrix and subdiagonal entry storage:
H = hilb(4);
n = length(H);
subdiag = [];
%Tridiagonalize H:
A = feval(@tridiag,H);
%Eigenvalue storage:
evals = zeros(n,1);
%Start iteration:
for i = n:-1:2
[lambda,vec,A] = feval(@qralg_ray,A(1:i,1:i));
evals(i) = lambda;
subdiag = [subdiag vec];
end

subdiag = [subdiag A(2,1)];

evals(1) = A(1,1);
%Plot results in semilogy axes:
figure(1)
semilogy(1:length(subdiag),subdiag),grid on,...
title('Absolute Subdiagonal Entry of A vs Interation Step'),...
xlabel('Iteration Steps'),ylabel('Ubsolute Subdiagonal Entry Value')

The absolute subdiagonal entries are then plotted in semilogy axes. The
results are as followed:

Notice that the graph of subdiagonal values vs. iteration steps obeys a
sawtooth like behavior.
(d) Modify qralg so that it uses the Wilkinson shift at each step. Turn in the
new sawtooth plot for the same example.
To implement the QR algorithm with Wilkinson shifts and deflation, we can
use the same layout as described in (c) but with qralg_ray change to the
function qralg_wil as follows:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Function Input: Symmetric tridiagonal matrix A
%
%Function Output: Eigenvalue lambda of A
%
%
Subdiagonal absolute values vec
%
%
Modified A using wilkinson shifts %
%
newA
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [lambda,vec,newA] = qralg_wil(A)

m = length(A);
vec = [];
vecdex = 1;
while abs(A(m,m-1)) >= 10^(-12)
%Wilkinson Shift:
delta = (A(m-1,m-1) - A(m,m))/2;
mu = (A(m,m) - sign(delta)*(A(m,m-1)^2))/...
(abs(delta) + sqrt(delta^2 + A(m,m-1)^2));
vec(vecdex) = abs(A(m,m-1));
[Q,R] = qr(A - (mu * eye(m,m)));
A = (R * Q) + (mu * eye(m,m));
vecdex = vecdex + 1;
end
lambda = A(m,m);
vec = vec;
newA = A;
end

With the following saw tooth semilogy plot as follows:

(e) Rerun your program for the matrix A = diag(15:-1:1) + ones(15,15) and
generate two sawtooth plots corresponding to shifts and no shifts. Discuss
the rates of convergence observed here and for the earlier matrix. Is the
convergence linear, superlinear, quadratic, cubic? Is it meaningful to speak
of a certain number of QR iterations per eigenvalue?
To do the QR algorithm without shifts but with deflation, I used the function
qralg as described in (b) but with more outputs as follows:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%Function Input: Symmetric tridiagonal matrix A
%

%Function Output: Eigenvalue lambda of A

%
%
Subdiagonal absolute values vec
%
%
Modified A using QR algorithm
%
%
without shifts newA
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [lambda,vec,newA] = qralg(A)
m = length(A);
vec = [];
vecdex = 1;
while abs(A(m,m-1)) >= 10^(-12)
vec(vecdex) = abs(A(m,m-1));
[Q,R] = qr(A);
A = R * Q;
vecdex = vecdex + 1;
end
lambda = A(m,m);
vec = vec;
newA = A;
end

The corresponding plot of absolute subdiagonal entries are as follows:

Now by rerunning the same matrix but instead using the Wilkinson shifts as
described in (d), we get the following plots:

First thing to notice about these plots is that QR algorithm without shifts
actually require significantly more steps than QR algorithm with Wilkinson
shifts for the subdiagonal entry to converge to 0. Another thing to notice is
that since Ak+1 = RkQk with Ak = QkRk then Ak+1 = QkTAkQk and hence, the QR
algorithm without shifts resembles power iteration as k approaches infinity
with A converging to diagonal matrix since the initial A is tridiagonal. Hence
the property of power iteration convergence applies to this particular case,
and so QR algorithm without shifts converges linearly with subdiagonal
elements converging to zero in a decaying exponential manner. As for the QR
algorithm with Wilkinson shift, it is a well-known fact that QR algorithms with
Wilkinson shifts converge cubically, hence so as for these cases. Hence, due
to these difference in convergence rates, it is then meaningful to speak of
total number of QR iterations for an eigenvalue.