engg7302

1/8

Advanced Computational

engg7302
Techniques in Engineering
Lecture LA01: Matrix-Vector Multiplication
This lecture:
1. Notation. 
2. Matrix-Vector Multiplication.

3. Matrix Inverse. 

Ref: TB Lecture 1.



Notation
2/8
• We’ll use bold face for matrices and vectors.

engg7302
• Upper case for matrices: A.
• Lower case for vectors: v.
• Scalars in italic and without bold: c.
• In general, vectors and matrices are complex-valued.
• Consider a matrix A ∈ Cm×n :
– The columns are denoted ai , i = 1, . . . , n.
– The rows are āi , i = 1, . . . , m. 
– The elements are aij .






Matrix-Vector Multiplication
3/8
Our first aim is to reinterpret the equation b = Ax as a linear com-
bination of the columns of A.

engg7302
• Recall that the basic definition of matrix multiplication is
n
X
bi = aij xj , i = 1, . . . , m.
j=1

• Consider a linear function f : Cn 7→ Cm where by linear we mean

that
– f(x + y) = f(x) + f(y) and

– f(αx) = αf(x)


for any x, y ∈ Cn and for any α ∈ C.

Theorem. f(x) is linear if and only if 
f(x) = Ax 
for some matrix A ∈ Cm×n . 

Matrix Times a Vector
To see that a matrix-vector multiplication can be interpreted as a
4/8
linear combination of matrix columns, we write
n

engg7302
X
b = Ax = xj aj .
j=1

• Schematically,  
!    ! x1  ! ! !
↑ ↑  ↑ 
 
↑
  x 2
 ↑ ↑ ↑
b = a1 a2  · · · an  . = x1 a 1 + x2 a 2 + · · · + xn an .
↓ ↓  ↓   ↓  .. 
 
↓ ↓ ↓
xn

Matrix Times a Matrix 

Similarly, we can interpret matrix-matrix multiplication as a linear
combination of matrix columns: 
n
X 
B = AX ⇒ bi = Axi = xij aj . 
j=1


Matrix Inverse
5/8
Range and Nullspace

engg7302
The range of a matrix A ∈ Cm×n is defined as

range(A) = {b ∈ Cm | b = Ax; x ∈ Cn }.

Theorem. range(A) is the space spanned by the columns of A.

• Hence, the range of A is also called the column space of A.

The nullspace of A is defined as


null(A) = {x ∈ Cn | Ax = 0}.



Rank

The column rank of a matrix is the dimension of its column space. 
• Similarly, the row rank is the dimension of its row space. 

 • Column rank always equals row rank.
– Hence, usually just use the term rank and write rank(A).
6/8
• Full rank means the matrix has the maximum possible rank, i.e.,
min {m, n}.

engg7302
Theorem. A matrix A ∈ Cm×n with m > n has full rank if and only
if it maps no two distinct vectors to the same vector.

⇒ A one-to-one linear function must be defined in terms of a full

rank matrix.

Inverse
A non-singular or invertible matrix is a square matrix of full rank. 


• If A ∈ Cn×n is invertible then the range of A is Cn .

⇒ Any b ∈ Cn can be expressed as a linear combination of the aj ,

j = 1, . . . , n.



⇒ There exists some zij , j = 1, . . . , n such that
n
X
ei = zij aj 7/8
j=1

engg7302
where ei is all zero except for a 1 in the ith position.
⇒ Since we can do this for each ei , we can write
   !
↑  ↑ 
  ↑
e1 e2  · · · en = I = AZ

↓  ↓  ↓

where I is known as the identity.

• Z is the inverse of A and is written A−1 .

• It is unique and satisfies

AA−1 = A−1 A = I. 




Theorem. For A ∈ Cn×n , the following conditions are equivalent:
1. A has an inverse A−1
8/8
2. rank(A) = n,

engg7302
3. range(A) = Cn ,
4. null(A) = {0},
5. 0 is not an eigenvalue of A,
6. 0 is not a singular value of A,
7. det A ≠ 0.

Matrix Inverse Times a Vector 

The important point is that multiplication by a matrix inverse is really

a change of basis operation. 
• A−1 b is the vector of coefficients of the expansion of b in the basis 
of columns of A.