Euclidean Structure

Kevin James

Kevin James Euclidean Structure

Definition
Let X be any n-dimensional Euclidean space (-i.e. vector space over R).
1 Given x ∈ X , we define the length of x denoted ||x|| to be the
distancepfrom x to 0. By the Pythagorean theorem we have
||x|| = x12 + · · · + xn2 .
2 Given x, y ∈ X , we define the inner (or dot) product of x and y as
Pn
(x, y ) = i=1 xi yi .

Note
1 ||x||2 = (x, x),
2 The dot product is symmetric and bilinear,
3 ||x + y ||2 = ||x||2 + 2(x, y ) + ||y ||2 ,
4 ||x − y ||2 = ||x||2 − 2(x, y ) + ||y ||2 . (Note that the last two are
independent of coordinates (-i.e. there is a geometric description).)
(x,y )
5 cos(θ) = ||x||·||y || .

Kevin James Euclidean Structure

Definition
A Euclidean Structure in a real vector space is endowed by an
inner product, which is symmetric bilinear form with the additional
property that (x, x) ≥ 0 with equality if and only if x = 0.

Assumption
Throughout we will assume that X is an n-dimensional real inner-product
space.

Definition
The norm of x ∈ X is ||x|| = (x, x)1/2 , and the distance between
x, y ∈ X is ||x − y ||.

Theorem (Cauchy-Schwartz)
For all x, y ∈ X , |(x, y )| ≤ ||x|| · ||y ||.

Corollary
(x,y )
For any x, y ∈ X , −1 ≤ ||x||·||y || ≤ 1.

Kevin James Euclidean Structure

Definition
Given x, y ∈ X , we define the angle between x and y to be
(x,y )
θ = cos−1 ||x||·||y || .

Theorem
||x|| = max{(x, y ) | ||y || = 1}.

Theorem (Triangle Inequality)

For all x, y ∈ X , ||x − y || ≤ ||x|| + ||y ||.

Kevin James Euclidean Structure

Definition
Two vectors x, y ∈ X are orthogonal if (x, y ) = 0. We write this as
x ⊥ y.

Remark (Pathagorean Theorem)

If x ⊥ y , then ||x + y ||2 = ||x||2 + ||y ||2 .

Definition
A subset {x1 , · · · , xn } ⊆ X is orthogonal if whenever i 6= j we have
(xi , xj ) = 0.
An orthogonal set is called an orthonormal set if each vector has length 1.

Fact
Any orthogonal set is linearly independent.

Kevin James Euclidean Structure

Theorem (Gram-Schmidt process)
Suppose W ≤ X has a basis {x1 , . . . , xp }. Then,

1
y1 = x1 and v1 = y1
||y1 ||
1
y2 = x2 − (x2 , v1 )v1 and v2 = y2
||y2 ||
1
y3 = x3 − (x3 , v1 )v1 − (x3 , v2 )v2 and v3 = y3
||y3 ||
..
.
1
yp = xp − (xp , v1 )v1 − · · · − (xp , vp−1 )vp−1 and vp = yp
||yp ||

is an orthonormal basis for W .

Also,
Span(v1 , . . . , vk ) = Span(x1 , . . . , xk )
for 1 ≤ k ≤ p.

Kevin James Euclidean Structure

Remark
An orthogonal matrix is a square matrix whose columns and rows are
orthogonal unit vectors (i.e., orthonormal vectors), i.e.

Q T Q = QQ T = I .

In particular, an orthonormal basis {u1 , · · · , un } for Rn forms an

orthogonal matrix
[u1 u2 · · · un ].

Kevin James Euclidean Structure

Remark
Any real square matrix A may be decomposed as A = QR, where Q is an
orthogonal matrix and R is an upper triangular matrix.
This is called the QR-factorization of A.
In particular, the Gram-Schmidt theorem gives a QR-factorization of
A = [x1 x2 · · · xn ], where x1 , · · · , xn is a basis of X ,
 
(x1 , u1 ) (x2 , u1 ) (x3 , u1 ) · · ·
 
 
 0 (x 2 , u2 ) (x 3 , u 2 ) · · · 
[x1 x2 · · · xn ] = [u1 u2 · · · un ]  .
 
 ..

0 (x3 , u3 ) · · ·
.. ..
 
..
. . 0 .

Kevin James Euclidean Structure

Remark
Any n-dimensional Euclidean space with an inner product is isomorphic
to Rn with the standard dot product.
If x1 , · · · , xn is an orthonormal basis, then for any x ∈ X we can write
n
X
x= aj xj ,
j=1

n
X
where aj = (x, xj ). Moreover, if y = bk xk , then
k=1
 
Xn n
X n
X n
X
(x, y ) =  aj xj , bk xk  = (aj xj , bj xj ) = aj bj .
j=1 k=1 j=1 j=1

The mapping X → Rn , x 7→ (a1 , · · · , an ), where aj = (x, xj ), is an

isomorphism that carries the inner product of X to the standard dot
product of Rn .

Kevin James Euclidean Structure

Theorem
Every linear function ` ∈ X 0 can be written as `(x) = (x, y ) for some
y ∈ X . The mapping ` 7→ y is an isomorphism of the Euclidean space X
with its dual.

Proof.
Let x1 , · · · , xn be an orthonormal basis, and let bk = `(xk ). Put
n
X
y= bk xk .
k=1

One can check for all 1 ≤ k ≤ n, we have `(xk ) = (xk , y ), since

n
!
X
(xk , y ) = xk , bk xk = bk = `(xk ).
k=1

If two linear functions have the same value for all vectors that form a
basis, they have the same values for all vectors x.

Kevin James Euclidean Structure

Definition
Let Y be a subspace of X . The orthogonal complement of Y is the set
{x ∈ X : (x, y ) = 0 for any y ∈ Y }. We denote it by Y ⊥ .

Remark
This is consistent with our previous definition of

Y ⊥ = {` ∈ X 0 : (`, y ) = 0 for any y ∈ Y }.

Theorem
For any subspace Y of X , we have X = Y ⊕ Y ⊥ .

Definition
The map PY : X → X , x = y + y ⊥ 7→ y , is called the
orthogonal projection of x into Y .

Kevin James Euclidean Structure

Theorem
PY is linear and idempotent (i.e., PY2 = PY ).

Theorem
Let Y ⊂ X be a subspace. Then,

||x − PY (x)|| = min{||x − w || : w ∈ Y }.

Proof.
Suppose that x ∈ X .
Write x = y + y ⊥ (-i.e. y = PY (x) ).
Then for any w ∈ Y , we have x − w = (y − w ) + y ⊥ .
Applying the Pythagorean theorem, we have

||x − w ||2 = ||y − w ||2 + ||y ⊥ ||2 ≥ ||y ⊥ ||2

with equality when w = y = PY (x).

Thus ||x − PY (x)|| = ||y ⊥ || = min{||x − w || : w ∈ Y }.

Kevin James Euclidean Structure

Definition
Let A : X → U be a linear map between Euclidean spaces. For any
u ∈ U, `(x) = (Ax, u) is a linear function X → R. By our previous
theorem, for some y ∈ X , `(x) = (x, y ) = (Ax, u).
The vector y ∈ X depends on u ∈ U, i.e., for some function A∗ : U → X ,
y = A∗ u. Thus, we have

(x, A∗ u) = (Ax, u).

We call A∗ the adjoint of A.

Note
If A ∈ Rm×n , and X = Rn , Y = Rm , and we identify X 0 and Y 0 with
R1×n and R1×m respectively, we have A∗ =T A.

Kevin James Euclidean Structure

Theorem
1 If A, B : X → U are linear, then (A + B)∗ = A∗ + B ∗ .
2 If A : X → U, C : U → V are linear, then (CA)∗ = A∗ C ∗ .
3 If A : X → X is 1 − 1, then (A−1 )∗ = (A∗ )−1 .
4 (A∗ )∗ = A.

Proof.
Let x ∈ X , u ∈ U, v ∈ V .
1 ((A + B)x, u) = (Ax, u) + (Bx, u) = (x, A∗ u) + (x, B ∗ u)
= (x, (A∗ + B ∗ )u).
2 (CAx, v ) = (Ax, C ∗ v ) = (x, A∗ C ∗ v ).
3 I = I ∗ = (A−1 A)∗ = A∗ (A∗ )−1 .
4 (Ax, u) = (u, Ax) = (A∗ u, x) = (u, (A∗ )∗ x) = ((A∗ )∗ x, u).

Kevin James Euclidean Structure

Definition
Let A be an m × n matrix, where m > n. Let p = (p1 , · · · , pm ), where
p1 , · · · , pm are the measured values. The system of equations given by
Ax = p where the number m of measurements exceeds the number n of
quantities in general does not have a solution. Such a system of
equations is overdetermined.

Remark
In such a situation, we seek that vector x that comes closest to satisfying
all the equations in the sense that makes ||Ax − p||2 as small as possible.
This is called a least square problem.

Kevin James Euclidean Structure

Theorem
Let A be an m × n matrix, m > n, and suppose that A has only the trivial
nullvector ~0. The vector x that minimizes ||Ax − p||2 is the solution z of

A∗ Az = A∗ p.

Proof.
Take p̂ = PRA (p).
Then ∃z such that Az = p̂ and z minimizes ||Ax − p||.
Also (p − p̂) ⊥ RA .
Thus ∀x ∈ X , we have 0 = ((p − p̂), Ax) = (A∗ (p − Az), x).
Thus A∗ p − A∗ Az = 0.
This proves that the solution z to A∗ Az = A∗ p minimizes ||Ax − p|| as
desired.
Suppose that z 0 is another solution to the above equation.
Then, taking y = z − z 0 we have that A∗ Ay = 0
Thus 0 = (A∗ Ay , y ) = (Ay , Ay ) = ||y ||2 .
Thus z − z 0 = y = 0.

Kevin James Euclidean Structure

Theorem
If PY is the orthogonal projection onto Y , the

PY∗ = PY .

Proof.
For all w , y ∈ Y , we have

(y , PY∗ (w )) = (PY (y ), w ) = (y , w ) = (y , PY (w )).

Thus, PY∗ (w ) = PY (w ) for all w ∈ Y .

Definition
A function M : X → X is an isometry if for all x, y ∈ X ,
||Mx − My || = ||x − y ||. (“Distance-preserving.”)

Example
Any translation Mx = x + a is an isometry.

Kevin James Euclidean Structure

Theorem
Let M be an isometric mapping of a Euclidean space into itself such that
M(0) = 0. Then,
1 M is linear
2 M ∗ M = I . Conversely if M ∗ M = I then M is an isometry.
3 M is invertible and M −1 is an isometry.
4 det(M) = ±1.

Definition
A matrix that maps Rn onto itself isometrically is called orthogonal.
The orthogonal matrices (for fixed n) form a group under matrix
multiplication called the orthogonal group denoted O(n).
The subgroup of matrices with determinant 1 is called the
special orthogonal group, denoted SO(n).

Kevin James Euclidean Structure

Proposition
A matrix M is orthogonal iff its columns form an orthonormal set.

Definition
We define the norm of a map A : X → U as ||A|| = sup||x||=1 ||Ax||.

Theorem
Let A : X → U be a mapping of Euclidean spaces. Then,
1 ||Az|| ≤ ||A||||z|| for all z ∈ X .
2 ||A|| = sup||x||=1;||v ||=1 (Ax, v ).

Kevin James Euclidean Structure

Theorem
Suppose we have linear maps A, B : X → U, C : U → U of Euclidean
spaces. Then,
1 ||kA|| = |k|||A||,
2 ||A + B|| ≤ ||A|| + ||B||,
3 ||CA|| ≤ ||C || · ||A||,
4 ||A∗ || = ||A||.

Theorem
Let A : X → X be an invertible linear map of Euclidean space into itself.
Suppose B : X → X has the property that ||A − B|| < ||A1−1 || . Then B is
invertible.

Note
We have assumed throughout the proof that dim(X ) < ∞. However, the
result also holds when dim(X ) = ∞.

Kevin James Euclidean Structure

Definition
A sequence of vectors xk in a Euclidean space X converges to the limit x
written limk→∞ xk = x if ||x − xk || → 0 as k → ∞.

Definition
A sequence of vectors {xk } in a Euclidean space X is called a
Cauchy sequence if ||xk − xj || → 0 as k, j → ∞.
A sequence {xk } of vectors in a Euclidean space X is called bounded if
||xk || ≤ R for all k where R is some real number.

Theorem
Let X be a finite dimensional Euclidean space.
1 X is complete (-i.e. all Cauchy sequences converge in X ).
2 X is locally compact (-i.e. every bounded sequence contains a
convergent subsequence).

Kevin James Euclidean Structure

Remark
We defined ||A|| = sup{||Ax|| | ||x|| = 1} but in this case its just
max{||Ax|| | ||x|| = 1}. Take a subsequence {xk } (with ||xk || = 1) for
which ||Axk || → ||A||. by theorem 7.16 this sequence has a subsequence
{xki } that converges to some x ∈ X . Now, ||Ae|| ≤ ||Ax|| for all unit
vectors e ∈ X . So, ||A|| = ||Ax||.

Theorem
If a Euclidean space X is locally compact then dim X < ∞.

Kevin James Euclidean Structure

Definition
A sequence {Ak } of maps X → U converges to a limit A if
limk→∞ ||An − A|| = 0.

Proposition
If dim X < ∞, then An → A iff An x → Ax for all x ∈ X .

Remark
This is not true if dim X = ∞.

Kevin James Euclidean Structure

Definition
Suppose that X is a finite dimensional vector space over C. For
z, w ∈ X , we define (z, w ) =T w̄ z =:H yx.

Proposition
1 Linear in x (-i.e. (kx, y ) = k(x, y )).
2 Skew linear in y (-i.e. (x, ky ) = k̄(x, y )).
3 Skew symmetric: (x, y ) = (y , x).
4 (x, x) > 0 for all x 6= 0. (positivity)
5 ||x + y ||2 = ||x||2 + 2Re((x, y )) + ||y ||2 .

Definition
The adjoint of a linear map A : X → U is defined to be the map
A∗ : U → X such that (x, A∗ u) = (Ax, u), ∀x ∈ X ; u ∈ U.

Kevin James Euclidean Structure

Definition
We define the adjoint as A∗ =H A =T Ā .


1

2 We say the set {qk } is orthonormal if H qi qj = δij .

3 An isometry of a complex Euclidean space is a map M : X → X
such that ||Mx − My || = ||x − y ||, ∀x, y ∈ X (as before). Such a
map is also called unitary.

Proposition
If M : X → X is unitary, then
1 M ∗M = I ,
2 M −1 is unitary,
3 The set of untiary maps of X forms a group,
4 | det(M)| = 1.

Definition
The norm of a linear map between complex spaces is defined as in the
real case: ||A|| = sup||x||=1 ||Ax||.

Kevin James Euclidean Structure