Notes on linear operators

Michael Snarski February 20, 2012

Abstract Linear algebra kicks ass, but we only know it in nite dimensionas. While the concepts are similar in both the nite and innite dimensional cases, and though one can draw intuition about the meaning of some operations from the nite case, the properties of operators may be drastically dierent in the innite case. The following attempts to provide some motivation and gives a brief and incomplete overview of what weve seen in Math 354 regarding linear operators. More specifically, we begin with the concept of operator norm, its implications, treat dual spaces a bit, and then give a few results and examples on compact operators.

Contents 1
1.1

Operator basics
Norm

Operator norm. Let (X, ), (Y, ) be normed vector spaces over a eld F, and let A : X Y be a linear operator. The space of bounded, linear operators from X to Y is denoted L(X, Y ). This is a vector space, and A = sup A(x) Y
xX x 1

is a norm on L(X, Y ). The motivation for this norm is straightforward. A natural way to measure the size of a function (an operator) is to see how much it scales elements of our space. If we think of an invertible matrix A : V V for some nitedimensional vector space V , in some ways det(A) measures this scaling (I am convinced there is a relationship between operator norms and the determinant, but I do not know what it is). If we simply took A = sup Ax Y ,
xX

without imposing the condition that x 1, the value Ax would depend more on the element x than the eect of A. Instead, we need to take a relative measurement, that is, A = sup
xX

Ax Y x X

or, in other words, sup

xX

Ax Y 1 = sup Ax x X x X xX x = sup A x xX = sup

xX x 1

Ax

which is our norm. Note that in the above calculation, x X F passes into the Y norm since both X and Y are over the same eld F. Moreover, this crucially depends on linearity of the operator. It is natural to study bounded operators, as these are naturally continuous operators. Proposition 1. If A : X Y is a linear operator, T.F.A.E.: i. A is bounded, i.e. A < . ii. A is continuous. Proof. (i. ii.) If A is bounded, there is M F such that A M , that is, Ax Y M for all x X. Then, Ax Ay = A(x y) = xy xy A A xy xy

M xy so that A is uniformly continuous, hence continuous. (ii. i.) If A is continuous, in particular it is continuous at 0X . Choose = 1, let be given, and consider some x X with x = 1. We have x 2 = 2 < , so A x 2 Ax < < 1(= ) 2 < 2

A = sup Ax <
xX

so A is bounded. Next, we note that if X is a Banach space, then the underlying eld F must also be Banach, where the norm is given by the absolute value ||. This is immediate since a vector space is closed under scalar multiplication, and any Cauchy sequence (an ) in F can be represented by a corresponding n=1 Cauchy sequence (an x) X for some x X with x = 1. More interestingly, if Y is Banach, then L(X, Y ) is also Banach.

Theorem 1. Suppose (X, ) is a normed space over F and (Y, ) is a Banach space over F. Then the normed vector space L(X, Y ) is complete, hence Banach. Proof. Let (An ) be a Cauchy sequence in L(X, Y ) (that is, each An is a n=1 linear operator from X to Y ). Then, for any > 0, there is N N such that n, m N , An Am < . As such, for any x X, An x Am x = (An Am )x An Am x , so (An x) is Cauchy in Y . Since Y n=1 is complete, lim An x exists. Denote this limit Ax. We need to show two n things: i. A is an actual element of L(X, Y ); ii. An A in L(X, Y ), i.e., An converges to A under the operator norm i. To show A L(X, Y ), we need to show A is linear and bounded. For linearity, let , F, x, y X.
n

lim An (x + y) = A(x + y) An (x + y) = An (x) + An (y) lim An (x + y) = lim An (x) + lim An (y)

n n

= A(x) + A(y) so A is linear. For boundedness, let M = sup An . For any x X with
nN

x = 1, An x An x M x =M
n

Then, since we dened A by Ax = lim An x, we have Ax An x Ax + An x An A

n

x +M x

lim An A x + An x M x = M so that Ax M x = M (because x = 1). Since x is an arbitrary element of norm 1 and this is a xed upper bound, we can take the supremum on the left hand side to nd sup
xX x 1

Ax M

and this is our norm on (X, Y ), so indeed A (X, Y ). ii. Let > 0, and get N such that An Am < 2 . For all x X with x = 1, An x Am x An Am x < 2 . Then, An x Ax An x Am x + Am x Ax < + Am A 2 Taking m on the right-hand side yields the desired result.

Clinotes and motivation. We have now seen that if X is a Banach space, both the base eld F and L(W, X) must be Banach, where W is any normed space. It then follows that L(X, F) is a Banach space. What does this mean? L(X, F) is the space of bounded linear operators sending elements of X to the base eld, F. We call these particular operators linear functionals, and L(X, F) the dual space of X, denoted X .

1.2

Dual spaces and dual operators.

Dual space. Let X be a vector space over a eld F. The (algebraic) dual of X is denoted by X . It is the space of all linear functionals on X: X = {f : X F | f (x + y) = f (x) + f (y), x, y X, , F} In the case where X is a Banach space, we consider the dual of X to be the space of bounded linear functionals, where the norm of the linear functional f : X F is given by f = sup f (x) X
xX x 1

The reason for which we treat the space of bounded linear operators (as opposed to the whole space of functionals, which is the algebraic dual) is simple: our space would not be Banach under our desired norm if we allowed for unbounded operators. Since boundedness is equivalent to continuity for linear operators, the subspace we treat is sometimes called the continuous dual. One can draw some basic intuition on the dual spaces by thinking of the nite-dimensional case. Given an ndimensional vector space V over a eld F, there exists a basis B = {v1 , . . . , vn } of precisely n vectors. The transposes vi of the vectors act as linear functionals on V by left multiplication, dened as for 1 n matrices. Alternatively, one can think of X as Hom(X, F), since any linear operator is a homomorphism of Fmodules (vector spaces over F). Note that this should only serve as an illustration of what an element of x does (send an element x X to F by x (x)); it does not accurately reect any of the properties of the innite-dimensional case. Dual of an operator. Let (X, ) and (Y, ) be two Banach spaces and let A : X Y be a bounded operator. The dual of A is denoted by A , and is dened by (A y )(x) = y (Ax). In particular, A is a bounded linear operator from Y to X . In other words, A sends linear functionals y in Y (linear operators which send elements of the space Y to the base eld, usually R or C) to linear functionals in X (which sends elements of X to the same base eld). Consider also how else we could possibly dene A . We want it to be an operator on dual spaces, that is, send linear functionals to linear functionals, and we already have a relationship between elements of X and Y (namely, Ax = y). It seems only natural that we would want to relate an element of X with a functional on Y by the eect of the functional on x under the action of A. 4

As in the nite-dimensional case, if A is represented by a matrix (aij ), A turns out to be the transpose of A, i.e. if (bij ) is the matrix representation of A , then aij = bji for all i, j. To see this, suppose A = (aij ), and A : V W . Let y W . The ith coordinate of (A y ) is gotten by taking x = ei , i.e. (A y )(ei ), just like 0 . . . (y1 , y2 , . . . , yn ) 1 = yi . . . . 0 Then, if (bij ) is the matrix representation of A , we have (A y ) = (By ) y1 ... y . . . 2 . . . . . . bij . . . yj . . . . . . + b y + + b y + . . . b11 y1 12 2 1j j j=1 b1j yj b21 y + b22 y + + b2j y + . . . b2j y 1 2 j j j=1 . . . . = = . . bij y + b22 y + + bij y + . . . bij yj 1 2 j j=1 . . . . . . b11 b12 . . . b21 b22 . . . . . . = bi1 bi2 . . . . . . b1j b2j

Thus, the ith coordinate of (By ) is

j=1

bij yj . But we also have that (A y )(x) =

y (Ax), so taking x = ei as before, we get a11 a12 . . . a1j . . . a1j a2j a21 a22 . . . a2j . . . 0 . . . . . . . . = . . . . Aei = ai1 ai2 . . . aij . . . 1 aij . . . . . . . . . . . . We then have that y (Aei ) =
Example Keeping this in mind, we see that relative to the basis B, vi (vj ) = j j j i , where i is the Kronecker-delta symbol (i = 1 if i = j and 0 otherwise). This example also illustrates why there is no canonical (coordinate-free) isomorphism from V to V , even in the nite-dimensional case where V and V are always isomorphic. In other words, one must always pick a basis for V (or V ) to construct an isomorphism.

Lemma 1. If A L(X, Y ), then for Ax = 0, sup |y (Ax)| = Ax .

y =1

Proof. sup |y (Ax)| sup

y =1 y =1

Ax = Ax

Now, Ax Y , and Ax = 0. By a Corollary to the Hahn-Banach theorem, if 0 = y0 Y , there is f Y such that |f (y0 )| = y0 and f = 1. Letting y0 = Ax, there is y Y such that |y (Ax)| = Ax so that the upper bound is achieved, and Ax = sup |y (Ax)|.
y =1

Proposition 2 (Duals preserve norm). Suppose A : X Y is a bounded linear operator, that is, A L(X, Y ) (X, Y Banach). Then A is a bounded linear operator from Y to X , i.e. A L(Y , X ), and A = A . Proof. First, recall that A is dened by (A y )(x) = y (Ax). By the above lemma, (note that it applies, since Ax = 0) A = sup
x =1

Ax sup |y (Ax)| sup |(A y )(x)|

= sup = sup

x =1 y =1 x =1 y =1

= sup sup |(A y )(x)|

y y x =1

= sup A y = A It follows that A is bounded. Linearity is a typing exercise.

Compact operators.

All spaces are assumed to be Banach. Denition An operator A L(X, Y ) is compact if cl{Ax | x X 1} is a compact subset of Y . In other words, the image of the closed unit ball is a precompact subset of Y (pre-compactness meaning before you take the closure). Note that, by Heine-Borel, any closed and bounded subset of a nite-dimensional space is compact. Proposition 3 (Compact operators form a closed subspace). Let X and Y be Banach spaces and An : X Y a sequence of operators such that lim An A = 0. If each An is compact, then A is also compact.

Proof. To show that A is compact, we need to show that {Ax | x 1} is pre-compact. Let (xn ) be a sequence in X such that xn 1 for each n. n=1 Consider the sequence (A1 xn ) . A1 is compact, so there is a convergent n=1 subsequence A1 x11 , A1 x12 , . . . , A1 x1k , = (A1 x1k ) . Next, look at the k=1 sequence (A2 x1k ); there is a convergent subsequence A2 x21 , . . . , A2 x2k , . . . . By the diagonal argument, we have a convergent subsequence xkk such that 6

(Aj xkk ) converges for all j. Thus, for any j, lim Aj xkk exists. Denote (xkk )
k

by (xnk ); we have by the triangle inequality and Ax A

Axnk Axnk = (A Aj )xnk + Aj (xnk xnk ) + (Aj A)xnk A Aj + Aj (xnk xnk ) + Aj A

Let then > 0, and choose j N such that A Aj < 3 . Fix this j = J; since lim Aj xnk exists for each j, for AJ there is K such that for any k, k K, AJ (xnk xnk ) AJ xnk xnk < 3 . We can then bound the above by 3 + 3 + 3 = and conclude that (Axnk ) is Cauchy, hence converges in Y since Y is Banach. k

Proposition 4 (Duals preserve compactness). If A L(X, Y ) is compact if and only if A is compact.

Proof. Let A be compact, and let (yn ) be a sequence in {y Y | y 1 Y . To show that A is compact, we must show that the sequence 1} = B (A yn ) X has a convergent subsequence. n=1 (The construction for this may seem a little convoluted; the most dicult part is simply keeping track of what is where, so it can be useful to draw diagrams to keep things straight.) Consider K = cl{Ax | x X, x 1}. Since A is compact, {Ax | x X} is relatively compact, and thus the closure is compact. The Arzela-Ascoli theorem then applies to the set K: a subset W of C(K) (the space of continuous functions on K) is compact if and only if W is closed, bounded and equicontinuous. To this end, dene a sequence fn : Y C by fn (y) = yn (y), where yn is of course the given sequence. Once we show this denes an equicontinuous family of functions, we can restrict fn to K (where the family remains equicontinuous), and by compactness we get a convergent subsequence (fnk ) for free. This means that lim fnk = f exists, so lim fnk (y) = lim ynk (y) = y (y)

exists, for each y Y . But then, since Ax = y Y ,

x 1

A ynk A ynl = sup |ynk (Ax) ynl (Ax)|

= sup |fnk (Ax) fnl (Ax)|

x 1 yK

sup |fnk (y) fnl (y)| < for suciently large k and l. It remains to show that the family is actually equicontinuous, but this is easy. For any n,
|fn (y) fn (y )| = |yn (y) yn (y )| = |yn (y y )| yn

yy

yy so fn is uniformly bounded by y y , which is independent of n. 7

2.1

Compact operators on

(N) spaces

Consider a doubly indexed sequence of numbers (aij ); this denes a linear operator A : p p , which can be visualized as an innite dimensional matrix a11 a12 a13 . . . a1j . . . a21 a22 a23 . . . a2j . . . a31 a32 a33 . . . a3j . . . . . . . . . ai1 ai2 ai3 . . . aij . . . . . . . . . For x
p,

Ax is dened as for matrix-vector multiplication: ... ... ... ... a1j a2j a3j . . . aij . . . ... x1 . . . x2 . . . x3 . = . . . . . xi . . . j=1 a1j xj j=1 a2j xj j=1 a3j xj aij xj j=1 . . . . . .

a11 a12 a13 a21 a22 a23 a31 a32 a33 . . . ai1 ai2 ai3 . . .

so that yi =
j=1

aij xj . It is a good idea to convince yourself that in the

nite case, A is the largest row sum (that is, sum all the elements of a row the largest such sum is A ) and A 1 is the largest column sum. This will give some intuition for the following examples and will make the conditions for compactness easier to remember. Example Let A = (aij ) as above, and take p = . Let

L = sup
i j=1

|aij | <

It is an important exercise to show that L = A . Note that this is exactly the same norm as in the nite case (that is, A = largest row sum), except now we have to take the maximum over all row sums; there are an innite amount, so we take the supremum. It is important to convince oneself this norm makes sense, as it can become confusing to work with the sums. Let (si ) be a sequence of positive numbers, and let K = {(xi )

(N) | |xi | si }.

We use the subscript i to emphasize that si should remind us of a bound on the sum over the ith row. It is another important exercise to show that K is compact if and only if lim si = 0.
i

If we now impose the overkill condition that

j=1

|aij | si for each i, we

will have that Ax lands in K, and hence A will be compact.

To see this, recall that A is compact if the image of the unit ball A(B) = {Ax | x

1} = {Ax = y | |xj | 1 j}

is relatively compact. This is easy once we note that

|yi | =
j=1

|aij ||xj | |aij |

j=1

si

Thus, A(B) K; since K is compact, it is closed, so clA(B) K. Then, since the closed subset of a compact set is compact, clA(B) is compact and so A is a compact operator.

We have thus shown that if

j=1

|aij | si for each i, A is compact. Note

that this means lim

|aij | = 0 (since lim si = 0).

j=1 i

Consider now the same situation, but in 1 instead of . Under which conditions do we get a compact set? It turns out that for an 1 , the set K = {(xn )
1

| |xn | |an | n}

is compact, just as in the p = case. In fact, this holds for any p [1, ]. However, the more general formulation of this is that a set is compact if we have a uniform bound on all the elements of the set. More precisely, a closed and bounded set A p is compact if, given > 0, there is N N such that

1 p

for all (xn ) A,

n=N

|xn |p

< .

One should consider this condition as a kind of analogue to the ArzelaAscoli theorem, which states that for (X, d) a compact metric space, a subset D C(X) (the space of continuous functions on X) is itself compact in C(X) if and only if it is closed, bounded and equicontinuous. The rst two properties are obvious (compact sets are always bounded, and closed in a Hausdor space), but equicontinuity is non-trivial. Constrast the denition of equicontinuity and uniform bound: > 0, > 0 : f D, d(x, y) < d(f (x), f (y)) <

1 p

> 0, N : x K,
n=N

|xn |p

<

These two denitions exemplify what one means by tightness at innity. We demonstrate this for the 1 case:

Example An operator A :

1,

A = (aij ) is compact if lim

|aij | =
i=1

0. This will ensure that the image set of the closed unit ball will satisfy the uniform bound condition above. Note that this is simply a sucient condition, not a necessary one. We need to show that for all > 0, there is an N such that for all y A(B) = {Ax | x 1 1}, |yn | < . Let then > 0 be given. We n=N rst get control over the innite tail by choosing J such that, for all j J,

|aij | < .
i=1 i=1

We now have j = 1, 2, . . . , J 1 to handle. Since for each of these,

|aij | is nite, there is Ij for each j such that

i=Ij

|aij | < for the given .

Hence, let I = max {Ij }, so that

1j<J i=I

|aij | < for all j = 1, . . . , J 1. Thus,

we have control over all j N. Everything up until now has been independent of (yn ). Let then (yn ) A(B) be given. We have y = Ax for some x B, so:

|yi | =
i=I i=I j=1

aij xj

|aij ||xj | =
i=I j=1 i=I j=1

|aij ||xj | |xj |

j=1

=
j=1

|xj |
i=I

|aij | <

=
j=1

|xj | = x

Thus, the set A(B) satises the uniform bound condition, and it is clearly bounded. Taking the closure will yield a closed, bounded, equibounded set, hence compact.

10