sigproc

October 12, 2013

Contents
1 Intro/ Review of background math 1.1 Discrete Time Signals . . . . . . . . . . . . . . . . . . . . . . 1.2 Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 4

Intro/ Review of background math

Remember that given a basis B = [b1 b2 . . . bn ] of a vector space (where the basis vectors are organized as columns of a matrix), we can nd the coordinates w.r.t the new basis of a vector x = [x1 x2 . . . xn ]T (where x has coordinates relative to the standard basis) by computing B 1 x. Key idea: the matrix B represents a mapping from vectors with coordinates w.r.t. the basis {bi } to vectors with coordinates given w.r.t. the standard basis. Hence the vector [1, 0, , 0]T B with respect to the B basis is the vector b1 w.r.t. the standard basis. Thus B 1 represents a mapping from vectors w.r.t the standard basis to vectors w.r.t the B basis If xB is the vector computed by B 1 x, then we can express x as a linear combination of the B basis: x = xB,1 b1 + xB,2 b2 + . . . + xB,n bn . This is called an expansion of x w.r.t B . If B = {b1 , . . . bn } is an orthonormal basis, then the coecients of the expansion of x w.r.t. B are xB,i = x, bi i } of B 1 are called the dual basis of B . The basis The rows {b

and its dual are a biorthogonal pair of sets of indexed vectors: i , bj = ij , where ij = 1 if i = j , 0 otherwise. b

General inner product satises 1

1. Distributivity 2. Linearity in the rst argument : x, y = x, y 3. Hermitian symmetry: x, y 4. Positive Deniteness : x=0

= y, x

x, x 0x V with x, x = 0

Note that (2) and (3) above imply the inner product is "conjugatelinear" in the second argument: x, y = x, y :

x, y = y, x = y, x

= y, x = x, y Complex Exponential

ej (n+) has (angular )frequency (in units radians), initial phase Can rotate a complex point z through radians CCW by multiplying by ej : z = zej = Re(z)Re(ej )Im(z)Im(ej )+j Re(z)Im(ej ) + Im(z)Re(ej ) = cos sin sin cos Re(z) Im(z)

Periodicity of discrete complex exponential: In discrete time the samples generated recursively by x[n] = ej x[n 1] (x[0] = ej ) may not be periodic discrete time exponential ejn is periodic = jM n M N = 1 n = kN, k N N 2, M, N N Easy to see, since e (hence for n = kN x[n] is equal to the initial condition)

1.1

Discrete Time Signals

A discrete-time signal is a complex-valued sequence indexed by integers. 2

 "sequence index n is best thought of as a measure of dimensionless time " A discrete time signal may arise from periodic sampling of an analog sample, so that x[n] = f (nT ) where f is a continuous function of time 1 representing the analog signal, and T is the sampling period and T the sampling frequency We often synthesize discrete time signals via the discrete time exponential x[n] = ejn+ or discrete samples of sinusoids (e.g. x[n] = A cos(0 n + )). Again, n is a dimensionless quantity, and is measured in radians. Clearly, complex exponential sequences with frequencies (0 + 2r), r Z) are indistinguishable from one another, so they can only have distinct frequencies in the range 0 0 < 2 . Discrete time signals and periodicity: In order for a discretetime complex exponential to have periodicity with period N , is must be the case that ej0 (n+N ) = ej0 n , which is only true for 0 N = 2k, k Z. Consequently, complex exponential and sinusoidal sequences are not necessarily periodic in n with period 2/0 , and will fail to be periodic at all if 0 is not a rational number. For example, consider the signals x1 [n] = cos(n/4), x2 [n] = cos(3/8). x1 clearly has period 8, but the smallest integer N for which x2 [n + N ] is N = 16, thus x2 has a period of N = 16, a longer period than that of x1 despite having a larger frequency. Integer restriction on n results in some discrete-time sinusoidal signals not being periodic at all. For example, there is no integer N such that the signal x3 [n] = cos(n) satises x3 [n + N ] = x3 [n] for all n. Dene energy and power of a discrete-time signal:

Ex = x

2 2

=
n= N 1

|x[n]|2

Px = lim

1 N 2N 3

|x[n]|2
N

 Discrete time signals with nite support have nite energy, and zero power. Those with innite support have innite energy. Many periodic signals have innite energy and nite power: For periodic sequences with period N , we dene power to be 1 Px = 2N
N 1

|x[n]|2
N

Note that this denition is required for periodic signals this additional denition is required because the sum in the original denition of power is undened.

1.2

Hilbert Spaces

Denition of Hilbert Space: A vector space H with an inner product f, g such that the norm dened by f = f, f

turns H into a complete metric space. That is, for any Cauchy sequence {xn }n H , limn xn = x H ("Cauchy implies convergent"). A sequence {xn }n is Cauchy convergent if > 0, N > 0 such that n, m > N = xn xm < Parsevals Theorem ("norm conservation"): an extension of the Pythagorean theorem to innite dimensional spaces. Orthogonal projection/"Orthogonality principle": given an orthonormal basis {bk }K k=1 of a subspace S (of potentially smaller dimen = sion than the superspace), the approximation of a vector x by x K x , b b is the minimizer arg min x y = x . This is yS k k k=1 k , x x =0 called the "orthogonal projection" because x Example of orthogonal basis vectors over the space of polynomial func1 tions on [1, 1]: b0 (x) = 1, b1 (x) = x. Then b0 , b1 = 1 1 xdx = 0. In general, a function that is symmetric (an even function) is orthogonal to a function that is antisymmetric (an odd function) "Finite-length and periodic signals live in CN , whereas innite-length signals live in 2 (Z)". 2 (Z) is the space of all square-summable funcn tions on Z (the set of all f such that i= f (i) < ). Both C and 2 (Z) t into the more category of Hilbert spaces. 4

 Lecture 3.3 had a pretty neat example of approximating f (t) = sin(t) 2 ([1, 1]) (where L2 [1, 1] is the space of square- integrable functions on [1, 1]) with an orthonormal basis for PN [1, 1] 2 (where PN [1, 1] is the space of polynomials of degree at most N 1 on [1, 1]). Used Gram-schmidt orthogonalization process to get the basis for PN , showed that this orthogonal projection was better than the taylor approximation of same degree (expected, since we said an orthogonal projection onto this space of polynomials minimizes the squared error over all projections onto the same space)