Below is a list of topics and concepts that will be used throughout the course.

They should be part of your undergraduate curriculum, but it is frequently the c

ase that these concepts have not been used recently, especially by CS majors. So
you may need to freshen up. This list provides a guideline to do so.
1. Linear algebra.
- linear (vector) spaces: what they are, how they are represented, how they are
transformed one onto the other
- linear combinations: what it takes for a space to be a linear space.
- linear independence: what it means, what happens when a set of vectors is not
linearly independent; how an it be transformed into another set that is "equival
ent" and yet is made of linearly independent elements
- basis: what it is, how to transform from one to the other
- inner products, orthogonality: what it means; how to transform from an inner p
roduct to another; abstract definition, generalization to vector spaces beyond R
^n
- orthogonal matrices: definitions, properties
- Gram-Schmidt orthogonalization: procedure and its meaning
- range space, null space, rank: definitions, interpretation, calculation: How d
o you compute the (range/null/rank) of a matrix?
- eigenvalues, eigenvectors: definitions, interpretation; spectrum of a matrix
- symmetric matrices: definitions, properties
- solving linear systems of equations of the form Ax = b: you must be able to so
lve a linear system of equations by hand, provided it is solvable. You must also
be able to ...
- conditions under which the system above can be solved, and can be solved uniqu
ely: determine whether a linear system of equation is solvable (has at least one
solution), and whether it has multiple solutions; in the latter case, you must
be able to represent the set of all possible solutions analytically
- least-squares approximation to the solution of a linear system: If a linear sy
stem does not admit a solution, how do you "relax" it so that it admits infinite
ly many, and how can you determine a criterion to choose one particular solution
, among the infinitely many, that is in some sense "sensible".
- (optional: SVD)
2. Calculus and numerical analysis.
- limits: you must understand the concept of limit; epsilon-delta and all that;
be facile with computing limits and derivatives
- differentiation
- Riemann integral: how to compute the integral of a function that is represente
d using samples, as opposed to analytically.
- Gauss-Newton methods, conjugate gradient, Levemberg-Marquardt: how to minimize
a scalar-valued smooth function iteratively; conditions under which you are gua
ranteed to converge to the global minimum; necessary conditions for a local mini
mum
- Constrained optimization, Lagrangian multipliers: how to frame a constrained o
ptimization problem into an unconstrained one.
- trigonometric functions, Fourier series: series of functions (polynomials, sin
/cos, exponentials)
- solution of linear system of ordinary differential equations x-dot = A x; you
must be able to solve a linear system of equations with constant coefficients; y
ou must have a procedure to compute, or at least approximate, matrix powers and
matrix exponentials
3. Basic probability and stochastic processes
- probability space: what it is, examples
- random variables: definition, interpretation, understand the difference betwee
n a random variable, its probabilistic description (distribution, density, momen

ts etc.), and its realization (sample)

- expectation: definition, interpretation, examples
- conditional expectation: definition (it is a random variable), interpretation
- moments: definition, interpretation; second- and third-order moments
- marginalization: how to "integrate out" a random variable.
- Bayes' rule: definition, interpretation
- law of large numbers, central limit theorem: definition, interpretation, impli
cations
- stochastic process: definition, interpretation, understand the difference betw
een a random process, its probabilistic description, its realizations
- correlation function: definition, interpretation