Outline of the Course 11.

Overtting (May 8)

Regularization (May 10 )
The Learning Problem
12.

Validation
1. (April 3 )
(May 15 )
Is Learning Feasible?
13.

2. (April 5)
14. Support Ve tor Ma hines (May 17 )
3. The Linear Model I (April 10 )
15. Kernel Methods (May 22 )
4. Error and Noise(April 12 )
16. Radial Basis Fun tions (May 24 )

5. Training versus Testing (April 17 )

17. Three Learning Prin iples (May 29 )

6. Theory of Generalization (April 19 )

18. Epilogue (May 31 )

7. The VC Dimension (April 24 )

8. Bias-Varian e Tradeo (April 26 ) theory; mathemati al

The Linear Model II te hnique; pra ti al
9. (May 1)
analysis; on eptual
10. Neural Networks (May 3)
 Learning From Data
Yaser S. Abu-Mostafa
California Institute of Te hnology

Le ture 1: The Learning Problem

Sponsored by Calte h's Provost O e, E&AS Division, and IST Tuesday, April 3, 2012
 The learning problem - Outline
Example of ma hine learning

Components of Learning

A simple model

Types of learning

Puzzle

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 2/19
 Example: Predi ting how a viewer will rate a movie
10% improvement = 1 million dollar prize
The essen e of ma hine learning:

A pattern exists.

We annot pin it down mathemati ally.

We have data on it.

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 3/19
 Movie rating - a solution
r s ? ?
y ? n? kbuste ru i s e
d
e tio blo m C
m
o s a rs To
s s
like like prefe like

viewer

Mat h movie and add ontributions predi ted

viewer fa tors from ea h fa tor rating

movie

Tom
om
a t edy o
blo on nt
kb ten
ion nte

Cru
uste t

ise
r?

in i
t?
AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 4/19
 The learning approa h
top

viewer

movie

LEARNING

rating bottom

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 5/19
 Components of learning
Metaphor: Credit approval
Appli ant information:
age 23 years
gender male
annual salary $30,000
years in residen e 1 year
years in job 1 year
urrent debt $15,000

Approve redit?
AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 6/19
 Components of learning
Formalization:
Input: x ( ustomer appli ation)

Output: y (good/bad ustomer? )

Target fun tion: f : X Y (ideal redit approval formula)

Data: (x1, y1), (x2, y2), , (xN , yN ) (histori al re ords)

Hypothesis: g : X Y (formula to be used )

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 7/19
 UNKNOWN TARGET FUNCTION
f: X Y

(ideal credit approval function)

TRAINING EXAMPLES
( x1 , y1 ), ... , ( xN , yN )

(historical records of credit customers)

LEARNING FINAL
HYPOTHESIS
ALGORITHM g~
~f
A
(final credit approval formula)

HYPOTHESIS SET
H

(set of candidate formulas)

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 8/19
 Solution omponents
The 2 solution omponents of the learning UNKNOWN TARGET FUNCTION
f: X Y

problem: (ideal credit approval function)

TRAINING EXAMPLES

The Hypothesis Set ( x1 , y1 ), ... , ( xN , yN )

(historical records of credit customers)

H = {h} gH LEARNING FINAL

HYPOTHESIS
ALGORITHM g~
~f
A
The Learning Algorithm (final credit approval formula)

HYPOTHESIS SET
H

Together, they are referred to as the learning (set of candidate formulas)

model .

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 9/19
 A simple hypothesis set - the `per eptron'
For input x = (x1, , xd) `attributes of a ustomer'

d
Approve redit if wixi > threshold,
P
i=1
d
Deny redit if wixi < threshold.
P
i=1

This linear formula h H an be written as

d
! !
X
h(x) = sign wixi threshold
i=1

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 10/19
 d
! !
X
h(x) = sign wi xi + w0
i=1

Introdu e an arti ial oordinate x0 = 1: _ + _ +

_ _
+ +
+ +
d
!
X _ + _ +
h(x) = sign wi xi + +
_ _
i=0

`linearly separable' data

In ve tor form, the per eptron implements

h(x) = sign(wTx)

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 11/19
 A simple learning algorithm - PLA
The per eptron implements
h(x) = sign(wTx) y= +1 w+y x

Given the training set: x

w
(x1, y1), (x2, y2), , (xN , yN )

pi k a mis lassied point:

sign(wTxn) 6= yn y= 1 w

and update the weight ve tor: x

w+y x
w w + ynxn
AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 12/19
 Iterations of PLA
One iteration of the PLA:
w w + yx
where (x, y) is a mis lassied training point. _ +
_ +

At iteration t = 1, 2, 3, , pi k a mis lassied point from +

_ +
(x1, y1), (x2, y2), , (xN , yN )
+
and run a PLA iteration on it. _

That's it!

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 13/19
 The learning problem - Outline
Example of ma hine learning

Components of learning

A simple model

Types of learning

Puzzle

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 14/19
 Basi premise of learning
using a set of observations to un over an underlying pro ess

broad premise = many variations

Supervised Learning
Unsupervised Learning

Reinfor ement Learning

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 15/19
 Supervised learning
Example from vending ma hines  oin re ognition

25 25

Mass
Mass

5 5
1 1

10 10

Size Size

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 16/19
 Unsupervised learning
Instead of (input, orre t output), we get (input, ? )

Mass

Size

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 17/19
 Reinfor ement learning
Instead of (input, orre t output),
we get (input,some output,grade for this output)

The world hampion was

a neural network!

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 18/19
 A Learning puzzle

f = 1

f = +1

f =?

AM
L Creator: Yaser Abu-Mostafa - LFD Le ture 1 19/19