Вы находитесь на странице: 1из 29

# Institut fr Informatik

Classification using
Logistic Regression
Ingmar Schuster
Patrick Jhnichen
using slides by Andrew Ng
Logistic regression 2
This lecture covers

Logistic regression
hypothesis

Decision Boundary

Cost function
(why we need a new one

## !implified Cost function "

\$dvanced %ptimi&ation
\$lgorithms

'ulticlass classification
Logistic regression 3
Logistic regression
Hypothesis Representation
Logistic regression 4
Classification (ro)lems

Classification

Spam or Ham

## Human ace or no human ace

Positi!e Sentiment"

(in most simple case

*ant

if close to +

## Doesn,t )elong to class

if close to -
Logistic regression 5
Logistic .unction (!igmoid .unction

\$ asymptote for

## & asymptote for

!igmoid .unction (!/shape
Logistic .unction
Logistic regression 6

0ypothesis

Interpretation

## If classify as positive sentiment, malignant tumor, ...

Logistic regression 7
Logistic regression
(ecision boundary
Logistic regression 8

If
or e4uivalently
predict y ) &

If
or e4uivalently
predict y ) \$
Logistic regression 9
56ample

If
and

(rediction y 7 + whenever
Logistic regression 10
56ample

If
and

(rediction y 7 + whenever
Logistic regression 11
Logistic regression
*ost +unction
Logistic regression 12
Training and cost function

## Training data wih m datapoints1 n features

where

\$verage cost
13
Reusing Linear Regression cost

## Cost from linear regression

with logistic regression
hypothesis

cost

## Conve6 J easier to optimi&e

(no local optima
All unction !alues below
intersection with any line
All unction !alues below
intersection with any line
Logistic regression 14
Logistic Regression Cost function

## If y ) & and h,-. ) &1 *ost ) \$

But for

Corresponds to intuition8
if prediction is h,-. ) \$ )ut
actual value was y ) &1
learning algorithm will )e
penali&ed )y large cost
Logistic regression 15
Logistic Regression Cost function

## If y ) \$ and h,-. ) \$1 *ost ) \$

But for
Logistic regression 16
Logistic regression
Simpliied *ost +unction /
Logistic regression 17
!implified Cost .unction (+

## Because we always have y = 0 or y = 1 we can simplify

the cost function definition to

## To convince yourself1 use the simplified cost function to

calculate
Logistic regression 18
!implified Cost .unction (9

## To make predictions given new x output

Logistic regression 19

## #radient Descent to minimi&e logistic regression cost function

with identical algorithm as for linear regression
Logistic regression 20
Logistic regression 21
\$dvanced %ptimi&ation \$lgorithms

\$dvantages

## :o learning rate to choose

Comple6
%ptimi&ation \$lgorithms

## B.#! " L/B.#!

Logistic regression 22
(reimplemented \$lorithms

## \$dvanced optimi&ation algorithms e6ist already in 'achine

Learning packages for important languages

%ctave<'atla)

=ava

## Rapidminer > under the hood

Logistic regression 23
5ulticlass *lassiication
,by cheap trickery.
Logistic regression 24
'ulticlass classification pro)lems

## :um)er classes as +1 91 ?1 @@@

Logistic regression 25
Binary vs@ 'ulticlass Classification
Logistic regression 26
%ne versus all
Logistic regression 27

## Train logistic regression classifier

for each class i to predict pro)a)ility of y = i

## %n new x predict class i which satisfies

Logistic regression 28
This lecture covered

Logistic regression
hypothesis

Decision Boundary

new one

## !implified Cost function "

\$dvanced %ptimi&ation
\$lgorithms

'ulticlass classification
Machine Learning Introduction 29
(ictures

## Tumor picture by lickr2user bc

the path6 License CC SA NC

## Lightbulb picture rom

openclipart7org6 public domain