2.

SIGNAL DETECTION THEORY

J. Elder PSYC 6256 Principles of Neural Coding

Signal Detection Theory

2

Probability & Bayesian Inference

Provides a method for characterizing human performance in detecting, discriminating and estimating signals. For noisy signals, provides a method for identifying the optimal detector (the ideal observer) and for expressing human performance relative to this. Origins in radar detection theory Developed through the 1950s and on by Peterson, Birdsall, Fox, Tanner, Green & Swets

PSYC 6256 Principles of Neural Coding

J. Elder

Example 1
3

Probability & Bayesian Inference

The observer sits in a dark room On every trial, a dim light will be flashed with 50% probability. The observer indicates whether she believes the light was flashed or not. This is a yes-no detection task.

PSYC 6256 Principles of Neural Coding

J. Elder

Noise
4

Probability & Bayesian Inference

In this example, the information useful for the task is the light energy of the stimulus. By the time the stimulus information is received by decision centres in the brain, it will be corrupted by many sources of noise:

photon noise isomerization noise neural noise

Many of these noise sources are Poisson in nature: the dispersion increases with the mean.
PSYC 6256 Principles of Neural Coding J. Elder

Equal-Variance Gaussian Case

5

Probability & Bayesian Inference

It is often possible to approximate this noise as Gaussian-distributed, with the same variance for both stimulus conditions. Then the noise is independent of the signal state.

PSYC 6256 Principles of Neural Coding

J. Elder

Discriminability d
6

Probability & Bayesian Inference

p x | S = sH =

x H exp 2 2 2 1 x L exp 2 2 2 1

) )

p x | S = sL =

signal separation H L d'= = signal dispersion

H L

PSYC 6256 Principles of Neural Coding

J. Elder

Criterion Threshold
7

Probability & Bayesian Inference

The internal response is often approximated as a continuous variable, called the decision variable. But to yield an actual decision, this has to be converted to a binary variable (yes/no). A reasonable way to do this is to define a criterion threshold z:
x z ' yes' x < z 'no'
x z

PSYC 6256 Principles of Neural Coding

J. Elder

Effect of Shifting the Criterion

8

Probability & Bayesian Inference

PSYC 6256 Principles of Neural Coding

J. Elder

How did we calculate these numbers?

9

Probability & Bayesian Inference

p x | S = sH =

x H exp 2 2 2 1 x L exp 2 2 2 1

) )

p x | S = sL =

d ' = zFA zHIT

H L

PSYC 6256 Principles of Neural Coding

J. Elder

What is the right criterion to use?

10

Probability & Bayesian Inference

Suppose the observer wants to maximize the expected number of times they are right. Then the optimal decision rule is to always select the state s with higher probability for the observed internal response x:

( ) 1 ' yes ' p (x | s ) p (x | s ) < 1 ' no ' p (x | s )

p x | sH
L H L

The likelihood ratio test

This is the maximum likelihood detector. For the equal-variance case, this means that the criterion is the average of the two signal levels:
1 z = L + H 2

PSYC 6256 Principles of Neural Coding

J. Elder

Optimal Performance
11

Probability & Bayesian Inference

The performance of the maximum likelihood observer for this yes/no task is given by
d p(correct) = p(HIT) = p(CORRECT REJECT) = erfc 2 2

PSYC 6256 Principles of Neural Coding

J. Elder

Bias
12

Probability & Bayesian Inference

For this optimal decision rule, the different types of errors are balanced: p(FA) = p(MISS) For observers that use a different criterion, the different types of errors will be unbalanced. Such observers have lower p(correct) and are said to be biased.

PSYC 6256 Principles of Neural Coding

J. Elder

ROC Curves
13

Probability & Bayesian Inference

Suppose the experiment is repeated many times under different instructions. The first time, the observer is instructed to be extremely stringent in their criterion, only reporting yes when they are 100% sure the light was flashed. On subsequent repetitions, the observer is instructed to gradually relax their criterion.

PSYC 6256 Principles of Neural Coding

J. Elder

ROC Curves
14

Probability & Bayesian Inference

As the criterion threshold is swept from right to left, p(HIT) increases, but p(FA) also increases. The resulting plot of p(HIT) vs p(FA) is called a receiveroperating characteristic (ROC).

Increasing d
d = 0

PSYC 6256 Principles of Neural Coding

J. Elder

ROC Curves
15

Probability & Bayesian Inference

Note that d remains fixed as the criterion is varied! Thus d is criterion-invariant, and is thus a pure reflection of the signal-to-noise ratio.

PSYC 6256 Principles of Neural Coding

J. Elder

Example 2: Motion Direction Discrimination

16

Probability & Bayesian Inference

Britten et al (1992) Random dot kinematogram Signal dots are either all moving up or all moving down Noise dots are moving in random directions

PSYC 6256 Principles of Neural Coding

J. Elder

100% Coherence
17

Probability & Bayesian Inference

PSYC 6256 Principles of Neural Coding

J. Elder

30% Coherence
18

Probability & Bayesian Inference

PSYC 6256 Principles of Neural Coding

J. Elder

5% Coherence
19

Probability & Bayesian Inference

PSYC 6256 Principles of Neural Coding

J. Elder

0% Coherence
20

Probability & Bayesian Inference

PSYC 6256 Principles of Neural Coding

J. Elder

The Medial Temporal Area (V5)

21

Probability & Bayesian Inference

PSYC 6256 Principles of Neural Coding

www.thebrain.mcgill.ca

J. Elder

Experimental Details
22

Probability & Bayesian Inference

Signal direction always in preferred or antipreferred direction for cell. What kind of task is this? Note that now there is external noise as well as internal noise. To calculate neural discrimination performance, assumed neuron paired with identical neuron, tuned to opposite direction of motion.

PSYC 6256 Principles of Neural Coding

J. Elder

Behaviour

Anti-Preferred Direction

Neuron

Preferred Direction

Hit Rate

False Alarm Rate

Priors
25

Probability & Bayesian Inference

Note that if the probabilities of the two signal states are not equal, the maximum likelihood observer will be suboptimal. In this case we must make use of the posterior ratio.

( ) 1 ' yes ' p (s | x ) p (s | x ) < 1 ' no ' p (s | x )

p sH | x
L H L

Maximum a posteriori (MAP) rule

PSYC 6256 Principles of Neural Coding

J. Elder

MAP Inference
26

Probability & Bayesian Inference

Using Bayes rule, we obtain:

p sH | x
L

( ) = p ( x | s ) p (s ) p (s | x ) p ( x | s ) p (s )
H H L L

Thus we simply scale the likelihoods by the priors.

PSYC 6256 Principles of Neural Coding

J. Elder

Loss and Risk

27

Probability & Bayesian Inference

Maximizing p(correct) is not always the best thing to do. How would you adjust your criterion if you were

venture capitalist trying to detect the next Google? A pilot looking for obstacles on a runway?

PSYC 6256 Principles of Neural Coding

J. Elder

Loss Function
28

Probability & Bayesian Inference

In general, different types of correct decision or action will yield different payoffs, and different types of errors will yield different costs. These differences can be accounted for through a loss function:
Let a( x) represent the action of the observer, given internal response x.
Then L s, a( x) represents the cost of taking action a, given world state s.

PSYC 6256 Principles of Neural Coding

J. Elder

The Ideal Observer

29

Probability & Bayesian Inference

The Ideal Observer uses the decision rule that minimizes the Expected Loss, aka the Risk R(a|x):
R(a | x) = L s, a( x) p(s, x) = L s, a( x) p( x | s )p(s )
s s

PSYC 6256 Principles of Neural Coding

J. Elder

Example 3: Slant Estimation

30

Probability & Bayesian Inference

PSYC 6256 Principles of Neural Coding

J. Elder