[MUSIC]

So, during the course, we discuss the

functional role of various brain regions.
So, we know now that the ventral striatum
plays
an important role in the processing of
anticipated values.
Also, the orbitofrontal cortex integrate
values into our decision
values and also plays an important role in
the learning.
Insular cortex meets an emotional arousal
signal that
is taken into account during the decision
making process.
Amygdala plays a vital role in the
calculation of the anticipated cost of our
decisions.
So dorsolateral prefrontal cortex is a key
region for the self-control during the
decision making.
But today, we will discuss the role of
these regions in decisions and the risk.
So already, the expected utility theory
suggested that we take
to account the uncertainty of the outcomes
during the decision making process.
We are never certain about the outcomes of
our decisions.
So really, the expected utility theory
suggests that expected utilities are a
product of the probability of the outcome
and the utility of the outcome.
So, it acknowledges that we are never
certain about the outcomes of our
decisions.
As Benjamin Franklin nicely stated, in
this world, nothing
can be said to be certain, except death
and taxes.
So, we're never certain about the outcomes
of our decisions.
Let's come back for a moment to the very
beginning of our course.
I hope that you remember the activity of
LIP neurons.
The decision making neurons that are
critical for the perceptual decisions.
In this case the monkey is making the
decision, a perceptual decision.
This monkey has to switch gates left or
right,
depending on the visual material presented
on the screen.
So as you remember that activity of the
LIP
neurons is rising slowly til a certain
decision's threshold.
We see how this neuron's accumulating
information
over time and the activity of these
neurons
increases faster if the decision is easier
and
more slower if the decision is more
difficult.
But in all cases, the activity of these
neurons reach a certain decision threshold
when the decision is finalized and monkey
switches his gaze in a certain direction.
Two prominent neuroeconomists, Platt and
Glimcher,
first investigate the activity of LIP
neurons.
So they conducted another study that
simplifies part of that.
This program, monkeys had to switch gates
left or
right depending on the color of the visual
cue.
So when cue changed color animals had to
switch gates, left or right.
And their correct decisions were rewarded
by an amount of juice.
In condition one, you're economist
manipulated the
amount of juice cessated with certain
decisions.
So, some decisions were cessated with
large
gains, with a larger amount of juice.
Other decisions were associated with the
smaller
gain, with the smaller quantity of juice.
So on the left side of the graph you see
the histogram
of the activity of this LIP neurons during
the decisions making process.
So, these results clearly show that
activity
of the LIP neurons during the decision
making process, when monkey anticipates a
large
gain, he's dramatically modulated by his
expectation.
So activity of the LIP neurons is
stronger, and
as you see here in this graph indicated by
the black line, during the expectation of
the larger
viewpoint as compared to the expectation
of the smaller viewer.
In the second condition, neuroeconomists
manipulate the probabilities
of the outcomes associated with their
correct decisions.
So some decisions were associated with a
high probability of the reward.
Others were associated with a small
probability of reward.
You see on the right side of this graph
the activity of the LIP neurons
was also stronger when monkeys expected
the rewards with a higher probability.
So, already in the decision making
process, a very fundamental decision
making process,
the neuron level is modulated both
by expected gains and probabilities of
outcomes.
So, our decision making metrics do take
into account
the variation of the outcomes during the
decision making process.
But the simple concept of the probability
doesn't capture other aspects of the
outcome variation.
For example, it doesn't really nicely
capture the idea of risk.
So, we do make decisions on the risk
and the outcome uncertainty, dramatically
affects our decisions.
So, we do make decisions on the risk.
For example, when we're playing roulette.
Or we make our decisions under very strong
uncertainty, when we invest money into the
stock market.
We are never certain in real life about
our outcomes, and this uncertainty
dramatically affects our decisions.
So, we can define risk in various ways.
For example, we can use a general common
sense definition of risk.
So risk increases with the perceived
chance that the bad outcome will occur.
According to this definition, animals also
experience risk and for them risk
increases with the perceived chance of
death, either through predation or
starvation.
But for economists, and also for decision
theorists, the concept
of risk is closely related to the concept
of uncertainty.
So as illustrated by this graph, the risk
is highest around the probability 0.5.
And it makes, because the risk is really
small
at 0 around the probability 0 of
probability 1.
Because we are almost certain about the
outcome of our decisions.
Risk here, is an inversely quadratic
function of probability,
that is minimal at p equal to 0, or 1.
And maximal at p equal to 0.5.
So risk is a form of uncertainty or
variance of outcome.
But now let's come back to the studies
of the dopamine neurons during the
processing of rewards.
I hope that you remember this experiment
when the
activity of dopamine neurons was recorded
during the conditioning procedure.
So here, on this graph, the activity of
the dopamine urine is represented during
the conditioning procedure.
You remember that at the beginning of the
conditioning
procedure, neurons primarily reacted to
the presentation of the reward.
But during the conditioning procedure
activity of these neurons jumped,
shift to the presentation of the cue
indicating the reward.
So at the end there was a conditioning
procedure dopamine
neurons primarily react to the cue that
predicts the reward.
So, it gives us an idea that neurons,
dopamine neurons are encoding the
expectation of the
reward as if, you remember, the activity
of
the dopamine neurons is proportional to
the anticipated gain.
So, here on this slide, you see a
histogram of the activity of the dopamine
neuron.
So you see a peak of activity after the
presentation of the condition
stimulus indicating the presentation of
the reward later on.
So, we nicely know this fact, this
transient activity
of the dopamine neurons, to the cue
predict angry words.
But also, what we can notice with this
graph, there is a sustained slow
activity later on, somewhere between the
conditioned stimulus and the reward.
We see a slow increase of activity of
the dopanuerologic neuron til the
presentation of the reward.
If we will make a closer look to this
sustained slowed activity.
We will find that this activity encodes
risk.
So, in an interesting new economic study,
the probability of the outcome was
manipulated.
So, here, you see activity of the dopamine
neurons, in different conditions.
When the q predicted reward was 0, 0.25,
0.5, 0.75, and was a probability of 1.
So we see here is this first, the dopamine
neurons react to
the presentation of the cue, of the
condition stimulus predicting the reward.
And this reaction, this transient, very
fast reaction is proportional
to the expected value, but later, as
indicated by the red
arrow, we see in the sustained slow
activity produced by dopamine neurons,
and this activity peaks as a probability
of 0.5.
So this activity, slow, sustained activity
dopaminergic neurons between the
transient response to the cue, and the
reward encodes risk.
To sum up, the sustained risk-related
response occurs between
the stimulus, between the conditions
stimulus and the reward.
And it occurs after this phasic, fast
response to the conditioned stimulus.
And this activity is sustained very slow,
and
this activity encodes the risk associated
with this cue.
We can also study neuron correlates of
risk in human brain.
So, we can apply neuron imaging
techniques and, quite simply, your
economics paradigms.
For example, here's this gate.
Subject has to guess which of two cards is
higher than the other one.
So, at the beginning of the trial subject
has to place a $1 bet on
one of the options, second card higher or
second card lower than the first card.
So, next subject is exposed to the first
card,
and after a few seconds subject sees a
second card.
So, if subject guessed correctly, subject
collects $1.
This is a very simple paradigm, but it's
already clear that the first card
immediately codes the risk associated with
the decision of the subject.
Of course, in this case for example,
subject decided
the second card will be lower than the
first one.
If the first card will be nine, it
indicates a very high
chance of the second card will indeed be,
be lower than this card.
Of course cards used in this study are
between one and ten.
So, the first card nicely signals the risk
associated with the subjects decision.
And the risk is highest around the
probability of outcome of 0.5.
So basically risk will be highest if the
first card will be five.
So we can discuss the brain activity
associated
with different levels of the risk in this
study.
Neureconomists separately analyzed early
and delayed
neuro activity in the ventral striatum,
and
here you see the results for the early
activity in the ventral striatum.
You see that the activity is proportional
to the expected value.
In this case it is proportional both to
the expected value and
to the probability of the outcome, because
the outcome was fixed $1.
You see here on the graph where blue dots
indicate the activity
of the ventral striatum, the activity is
proportional to the expected value.
If we will analyze the delayed activity in
the ventral striatum
we will see that this activity peaks
around the probability 0.5.
So, delayed activity indeed, encodes the
risk of variance, of the outcomes.
So, it supports the idea, that
dopaminergic neurons both encodes
their expected value and the risk
associated with the decision.
So a really fast, transient response
of the dopaminergic neurons encodes the
expected
values and later sustained slow activity
encodes the risk associated with the
decisions.
So, expected reward is immediately encoded
in the ventral striatum.
Risk also is coded in the reward-sensitive
dopamine neurons, but this signal is
delayed.
So, overall, dopamine neurons show a slow,
sustained reaction to the risk.
So, lets now introduce a formal
definition of risk.
So economists and decision theorists, they
divide uncertainty into two distinct
concepts.
Risk is the situation when probability of
the outcome's unknown and ambiguity is the
situation when we do not know precisely
the probabilities of our outcomes.
So, the situation when you play roulette
will be a decision on the risk.
Our situation when you invest money into
the
stock market will be a decision on
ambiguity.
So in the first case you fully know the
probabilities of the outcomes.
In the second case, in the stock market,
you didn't know the probabilities of the
outcomes.
So, to make the story even more
complicated, [COUGH] we can say that
uncertainty can
be divided into ambiguity and risk, so
depending whether the probabilities are
known or unknown.
But risk can be determined by various
aspects of the outcome variability.
For example, risk can be modulated by the
variance of the outcomes.
Also, risk can be modulated by the
symmetry of outcomes.
By the skewness of outcomes.
Or by the peakness of outcomes, by
kurtosis.
So different aspects of the outcome of the
ambiguity can modulate risk.
But most of the studies talk with some of
the variants as a determinant of the risk.
And finally we have to say that concepts
of ambiguity and risk are closely related.
So all animal's for example, they do not
have
enough medical skills to calculate the
probabilities of outcomes.
Animals, and also humans, have to learn
the probabilities
of outcomes through repeated sampling
gradually turning ambiguity into risk.
[MUSIC]
[BLANK_AUDIO]