[FLORENTINA ANDRE/29114312]

Methods for Eliciting Probabilities

Preparing for probability assessment
Because of the danger of bias in probability assessment it is a good idea for the analyst to
prepare the ground carefully before eliciting the probabilities themselves.
Motivating
This phase is designed to introduce the decision maker to the task of assessing probabilities
and to explain the importance and purpose of the task. Sensitivity analysis should be used by
the analyst to identify those probabilities which need to be assessed with precision. Deliberate
bias is, of course, an undesirable characteristic of an input to an analysis whose intended
outcome is improved decision making.
Structuring
In the structuring phase the quantity to be assessed should be clearly dened and structured.
When the decision maker thinks that the quantity to be assessed depends on other factors it
may be simpler to restructure the assessment task, possibly by making use of a probability
tree.
Conditioning
The objective of this phase is to identify and thereby avoid the biases which might otherwise
distort the decision makers probability assessments. It involves an exploration of how the
decision maker approaches the task of judging probabilities.
Assessment methods
A number of different methods have been developed for assessing probabilities. Some of
these require a direct response from the decision maker in the form of a probability or
quantity, while others allow the probability to be inferred by observing the decision makers
choice between bets.
Assessment methods for individual probabilities
Direct assessments
The simplest way to elicit a probability from a decision maker is to pose a direct question
such as What is the probability that the product will achieve a break-even sales level next
month? Unfortunately, many people would feel uncomfortable with this sort of approach,
and they might be tempted to give a response without sufcient thought. Asking the
individual to mark a point on a scale which runs from 0 to 1 might be preferred because at
least the scale enables the probability to be envisaged.
The probability wheel
A probability wheel is a device like that shown in Figure 10.1, and it consists of a disk with
two different colored sectors, whose size can be adjusted, and a xed pointer.

Assessment methods for probability distributions

The probability method
There is evidence that, when assessing probability distributions, individuals tend to be
overcondent, so that they quote too narrow a range within which they think the uncertain
quantity will lie. Some assessment methods fail to counteract this tendency.
Graph drawing

In one approach the analyst produces a set of graphs, each representing a different probability
density function (pdf), and then asks the decision maker to select the graph which most
closely represents his or her judgment.
A comparison of the assessment methods
The main implication of these studies is that a variety of different methods should be used
during the elicitation process. Nevertheless, certain types of approach will obviously be more
acceptable than others to particular individuals.
Consistency and coherence checks
The use of different assessment methods will often reveal inconsistencies that can then be fed
back to the decision maker. These inconsistencies should act as a stimulant to more intense
thought which, hopefully, will result in greater insight and improved judgment.
Assessment of the validity of probability forecasts
A major measure of the validity of subjective probability forecasts is known as calibration.
By calibration we mean the extent to which the assessed probability is equivalent to
proportion correct over a number of assessments of equal probability. Perfect testretest
consistency is a necessary but not sufcient condition for perfect coherence, and perfect
coherence in turn is a necessary but not sufcient condition for perfect calibration. Given that
assessed probabilities are consistent and coherent then validity becomes an issue. Although
perfect calibration is the most desirable aspect of judgmental probability forecasts, in most
practical circumstances it may not be possible to measure this aspect of validity.
Assessing probabilities for very rare events
Such events are often of interest because of the disastrous consequences which may be
associated with their occurrence. Obviously, a probability wheel would be of little use in
assessing probabilities like these. There are, however, a number of ways in which the
problems of assessing very low probabilities can be tackled. Event trees and fault trees allow
the problem to be decomposed so that the combinations of factors which may cause the rare
event to occur can be identied. Each of the individual factors may have a relatively high
(and therefore more easily assessed) probability of occurrence. A log-odds scale allows the
individual to discriminate more clearly between very low probabilities.
Event trees
Each stage of the tree represents a factor which might, in combination with others, lead to the
catastrophe. Rather than being asked to perform the difcult task of directly estimating the
probability of catastrophic failure, the decision maker is asked instead to assess the
probability that each factor will contribute to it.
Fault trees
Sometimes it is easier to consider the problem from a different point of view. In contrast to
event trees, fault trees start with the failure or accident and then depict the possible causes of
that failure.
Using a log-odds scale
Because people generally have problems in distinguishing between probabilities like 0.001
and 0.0001 some analysts prefer to use what is known as a log-odds scale to elicit the
probabilities of rare events. An analyst using the scale would ask the decision maker to mark
the point that represents his or her assessment of the likelihood of the event occurring. The
scale also ranges from minus to plus innity, which makes it very difcult for individuals to
assert that particular events are impossible or certain to occur.