Prosecutor's fallacy - Wikipedia, the free encyclopedia

1 of 6

https://en.wikipedia.org/wiki/Prosecutor's_fallacy

Prosecutor's fallacy

From Wikipedia, the free encyclopedia

The prosecutor's fallacy is a fallacy of statistical reasoning, typically used by the prosecution to argue for the
guilt of a defendant during a criminal trial. Although it is named after prosecutors it is not specific to them, and
some variants of the fallacy can be utilized by defense lawyers arguing for the innocence of their client. At its
heart the fallacy involves assuming that the prior probability of a random match is equal to the probability that
the defendant is innocent. For instance, if a perpetrator is known to have the same blood type as a defendant and
10% of the population share that blood type, then to argue on that basis alone that the probability of the
defendant being guilty is 90% makes the prosecutor's fallacy (in a very simple form).

Contents
1 Concept
2 Examples of prosecutor's fallacies
2.1 Conditional probability
2.2 Multiple testing
3 Mathematical analysis
4 Legal impact
5 Defense attorney's fallacy
5.1 Possible examples of fallacious defense arguments
6 The Sally Clark case
7 See also
8 References
9 External links

Concept
The terms "prosecutor's fallacy" and "defense attorney's fallacy" were originated by William C. Thompson and
Edward Schumann in 1987.[1][2] The fallacy can arise from multiple testing, such as when evidence is compared
against a large database. The size of the database elevates the likelihood of finding a match by pure chance
alone; i.e., DNA evidence is soundest when a match is found after a single directed comparison because the
existence of matches against a large database where the test sample is of poor quality may be less unlikely by
mere chance.

The basic fallacy results from misunderstanding conditional probability and neglecting the prior odds of a
defendant being guilty before that evidence was introduced. When a prosecutor has collected some evidence
(for instance a DNA match) and has an expert testify that the probability of finding this evidence if the accused
were innocent is tiny, the fallacy occurs if it is concluded that the probability of the accused being innocent must
be comparably tiny. If the DNA match is used to confirm guilt which is otherwise suspected then it is indeed
strong evidence. However if the DNA evidence is the sole evidence against the accused and the accused was
picked out of a large database of DNA profiles, the odds of the match being made at random may be increased,
and less damaging to the defendant. The odds in this scenario do not relate to the odds of being guilty, they
relate to the odds of being picked at random. While the odds of being picked at random may be low for an
individual condition implying guilt, ie a positive DNA match, the probability of being picked at random for any
5/2/2016 3:10 PM

Prosecutor's fallacy - Wikipedia, the free encyclopedia

2 of 6

https://en.wikipedia.org/wiki/Prosecutor's_fallacy

condition diverges to the infinite as more conditions are considered, as is the case in multiple testing. It is often
the case that both innocence and guilt (ie, accidental death and murder) are both highly improbable, though
naturally one must be true, so the ratio of the likelihood of the "innocent scenario" to the "guilty scenario" is
much more informative than the probability of the "guilty scenario" alone.

Examples of prosecutor's fallacies

Conditional probability

Argument from rarity Consider this case: a lottery winner is accused of cheating, based on the improbability
of winning. At the trial, the prosecutor calculates the (very small) probability of winning the lottery without
cheating and argues that this is the chance of innocence. The logical flaw is that the prosecutor has failed to
account for the large number of people who play the lottery. While the probability of any singular person
winning is quite low, the probability of any person winning the lottery, given the number of people who play it,
is very high.
Berkson's paradox mistaking conditional probability for unconditional led to several wrongful convictions
of British mothers, accused of murdering two of their children in infancy, where the primary evidence against
them was the statistical improbability of two children dying accidentally in the same household (under
"Meadow's law"). Though multiple accidental (SIDS) deaths are rare, so are multiple murders; with only the
facts of the deaths as evidence, it is the ratio of these (prior) improbabilities that gives the correct "posterior
probability" of murder.[3]

Multiple testing

In another scenario, a crime-scene DNA sample is compared against a database of 20,000 men. A match is
found, that man is accused and at his trial, it is testified that the probability that two DNA profiles match by
chance is only 1 in 10,000. This does not mean the probability that the suspect is innocent is 1 in 10,000. Since
20,000 men were tested, there were 20,000 opportunities to find a match by chance.
Even if none of the men in the database left the crime-scene DNA, a match by chance to an innocent is more
likely than not. The chance of getting at least one match among the records is:

So, this evidence alone is an uncompelling data dredging result. If the culprit was in the database then he and
one or more other men would probably be matched; in either case, it would be a fallacy to ignore the number of
records searched when weighing the evidence. "Cold hits" like this on DNA databanks are now understood to
require careful presentation as trial evidence.

Mathematical analysis

Finding a person innocent or guilty can be viewed in mathematical terms as a form of binary classification. If E
is the observed evidence, and I stands for "accused is innocent" then consider the conditional probabilities:
P(E|I) is the probability that the "damning evidence" would be observed even when the accused is
innocent (a "false positive").

5/2/2016 3:10 PM

Prosecutor's fallacy - Wikipedia, the free encyclopedia

3 of 6

https://en.wikipedia.org/wiki/Prosecutor's_fallacy

P(I|E) is the probability that the accused is innocent, despite the evidence E.

With forensic evidence, P(E|I) is tiny. The prosecutor wrongly concludes that P(I|E) is comparatively tiny. (The
Lucia de Berk prosecution is accused of exactly this error,[4] for example.) In fact, P(E|I) and P(I|E) are quite
different; using Bayes' theorem:

Where:

P(I) is the probability of innocence independent of the test result (i.e. from all other evidence) and
P(E) is the prior probability that the evidence would be observed (regardless of innocence):
P(E|~I) is the probability that the evidence would identify a guilty suspect (not give a false negative).
This is usually close to 100%, slightly increasing the inference of innocence over a test without false
negatives. That inequality is concisely expressed in terms of odds:

The prosecutor is claiming a negligible chance of innocence, given the evidence, implying Odds(I|E) -> P(I|E),
or that:
A prosecutor conflating P(I|E) with P(E|I) makes a technical error whenever Odds(I) >> 1. This may be a
harmless error if P(I|E) is still negligible, but it is especially misleading otherwise (mistaking low statistical
significance for high confidence).

Legal impact

Though the prosecutor's fallacy typically happens by mistake,[5] in the adversarial system lawyers are usually
free to present statistical evidence as best suits their case; retrials are more commonly the result of the
prosecutor's fallacy in expert witness testimony or in the judge's summation.[6]

Defense attorney's fallacy

Suppose there is a one-in-a-million chance of a match given that the accused is innocent. The prosecutor says
this means there is only a one-in-a-million chance of innocence. But if everyone in a community of 10 million
people is tested, one expects 10 matches even if all are innocent. The defense fallacy would be to reason that
"10 matches were expected, so the accused is no more likely to be guilty than any of the other matches, thus the
evidence suggests a 90% chance that the accused is innocent." and "As such, this evidence is irrelevant." The
first part of the reasoning would be correct only in the case where there is no further evidence pointing to the
defendant. On the second part, Thompson & Schumann wrote that the evidence should still be highly relevant
because it "drastically narrows the group of people who are or could have been suspects, while failing to
exclude the defendant" (page 171).[1][7]
5/2/2016 3:10 PM

Prosecutor's fallacy - Wikipedia, the free encyclopedia

4 of 6

https://en.wikipedia.org/wiki/Prosecutor's_fallacy

Possible examples of fallacious defense arguments

Authors have cited defense arguments in the O. J. Simpson murder trial as an example of this fallacy regarding
the context in which the accused had been brought to court: crime scene blood matched Simpson's with
characteristics shared by 1 in 400 people. The defense argued that a football stadium could be filled with
Angelenos matching the sample and that the figure of 1 in 400 was useless.[8][9]

Also at the O. J. Simpson murder trial, the prosecution presented evidence that Simpson had been violent
toward his wife, while the defense argued that there was only one woman murdered for every 2500 women who
were subjected to spousal abuse, and that any history of Simpson being violent toward his wife was irrelevant to
the trial. However, the reasoning behind the defense's calculation was fallacious. According to author Gerd
Gigerenzer, the correct probability requires the contextthat Simpson's wife had not only been subjected to
domestic violence, but subjected to domestic violence and murderedto be taken into account. Gigerenzer
writes "the chances that a batterer actually murdered his partner, given that she has been killed, is about 8 in 9 or
approximately 90%".[10] While most cases of spousal abuse do not end in murder, most cases of spousal murder
have a history of spousal abuse.

The Sally Clark case

Sally Clark, a British woman who was accused in 1998 of having killed her first child at 11 weeks of age and
then her second child at 8 weeks of age. The prosecution had expert witness Sir Roy Meadow, a professor and
consultant paediatrician,[11] testify that the probability of two children in the same family dying from SIDS is
about 1 in 73 million. That was much less frequent than the actual rate measured in historical data Meadow
estimated it from single-SIDS death data, and the assumption that the probability of such deaths should be
uncorrelated between infants.[12]

Meadow acknowledged that 1-in-73 million is not an impossibility, but argued that such accidents would
happen "once every hundred years" and that, in a country of 15 million 2-child families, it is vastly more likely
that the double-deaths are due to Mnchausen syndrome by proxy than to such a rare accident. However, there
is good reason to suppose that the likelihood of a death from SIDS in a family is significantly greater if a
previous child has already died in these circumstances (a genetic predisposition to SIDS is likely to invalidate
that assumed statistical independence[13]) making some families more susceptible to SIDS and the error an
outcome of the ecological fallacy.[14] The likelihood of two SIDS deaths in the same family cannot be soundly
estimated by squaring the likelihood of a single such death in all otherwise similar families.[15]

1-in-73 million greatly underestimated the chance of two successive accidents, but, even if that assessment were
accurate, the court seems to have missed the fact that the 1-in-73 million number meant nothing on its own. As
an a priori probability, it should have been weighed against the a priori probabilities of the alternatives. Given
that two deaths had occurred, one of the following explanations must be true, and all of them are a priori
extremely improbable:
1. Two successive deaths in the same family, both by SIDS
2. Double homicide (the prosecution's case)
3. Other possibilities (including one homicide and one case of SIDS)

It's unclear that an estimate for the second possibility was ever proposed during the trial, or that the comparison
of the first two probabilities was understood to be the key estimate to make in the statistical analysis assessing
the prosecution's case against the case for innocence.
5/2/2016 3:10 PM

Prosecutor's fallacy - Wikipedia, the free encyclopedia

5 of 6

https://en.wikipedia.org/wiki/Prosecutor's_fallacy

Mrs. Clark was convicted in 1999, resulting in a press release by the Royal Statistical Society which pointed out
the mistakes.[16]
In 2002, Ray Hill (Mathematics professor at Salford) attempted to accurately compare the chances of these two
possible explanations; he concluded that successive accidents are between 4.5 and 9 times more likely than are
successive murders, so that the a priori odds of Clark's guilt were between 4.5 to 1 and 9 to 1 against.[17]
After it was found that the forensic pathologist who had examined both babies had withheld exculpatory
evidence, a higher court later quashed Sally Clark's conviction, on 29 January 2003. [18]

Sally Clark, a practising solicitor before the conviction and her three year imprisonment, developed a number of
serious psychiatric problems including serious alcohol dependency and died in 2007 from acute alcohol
poisoning.[19][20]

See also

Base rate fallacy

Representativeness heuristic
False positive
False positive paradox
Likelihood function
Howland will forgery trial
People v. Collins
Conditional probability fallacy
Lucia de Berk
Simpson's paradox
Data dredging

References

1. Thompson, E.L.; Shumann, E. L. (1987). "Interpretation of Statistical Evidence in Criminal Trials: The Prosecutor's
Fallacy and the Defense Attorney's Fallacy". Law and Human Behavior 2 (3): 167. doi:10.1007/BF01044641.
JSTOR 1393631.
2. Fountain, John; Gunby, Philip (February 2010). "Ambiguity, the Certainty Illusion, and Gigerenzer's Natural
Frequency Approach to Reasoning with Inverse Probabilities" (PDF). University of Canterbury. p. 6.
3. Goldacre, Ben (2006-10-28). "Prosecuting and defending by numbers". The Guardian. Retrieved 2010-05-22. "rarity
is irrelevant, because double murder is rare too. An entire court process failed to spot the nuance of how the figure
should be used. Twice."
4. Meester, R.; Collins, M.; Gill, R.; van Lambalgen, M. (2007-05-05). "On the (ab)use of statistics in the legal case
against the nurse Lucia de B". Law, Probability & Risk 5 (34): 233250. arXiv:math/0607340. doi:10.1093/lpr
/mgm003. "[page 11] Writing E for the observed event, and H for the hypothesis of chance, Elffers calculated P(E |
H) < 0.0342%, while the court seems to have concluded that P(H | E) < 0.0342%"
5. Rossmo, D.K. (October 2009). "Failures in Criminal Investigation: Errors of Thinking". The Police Chief. LXXVI
(10). Retrieved 2010-05-21. "The prosecutor's fallacy is more insidious because it typically happens by mistake."
6. "DNA Identification in the Criminal Justice System" (PDF). Australian Institute of Criminology. 2002-05-01.
Retrieved 2010-05-21.
7. N. Scurich (2010). "Interpretative Arguments of Forensic Match Evidence: An Evidentiary Analysis". The Darmouth
Law Journal 8 (2): 3147. SSRN 1539107. "The idea is that each piece of evidence need not conclusively establish a
proposition, but that all the evidence can be used as a mosaic to establish the proposition"

5/2/2016 3:10 PM

Prosecutor's fallacy - Wikipedia, the free encyclopedia

6 of 6

https://en.wikipedia.org/wiki/Prosecutor's_fallacy

8. Robertson, B., & Vignaux, G. A. (1995). Interpreting evidence: Evaluating forensic evidence in the courtroom.
Chichester: John Wiley and Sons.
9. Rossmo, D. Kim (2009). Criminal Investigative Failures. CRC Press Taylor & Francis Group.
10. Gigerenzer, G., Reckoning with Risk: Learning to Live with Uncertainty, Penguin, (2003)
11. http://reporter.leeds.ac.uk/428/mead.htm
12. The population-wide probability of a SIDS fatality was about 1 in 1,303; Meadow generated his 1-in-73 million
estimate from the lesser probability of SIDS death in the Clark household, which had lower risk factors (e.g.
non-smoking). In this sub-population he estimated the probability of a single death at 1 in 8,500. See: Joyce, H.
(September 2002). "Beyond reasonable doubt" (pdf). plus.maths.org. Retrieved 2010-06-12.. Professor Ray Hill
questioned even this first step (1/8,500 vs 1/1,300) in two ways: firstly, on the grounds that it was biased, excluding
those factors that increased risk (especially that both children were boys) and (more importantly) because reductions
in SIDS risk factors will proportionately reduce murder risk factors, so that the relative frequencies of Mnchausen
syndrome by proxy and SIDS will remain in the same ratio as in the general population: Hill, Ray (2002). "Cot Death
or Murder? Weighing the Probabilities". "it is patently unfair to use the characteristics which basically make her a
good, clean-living, mother as factors which count against her. Yes, we can agree that such factors make a natural
death less likely but those same characteristics also make murder less likely."
13. Gene find casts doubt on double 'cot death' murders (http://observer.guardian.co.uk/print
/0,,4221973-102285,00.html). The Observer; July 15, 2001
14. Vincent Scheurer. "Convicted on Statistics?". Retrieved 2010-05-21.
15. Hill, R. (2004). "Multiple sudden infant deaths coincidence or beyond coincidence?" (PDF). Paediatric and
Perinatal Epidemiology 18 (5): 321. doi:10.1111/j.1365-3016.2004.00560.x.
16. "Royal Statistical Society concerned by issues raised in Sally Clark case" (PDF). 23 October 2001. "Society does not
tolerate doctors making serious clinical errors because it is widely understood that such errors could mean the
difference between life and death. The case of R v. Sally Clark is one example of a medical expert witness making a
serious statistical error, one which may have had a profound effect on the outcome of the case"
17. The uncertainty in this range is mainly driven by uncertainty in the likelihood of killing a second child, having killed
a first, see: Hill, R. (2004). "Multiple sudden infant deaths coincidence or beyond coincidence?" (PDF). Paediatric
and Perinatal Epidemiology 18 (5): 322323. doi:10.1111/j.1365-3016.2004.00560.x.
18. http://www.bailii.org/ew/cases/EWCA/Crim/2003/1020.html
19. Shaikh, Thair (March 17, 2007). "Sally Clark, mother wrongly convicted of killing her sons, found dead at home".
London: The Guardian. Retrieved 2008-09-25.
20. http://news.bbc.co.uk/1/hi/england/essex/7082411.stm

External links
Discussion of the prosecutor's fallacy (http://web.archive.org/web/20030528061717/http:
//www.colchsfc.ac.uk:80/maths/dna/discuss.htm)
Forensic mathematics of DNA matching (http://dna-view.com/profile.htm)
A British statistician on the fallacy (http://www.ted.com/index.php/talks
/peter_donnelly_shows_how_stats_fool_juries.html)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Prosecutor%27s_fallacy&oldid=707236456"
Categories: Bayesian statistics Logical fallacies Forensic statistics Misuse of statistics
This page was last modified on 27 February 2016, at 17:27.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may
apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia is a registered
trademark of the Wikimedia Foundation, Inc., a non-profit organization.

5/2/2016 3:10 PM