Prosecutor's fallacy

The prosecutor's fallacy is a fallacy commonly occurring in criminal trials and elsewhere. A prosecutor[?] has collected some evidence (for instance a DNA match) and has an expert testify that the probability of finding this evidence if the accused were innocent is tiny. The fallacy is committed if one then proceeds to claim that the probability of the accused being innocent is comparably tiny.

The fallacy lies in the fact that the a priori probability of guilt is not taken into account. If this probability is small, then the only effect of the presented evidence is to increase that probability somewhat, but not necessarily dramatically.

Examples

Consider for instance the case of Sally Clark, who was accused in 1998 of having killed her first child at 11 weeks of age, then conceived another child and killed it at 8 weeks of age. The prosecution had an expert witness testify that the probability of two children dying from sudden infant death syndrome[?] is about 1 in 73 million. To provide proper context for this number, the probability of a mother killing one child, conceiving another and killing that one too, should have been estimated and compared to the 1 in 73 million figure, but it wasn't. Ms. Clark was convicted in 1999, resulting in a press release by the Royal Statistical Society[?] which pointed out the mistake. (See link at end of article.) Sally Clark's conviction was eventually quashed on appeal on 29th January 2003.

In another scenario, assume a rape has been committed, and all the males of the town are rounded up for DNA testing. Finally one man whose DNA matches is arrested. At the trial, it is testified that the probability of finding a DNA match is only 1 in 10,000. This does not mean that the suspect is innocent with the tiny probability of 1 in 10,000. If for instance 20,000 men were tested, then we would expect to find two matches, and the suspect is innocent with probability at least 1 in 2.

Another instance of the prosecutor's fallacy is sometimes encountered when discussing the origins of life: the probability of life arising at random out of the physical laws is estimated to be tiny, and this is presented as evidence for a creator, without regard for the possibility that the probability of such a creator could be even tinier.

Mathematical explanation

We start with a thought experiment. I have a big bowl with one thousand balls, some of them made of wood, some of them made of plastic. I know that 100% of the wooden balls are white, and only 1% of the plastic balls are white, the others being red. Now I pull a ball out at random, and observe that it is actually white. Given this information, how likely is it that the ball I pulled out is made of wood? Is it 99%? No! Maybe the bowl contains only 10 wooden and 990 plastic balls. Without that information (the a priori probability), we cannot make any statement. In this thought experiment, you should think of the wooden balls as "accused is guilty" or "life originated from a creator", the plastic balls as "accused is innocent" or "life emerged without a creator", and the white balls as "the evidence is observed" or "life developed".

The fallacy can be analyzed using conditional probability: Suppose E is the evidence, and G stands for "guilt". We are interested in Odds(G|E) (the odds that the accused is guilty, given the evidence) and we know that P(E|~G) (the probability that the evidence would be observed if the accused were innocent) is tiny. One formulation of Bayes' theorem then states:

Odds(G|E) = Odds(G) · P(E|G)/P(E|~G)

Without knowledge of the a priori odds of G, the small value of P(E|~G) does not necessarily imply that Odds(G|E) is large.

The prosecutor's fallacy is therefore no fallacy if the a priori odds of guilt are assumed to be 1:1. In an Bayesian approach to personal probabilities, where probabilities represent degrees of belief of reasonable persons, this assumption can be justified as follows: a completely unbiased person, without having been shown any evidence and without any prior knowledge, will estimate the a priori odds of guilt as 1:1.

In this picture then, the fallacy consists in the fact that the prosecutor claims an absolutely low probability of innocence, without mentioning that the information he conveniently omitted would have led to a different estimate.

External links

• Press release by the Royal Statistical Society about the Sally Clark case: http://www.rss.org.uk/archive/reports/sclark
• http://www.colchsfc.ac.uk/maths/dna/discuss.htm
• http://dna-view.com/profile.htm