The prosecutor’s fallacy

Note:The following scenario is entirely fictional, but the issues addressed are very real. Follow the link at the end to see how they affect real-life court cases.

The Scenario

One morning a woman walking in an expensive part of London had her bag snatched – unfortunately it contained a large amount of cash. She and an eyewitness both described the thief as a very tall man (over 2 metres), between 20 and 30 years old, with red hair and a pronounced limp.

Later that day an eagle-eyed policeman spotted a man fitting this description limping down Oxford Street with a large plasma-screen TV. He was arrested, but denied being the bag-snatcher. He was unable to provide an alibi, claiming to have been at home on his own when the crime was committed. Further investigation revealed that the TV was paid for in cash.

In court the prosecutor said “This case is quite unusual. Because of the nature of the crime, there is no forensic evidence. The thief was wearing gloves, there were no footprints, etc. But unfortunately for the perpetrator we can show his guilt using other methods. To illustrate this we call an expert witness.”

He produced a criminologist who quoted some statistics from demographic tables. “In London, the probability of:

“And because these are all independent of each other, we can multiply them together to obtain the probability of one person having all these characteristics:

He announced with a flourish “The chance of any random individual sharing all these characteristics is vanishingly small – only 0.000002. The prisoner has them all.”

“So how certain are you that this is the man described by the victim and the eyewitness?”

“If the witnesses had described a man of average height and colouring between the ages of 20 and 40 who walked easily, then the probabilities of finding a match would be nowhere near zero, and therefore a case of the ‘usual suspects’ rather than what we have got here, which is an unusual suspect. But because the accused has this very unique combination of characteristics I am as certain as I can be that this is the man. To get such a match is really quite extraordinary. The probability of it is vanishingly small – 0.000002 – so I am very comfortable that this is the thief. The chance of him being innocent is so very small. And I commend both eyewitnesses on the precision of their beautifully matching descriptions.”

The jury was impressed, and things were looking bad for the red-headed man. Even his defence lawyer was convinced; shaking his head he muttered that he wouldn’t bet on those odds. With his now less than enthusiastic defence, the red-headed man went down for a stretch.

However, he knew he was innocent. Eventually he consulted a statistician who helped to bring an appeal. He said “Of course, this sort of thing happens a lot: it is known as the prosecutor’s fallacy.”

In the appeal court the red-headed man had a new defence lawyer, much better informed on the prosecutor’s fallacy thanks to the statistician. She called the original expert witness back to the stand.

She asked him “Am I correct in saying that in the original trial you stated that the chance of any random individual having this set of characteristics was 0.000002 – that this grouping of characteristics is so rare that you will only come across it in one in half a million people?”

“Yes, I am, which is what points towards him being the perpetrator.”

“What is the population of London?”

He looked startled, but replied “I think it is about 10 million.”

“So based on your statistics, how many people in London have this set of characteristics?”

He blustered a bit, but was forced to admit that there should be 20.

“Given that your evidence is based solely on a description and that you have admitted that there are 20 people in London who fit this description, this must mean that the probability of my client’s guilt is very small, only only 1 in 20. Or to put it another way, the chance of his innocence is 19 in 20, not 1 in half-a-million.”

“Yes, I suppose so.”

In the absence of any other evidence against him, the red-headed man was acquitted, and the police initiated a search for tall red-headed limping men.

What happened here?

The expert witness confused two things:

• the probability of an individual matching the description
• and the probability of an individual who does match the description being guilty

They are not the same!

It is easier to see the fallacy as soon as the probability of 0.000002 is turned into numbers of real people. When you bear in mind that the population of possible suspects is 10 million, 1 in half a million easily translates into 20 possible suspects – the accused is only one of this group, and if we are to be convinced of his guilt with no other evidence we would want to know that the other 19 had been excluded.

And that is without even considering people who might have come up to London for the day!

The defendant’s fallacy

The red-headed man was able to be acquitted because there was no other evidence for the prosecution to bring. The fact that he is only 1 of 20 possible suspects does not mean that he is necessarily innocent. The guilty man is one of these 20 – and it could still be him.

Links

How this affects real cases:

One in millions,
billions and trillions: lessons from People v. Collins (1968) for People v. Simpson (1995)
Jonathan J. Koehler, Journal of Legal Education 47, 214-223