Norman Fenton, 28 January 2016
This
post is primarily about how DNA match evidence is often presented in a
way that is highly misleading (it is an important issue in an ongoing
case I'm involved with). But in order to illustrate the point it turns
out that we can use a simple analogy based loosely on
the current lottery story
that is getting a lot of media attention in the UK. This concerns an
unverified £33 million winning ticket from a recent draw. About 200
people are claiming to have bought the (single) winning ticket but,
until today*, none had actually provided proof of possessing such a
ticket. The claim of one  Miss Susan Hinte  is the one that has
grabbed media attention because she has produced a ticket in which key
identifying information cannot be read because, she claims, the ticket
was put through a washing machine.
But first let's look at the DNA issue, which is concerned with the following generic problem:
 The prosecution claims that defendant Joe was at the crime scene. This hypothesis is denoted as Hp.
 A tiny trace of DNA from the crime scene has been analysed and found
to match the profile of Joe. This evidence (of the match) is denoted E.
Typically the defence will argue that Joe was not at the crime
scene and that any DNA matching Joe  especially as it was a tiny trace 
got there through secondary transfer or other means. So the defence
hypothesis
Hd is simply the negation of Hp.
The DNA experts have correctly recognised that, in determining the
probative value of the evidence E, they have to use the ‘likelihood ratio’
approach [1]. This means they have to consider
both of the following probabilities:
 The probability that E is the result of the prosecution hypothesis Hp being true  formally we write this as P(E given Hp)

The probability that E is the result of the defence hypothesis Hd being false  formally we write this as P(E given Hd)
If probability 1 is greater than probability 2 then the evidence
E supports Hp over Hd and vice versa. The likelihood ratio is simply 1
divided by 2 and provides a simple and compelling measure of probative
value of evidence. If the ratio is greater than one the evidence E
supports Hp, with higher values indicating stronger support. If the
ratio is less than one the evidence E supports Hd, with smaller values
indicating stronger support. However, for reasons explained in [1], this
whole notion of probative value is not meaningful if the defence
hypothesis Hd is not the
negation of the prosecution hypothesis Hp. One of the common errors made
by DNA experts is to replace Hd with a
different hypothesis,
namely Hd': "DNA from Joe got there by secondary transfer". In this
case Hd' excludes other possibilities of observing E even though Joe was
not at the crime scene (such as errors or contamination during the DNA
testing, or the DNA belonging to a different person with the same
profile etc) and is not even mutually exclusive to Hp since Joe may have
been at the crime scene even though the trace sample was there through
secondary transfer. But, while this common error is serious, it is not
the real concern I wish to raise here. In fact, let's suppose that no
such error is made and that the expert considers the correct Hd.
.
The real concern is how a jury member reacts when the DNA expert now makes the following assertions:
 “The findings are what I would have expected if Hp were true.” i.e. P(E given Hp) is very high

“The probability of the findings are considerably more likely to have been the result of Hp rather than Hd” i.e. P(E given Hp) is much higher than P(E given Hd)
Notwithstanding the unnecessary redundancy of statement 1, these
assertions sound very important and suggest very strong support for the
prosecution hypothesis, especially as most people would already have
assumed (wrongly) that the DNA 'match' means the trace certainly belongs
to Joe.
But to demonstrate how misleading they are I
will return now to the lottery example. For simplicity I will assume the
old 6ball lottery with 49 numbers. Suppose the winning numbers were:
1, 7, 21, 28, 40, 46
Mrs
Smith has a damaged ticket that she claims has the winning numbers. The
evidence E is that the first number (which is the only number clearly
visible) is 1.
Our hypotheses are:

Hp: “Mrs Smith's ticket is the winning ticket”

Hd: “Mrs Smith ticket is not the winning ticket”
In this case we know the following:

P(E given Hp) = 1 (it is certain that the first number on the ticket would be 1 if it was the winning ticket)

P(E given Hd) is 0.122 (this is the proportion of nonwinning tickets that have 1 as the first number)
So we could certainly make
exactly the same assertions in this case as the DNA experts above:
 “The findings are what I would have expected if Hp were true.” (since the probability of E given Hp is 1)

“The probability of the findings are considerably more likely to have
been the result of Hp rather than Hd” (since 1 is considerably
greater than 0.122).
However, despite these (correct) assertions
it is almost certain that Hd
rather than Hp is true  Mrs Smith's ticket is not the winning ticket. In fact, the probability of Hp being true is
less than one in 1.7 million (because there are over 1.7 million
nonwinning combinations in which the first number is 1).
So what is the moral of this story? The likelihood ratio of the
evidence might often suggest the evidence is highly probative in favour
of one of the hypotheses, but if the prior probability of the
alternative hypothesis was much higher to start with then the evidence
will not ‘overturn’ the prior belief in favour of the alternative.
Lay
people ignore this in connection to DNA evidence. Because the random
match probability associated with a DNA match is typically less than one
in a billion, the very fact that the evidence E is a "DNA match"
already puts into their mind the notion that this 'must tie the
defendant to the crime scene'. But the random match probability is
almost irrelevant in this case  it only accounts for a tiny proportion
of P(E given Hp). Lay people can also easily be tricked into believing
that the (redundant) assertion 1 “The findings are what I would have
expected if Hp were true” provides additional weight to assertion 2.
Unfortunately,
this type of evidence is increasingly prejudicing juries and, I
believe, leading to serious miscarriages of justice.
*The
real winner has now been found, and since their ticket was not damaged it can not have been Miss Hinte
[1] Fenton, N. E., D. Berger, D. Lagnado, M. Neil and A. Hsu, (2014). "When
‘neutral’ evidence still has probative value (with implications from
the Barry George Case)", Science and Justice, 54(4), 274287
http://dx.doi.org/10.1016/j.scijus.2013.07.002. (prepublication draft
here)