I have written many times before on this blog about the benefits and (extreme) limitations of using the likelihood ratio (LR) to determine the strength of forensic match evidence - such as DNA evidence. The LR is the probability of the evidence (i.e. 'finding a match') under the prosecution hypothesis divided by the probability of the evidence under the defence hypothesis.
When a forensic expert calculates a high LR for the prosecution hypothesis "DNA found at the crime scene matches the DNA profile of the suspect" they typically conclude that ‘this provides strong support for the prosecution hypothesis’. However, it is well known that a high LR does not necessarily mean a high (posterior) probability that the hypothesis is true because this depends on the prior probability of the hypothesis. Assuming that it does is an example of what is called the prosecutor's fallacy; judges are expected to warn both lawyers and expert witnesses when this mistake is made in court. What is less well understood is that, in order to draw any rational conclusions at all about the probative value of a high LR, the defence hypothesis used to determine the LR has to be the negation of
the prosecution hypothesis (formally we say the hypotheses must be mutually exclusive and exhaustive). So, in the above example the defence hypothesis has to be ‘DNA does not come
from the suspect’. Yet in practice, it is common to use ‘DNA comes
from a person unrelated to the defendant’ which of course is not the negation of the prosecution hypothesis. When the defence hypothesis is not the negation, then a high LR could actually mean the exact opposite of what is claimed: it could provide more support for the hypothesis that the DNA does not come
from the suspect than it does for the prosecution hypothesis.
While the problem of using the LR with hypotheses that are not mutually exclusive and exhaustive is known (but not widely understood) - for 'single' DNA profiles (i.e. those where the DNA found can only possibly come from a single person) it is even more serious for DNA mixture profiles (i.e those where the DNA is a mixture of at least two people). However, for mixture profiles, not having mutually exclusive and exhaustive hypotheses when using the LR is just one of several serious problems. A new paper shows the extent to which a very high LR for a ‘match’ from a DNA mixture profile – typically computed from probabilistic genotyping software - can be misleading even if the hypotheses are mutually exclusive and exhaustive. The paper shows that, in contrast to single profile DNA ‘matches’ where the only residual uncertainty is whether a person other than the suspect has the same matching DNA profile, it is possible for all the genotypes of the suspect’s DNA profile to appear at each locus of a DNA mixture, even though none of the contributors has that DNA profile. In the absence of other evidence, the paper shows it is possible to have a very high LR for the hypothesis ‘suspect is included in the mixture’ even though the posterior probability that the suspect is included is very low. Yet, in such cases a forensic expert will generally still report a high LR as ‘strong support for the suspect being a contributor’.
The problems are especially acute where there are very small amounts of DNA in the mixture (so-called low template DNA abbreviated to LTDNA). In certain circumstances, the use of the LR may have led lawyers and jurors into grossly overestimating the probative value of a LTDNA mixed profile ‘match’.
Full paper (pdf):
See also
- Problems with DNA mixed profile evidence: the case of Florencio Jose Dominguez
- Problem with likelihood ratio for DNA mixture profiles
- Confusion over the Likelihood ratio
- Problems with the Likelihood Ratio method for determining probative value of evidence: the need for exhaustive hypotheses
- Misleading DNA evidence
- Barry George case: new insights on the evidence
- Sally Clark revisited: another key statistical oversight?
- Prosecutor fallacy in Stephen Lawrence case?
- Prosecutor fallacy in media reporting of Burgess DNA case
- Flaky DNA: Prosecutors fallacy yet again
- Prosecutors fallacy just will not go away
- Misleading DNA evidence
No comments:
Post a Comment