Flaky DNA (the prosecutor's fallacy yet again and much more to be worried about)

In August 2012 David Butler (who had been jailed for the 2005 murder of Anne Marie Foy) was freed when it was discovered that the DNA evidence - which had essentially been the only evidence against him - was flaky in more senses than one. Tiny traces of DNA, whose profile matched that of Butler, were discovered under Foy's fingernails. A sample of Butler's DNA had been previously stored in a database (the police had mistakenly assumed it belonged to the person who burgled his mother's house). It was a search of this database that revealed his DNA matched that under Foy's fingernails.

The reports here and here give a good overview of the case, focusing on the critical observation that some people - such as Butler - have especially dry skin making it extremely likely to shed tiny amounts of DNA wherever they go. This means that Butler - a cab driver - could have easily transferred his cells simply by handling money that was later passed on to either the victim or the real attacker. A more recent US case - described here - also provides an example of how easily DNA can be innocently 'transferred' to a crime scene and mistakenly assumed to belong to the person who committed the crime.

The reporting of these cases highlights just one important scenario under which the probative value of DNA evidence can be massively exaggerated, namely the fact that there are multiple opportunities for DNA to be 'transferred'. This means that DNA found at a crime scene or on a victim could have come from multiple innocent sources.

But there are many other, less well understood, scenarios under which the probative value of DNA evidence can be massively exaggerated, and the Bulter case actually highlights all of few of them:

1.  Incorrectly reporting the probabilistic impact: In reporting the impact of the DNA evidence it appears (based on the Telegraph report) that the prosecuting QC has yet again committed the prosecutor's fallacy. The statement that there is “a one billion-to-one chance that the DNA belongs to anyone else" is wrong (just as it was  here here and here). In fact, if the DNA profile was indeed such that it is found in one in a billion people, then it is likely to be shared with about six other (unknown and unrelated) people in the world. In the absence of any other evidence against the defendant there is actually therefore a 6/7 chance that it belongs to 'anyone' else.
2. The impact of a database search: Finding the matching DNA as a result of a database search, rather than as a result of testing a suspect on the basis of some other evidence, completely changes the impact of the evidence. This is especially devastating when there is so-called 'low-template DNA' - where the random match probabilities are nothing like as low as 1 in a billion.  Let's suppose the DNA at the crime scene is such that it is found in one in every 10,000 people. Then even in a fairly small database - say of 5,000 individuals' DNA samples - there is a good chance (about 50%) that we will find a match to the crime scene DNA. Suppose we find a 'match'. Have we 'got our man'. Almost certainly not. In the UK alone we would expect 6000 people to have the matching DNA. In some cases low-template DNA profiles have a match probability of 1 in 100. In such situations a database match tells us nothing at all. If we charged the first matching person we found we would almost certainly have the wrong person.
3. The potential for errors in DNA analysis and testing. It is not just the potential for 'innocent transfer' that we have to consider when we think about 'human error'.  Brian McKeown, chief scientist representative from LGC Forensics says:
        "..the science is flawless and must not be ignored. If you do it right you get the right result.”.
    Yet LGC have themselves committed high-profile critical DNA testing errors, such as those reported here and here. When their scientists report the probabilistic impact of DNA matches they never incorporate the very real probability of errors that can be introduced at numerous stages in the process. As we explained here (and we will be reporting much more extensively on this in upcoming papers) when sensible allowance is made for human error, the DNA 'statistics' become very different.
4. The critical importance of absence of DNA evidence. If a person - especially one who easily sheds DNA - really did rape and strangle the victim then, the fact that only tiny cells of DNA matching theirs are discovered on the victim is actually two pieces of evidence. One is made explicit - that the DNA matches - and it supports the prosecution case. But the other - that no substantive DNA from the defendant was found - is typically ignored; and it may provide very strong support for the defence case. This 'evidence' of  'relative absence of DNA evidence' has been a key (previously ignored) factor in cases I have been recently involved in, so hopefully soon I will be able to reveal more about its impact.
5. The entire theoretical basis for DNA 'match probabilities' and sampling is itself extremely flaky. This is something I am currently looking at with colleagues and will be writing about soon.

Unlike some others, I am not suggesting the imminent future demise of DNA in the courtroom. However, I am convinced that a far more critical approach to both the presentation and evaluation of DNA evidence is urgently required to avoid future miscarriages of justice. And I am convinced that many - as yet undiscovered - errors in DNA analysis means that innocent people are in jail and guilty people are at large.