Transforming Legal Reasoning through Effective use of Probability

Recent reports have highlighted the difficulties faced by the criminal justice system in adequately responding to the dramatic increase in the amount and complexity of forensic science, particularly given its (not infrequently) questionable value. Despite the growing consensus that the role of experts should be limited to making statements about the probability of their findings under competing hypotheses (instead of, for example, making categorical source attributions), and the ability of Bayes’ theorem to encapsulate the proper or normative effect of probabilistic evidence, Bayesian reasoning has been largely ignored or misunderstood by criminal justice professionals.

Proper use of probabilistic reasoning has the potential to improve dramatically the efficiency, transparency and fairness of the criminal justice system and the accuracy of its verdicts, by enabling the value of any given piece of evidence to be meaningfully evaluated and communicated. Bayesian reasoning employs the likelihood ratio (which is the probability of seeing the evidence given the prosecution hypothesis divided by the probability of seeing the evidence given the defence hypothesis), to illustrate the relevance and strength of each piece of evidence. Bayesian reasoning can therefore help the expert formulate accurate and informative opinions; help the court in determining the admissibility of evidence; help identify which cases should and should not be pursued and help lawyers explain, and jurors to evaluate the weight of evidence during a trial. It would also help identify error rates and unjustified assumptions entailed in expert opinions, which would in turn contribute to the transparency and legitimacy of the criminal justice process.

Unfortunately, there is widespread disagreement about the kind of evidence to which Bayesian reasoning should be applied and the manner in which it should be presented. Much of the disagreement over when it should be applied arises from fundamental misunderstandings about the way Bayes’ reasoning works, whereas disagreement over the manner in which it should be presented could be resolved by empirical research. Misunderstandings in the criminal justice system are exacerbated by the fact that in the few areas where Bayesian reasoning has been applied (such as DNA profiling ) its application has often been faulty and its ramifications poorly communicated. This has further resulted in widespread recourse to probabilistic fallacies in legal proceedings.

A dramatic and worrying example of this was a recent appeal court decision ((2010). R v T. EWCA Crim 2439 , see our draft article about this here) which appears to reject the use of Bayesian analysis and likelihood ratios for all but a very narrowly defined class of forensic evidence. Instead of being accepted as a standard tool of the forensic science trade, Bayesian analysis is perceived by much of the legal profession as an exotic, somewhat eccentric method to be wheeled out for occasional specialist appearances whereupon a judge or lawyer will cast doubts on, and even ridicule, its integrity (hence ensuring it is kept firmly locked in the cupboard for more years to come).

Ultimately this represents a failure by the community of academics, expert witnesses and lawyers who understand the potentially crucial and wide role that can be played by Bayesian analysis and likelihood ratios in legal arguments. This failure must be attributed to our inability to communicate the core ideas effectively. Resorting to the formulas and calculations in court is a dead-end strategy since these will never be understood by most lawyers, judges and juries.