Challenging claims that probability theory is incompatible with legal reasoning

 
A new paper published in Science and Justice exposes why common claims that probability theory is incompatible with the law are flawed.

One of the most effective tactics that has been used by legal scholars to 'demonstrate' the 'limitations' and 'incompatibility' of probability theory (and particularly Bayes theorem) with legal reasoning is the use of puzzles like the following:

Fred is charged with a crime. A reliable eye witness testifies that someone exactly matching Fred’s appearance was seen fleeing the crime scene. But Fred is known to have an identical twin brother. So is the evidence relevant?"**

The argument to suggest that this example demonstrates probability theory is incompatible with legal norms goes something like this:

Both intuitively and legally it is clear that the evidence should be considered relevant. But according to probability theory (Bayes' theorem), the evidence has 'no probative value' since it provides no change in our belief about whether Fred is more likely than his twin brother to have been at the crime scene. Hence, according to probability theory the evidence is wrongly considered inadmissible.

Specifically, such problems are intended to show that use of probability theory results in legal paradoxes. As such, these problems have been a powerful detriment to the use of probability theory  in the law.

The new paper  shows that all of these puzzles only lead to ‘paradoxes’ under an artificially constrained view of probability theory and the use of the so-called likelihood ratio, in which multiple related hypotheses and pieces of evidence are squeezed into a single hypothesis variable and a single evidence variable. When the distinct relevant hypotheses and evidence are described properly in a causal model (a Bayesian network), the paradoxes vanish. Moreover, the resulting Bayesian networks provide a powerful framework for legal reasoning.

Full reference details of the paper:

de Zoete, J., Fenton, N. E., Noguchi, T., & Lagnado, D. A. (2019). "Countering the ‘probabilistic paradoxes in legal reasoning’ with Bayesian networks". Science & Justice 10.1016/j.scijus.2019.03.003. 

The pre-publication version (pdf)

The models (which can be run using AgenaRisk)

Two other papers just accepted (details to follow) also demonstrate the power of Bayesian networks in legal reasoning:

Fenton, N. E., Neil, M., Yet, B., & Lagnado, D. A. (2019). "Analyzing the Simonshaven Case using Bayesian Networks". Topics in Cognitive Science, 10.1111/tops.12417.  (Update: this had now been published; the published version can be read https://rdcu.be/bqYxp )

Neil, M., Fenton, N. E., Lagnado, D. A. & Gill, R. (2019), "Modelling competing legal arguments using Bayesian Model Comparison and Averaging". to appear Artififical Intelligence and Law . The full published version can be read here.

**This particular puzzle is easy to 'resolve'. The 'non-probative' Bayes conclusion is only correct if we assume that the only people who could possibly have committed the crime are Fred and his twin brother. In practice we have to consider the possibility that neither committed the crime. While the eye witness evidence fails to distinguish between which of Fred and his twin was at the crime scene the evidence results in the probability that Fred was at the crime scene increasing in relation to the hypothesis that Fred was not at the crime scene.