What chance the next roll of the die is a 3?

In response to my posting yesterday a colleague posed the following question:

The die has rolled 3 3 3 3 3 3 3 in the past. What are the chances of 1 2 4 5 6 being rolled next? The mathematician will say: P(k)=1/6 for each number, forget that short-term evidence. What will the probability expert say? And the statistician? And the philosopher? 

I have provided a detailed solution to this problem here.

In summary, it is based on a Bayesian network in which (except for the 'statistician') it all comes down to what priors they are assuming for the probability of each P(k).

• The mathematician's prior is that the probability of each P(k) is exactly 1/6.
•  One type of probability expert (including certain types of Bayesians) will argue that, in the absence of any prior knowledge of the die, the probability distribution for each P(k) is uniform over the interval 0-1 (meaning any value is just as likely as any other).
• Another probability expert (including most Bayesians) will argue that the prior should be based on dice they have previously seen. They believe most dice are essentially 'fair' but there could be biases due to either imperfections or deliberate tampering. Such an expert might therefore specify the prior distribution for P(k) to be a narrow bell curve centred on 1/6.
•  A philosopher might consider any of the above but might also reject the notion that 1,2,3,4,5,6 are the only outcomes possible.

Anyway, when we enter the evidence of seven 3's in 7 rolls, the Bayesian calculations (performed using AgenaRisk) result in an updated posterior distribution for each of the P(k)s.

The mathematician's posterior for each P(k) is unchanged: i.e. each P(k) is still 1/6.So there is still just a probability of 1/6 the next roll will be a 3.

For the probability expert with the uniform priors, the posterior for P(3) is now a distribution with mean 0.618. The other probabilities are all reduced accordingly to distributions with mean about 0.079. So in this case the probability of rolling a 3 next time is about 0.618 whereas each of the other numbers has a probability about 0.079

For the probability expert with the bell curve priors, the posterior for P(3) is now a distribution with mean 0.33. The other probabilities are all reduced accordingly to distributions with mean about 0.13. So in this case the probability of rolling a 3 next time is about 0.33 whereas each of the other numbers each has a probability about 0.13.

And what about the statistician? Well a classical statistician cannot give any prior distributions so the above approach does not work for him. What he might do is propose a 'null' hypothesis that the die is 'fair' and use the observed data to accept or reject this hypothesis at some arbitrary 'p-value' (he would reject the null hypothesis in this case at the standard p=0.01 value). But that does not provide much help in answering the question. He could try a straight frequency approach in which case the probability of a three is 1 (since we observed 7 out of 7 threes) and the probability of any other number is 0.

Anyway the detailed solution showing the model and results is here. The model itself - which will run in AgenaRisk is here.

"No such thing as probability" in the Law?

David Spiegelhalter has posted an important article about a recent English Court of Appeal judgement in which the judge essentially suggests that it is unacceptable to use probabilities to express uncertainty about unknown events. Some choice quotes David provides from the judgement include:

"..and to express the probability of some event having happened in percentage terms is illusory.

....The chances of something happening in the future may be expressed in terms of percentage. ... But you cannot properly say that there is a 25 per cent chance that something has happened... Either it has or it has not. "

What is interesting about this is that the judge has used almost the same words that we said (in- Chapter 1 of our book Risk Assessment and Decision Analysis with Bayesian Networks) we had heard from several lawyers. One of the quotes we gave there from an eminent lawyer was:

“Look the guy either did it or he didn’t do it. If he did then he is 100% guilty and if he didn’t then he is 0% guilty; so giving the chances of guilt as a probability somewhere in between makes no sense and has no place in the law”. 

Of course, as we show in the book (Chapter 1 is freely available for download) you can actually prove that the this kind of assertion is flawed in the sense that it inevitably leads to irrational decision-making.

The key point is that there can be as much uncertainty about an event that has yet to happen (e.g. whether or not your friend Naomi will roll a 6 on a die) as one that has happened (e.g. whether or not Naomi did roll a six on the die). It all depends on what information you know about the event that has happened. If you did not actually see the die rolled in the second case your uncertainty about the outcome is no different than before it was rolled, even though Naomi knows for certain whether or not it was a six (so for her the probability really is either 1 or 0). As you discover information about the event that has happened (for example, if another reliable friend tells you that an even number was rolled) then your uncertainty changes (in this case from 1/6 to 1/3). And that is exactly what is supposed to happen in a court of law where, typically, nobody (other than the defendant) knows  whether the defendant committed the crime; in this case it is up to the jury to revise their belief in the probability of guilt as they see evidence during the trial.

David Spiegelhalter points out that the judge is not just 'banning' Bayesian reasoning, but also banning the Sherlock Holmes approach to evidence. But it is even worse, because the judge is essentially banning the entire legal rationale for presenting evidence (which is ultimately about helping the jury to determine the probability that the defendant committed the crime).

p.s. There are other aspects of the case which are troubling, notably the assumption that there were just three possible potential causes of the fire (other as yet unknown/unknowable potential causes would have non-zero prior probabilities). However, the judge got some things right including his line of reasoning about the relative likelihood of two unlikely events (the arcing or the smoking) demonstrated that, if these are exhaustive, then the smoking was the most likely cause.