Proving referee bias with Bayesian networks

An article in today's Huffington Post by Raj Persaud and Adrian Furnham talks about the scientific evidence that supports the idea of referee bias in football. One of the studies they describe is the recent work I did with Anthony Constantinou and Liam Pollock** where we developed a causal Bayesian network model to determine referee bias and applied it to the data from all matches played in the 2011-12 Premier League season. Here is what they say about our study:

Another recent study might just have scientifically confirmed this possible 'Ferguson Factor', entitled, 'Bayesian networks for unbiased assessment of referee bias in Association Football'. The term 'Bayesian networks', refers to a particular statistical technique deployed in this research, which mathematically analysed referee bias with respect to fouls and penalty kicks awarded during the 2011-12 English Premier League season.

The authors of the study, Anthony Constantinou, Norman Fenton and Liam Pollock found fairly strong referee bias, based on penalty kicks awarded, in favour of certain teams when playing at home.
Specifically, the two teams (Manchester City and Manchester United) who finished first and second in the league, appear to have benefited from bias that cannot be explained by other factors. For example a team may be awarded more penalties simply because it's more attacking, not just because referees are biased in its favour.

The authors from Queen Mary University of London, argue that if the home team is more in control of the ball, then, compared to opponents, it's bound to be awarded more penalties, with less yellow and red cards, compared to opponents. Greater possession leads any team being on the receiving end of more tackles. A higher proportion of these tackles are bound to be committed nearer to the opponent's goal, as greater possession also usually results in territorial advantage.

However, this study, published in the academic journal 'Psychology of Sport and Exercise', found, even allowing for these other possible factors, Manchester United with 9 penalties awarded during that season, was ranked 1st in positive referee bias, while Manchester City with 8 penalties awarded is ranked 2nd. In other words it looks like certain teams (most specifically Manchester United) benefited from referee bias in their favour during Home games, which cannot be explained by any other possible element of 'Home Advantage'. 

What makes this result particularly interesting, the authors argue, is that for most of the season, these were the only two teams fighting for the English Premiere League title. Were referees influenced by this, and it impacted on their decision-making?  Conversely the study found Arsenal, a team of similar popularity and wealth, and who finished third, benefited least of all 20 teams from referee bias at home, with respect to penalty kicks awarded. With the second largest average attendance as well as the second largest average crowd density, Arsenal were still ranked last in terms of referee bias favouring them for penalties awarded. In other words, Arsenal didn't seem to benefit much at all from the kind of referee bias that other teams were gaining from 'Home Advantage'. Psychologists might argue that temperament-wise, Sir Alex Ferguson and Arsene Wenger appear at opposite poles of the spectrum.

**  Constantinou, A. C., Fenton, N. E., & Pollock, L. (2014). "Bayesian networks for unbiased assessment of referee bias in Association Football". To appear in Psychology of Sport & Exercise. A pre-publication draft can be found here.

Our related work on using Bayesian networks to predict football results is discussed here.

Daniel Kahneman at the Hebrew University Jerusalem

I have just returned from the workshop on "Behavioral Legal Studies - Cognition, Motivation, and Moral Judgment" at the Hebrew University in Jerusalem, Israel. I was especially interested in seeing Daniel Kahneman open the workshop with "Reflections on Psychology, Economics, and Law". Kahneman won the 2002 nobel prize in economics and was also recipient of the Presidential Medal of Freedom from President Obama in 2013.

Kahneman (centre) interviewed by Prof Zamir (left) and Prof Ritov (right)

Kahneman is, of course, very well known for his pioneering work with Amos Tversky and Paul Slovic (who also spoke at the workshop) on cognitive bias (which has greatly influenced our own work on probabilistic reasoning in the law) and also prospect theory (for which he won the Nobel prize). Kahneman's 2011 book "Thinking, Fast and Slow" which summarises much of his work, has sold over one and a half million copies. The book is based on the idea that, when it comes to assessment and decision-making, people are either system 1 (fast) thinkers or system 2 (slow) thinkers. The former act on instinct and often get things wrong while the latter are more likely to get things right because they think through all aspects of a problem carefully. While I think Kahneman's book is a very good read, I personally do not find the fast/slow classification of decision-makers to be especially helpful. Nevertheless, a lot of the speakers at the workshop used it to inform their own work.

Kahneman's presentation was in the form of an interview by Prof. Eyal Zamir and Prof. Ilana Ritov (both of the Law Faculty at the Hebrew University) asking the questions. Kahneman nicely summarised the main results and achievements of his career and was humble enough both to give credit to his co-researchers and also to admit that some of his theories (such as on gambling choices) had subsequently been proven to be false.

Audience at Kahneman interview

Kahneman touched on one of the key points in his book that I find problematic, namely his rejection of what he calls 'complex algorithms';  his argument is that any assessment/decision problem that involves expert judgment should not involve many variables because you can always get just as good a result with a simple model inolving no more than three variables. While I agree that any problem solution should be kept as simple as possible, a crude limit to the number of variables directly contradicts our Bayesian network approach, where models often necessarily involve multiple variables and relationships derived from both data and expert judgment. The important point is that the 'complex algorithms' we use are just Bayesian inference - of course if you had to do this 'by hand' then it would be disastrous, but the fact that there are widely available tools means the algorithmic complexity is completely hidden.  Crucially, we have shown many times (see for example this work on evaluating forensic evidence) that the Bayesian network solution provides greater accuracy and insights than the commonly used simplistic 'solutions'. 
 
Much of the theme of what Kahneman spoke about (and which was also a key theme of the workshop generally) was about 'moral judgment' - he cited the radically different legal responses to murder and attempted murder as an example of irrational (and possibly immoral) decision-making. The problem with 'moral judgment' - and the continually repeated notion of  'what is good for society' is that most academics have a particular view about these that they assume are both 'correct' and universally held. Hence, much of what I heard during the workshop was politicized and biased. This was also evident in Kahneman's answers to audience questions following the interview. I actually asked Kahneman what his rationale was for concluding that President Obama was a system 2 thinker. Bearing in mind that system 2 thinkers are supposed to be 'good' decision makers compared with system 1 thinkers, his response was clearly popular with many in the audience, but actually surprised me because it seemed to be purely political;  he basically said something like "you only have to compare him with the previous guy (Bush) to know the difference".

Kahneman also gave his views on how conflicts (like that of Israel and its enemies) could be solved, which I found were naive and possibly contradictory to his own work in psychology. His theory is that both 'sides' in a conflict are rational, but believe they are responding to the actions of the other side - so all you need to do is to make both sides aware of this.

There was a very nice reception for invited workshop participants after Kahneman's interview, but Kahneman himself had to rush off to another meeting and he took no further part in the workshop. 

"

Dave Lagnado (UCL) - who we have worked with on Bayesian networks and the law - giving an excellent talk on "Spreading the blame" (he presented a framework for intuitive judgements and blame)

My trip was partially funded under ERC Grant number: 339182 (BAYES-KNOWLEDGE) and I gratefully acknowledge the ERC contribution.