Most people watching this weekend's Premier League matches were genuinely concerned about the penalty decisions that were awarded against Spurs (v Newcastle) and for Manchester Utd (v Brighton).
As a result today there have been several posts with the above "Team Fortune Ladder" from UREF. This is a record of the net VAR decisions that went in favour of each team during last season (2019-20). And it seems that the 'good fortune' of Man Utd and the 'bad fortune' of Spurs is a continuation of a pattern from last season, with Man Utd clear at the top with a net total 17 decisions in their favour and Spurs clear at the bottom with a net 12 against.
But are these results through luck or bias? We can do a simple statistical analysis (full details below for those interested) to see if the assumption that there was no bias is realistic. Let's assume there were on average 2 VAR decisions per game last season. Then, for any given team this means that over the season they were involved in 76 VAR decisions (as they each played 38 games). With the 'no bias' assumption we would expect VAR decisions to 'balance out', i.e. that the net total decisions in favour should be 0 (like Sheffield Utd in the table). However, because of inevitable statistical variation some teams will have a positive net number and some teams negative; we'd expect most teams to be between plus or minus 5. In fact, we can calculate the probability that any given team would end up over the season with a net total of more than 16 decisions in favour. The answer is just 2.5%, i.e. this is highly unlikely; similarly we can calculate the probability that any given team would end up over the season with a
net total of more than 11 decisions against them. The answer is just over 10%, i.e. this is also unlikely.
However, we have to take account of the fact that there are 20 teams. It turns out that the probability that at least one team will end up with net total of more than 16 decisions in their favour is actually 40%. So, in fact this is not that unlikely; would expect to see something like this happen in 2 out of every 5 seasons.
Similarly, the probability that at least one team will end up with net total of more than 11 decisions against them is about 89%, which is very likely, i.e. we would expect to see something like this happen in about 9 out of 10 seasons.
So, while individually Man Utd were exceptionally lucky and Spurs very unlucky, the table for 2019-20 provides only limited evidence of bias. However, that fact that the trend in favour of Man Utd and against Spurs has continued into the 2020-21 season means that this conclusion could change once I add the new data (I understand Spurs have a net 5 against in just 3 games this season). Moreover, it is interesting to note that a while ago in published work done with my colleague Anthony Constantinou we did find real evidence of referee bias in favour of Man Utd.
And let's not forget this - from Spurs againt Man Utd in 2005 (before VAR) - that was ruled to be not over the line.
See also:
Constantinou, A. C., Fenton, N. E., & Pollock, L. (2014). "Bayesian networks for unbiased assessment of referee bias in Association Football". Psychology of Sport & Exercise, 15(5) 538–547, http://dx.doi.org/10.1016/j.psychsport.2014.05.009. Pre-publication draft here.
No comments:
Post a Comment