Thursday, 13 January 2022

Debunking the hypothesis that the "healthy vaccinee" effect explains anomalies in ONS mortality data

In our previous report led by Martin Neil, we showed that ONS England data from November did NOT support vaccine efficacy claims once we adjust for obvious anomalies in the ONS data. Some, including the ONS themselves in their December report, imply that our conclusions were wrong because the anomalies we identified are caused by the so-called "healthy vaccinee" effect. 

We examined the new ONS data and found NO evidence to support this claim. We have therefore produced a new and signifiantly revised report with our updated analysis.

The healthy vaccinee effect is the hypothesis that people closer to death are too ill to be vaccinated and so become concentrated in a shrinking unvaccinated population, thus increasing the group’s mortality rate.  This hypothesis to explain their anomalous data is stated on Page 5 of the ONS report:

Page 5: “The all-cause ASMRs for the year-to-date were lower in the first three weeks after a vaccine dose than in subsequent weeks after that dose. This could be because of a “healthy vaccinee effect” where people who are ill (either due to COVID-19 or another relevant illness) are likely to delay vaccination. Therefore, the people who have been recently vaccinated are, in the short term, in better health than the general population.”

However, this not only contradicts NHS guidance report ("Joint Committee on Vaccination and Immunisation: advice on priority groups for COVID-19 vaccination", Page 3), which states: 

but is also contradicted by ONS on page 8 of their own report:

Page 8: "…the vaccination roll-out was also prioritised by health status of individuals, with the extremely clinically vulnerable and those with underlying health conditions being vaccinated earlier…

If the ONS hypothesis was correct then we would see: 

a) The percentage of the unvaccinated in poor health rise as vaccine rollout progresses 

b) A steady non-Covid mortality rate among the unhealthy (because they are dying at same rate as they always have done).

To support their claim the ONS released the percentage of 70-79 age group with "very poor" health in each vaccination category. Oddly, the vaccinated population contains a higher percentage of those in very poor health and this increases over time. Surprisingly the unvaccinated population has the LOWEST concentration of the unhealthy and the percentage declines over time:


In this unhealthy sub-population we found the non-Covid mortality rate for the unvaccinated is HIGHER than for the vaccinated. Both rates should be equivalent Again, we see unnatural spikes in non-Covid mortality just after vaccine roll out as seen before in whole population:

 

Therefore, those in poorest health were NOT more likely to remain unvaccinated. Also, there is a rise in non-Covid mortality, coincidental with vaccine rollout that is not only seen in the population as a whole but is also seen in those with the poorest health.  

We conclude that the "healthy vaccine effect" cannot explain the anomalies we discovered in the ONS data and believe it is up to advocates for this hypothesis to now prove their case using the released data.

Full report:

Martin Neil, Norman Fenton, Joel Smalley, Clare Craig, Joshua Guetzkow, Scott McLachlan, Jonathan Engler, Dan Russell and Jessica Rose (2021), “Official mortality data for England suggest systematic miscategorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination”, http://dx.doi.org/10.13140/RG.2.2.28055.09124 (this is a significantly revised version of http://dx.doi.org/10.13140/RG.2.2.14176.20483)

Monday, 3 January 2022

No fancy statistics: a simple plot of vaccination rate against Covid death rate for all countries in the world

Using the "Our Word in Data" website we have extracted the latest snapshot for each country of total vaccinations per hundred people and total 'covid deaths' per million. The full data by country - in order of vaccinations - is listed at the bottom of this page (all numbers rounded to 0 decimal places).

We use inverted commas for 'covid deaths' because (as readers of this blog will know) this is a very vague metric and we have no confidence that it is accurate or consistently collected for any country in the world. If the data were accurate, and if the vaccines worked as claimed, then what we should see when we plot the vaccinations against deaths is something like this:


 i.e. the more vaccinations in a country the fewer deaths.

Obviously there are multiple confounding factors (other than inconsistent reporting) that can impact on the relationship (timing when covid first hit, average population age, population density, geographial location, access to healthcare, etc) not to mention all the missing factors previously discussed**. Ideally the deaths should also be restricted to post-vaccination roll out (difficult to do that using the Our World in Data spreadsheet). But it is still surprising that the following is the actual plot:


All pretty random*** but note the high number of low vaccination, low covid death countries (mainly in Africa) as shown in this map:

But what it really shows more than anything is how poor all the 'official' covid data are (look at the laughable China data) and, because of the universally poor data, how little evidence there is of either the severity of Covid or the effectiveness of any covid interventions.

**As we have been saying since March 2020 all of the 'official' Covid data are essentially useless because they do not provide us with the necessary information to take account of all of the causal explanations for what is observed:

 

Country total vaccinations per hundred total deaths per million
Gibraltar 322 2968
Cuba 268 735
Chile 230 2035
United Arab Emirates 224 216
Iceland 209 108
Denmark 209 560
Isle of Man 208 784
Malta 208 922
South Korea 202 111
Uruguay 200 1771
China 197 3
Cayman Islands 196 165
Faeroe Islands 196 285
United Kingdom 195 2181
Ireland 194 1186
Portugal 191 1864
Belgium 186 2434
Seychelles 185 1324
Bahrain 185 797
Spain 184 1909
Italy 184 2278
France 183 1830
Austria 182 1520
Bermuda 182 1707
Canada 181 798
Israel 180 887
Cambodia 180 178
Brunei 179 222
Germany 178 1336
Norway 178 239
Qatar 178 211
Malaysia 176 961
Singapore 175 149
Finland 175 282
Sweden 173 1507
Cyprus 172 698
Argentina 168 2569
Greece 167 2005
Australia 165 88
Luxembourg 165 1429
Liechtenstein 165 1804
Mongolia 161 619
Kuwait 160 570
Mauritius 160 188
New Zealand 160 10
San Marino 159 2793
Japan 158 146
Switzerland 158 1397
Sri Lanka 158 698
Hungary 156 3934
Netherlands 155 1193
Brazil 155 2894
Turkey 155 968
Lithuania 154 2752
United States 153 2476
Aruba 153 1689
Ecuador 153 1881
Vietnam 151 322
Costa Rica 151 1429
Andorra 150 1719
Bhutan 148 4
Peru 148 6071
Thailand 147 309
El Salvador 147 585
Taiwan 146 36
Maldives 145 482
Saudi Arabia 144 251
Czechia 144 3374
Turks and Caicos Islands 144 586
Fiji 140 772
Slovenia 140 2693
Greenland 138 18
Iran 137 1545
Latvia 137 2443
Morocco 135 397
Panama 135 1696
Anguilla 134 264
Curacao 132 1147
Hong Kong 132 28
Dominican Republic 129 388
Monaco 126 835
Colombia 126 2533
Poland 124 2581
New Caledonia 123 971
Antigua and Barbuda 123 1195
Serbia 120 1840
British Virgin Islands 118 1282
French Polynesia 116 2251
Nicaragua 116 32
Croatia 116 3072
Oman 116 787
Uzbekistan 115 44
Estonia 115 1458
Mexico 114 2299
Slovakia 112 3046
Azerbaijan 111 818
Wallis and Futuna 108 631
Belize 105 1462
Venezuela 105 183
India 104 346
Barbados 104 904
Saint Kitts and Nevis 102 523
Cape Verde 102 625
Tunisia 102 2141
Indonesia 101 521
Montenegro 101 3844
Russia 101 2080
Trinidad and Tobago 100 2054
Rwanda 99 102
Philippines 98 463
Guyana 97 1318
Honduras 95 1037
Paraguay 95 2301
Kosovo 94 1678
Kazakhstan 92 959
Botswana 92 1020
Timor 89 91
North Macedonia 84 3752
Romania 83 3072
Suriname 83 2009
Bolivia 82 1661
Belarus 82 570
Albania 81 1116
Jordan 80 1205
Bangladesh 80 169
Laos 79 19
Bahamas 76 1796
Nepal 74 390
Pakistan 70 128
Grenada 69 1770
Ukraine 65 2354
Lebanon 65 1350
Tajikistan 64 13
Palestine 63 932
Georgia 63 3457
Guatemala 62 883
Sao Tome and Principe 62 255
Montserrat 61 201
Comoros 59 170
Myanmar 58 350
Saint Lucia 57 1600
Saint Vincent and the Grenadines 55 728
Armenia 55 2676
Bulgaria 54 4492
Egypt 51 207
Vanuatu 49 3
Zimbabwe 48 332
Bosnia and Herzegovina 48 3590
South Africa 46 1515
Mozambique 46 62
Moldova 44 2556
Jamaica 41 833
Libya 39 819
Iraq 34 586
Angola 34 52
Eswatini 34 1102
Kyrgyzstan 34 423
Lesotho 32 308
Equatorial Guinea 31 121
Namibia 29 1398
Algeria 28 139
Togo 28 29
Gabon 25 125
Ghana 24 40
Congo 23 63
Guinea 21 29
Uganda 21 69
Guinea-Bissau 21 74
Djibouti 20 189
Kenya 18 98
Cote d'Ivoire 18 26
Liberia 17 55
Central African Republic 16 21
Benin 14 13
Senegal 13 110
Afghanistan 13 183
Sudan 12 72
Sierra Leone 11 15
Gambia 11 138
Syria 10 156
Ethiopia 9 59
Somalia 9 81
Zambia 9 197
Malawi 9 120
Nigeria 7 14
Papua New Guinea 6 65
Mali 5 32
Burkina Faso 5 15
Tanzania 4 12
Niger 4 10
Cameroon 4 67
Madagascar 3 34
Yemen 3 64
South Sudan 2 12
Chad 2 11
Haiti 2 66
Burundi 0 3
Mozambique 0 22

 

***For those who place value in correlation coefficients for such relationships (we don't) there is a significant positive correlation of 0.31 between number of vaccines and number of deaths

 

Wednesday, 29 December 2021

Covid-19: Definitions matter - and these are REAL


After posting the above on twitter there were some of the usual useful idiots saying it was all false without offering any evidence. Well, here are screenshots from the UK Government Covid dashboard website from today on definitions of cases and deaths:

 

It is also important to note that having a positive PCS test within 28 days of deaths is not the ONLY way to be counted as a Covid death. In fact the dashboard also has this plot, which is the one whose numbers are generally quoted:

If proper analyses of cause of death were undertaken then clearly this number should be much lower than the number who died within 28 days of a positive test. Unfortunately the numbers are consistently HIGHER - as of today the cumulative total of deaths within 28 days of a positive test is 148,089 whereas the cumulative total with Covid-19 on the death certificate is 171,801. This means that in addition to all of those classified as a Covid-19 death by virtue of a positive test within 28 days, another 23,000 deaths have been added. We suspect that many are like that of this twitter responder:

And (as the dashboard website has recently removed the definition of a Covid hospitalisation) here is the definition in the NHS England Report


For definition of a vaccinated and unvaccinated person see this CDC report (all the various studies of vaccine efficacy and safety from around the world that we have previously reported on use these definitions):

This CDC report also has the US definitions of cases, hospitalizations and deaths. The only difference to the UK definitions is that anybody dying within 60 days of a positive PCR test is classified as a Covid death:






14 month-old evidence submitted to UK Parliament on Covid-19 data transparency and accountability

Just discovered the statement below on the UK Government website (I’d totally forgotten about it). It’s the evidence we provided 14 months ago concerning data transparency and accountability during the COVID-19 crisis. A pdf version is here

Shame the Government totally ignored the evidence.


A Response to the Call for Evidence
Regarding COVID-19 Data
Transparency and Accountability

 

To: UK Parliament, Public Administration and Constitutional Affairs Committee, Commons Select Committee

 

1 November 2020

 

Professor Norman Fenton

Director of Risk Information Management Research Group

n.fenton@qmul.ac.uk

 

Professor Martin Neil

Professor in Computer Science and Statistics

 

Dr Scott McLachlan

Postdoctoral Researcher in Health Informatics (Queen Mary)

Fellow in Law (Birmingham Law School)

 

To:      UK Parliament

Public Administration and Constitutional Affairs Committee

Commons Select Committee

 

Honourable Members;

 

Re:      Data Transparency and Accountability: COVID-19.

 

Apropos the call for evidence concerning data transparency and accountability during the COVID-19 crisis.

We respond;

1.     We are a group of senior researchers in risk assessment, probability, statistics, and public health technologies based at Queen Mary University of London. Since March 2020, we have produced 23 articles/reports[1] (of which 5 have been published in peer reviewed journals) analysing the publicly available COVID-19 statistics and producing risk assessments and models.

2.     We believe that the statistics provided to and by the Government during the COVID-19  crisis have been inadequate and have been too easily used by influencers and decision-makers to fit particular narratives that have exaggerated the scale of the crisis.

3.     Statistics and data are observed phenomena arising from unobserved processes and their interactions (including causal explanations) as shown in Figure 1. The number of observed COVID-19 ‘cases’ clearly depends on how a ‘case’ is defined and the population infection rate, but it is also influenced by many (normally unreported) causal factors such as how many tests are being performed, who is being tested and why, and the accuracy of the testing. Similarly, while the number of observed COVID-19  ‘deaths’ clearly depends on how a COVID-19  death is defined and reported, it is also influenced by the population demographics, quality of healthcare etc. Hence, contrary to popular conception, data do not ‘speak for themselves’.

4.     For example, in March and April (as we pointed out in[2],[3],[4]), by focusing only on simple counts of ‘cases’, ‘hospitalisations’ and ‘deaths’, the public was misled into believing that the virus was more deadly than it really was. At that stage testing was essentially limited to those who were either already hospitalized with severe symptoms or were frontline healthcare workers. The reported high death (and hospitalisation) rates of those infected (calculated by simply dividing the number of deaths by the number of ‘cases’) were in part explained by the limited testing regime that was essentially only ‘finding’ the most severe ‘cases’.

5.     Similarly, the scale of the ‘second wave’ has been continually exaggerated by focusing on increased ‘cases’ without considering the simple causal explanation of massively increased testing. When this is done – as shown in the plots of Figure 2 –the trends for cases, hospitalizations and deaths look far less worrying than those presented at https://coronavirus.data.gov.uk using exactly the same data.

6.     At the root of the data problem there has been a fundamental misunderstanding about the meaning of terms ‘COVID-19 cases’ and ‘COVID-19 deaths’, and what can be interpreted from statistics that use these terms.  Even small changes in how these are defined and classified (as has happened several times since March) lead to very different trends and conclusions.

7.     The definition of a COVID-19 ‘case’ is especially concerning. In epidemiology, a case definition includes criteria for person (e.g. gender, race, age, or exclusion criteria), place (such as that associated with the outbreak of a disease), time (when illness started) and clinical features. Clinical features are initially normally simple and objective such as ‘sudden onset of fever and cough’ but should later be characterised by confirmed presence of specific laboratory findings, such as ‘ground-glass opacity on Chest CT and positive culture for SARS-CoV-2’. During this crisis, a positive PCR test has improperly become the surrogate replacing all four aspects of case definition. A PCR test may be positive: (i) before clinical features arise; (ii) long after clinical features have abated; or even (iii) when a person has simply come into contact with the disease but without them ever becoming infected. Some argue that reporting cycle threshold (Ct) values may help clinical decision-makers identify at which of these three stages an asymptomatic person may present; however, given that almost all so-called asymptomatic cases never develop active disease, if we leave aside issues with false positives (which increase for high Ct values), we submit that many ‘cases’ must be type (iii) and therefore did not meet the normal epidemiological standard to be classified or counted as a case.

8.     Confusion about the definition of a COVID-19 ‘death’ also persist. It is now clear that Government-reported deaths include not just those who died as a direct result of the disease, but also all of those who have died ‘with it’, thus leading to inflation of the fatality figures. Several studies have also suggested that reported deaths from other pneumonias, influenzas and even lung cancer have dropped well below normal annual levels since March. As such there are questions surrounding whether people who died of these similar conditions were incorrectly classified as COVID-19 deaths.

9.     With the massive increase in testing since August, uncertainty about the testing accuracy -  especially the false positive rate of PCR tests[5]   - means that almost nothing meaningful can be concluded about the increasing cases or fatality rate – see Figure 3.  The vast majority tested have no symptoms at all, so in the absence of data provided about the proportion of asymptomatic people who were tested and tested positive (as well as the other missing information shown in Figure 3), we do not know what proportion of new ‘cases’ and reported ‘deaths’ are people infected with COVID-19  at all. A false positive rate of even just 1% would, together with the massively increased testing, provide a causal explanation for the increase in cases even if the virus has largely subsided[6]. But, yet again, the narrative presented – and the one on which lockdown decisions are based – is that of a massive ‘second wave’.

10.  Closed loop thinking means that, once a particular narrative is ‘believed’, alternative explanations for the observed data are never entertained. Indeed, the lack of data, unscientific closed-shop models, fundamental misunderstandings by decision-makers, manipulation of underlying reporting processes, contradictory goals or the potential for malign intent are all feasible explanations for the observed data and chaotic analysis. The lack of data transparency gives credence to these explanations and leads to a lack of trust in government statistics and decisions made using those statistics.

11.  There are many examples of how the crude data, and failure to consider alternative causal explanations, has been used for inappropriate decision-making and even scare-mongering. These include:

i.      Using ‘100 new cases per 100,000 people’ as a threshold beyond which a local borough is required to move to lockdown. With this metric the threshold can be avoided or reached simply by decreasing/increasing the number of tests carried out.

ii.     As explained above (and in Figure 2) the headline figures and graphs – as presented for example, at https://coronavirus.data.gov.uk/ do not factor in the increase in testing. For example, the recent ‘exponential’ increase in number of cases – which has driven the ‘second wave narrative’ does not look at all serious when we plot it as number of cases per 1000 tests. The same is true of hospital admissions and deaths; for example,  contrary to the frightening ‘absolute’ increase in hospital cases since September, it turns out that the number of hospital admissions per 1000 cases has remained stable – and may even be decreasing when we factor in the false positives and those admitted for non-COVID reasons who happen to get a positive COVID test after admission.

iii.   The ONS report[7] on COVID-19 deaths by ethnicity is one of many that have produced misleading conclusions without even revealing all relevant data. This particular report exaggerated[8] the increased risk to people from the BAME community by using ‘relative risk’ to summarise the findings, rather than ‘absolute risk’ as continually  recommended for communicating risk to the public, by Royal Statistical Society Chairman (and member of SAGE) Professor Sir David Spiegelhalter[9]. Moreover, we noted[10], that the claims were almost certainly further exaggerated as they were likely based on out of date demographic information (the ONS failed to respond to our request to identify what data were used). Hence the ONS report – which was widely quoted in the media – was likely to create an unjustified level of fear and anxiety among the BAME community. Failure to identify causal explanations for data bias has also led to multiple well-publicised studies with exaggerated[11],[12] - or even flawed[13] -  claims that certain communities, or people with certain attributes or habits, are at much higher risk of COVID-19.

iv.   In early October news broke of under-reporting of almost 16,000 positive PCR tests and that, as a result, as many as 48,000 people may not have been informed of their exposure due to close contact with these undisclosed ‘cases’[14]. PHE blamed Microsoft’s Excel software[15], but this disingenuous admonition did more to highlight PHE’s: (a) reliance on almost 25yr old technology; (b) ignorance of and failure to maintain pace with technology; and (c) lack of any reliable approach to checking and validating data they collect and report. Data security experts describe this as one in a long string of data and information security failings by PHE and the Government and have used it to support eschewing use of the proposed NHSx track and trace apps[16].

v.     Removal or sanitising of flu incidence/death data from 1999 and all previous years from the ONS website making comparisons almost impossible and giving the impression that the ‘past is being rewritten or expunged’.

vi.   Constant changing of scales and metrics used in data reporting. For example, deaths were recorded as COVID-19 deaths if they occurred within 28 days of a positive test and this has recently been changed to 60 days if COVID-19 appears on the death certificate. This change was done in reaction to a recommendation that the period should be reduced to 21 days. The change was made with no accompanying   explanation of why it was increased rather than decreased[17].

12.  Ultimately the only way to achieve accurate estimates of the critical population infection rate at any given time is to provide the missing – but easy to obtain data shown in Figure 4.

13.  Decisions about lockdown require data to support the evidence shown in  Figure 5. If these data have been considered in Government decisions, they have certainly not been made public.

 

In summary, and supported by the arguments above our responses to the eight issues identified in the public call are:

 

14.  In response to Issue 1: Did the Government have good enough data to make decisions in response to Coronavirus, and how quickly were the Government able to gather new data?

Data provided by several departments including PHE, NHS and ONS for Government decision-making was observed to be ever-changing, unreliable, and of such poor quality and so inappropriately framed as to be insufficient to support the public health, policy and legislative decisions that resulted.

15.  In response to Issue 2: Was data for decision-making sufficiently joined up across departments?

Where multiple actors are responsible for collecting and reporting data that will be aggregated and used to direct public policy: definitions, thresholds and processes must observe a consistent standard. The central aggregator, in this case the ONS, should have been responsible for both dictating and enforcing that standard.

As evidenced by anomalies and misrepresentations identified above, the efforts of PHE, NHS and ONS were not sufficiently joined up, fell short of due standards, and severely undermined Government decision-making, independent scrutiny, and ultimately public confidence.

16.  In response to Issue 3: Was relevant data disseminated to key decision-makers in: Central and Local Government; other public services (like schools); businesses; and interested members of the public?

To be relevant, data must be capable of informing the decision-making process. Relevant data is that which is accurate, timely, indisputable, optimised and fit to inform the known purposes for which it may be used.

Given that Government was aware most members of the public consume only limited ‘views’ of such data as are presented in the media, its presentation, accuracy and fitness for purpose should have received greater consideration. While data was made available via the ONS website, for the reasons discussed above the relevance of this data has remained questionable.

17.  In response to Issue 4: Were key decisions (such as ‘lockdowns’) underpinned by good data and was data-led decision-making timely, clear and transparently presented to the public?

Government decisions impacting the liberty and freedoms of individuals appear to have been made haphazardly. While each came supported by justifications, it was claimed, that they were ‘led by the science’, more often it could be argued this was not the case.

Decision making has been presented as being the result of “the science” with the goal of delivering ‘consensus’. However, science does not operate as a consensus making mechanism and it is not monolithic. The current crisis has demonstrated that groups like SAGE and the Joint Biosecurity Centre are not following scientific norms of behaviour. Analysis and policy formulation need more stringent oversight in a way that invites and delivers scientific debate from both within and also outside the group.

18.  In response to Issue 5: Was data shared across the devolved administrations and local authorities to enable mutually beneficial decision-making?

If this were the case it has not been made clear to the public, and in any case, it is likely that the shared data suffered from all of the limitations we have highlighted.

19.  In response to Issue 6: Is the public able to comprehend the data published during the pandemic. Is there sufficient understanding among journalists and parliamentarians to enable them to present and interpret data accurately, and ask informed questions of Government?

It is difficult to ensure accurate comprehension in circumstances where, as discussed earlier, relevantly framed data has not been provided. Continued reliance on journalists to identify meaning from data has only resulted in sensational headlines that amplified public ignorance and promulgated fear.

What could have been done to improve understanding and who could take responsibility for this?

The current crisis has demonstrated Government must take additional steps to provide context and meaning capable of supporting differing interpretations they wish the public should draw from published data. The public should be trusted to understand nuance and scientific disagreement about what the data might be telling them.

20.  In response to Issue 7: Does the Government have a good enough understanding of data security, and do the public have confidence in the government’s data handling?

The policies and approaches of Government do not seem to have reflected prevailing opinions and wishes of the public. This has never been more obvious than during development and release of both versions of the NHSx Track and Trace smartphone app, and when vision-based population proximity monitoring AI systems were deployed around London suburbs, and once exposed in the media, hastily removed. It seems there is little public confidence in the current approach to securing public and personal data and indeed the potential for increased suspicion of the government’s motives in this regard.

21.  In response to Issue 8: How will the change in responsibility for Government data impact future decision-making?

It is not clear what the change in responsibility is and the motivation for it. Any change in responsibility might simply be akin to ‘rearranging the deckchairs on the Titanic’.

 

 

 

 

FIGURES

 

 

Figure 1 Causal model explaining observed data

 


Figure 2 Simple plots that take account of number of tests and cases

 

Figure 3 Why the daily reported data tell us almost nothing


Figure 4 Missing data needed to accurate estimation of population infection rate

 

 

Figure 5 The evidence we need to demonstrate why lockdowns are needed

 



[1] https://www.eecs.qmul.ac.uk/~norman/all_publications.htm#Papers

[2] Fenton, N. E., Neil, M., Osman, M., & McLachlan, S. (2020). "COVID-19 infection and death rates: the need to incorporate causal explanations for the data and avoid bias in testing". Journal of Risk Research, 1–4. https://doi.org/10.1080/13669877.2020.1756381

[3] Fenton, N. E., Osman, M., Neil, M., & McLachlan, S. (2020). Coronavirus: country comparisons are pointless unless we account for these biases in testing. The Conversation,  April 2, 2020  https://theconversation.com/coronavirus-country-comparisons-are-pointless-unless-we-account-for-these-biases-in-testing-135464

[4] Fenton, N.E., Hitman, G. A., Neil, M., Osman, M., & McLachlan, S. (2020). Causal explanations, error rates, and human judgment biases missing from the COVID-19 narrative and statistics. PsyArXiv Preprints. https://doi.org/10.31234/OSF.IO/P39A4

[5] Cohen, A. N., Kessel, B., & Milgroom, M. G. (2020). Diagnosing COVID-19 infection: the danger of over-reliance on positive test results. MedRxiv, 2020.04.26.20080911. https://doi.org/10.1101/2020.04.26.20080911

[6] https://probabilityandlaw.blogspot.com/2020/09/the-impact-of-covid-false-positives.html

[7] Office for National Statistics. (2020a). Coronavirus (Covid-19) related deaths by ethnic group, England and Wales -2 March 2020 to 10 April 2020.

[8] Fenton N. E, Neil M, McLachlan S, Osman M (2020), "Misinterpreting statistical anomalies and risk assessment when analysing Covid-19 deaths by ethnicity". DOI: 10.13140/RG.2.2.18957.5680

[9] https://www.regulation.org.uk/library/2017-Spiegelhalter-Risk_and_Uncertainty_Communication.pdf

[10] Fenton N. E, Neil M, McLachlan S, Osman M (2020), "Misinterpreting statistical anomalies and risk assessment when analysing Covid-19 deaths by ethnicity". DOI: 10.13140/RG.2.2.18957.5680

[11] Fenton N. E, Neil M, McLachlan S, Osman M (2020), "Misinterpreting statistical anomalies and risk assessment when analysing Covid-19 deaths by ethnicity". DOI: 10.13140/RG.2.2.18957.5680

[12] Fenton, N E. (2020). A Note on UK Covid19 death rates by religion: which groups are most at risk? http://arxiv.org/abs/2007.07083

[13] Fenton, N E (2020), "Why most studies into COVID19 risk factors may be producing flawed conclusions-and how to fix the problem", http://arxiv.org/abs/2005.08608 

[17] https://www.gov.uk/government/news/new-uk-wide-methodology-agreed-to-record-covid-19-deaths