On false positives in COVID19 testing again: we are being misled over confirmatory testing

 

It has long been claimed that many of the COVID19 'cases' reported (where a 'case' is simply a positive test result) are false positives. This short article addresses an aspect of the problem not widely discussed before and suggests that many of the very large number of new 'cases' reported amid great hysteria each day could be people who do not have COVID19.

One of the biggest misunderstandings about the notion of accuracy for a type of test (such as a PCR test) is the idea that there is a (single) false positive rate, namely the probability that the test will be positive for a person who does not have the virus. The false positive rate at any particular time - and in any particular testing lab - will depend on multiple factors. Some of these factors relate to the lab doing the testing, which includes: experience of staff, pressure staff are working under, probability of contamination; the particular equipment and chemicals used; and, of course, the much discussed Ct-value threshold used. Other factors relate to the source of the samples being tested: for example, whether they were primarily from people with symptoms, whether they were primarily from people in hospitals, whether they were primarily from people who had been in recent contact with a COVID19 positive person, etc.  If any of these factors change, the false positive rate is likely to change. Without information about all these factors there is nothing we can conclude with any kind of certainty from counts of tests and positives alone.

However, as I explained in this article, by far the biggest factor affecting the false positive rate is whether and how a positive test is repeated before actually declaring that a person has tested positive (this is called confirmatory testing). Let us suppose that on any single test the current false positive rate is 0.01 (i.e. 1%). Then, if a lab does no confirmatory testing, 0.01 can be crudely assumed to be the false positive rate (meaning 1 in every 100 without the virus would wrongly test positive). However, if the lab does confirmatory testing whereby a new sample is taken from the person who tested positive, then it reasonable to assume that the two tests are 'independent' and in that case the probability a person who does not have the virus tests positive both times is 0.01 times 0.01 which is 0.0001, i.e. 0.01% (so only 1 in every 10,000 without the virus would wrongly be classified as testing positive). 

Hence, for a Lab in which confirmatory testing is carried out rigorously, the false positive rate can be close to 0 even if 'the false positive rate' for individual tests is relatively high.

When I posted this article suggesting that - due to pressure of massively increased numbers of tests - it might be possible that confirmatory testing was not always carried out - a number of indignant respondents on Twitter ridiculed the idea, insisting that confirmatory testing was always done. But, in the absence of any clear information about this everything points to the likelihood that rigorous confirmatory testing may not have been widely conducted at all since mass testing began in late summer. 

The official NHS Guidance document on PCR testing says the following (page 7):

Positive results that are sent by NHS England and NHS Improvement pathology network laboratories for confirmation to a PHE laboratory will be considered presumptive positives until confirmed. (see list –Appendix 2). Confirmation is not required if network laboratories are confident in the test they have adopted and assured of an accurate result. If in any doubt, samples can be referred to a PHE regional laboratory local to the NHS testing laboratory for confirmatory testing, for an initial period until the NHS network laboratory is assured their testing is robust, accurate and safe. After this time confirmation by local PHE laboratories will no longer be required. Presumptive positive/positive results will be notified to the co-ordination center for contact tracing, which will start immediately.

So we know that confirmation was never required providing the labs were sufficiently confident that they could 'accurately test' first time round. Even if confirmatory testing is done there is also no definitive requirement to test a new sample, meaning that the second test may not be independent of the first; in that case there will not be such a great reduction in the false positive rate.  

The simple anecdotal evidence that independent confirmatory testing is now unlikely to be done routinely is that we know many people who have got a positive test result (outside of hospitals) and they have never been asked to do a repeat test.

The Government website that produces the daily number of  'new cases' is supposed to only count 'confirmed' cases. In other words, it should not include any positive test where that was the first positive test for the person sampled. As we now know that this is no longer true it means that the published 'number of cases' are not all 'confirmed' cases. Hence, it will include a greater percentage of false positives than previous periods when more confirmatory testing was done.

The massive increase in testing since September means that it is very likely an increasingly smaller proportion of confirmatory testing has been done, which in turn means the false positive rate would have increased.  Similarly, when testing is scaled back the false positive rate is likely to decrease. While there was also likely to have been a small resurgence of COVID19 in September, the changes in false positive rate are likely to be the most significant causal explanation for the close correlation between number of tests and test positivity (% of test that are positive) that has been observed in all areas of the UK since September.  

Number of daily tests and test positivity rate for England

In contrast to the 'number of cases' which has risen steadily as more tests are conducted (and which cannot be considered as a reliable measure of the infection rate), the data on NHS triages for COVID (see https://digital.nhs.uk/dashboards/nhs-pathways) - which includes all 999, 111 calls and 111-online reports also supports the hypothesis that the big increases in number  of 'cases' since September is due in part to increase in false positives; while the greater availability of tests could explain some of the decrease in people using the NHS triage path, the data below still provides evidence that, after a small resurgence in September, the number of people actually ill with COVID has remained quite steady and much lower than it was in the spring.

 

NHS daily COVID triage reports (999, 111 calls and 111-online)

To give some idea of the impact of a changing false positive rate, let us suppose for illustration that (as above) the individual test false positive is 0.01 and that therefore the independent confirmatory false positive rate is 0.0001. Then, with confirmatory testing if the population infection rate is 0.5% (1 in 200) then, assuming an 80% true positive test rate, it follows from Bayes Theorem that 97.6% of those testing positive really are positive. However, without confirmatory testing only 28.7% of those testing positive really are positive.

There is no getting away from the fact that the data on testing and cases provided by the Government is insufficient to make any definitive conclusions about the true status of COVID19. I have already listed the many additional pieces of information needed to make anything other than massively uncertain conclusions. As this article has explained we also need to know how many of the new 'cases' were subject to confirmatory testing. Given the incredible resources being spent on testing, if the Government cannot provide the additional information I have listed (which all must be available somewhere) there really is no point in carrying out these tests. 

 

Update: There is also the issue of the difference between PCR test results and LFT test results

See also previously on this blog:

 

      

• Pooled COVID19 testing makes the data on 'cases' even more dubious  

  Remarkable relationship between number of tests and positivity rate when we drill down into regions
  

  COVID-19 in the UK: the remarkable divergence between number of 'cases' and number of people reporting symptoms
  

• How to explain an increase in proportion testing positive if there is no increase in infection rate
• We are still not getting the basic testing data we need
  
• Why we know so little about COVID19 from the data provided
• Impact of false positives in Covid testing
• Covid19 hospital admissions data: evidence of exponential increase?
  
• Don't panic: limits to what we know about Covid-19 PC testing, inferred infection rates and alse positive rates
  
• A privacy-preserving Bayesian network model for personalised COVID19 risk assessment and contact tracing
  Covid-19: Infection rates are higher, fatality rates lower than widely reported 
• Coronavirus: country comparisons are pointless unless we account for these biases in testing 
• Why most studies into COVID19 risk factors may be producing flawed conclusions - and how to fix the problem
• Causal explanations, error rates, and human judgment biases missing from the COVID-19 narrative and statistics
• Covid-19 risk for the black and minority ethnic community: why reports are misleading and create unjustified fear and anxiety
• UK Covid19 death rates by religion: Jews by far the highest and atheists by far the lowest 'overall' - but what does it mean?
• All COVID articles on this blog