Pooled COVID19 testing makes the data on 'cases' even more dubious

This article by Clare Craig, Jonathan Engler, Mike Yeadon, and Christian McNeill lays out a convincing case why PCR testing for COVID19 has failed.

While I was aware of many of the issues they raise, until speaking with Clare, Jonathan and also Joel Smalley, Keith Johnson, Martin Neil and Scott McLachlan yesterday I was unaware of another major problem with PCR testing that is not discussed in the article. Specifically, when mass PCR testing for COVID19 started in the UK in late summer, so too did the idea to implement pooled testing as described in this official NHS document. It is not clear how extensively the idea has been implemented, but it could have created more problems.

The motivation for pooled testing for any virus is that you can do much more efficient testing when population infection rates are low. Instead of separately testing each sample you combine, say, 10 samples and test the pooled sample. When prevalence is low then the pooled sample will most likely test negative and so we can conclude that all of the individual samples in it were negative. On the relatively rare occasions when the pooled sample tests positive we conclude that at least one of the individual samples would be positive and so (only when this happens) we retest each of the individual samples to confirm which ones were positive.

A proposed implementation of the NHS Guidelines for pooled testing by McNally and Ball  is described here.

However, it is clear that there are the following fundamental problems with using pooled testing for COVID19. 

1. For a start, pooled testing - according to the NHS document - should be used only when the population infection rate is in the range 1% to 2% and when the infection rate among those being tested (the test positivity rate) is around 8%. But many areas currently have much higher test positivity rates (for example, as we can see below, London is now at 15% and the number tested there in the week ending 18 Dec was 410,463 - an increase of over 57% from the 261,075 just one week earlier). So, this should certainly rule out pooled testing. Moreover, the title of the NHS document suggests it should be used only on samples from asymptomatic people, which makes sense since those with symptoms will have a much higher probability of being infected. However, it is not clear if this has been the case. The NHS document says: 

 "It is also expected that as winter approaches the demand for COVID-19 testing will increase, putting more pressure on an already stretched system. Sample pooling strategies increase testing capacity when resources are limited.” 

If pooled testing was only used on asymptomatics, then testing data that differentiates between symptomatics and asymptomatics (data which I have argued is critical for determining the true infection rate) would be available, but we are constantly told it is not.

2. As explained here, calculating the optimum number of samples (and process) for pooled testing requires prior knowledge of: 

• The current true population infection rate
  
• The false positive test rate for asymptomatics (i.e. the probability an asymptomatic person without the virus tests positive)
• The false negative test rate for asymptomatics (i.e. the probability an asymptomatic person with the virus tests negative) 
  

Moreover, all of these values need to be low for pooled testing to be efficient and accurate. But, as I have made clear many times on this blog, NONE of these are known; indeed, one of the main purposes of testing asymptomatics is to find out what these really are.

So, while theoretically the concept of pooled testing (especially if you are testing asymptomatic people) for COVID19 is sound, practically it seems flawed before we start. 

3. Asymptomatics with the virus have low viral load, so even for a single sample it requires high Ct values to detect it. Because pooled samples are mixed from the original samples, even higher Ct values are needed to detect the virus if it is indeed there. But, while high Ct values decrease the false negative rate, they increase the false positive rate. This is tolerable when the population infection rate is very low because there will be sufficiently few positive samples so that they can all be retested individually. Suppose, for example, that the false positive rate for a pooled sample is 5% and the false positive rate for a single sample is 2%. Then the probability that an individual sample will wrongly test positive is the probability it will wrongly test positive both in a pooled sample and then individually. This is 0.05 times 0.02 which is 0.001, i.e. just 0.1%.  Similarly, if there is no pooled testing, but there is confirmatory testing for each individual sample that tests positive, then the probability that a sample will wrongly test positive twice is 0.02 times 0.02 which is 0.0004, i.e. 0.04%.

The false positive rates for a PCR test are illustrative. They could be higher or lower

 This explains why – in the summer when the infection rate was close to zero – the number of ‘positives’ (and necessarily the number of false positives) was also very low. All those people who have ridiculed the notion of false positive rates above 1% for PCR testing simply do not understand the role of confirmatory testing (whether in pooled or individual testing). But, while the conditions for pooled and/or confirmatory testing were OK for the summer (low population infection rate, lots of testing but mostly asymptomatic people) that changed as soon as the infection rate (as well as illnesses which have similar symptoms) increased in the autumn. 

4. Even if the population infection rate is stable any big increase in the number of tests conducted is likely to lead to a greater proportion of false positives whether or not pooled testing is used. This is because, as testing increases not only is there greater potential for various types of handling and contamination errors due to limited resources and time, but inevitably it seems fewer confirmatory tests can be carried out. If pooled testing is performed without confirmatory testing then the false positive rate (using the above example figures - see table) increases from 0.1% to 5%. That is a massive increase. If pooled testing is not applied then the false positive rate rises from 0.04% when confirmatory testing is used to 2% when it is not. 

It is difficult to find out just how much pooled testing has been going on and whether and when there has been a drop in confirmatory testing. It is likely that there will have been major differences between labs. But, given the massive increase in testing since September, if for whatever reason it is felt that confirmatory testing cannot be carried out, then this would explain  why we are seeing a very close correlation between number of tests and test positivity.

p.s. Jonathan Engler raises the following additional relevant points: 

"The PCR test has been worked up from bench-top to industrialised scale, in assay development terms, overnight. In particular, there is only quite limited testing of cross-reactivity with other viruses. What testing there was – as outlined in the Drosten paper – revealed some cross-reactivity.

Also from the Drosten paper they do report on 4 positive cases out of 310 negative controls containing other viruses: These examples of “weak initial reactivity” are somewhat brushed off, justified by them testing negative 2nd time around, but they do account for >1% of tests. (What if the 306 initial negatives had been tested again – might some of those then have then been positive on their 2nd test?). The point of this in relation to pooling is that there are clearly uncharacterised sources of false positivity, likely including but possibly not limited to unknown or undetected coronaviruses; even a single such instance could contaminate the entire pool, and then testing at high cycle rates could surely magnify such issues greatly."

London COVID testing data (from Govt website) 24 Dec 2020

 

 

 

See also:

• PCR based COVID testing has failed
  
• Pooling of asymptomatic SARS COV-2 COVID-19 samples for (PCR/or other) testing 
• Pooled testing for SARS-CoV-2 could provide the solution to UK’s testing strategy
• UK Covid Testing data: Remarkable relationship between number of tests and positivity rate when we drill down to regions