Non-Technical Summary
Botched covid-19 test results in Britain led to thousands of extra cases.
A new study shows that each false negative results in 0.6-1.6 subsequent reported infections.
Among the tools health officials have used to fight covid-19, the impacts of lockdowns, masks and vaccines have been well-studied. Testing, whose effect is hard to measure because it coincides with other pandemic-control policies, has received less attention. But thanks to a grim natural experiment, that is now starting to change.
In September covid seemed to come to a halt in south-western Britain. Overnight, the positivity rate of pcr tests near Bath and Bristol fell from 3% to 1%. Nowhere else in England enjoyed a similar decline.
By mid-month a local volunteer group said this apparent dip might stem from faulty tests. A few weeks later, the group and local scientists asked Britain’s Health Security Agency (ukhsa) to investigate.
On October 12th the ukhsa shut down a lab run by Immensa, a testing firm. The agency said that the firm had incorrectly told 43,000 infected people that they were virus-free. The cause of the error remains unclear. Immensa declined to comment.
As soon as the lab was closed, the region’s reported covid case rate soared. A spokesperson for the prime minister said that the lab errors did not cause this surge. On November 14th, however, Thiemo Fetzer of the University of Warwick released a paper showing that they probably did. It has not yet been peer-reviewed, but offers firm evidence that accurate testing does indeed slow covid’s spread, by letting infectious people know that they should isolate.
To estimate what might have happened were it not for the snafu, Mr Fetzer built a “synthetic control”: a group of areas whose prior rates of vaccination, testing, and covid cases and deaths matched those of the 13 most affected regions. The difference was stark. From September 2nd to October 12th, the areas in question recorded 13,000 fewer positive tests than the control. Afterwards, they registered 21,000 more.
This implies that each faulty test may have led to 0.6-1.6 extra cases (the range reflects uncertainty over how many people who tested positive received earlier false negatives). Based on Britain’s case-fatality rate, this translates to 100-250 deaths.
Surprisingly, this toll is not the highest that Mr Fetzer has attributed to technical glitches. In 2020 a spreadsheet error prevented Britain’s statistical service from reporting 15,000 covid cases to contact-tracers. In an earlier study, Mr Fetzer calculated that 1,500 people died as a result.
Measuring the Epidemiological Impact of a False Negative: Evidence from a Natural Experiment, CAGE Working Paper 596, 2021.
Abstract
Reliable COVID-19 testing remains a central pillar to manage the pandemic. Yet, the accuracy and reliability of tests and test equipment has regularly been brought into question. Both false-positive and false-negative test results convey costs. Yet, false negatives are likely more problematic due to the risk of onward transmission and the failure to break infection chains as a result. This paper studies the epidemiological impact of a false negative in the context of a high vaccine uptake country. Between 2 September and 12 October an estimated 43,000 PCR tests in the UK may have produced a false negative test result with individuals infected being told that they tested negative. These instances were particularly pronounced in the South West of England. Using a synthetic control method approach concentrating on the 13 most affected regions, this paper estimates that every false negative COVID-19 case is likely to have caused between 0.6 to 1.6 additional infections in the subsequent weeks.
Press Coverage & Non-Technical Summaries
2021
BBC Points West (Youtube) (UK) 🇬🇧, The Economist (UK) 🇬🇧, The Times (UK) 🇬🇧.
2022
BBC (UK) 🇬🇧.
Video Summary – 2021
Social Media Summary
What is the epidemiological impact of a #falsenegative #COVID test? An important question in a high vaxx/low NPI context, but one that cannot be studied in a experiment for obvious reasons. Enter the UK, a reliable supplier of #naturalexperiments. 🧵⬇️
➡️https://t.co/Zt0cHz9zoo pic.twitter.com/UIqWd1kCC8— Thiemo Fetzer 🇪🇺🇺🇦 – same handle elsewhere (@fetzert) November 15, 2021