Thiemo Fetzer | Professor of Economics

Measuring the Scientific Effectiveness of Contact Tracing: Evidence from a Natural Experiment

Mobile phone with contact tracing app showing.

Non-Technical Summary

NHS Test and Trace works, new research finds, but the data glitch which led to over 15,000 late referrals in October may have brought about more than 125,000 additional COVID-19 infections and over 1500 deaths.

On 3 October Public Health England revealed that, due to a technical error, at least 15,841 individuals who tested positive for COVID-19 between 20 September and 2 October had not been referred to the national contact tracing system. The technical error was due to an Excel spreadsheet reaching its maximum file size. New analysis shows that this data glitch, which by chance affected some regions more than others, led to a significant increase in new infections – and subsequently more deaths – in those areas.

In the new paper, ‘Does Contact Tracing Work? Quasi-Experimental Evidence from an Excel Error in England’, Dr Thiemo Fetzer of the CAGE Research Centre in the Economics Department at the University of Warwick and Dr Thomas Graeber of Harvard Business School consider the effectiveness of contact tracing by analysing the causal impact of the delays resulting from the data glitch. Key findings are:

· By chance, the loss of case information from was more severe for some areas of England than others.

· Areas that were more strongly affected by the lack of contact tracing subsequently experienced a sharp increase in COVID-19 infections and COVID-19 related deaths.

· The study’s most conservative estimates suggest that the failure to contact trace is associated with more than 125,000 additional infections and more than 1500 deaths.

· The delay in contact tracing was further associated with a subsequent worsening of the regional performance of contact tracing.

The research leverages daily data on the spread of COVID-19 at the Lower Tier Local Authority level, together with weekly data on NHS Test and Trace testing activity and contact tracing performance.

To demonstrate the causal connection between the delays in contact tracing and the spread of COVID-19, Dr Fetzer and Dr Graeber show that areas that had more delayed contact tracing cases saw a notable differential increase in infections ­– and subsequently in deaths – compared to other, otherwise comparable areas.

Dr Fetzer said, ‘Distinguishing correlation from causation in the analysis of non-pharmaceutical interventions is hard. This data glitch gives researchers a natural experiment to cast a light on whether contact tracing can make a difference as it resulted in a large number of cases that were not contact traced.’

The results strongly suggest that contact tracing is an important tool to contain the pandemic. Dr Graeber said, ‘The large size of the estimated effects implies that a well-functioning and robust contact tracing system may be even more important to fight COVID-19 than we previously thought’.

T. Fetzer & T. Graeber, Measuring the scientific effectiveness of contact tracing: Evidence from a natural experiment, Proceedings of the National Academy of Sciences, Volume 118 Issue 33, 2021.

🔓 Open access
🗃️ Replication archive
💾 Presentation Slides

Significance

Contact tracing constitutes the backbone of nonpharmaceutical public interventions against COVID-19, as it did with previous pandemics. Experts argue that its importance rises again as vaccination rates increase and the spread of COVID-19 slows, which makes tracing of individual cases possible. However, because randomized experiments on contact tracing are infeasible, causal evidence about its effectiveness is missing. This shortage of evidence is alarming as governments around the world invest in large-scale contact tracing systems, frequently facing a lack of cooperation from the population. Exploiting a large-scale natural experiment, we provide evidence that contact tracing may be even more effective than indicated by previous correlational research. Our findings inform current and future public health responses to the spread of infectious diseases.

Abstract

Contact tracing has for decades been a cornerstone of the public health approach to epidemics, including Ebola, severe acute respiratory syndrome, and now COVID-19. It has not yet been possible, however, to causally assess the method’s effectiveness using a randomized controlled trial of the sort familiar throughout other areas of science. This study provides evidence that comes close to that ideal. It exploits a large-scale natural experiment that occurred by accident in England in late September 2020. Because of a coding error involving spreadsheet data used by the health authorities, a total of 15,841 COVID-19 cases (around 20% of all cases) failed to have timely contact tracing. By chance, some areas of England were much more severely affected than others. This study finds that the random breakdown of contact tracing led to more illness and death. Conservative causal estimates imply that, relative to cases that were initially missed by the contact tracing system, cases subject to proper contact tracing were associated with a reduction in subsequent new infections of 63% and a reduction insubsequent COVID-19–related deaths of 66% across the 6 wk following the data glitch.

Press Coverage & Non-Technical Summaries

2020

Bloomberg School of Public Health Summary (US) 🇺🇸, City AM (UK) 🇬🇧, Die Welt (German) 🇩🇪, FocusTech (Italian) 🇮🇹, Le Monde (French) 🇫🇷, Medscape (UK) 🇬🇧, Neue Zuericher Zeitung (German) 🇩🇪, News Medical (Australia) 🇦🇺, Sky News (UK) 🇬🇧, The Guardian (UK) 🇬🇧, ZAP (Portugal) 🇵🇹.

2021

Atlantic (French) 🇫🇷, Financial Times (1) (UK) 🇬🇧, Financial Times (2) (UK) 🇬🇧, La Libre (French) 🇫🇷, Liberation (French) 🇫🇷, Svenske Dagblad (Swedish) 🇸🇪, Wired (US) 🇺🇸.

Previous Versions

Does Contact Tracing Work? Quasi-Experimental Evidence from an Excel Error in England, medrxiv pre-print, 2020.

Social Media Summary

Posted

in

,

by

Gillian

Tags: