Article Text
PDF
Abstract
Objective To systematically review evidence on effectiveness of contact tracing apps (CTAs) for SARS-CoV-2 on epidemiological and clinical outcomes.
Design Rapid systematic review.
Data sources EMBASE (OVID), MEDLINE (PubMed), BioRxiv and MedRxiv were searched up to 28 October 2020.
Study selection Studies, both empirical and model-based, assessing effect of CTAs for SARS-CoV-2 on reproduction number (R), total number of infections, hospitalisation rate, mortality rate, and other epidemiologically and clinically relevant outcomes, were eligible for inclusion.
Data extraction Empirical and model-based studies were critically appraised using separate checklists. Data on type of study (ie, empirical or model-based), sample size, (simulated) time horizon, study population, CTA type (and associated interventions), comparator and outcomes assessed, were extracted. The most important findings were extracted and narratively summarised. Specifically for model-based studies, characteristics and values of important model parameters were collected.
Results 2140 studies were identified, of which 17 studies (2 empirical, 15 model-based studies) were eligible and included in this review. Both empirical studies were observational (non-randomised) studies and at high risk of bias, most importantly due to risk of confounding. Risk of bias of model-based studies was considered low for 12 out of 15 studies. Most studies demonstrated beneficial effects of CTAs on R, total number of infections and mortality rate. No studies assessed effect on hospitalisation. Effect size was dependent on model parameters values used, but in general, a beneficial effect was observed at CTA adoption rates of 20% or higher.
Conclusions CTAs have the potential to be effective in reducing SARS-CoV-2 related epidemiological and clinical outcomes, though effect size depends on other model parameters (eg, proportion of asymptomatic individuals, or testing delays), and interventions after CTA notification. Methodologically sound comparative empirical studies on effectiveness of CTAs are required to confirm findings from model-based studies.
- COVID-19
- epidemiology
- microbiology
- statistics & research methods
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information. All data generated in this study were published, either as part of the manuscript, or its supplemental file.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first paper to provide a comprehensive overview and critical appraisal of studies assessing the effectiveness of contact tracings apps for SARS-CoV-2 on clinical and epidemiological outcomes.
Studies were retrieved using a large repository that is developed by a specific search string dedicated to identify studies on SARS-CoV-2 published in various underlying databases.
Critical appraisal was performed by reviewers from diverse backgrounds (ie, mathematical modelling, epidemiology, medicine, systematic reviews) using predefined customised templates for both empirical and model-based effectiveness studies.
Given the rapid execution and (preprint) publication of studies on effectiveness of contact tracing apps (CTAs) for SARS-CoV-2, this review is unlikely to include the most recent studies published after the search date.
Due to high heterogeneity across studies, it was not feasible to provide a pooled meta-analysis estimate of the effectiveness of CTAs for SARS-CoV-2 on the clinical and epidemiological outcomes.
Introduction
The SARS-CoV-2 outbreak has dominated worldwide news and scientific research throughout 2020. Since its emergence in Wuhan (People’s Republic of China) in early December 2019, reducing transmission of SARS-CoV-2 has been a worldwide priority. Digital technology could be applied for efficient contact tracing. Contact tracing applications (CTAs) are able to identify individuals who have recently been in close contact with infected individuals (and may have acquired infection as a consequence). After identification, the contact person can be instructed to go in self-quarantine, preventing further transmission and spread of the virus.
A substantial amount of research on CTAs for SARS-CoV-2 has been performed since the start of the pandemic. Summarising all evidence, including results from research that has not yet undergone, or is currently undergoing peer-review, is warranted to provide an overview of what is known regarding CTA effectiveness. Research that has not yet undergone peer-review is often published by authors through so-called preprint databases. However, identifying these articles, extracting data and drawing conclusions can be a challenge, as this requires knowledge on epidemiology, mathematical modelling, systematically appraising evidence and summarising that evidence.
A few overviews of evidence on effectiveness of CTAs have been published in recent time. Anglemyer et al provided an overview of study characteristics and quality appraisal of studies on effectiveness of CTAs and other digital contact tracing technologies.1 However, their data are based on both SARS-CoV-2 infections and other infections (eg, Ebola), and lack a quantitative effectiveness measure of CTAs on clinically relevant outcomes. Other systematic reviews focused only on user experience in using a CTA for SARS-CoV-2 detection,2 or only studied manual, as opposed to digital, contact tracing.3 One systematic review did look into studies on automated and semi-automated CTAs for SARS-CoV-2, but lacked reporting on CTA effectiveness on total number of infections, and hospitalisation or mortality rates.4
In this rapid systematic review, we aim to evaluate all (empirical and model based) studies addressing effectiveness of CTAs for SARS-CoV-2 on relevant epidemiological and clinical outcomes. We will provide descriptive characteristics, critical appraisal and a narrative summary of evidence of included studies.
Methods
Search strategy
The Bern COVID-19 Open Access Project (COAP) database was used for identification of relevant research. The COAP database is comprised research from EMBASE (OVID), MEDLINE (PubMed), BioRxiv en MedRxiv databases, specifically focused on SARS-CoV-2. On 28 October 2020 the COAP database was searched for scientific literature evaluating the effectiveness of CTAs for SARS-CoV-2 on epidemiological and clinical outcomes. The complete search strategy, as well as background information on the COAP database provided by Bern University, are provided in online supplemental file 1.
Supplemental material
Eligibility criteria
Empirical (both observational and experimental) and model-based studies evaluating effectiveness of CTAs for SARS-CoV-2 were eligible for inclusion. Peer-reviewed publications as well as preprint papers were considered.
CTAs were considered when they provided feedback about potential recent exposure to an infected individual, based on proximity measurements (eg, Bluetooth or GPS). Feedback should be provided directly to the individual through a CTA, although other feedback mechanisms, such as personal devices (eg, a smartwatch), were also considered. National emergency warning systems using SMS were also included, provided they used proximity data to inform individuals.
All epidemiologically or clinically relevant outcomes quantifying the impact of CTAs were considered, which include but are not limited to: the reproduction number (R), total number of infections, hospitalisation rate and mortality rate related to SARS-CoV-2. Studies investigating other relevant outcomes, such as prevention of outbreaks or a second infection wave of SARS-CoV-2, were also included. Studies solely assessing (determinants affecting) adoption rate of CTAs (ie, the proportion of citizens using, and following recommendations provided by, the CTA), temporal change in incidence SARS-CoV-2, or other non-epidemiological or clinical outcomes were excluded.
Study selection
Studies identified in the search were first screened independently on title and abstract by two reviewers. Relevant studies were included for full-text screening, and further selection of articles was performed by two independent reviewers. Any discrepancies were discussed and resolved. When consensus was not reached, a third reviewer was consulted to provide the final judgement.
Critical appraisal
Risk of bias was systematically assessed by two researchers using separate checklists for empirical and model-based studies. Discrepancies between researchers were discussed, and a final verdict was provided by a third reviewer if consensus was not reached. Empirical studies were appraised using a formal scoring method based on the Critical Appraisal Skills Programme and Cochrane’s Effective Practice and Organisation of Care checklists5 6 (online supplemental file 2). Risk of bias in model-based research was evaluated by assessing use of empirical input data for the model, number of scenarios analysed and transparency of model reporting (online supplemental file 3).
Data extraction
Data extraction was performed by one reviewer, and checked by a second reviewer. Descriptive characteristics on type of research, that is, empirical or model-based, sample size, (simulated) time horizon, study population, CTA properties and intervention, comparator, and epidemiological and clinical outcomes studied, were extracted from all included studies.
Specifically for model-based research, model characteristics (ie, type of model and distributions used) and values used for important model parameters were collected. Furthermore, CTA-specific properties were extracted, such as the method of contact tracing used by these apps. Forward tracing CTAs can only detect the ‘offspring’, that is, individuals the index case has infected. Bidirectional tracing CTAs also detect the ‘parents’, that is, the individual that infected the index case. Models were considered to use bidirectional (as opposed to forward) tracing when, after the index case is detected and registered, all contacts within a period of at least the incubation time are identified, such that the parent of the index case could be found.
Another CTA-specific property included the use of 1-step-tracing or sequential tracing. When a CTA-identified individual could only notify their contacts after testing positive themselves, this was considered 1-step-contact tracing. When notified contacts could subsequently also notify their own contacts, creating a cascade, even before that individual has shown symptoms or received a positive test result for SARS-CoV-2, this was considered sequential tracing.
The most important findings regarding effectiveness of CTAs for SARS-CoV-2 on epidemiological and clinical outcomes were extracted, synthesised and reported narratively. These outcomes were pooled quantitatively whenever it was feasible to do so.
Results
Study selection
A total of 2140 potential studies were identified by the search. After selection based on title and abstract, 2059 articles were excluded. Full texts of the 81 remaining studies were assessed, after which 17 articles were included for critical appraisal and data extraction (online supplemental file 4). The 64 excluded studies with their reasons for exclusion are summarised in online supplemental file 5.
Characteristics of included studies
Seventeen primary studies were included, of which two were empirical observational (non-randomised) studies, and 15 were model-based studies (table 1).
Descriptive characteristics of included studies
Six of the 17 studies were published preprints, meaning they had not (yet) gone through the peer-review process at the time of submitting this paper.7–12 Included studies focused predominantly on the general population, although some analysed the effectiveness of CTAs for specific populations such as hospital personnel, or school children.8 9 11 13–16 Especially in model-based studies, results were often presented graphically. Consequently, the effectiveness of CTAs on epidemiological and clinical outcomes was only partly, or not at all, reported in key numerical figures.
The model-based studies typically assessed the effectiveness of CTAs by simulating one or more scenarios based on certain baseline or input values (eg, proportion of asymptomatic infections). Table 2 provides an overview of characteristics and the most important input parameters used in models of the 15 included articles. Nine of the 15 model-based studies evaluated forward tracing CTAs,8 9 11 13–18 four studies analysed bidirectional tracing CTAs7 10 12 19 and one used an alternative method.20 Four studies used a CTA that used sequential tracing.7 10 12 19 All of these also used bidirectional CTAs, which are more effective than forward tracing CTAs in reducing R, but require quarantining many more contact persons. This is especially the case when a significant number of infections come from asymptomatic individuals (ie, transmission from a case who does not (yet) have symptoms), who are unaware they have SARS-CoV-2.19
Properties of model-based studies
The percentage of CTA adoption was varied in almost all studies, allowing for assessment of the impact of CTAs on epidemiological and clinical outcomes. Average incubation time, that is, the mean time between infection and symptom onset of SARS-CoV-2, was estimated to be 5–6 days for SARS-CoV-2.9 11–21 The proportion of asymptomatic SARS-CoV-2 infections, used as input parameter in model-based studies, was estimated at 20%–50% based on empirical data,8 9 16 18 but could vary between 18% and 86%.9 The baseline R value chosen in the model-based studies varied between 1.2 and 4.0.7–10 12 14–21
Furthermore, so-called superspreaders (ie, individuals that infect numerous other individuals, and consequently have a high individual R) were discussed in context of the SARS-CoV-2 pandemic. Tracing these superspreaders is key in containing outbreaks. Hence, it is warranted to use bidirectional CTAs to trace these superspreaders, and advise them to immediately enter quarantine on identification.14 22
Critical appraisal
Risk of bias in the two empirical studies was judged to be high (table 3).23 24 Confounding variables (such as smoking, work status and income) were insufficiently taken into account given the explanatory and observational nature of these empirical studies. It was also unclear how missing (outcome) data were dealt with.
Critical appraisal of empirical studies
Most model-based research was judged to have a low risk of bias (table 4). Three of the 15 studies had a high risk of bias due to the lack of use of empirical distributions for variables, the limited number of scenarios analysed and insufficient transparency regarding reporting of the model.11 20 21
Critical appraisal of model-based studies
Synthesis of results
Evidence from empirical studies
Two empirical comparative observational studies assessed the effectiveness of CTAs compared with a control group that did not use CTAs (table 1).23 24 One study looked at effectiveness of a text warning system used in 627 386 individuals who came in contact with a SARS-CoV-2 exposed population, and compared it to the general population of Taiwan who did not use such a warning system.17 They showed a reduction in incidence of respiratory syndrome from 19.23 to 16.87 per 1000 individuals. They also showed a reduction in pneumonia incidence from 3.81 to 2.36 per 1000 individuals.17 The second observational study investigated the introduction and adoption of a ‘Test and Trace’ app by 34 000 individuals living on the Isle of Wight (UK), and compared the estimated value of R in that region to that in the general UK population.24 The CTA marked individuals as positive based on self-reporting of symptoms. Individuals that came in contact with an individual marked as positive were provided with social distancing advice. The study found that R was reduced from 1.3 to 0.5 after implementation of the CTA. Within 2–3 weeks after implementation, incidence of SARS-CoV-2 diagnoses declined by around 90%.24
Evidence from model-based studies
Effect on R
Effectiveness of a 1-step-contact tracing in reducing R can be approached using the following formula:
Here, Rc is the reproduction number when a CTA is used, R is the reproduction number without the use of a CTA, p is the proportion of the population using the CTA and f is the combination of other factors that affect effectiveness of notification by the CTA. Such factors include, but are not limited to: delay between CTA notification and testing, delay between testing and test result, delay between reception of test result and entry of that result in the CTA, compliance to interventions (eg, self-quarantine), and the proportion of infections that occur presymptomatically or asymptomatically. Note that p occurs as a quadratic term, which reflects the fact that both infector and infectee have to use the CTA for the transmission to get traced.
Nine of the 15 model-based studies assessed the effect of CTAs on reduction of R.8 11 14–16 18 19 21 CTAs were able to control an ongoing outbreak or epidemic through quicker and more efficient feedback of a positive test result, and by notifying close contacts of a positively tested individual.15 16 19 This speed and efficiency were not feasible using traditional manual contact tracing.16 New outbreaks could be controlled (ie, Rc<1.0) by CTAs, by combining them with quarantine or self-isolation interventions, provided that hygiene and social distancing measures are maintained.8 14 18 21 CTAs were able to reduce R by 0.3 more than traditional manual contact tracing, provided that feedback about contact with a positively tested individual is given to all contacts of the index case of the preceding 7 days.19 Another model-based study demonstrated that a CTA with 20% adoption rate reduces R by 17.6% compared with no contact tracing, whereas traditional manual contact tracing reduced R by 2.5% compared with no contact tracing.15 This study also demonstrated that a CTA is able to reduce the R further, even when social distancing has already reduced R to 1.2. In this situation, R can be reduced further by 30%–0.8 when CTA adoption rate is 80%.15 Another model-based study determined that 60% adoption rate of a CTA could result in an R below 1.0.11 In one study, adoption rate of 53% resulted in a 47% reduction in R when the complete household of an individual with a positive test result is advised to be quarantined.14 The last study looking at effect of CTA on R showed that only at 60% adoption rate of the app a significant beneficial effect on R would become apparent.12 When R is high (eg, 3.0), and a considerable proportion of individuals is asymptomatic (eg, 40% of all infections), CTAs need to be combined with other interventions (such as social distancing and random testing) to be able to lower the R below 1.0.12 Potential for CTAs to reduce R is not only dependent on the adoption rate of the app, but also on (effectiveness of) various other measures that are provided after a positive notification, the delay between positive notification and opportunity for testing, and delay between receiving a positive test result and sharing that result through the CTA.5 6 10 One study found that the percentage of preventable infections by one individual strongly depends on the time delay between CTA notification and the ability to be tested.15 When there was no delay (ie, 0 days) 79.9% of infections could be prevented, compared with 41.8% and 4.9% for 3 and 7 days delay, respectively.
Effect on total number of infections
Eight of the 15 model-based studies assessed the effect of CTAs on reducing the total number of infections.8–11 13 17 18 20 Two studies indicated that the success of CTAs in reducing the total number of infections could only be ensured with a high adoption rate of that app.8 13 Another study showed that with a high CTA adoption rate of 75%, there would be no more new infections occurring within 3 months after implementation.11 It was found that adequate hygiene and social distancing measures are needed to enable CTAs to reduce the total number of infections.8 9 17 18 Especially in areas where there is low compliance to social distancing, a sufficiently high adoption rate of a CTA is essential to maintain control of an outbreak.9
The height of the peak number of new infections can, according to one study, be reduced by half with a 50% adoption rate of a CTA,18 whereas another study showed that this could be achieved with an adoption rate as low as 20%.20 Another study demonstrated that at 27% CTA adoption rate, a quarter of all new infections can be prevented.17 However, according to another study that used a similar adoption rate, the number of infections would stabilise, but the epidemic would be maintained by core groups in densely populated areas.18 There may be a period of time of more than 2 months between implementation of interventions (such as CTAs) and the effect of that implementation on the total number of SARS-CoV-2 infections.13
Effect on number of hospitalisations
None of the 15 model-based studies assessed the effect of CTAs on the number of hospitalisations due to SARS-CoV-2 infection, possibly because the number of hospitalisations is expected to be proportional to the number of infections, only with a time-delay. A German study did look into the effect of a CTA on the number of days that intensive care unit (ICU) capacity was exceeded.9 They found in their simulations that—based on the German population, and assuming an ICU capacity of 24 000 beds—a CTA adoption rate of 20% would prevent exceedance of ICU capacity at any point in time. In contrast, if no contact tracing (either manual or digital) would be used, ICU capacity would be exceeded on a quarter of days.
Effect on mortality rate
Three of the 15 model-based studies assessed the effect of CTAs on mortality rate.9 18 20 One study demonstrated that a high adoption rate (80%) of a CTA would result in an 85% reduction in mortality rate, over a period of 500 days.9 Another study found that a low CTA adoption rate (25%) is associated with a 10% decrease in mortality rate, an average adoption rate (50%) with 25% decrease, and a high adoption rate (75%) with 40%–60% decrease.18 A third study showed that at 40% adoption rate, during the peak of an outbreak, a reduction in number of deaths by 97% could be achieved.20
Discussion
Empirical evidence regarding the effectiveness of using CTAs for detection of SARS-CoV-2 is still limited. Currently, no randomised studies have been performed, and only two observational comparative studies were identified in this systematic review. Although some benefits of using CTAs for detection of SARS-CoV-2 were observed, both studies were deemed to be of low methodological quality. However, the results of these studies were in accordance with the 15 included, higher quality, model-based studies assessing effectiveness of CTAs. These studies showed that CTAs can be effective and a valuable addition to manual contact tracing. CTA use resulted in a lower R, lower total number of infections, and lower mortality rate. These reductions were already observed at relatively low adoption rates (eg, 20%), though higher adoption rates of CTAs resulted in greater reductions. Shortening delays between CTA notification and diagnostic testing may increase its effectiveness.
Strengths and limitations
This rapid systematic review assesses key features, quality, and main clinical and epidemiological outcomes of a set of studies, both empirical and model-based, on effectiveness of CTAs for SARS-CoV-2. To our knowledge, no such systematic review has been published, assessing these specific properties. Methodological quality of empirical studies was assessed using standardised tools. No such tool was available in literature for model-based studies, and as such a set of key features used in other systematic reviews on this topic was used. This set was validated by experts in mathematical modelling.
To fully appreciate the findings from this systematic review, some considerations should be taken into account. First, the set of studies identified in the literature search may not be comprehensive. Studies on SARS-CoV-2 are published at a rapid, almost daily, basis in various online repositories. Although we cannot ensure that all studies on the effectiveness of CTAs for SARS-CoV-2 have been identified, we believe that the set of included studies that we have identified is a representative sample.
Furthermore, effectiveness of CTAs for SARS-CoV-2 described in model-based studies is complex. Numerous input variables used in the models interact with one another, and consequently affect effectiveness of, for example, adoption rate of CTAs on clinical or epidemiological outcomes. Summarising these findings into a general effectiveness is difficult, and will always suffer from simplification of a system of complex interactions. Though we feel that providing some (conditional) findings from these studies will help provide some general insight in the impact CTAs can have on clinical and epidemiological outcomes for SARS-CoV-2.
Conclusion and implications for further research
Current evidence on the effectiveness of CTAs for SARS-CoV-2 is predominantly based on modelling studies, which indicate that there is potential in beneficially affecting key clinical and epidemiological outcomes. High-quality empirical evidence, either from experimental or methodologically sound observational studies, is needed in order to be able to draw more robust conclusions regarding effectiveness of CTAs for SARS-CoV-2.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information. All data generated in this study were published, either as part of the manuscript, or its supplemental file.
Ethics statements
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors Conception and design: KJ, KGMM, LH. Planning: KJ, MCJB, JAAGD, MSO, RWMV, RS, KGMM, MEEK, LH. Acquisition of data: RS. Conduct: KJ, MCJB, JAAGD, MSO, RWMV, RS, KGMM, MEEK, LH. Analysis: KJ, MCJB, KGMM, MEEK, LH. Interpretation of data: KJ, MCJB, KGMM, MEEK, LH. Reporting: KJ, MCJB, JAAGD, MSO, RWMV, RS, KGMM, MEEK, LH.
Funding This research was funded directly (no grant number) by the Dutch Ministry of Health, Welfare and Sport.
Disclaimer Researchers involved in this study contributed to the study’s conception and execution independently from the funding agency, and as such, it was in no way influenced by the funding agency.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.