Keywords
SARS-CoV-2, infection, outcomes, outcome measurement, COVID-19, prevention, trials
This article is included in the Emerging Diseases and Outbreaks gateway.
This article is included in the Coronavirus collection.
SARS-CoV-2, infection, outcomes, outcome measurement, COVID-19, prevention, trials
Coronavirus disease 2019 (COVID-19), resulting from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, has significant morbidity and mortality. Addressing human-to-human transmission of SARS-CoV-2 is a global research priority.1 Research to fully understand the modes of transmission of SARS-CoV-2 is ongoing. Studies to prevent the spread of SARS-CoV-2 are increasing, with more than 1000 studies registered between December 2020 and February 2021.2 Interventions are wide ranging, including public health measures such as lockdown of countries/cities, closure of schools and public places, social distancing measures and handwashing, interventions in primary and secondary care practice such as the use of personal protective equipment (PPE) and increased time for appointments/interventions, alongside the development of effective vaccines and drugs to prevent disease.
Evidence is accumulating to evaluate the effectiveness of prevention interventions, but limitations in evidence synthesis have been identified because of inconsistent selection, measurement and reporting of outcomes in the studies.3–5 Whilst studies primarily focus on mitigating the risk of contracting COVID-19, they may measure other outcomes relevant to the population of interest and type of intervention, for example, educational outcomes in studies of school closures6 and tolerance/acceptability in studies of PPE use in secondary care.5 However, unnecessary heterogeneity in the outcomes measured and reported in COVID-19 prevention trials leads to research waste and inefficiency. For example, a recent commentary highlighted the rarity of randomised trials of public health interventions for COVID-19, which is likely due to the need for unattainably large sample sizes.7 Fretheim argues that underpowered trials in public health should be regarded as contributions to the larger body of evidence on COVID-19. However, for underpowered trials to be collectively assessed and synthesised in systematic reviews they need to be measuring the same outcomes.
A solution to this issue is to develop a core outcome set (COS), which is defined as an agreed standardised set of outcomes that should be measured and reported, as a minimum, in all clinical research studies in a specific area of health care to address inconsistencies in outcome measurement.8 The COS-COVID-P study, in collaboration with an international and multidisciplinary committee of experts and public contributors, has developed a COS for studies of interventions to prevent SARS-CoV-2 infection. During the COS development process, online workshops were held to consider the evidence on COVID-19 prevention and the issue of outcome heterogeneity. More than 70 key international stakeholders, including clinicians, researchers, public representatives, methodologists, systematic reviewers, guideline developers, and regulatory agency representatives shared their expert opinions. Agreement on the final COS was reached in August 2020. Two outcome domains were identified as being essential to all COVID-19 prevention studies: infection and intervention-specific harms.9 How to measure these outcomes now requires consideration and led to this paper.
SARS-CoV-2 infection can be measured through laboratory testing, symptom assessment or a combination of both.2 However, there is no single standalone diagnostic test nor agreement on diagnostic symptoms.10,11 Thus, there is a pressing need to consider how to assess SARS-CoV-2 infection uniformly across studies to optimise data reporting and synthesis. The aim of this study is to review SARS-CoV-2 infection outcomes in registered COVID-19 prevention trials and examine the reported rationales for choice of outcome measurement. Our intention is that this will inform optimal methods for the design of future studies.
For the purposes of the review presented here, we searched the Cochrane COVID-19 Study Register https://covid-19.cochrane.org/ on 24 November 2020. The Cochrane COVID-19 Study Register is a freely available, annotated reference collection of primary research studies on COVID-19, including interventional, observational, diagnostic, prognostic, epidemiological and qualitative designs. The register is continually updated to support rapid evidence synthesis, and includes studies from multiple data sources including Clinicaltrials.gov, the Cochrane Central Register of Controlled Trials (CENTRAL), Embase, medRxiv, PubMed and the WHO International Clinical Trials Registry Platform (ICTRP). We restricted our Cochrane COVID-19 Study Register search results by study type (“interventional”), study design (“parallel/crossover”), intervention assignment (“randomised”) and study aim (“prevention”). The Cochrane COVID-19 Study Register defines prevention studies as those aiming to assess one or more interventions for preventing the development of a specific disease or health condition.
We included phase three and four trials of prevention interventions for COVID-19. We excluded phase one and two trials, including those that used other terms to imply that they were early phase (e.g. “explorative”), because these trials focus on safety, rather than the effectiveness of an intervention. Studies relating to the transmission, treatment or management of COVID-19 were also excluded. Studies identified from the Cochrane COVID-19 Study Register search were split equally between five experienced systematic reviewers (KA, SD, SG, NH, RM), who assessed the eligibility of their cohort of studies according to these eligibility criteria.
Data from eligible trials were extracted using a piloted, standardised data extraction spreadsheet, with the work split equally between five experienced systematic reviewers (KA, SD, SG, NH, RM). In cases of any ambiguity or doubt, queries were assessed by SD. Any queries were discussed with the whole team who collectively reached agreement on how to progress. The following data were extracted for each study: intervention type (e.g. drug, vaccine, behavioural), intervention (as described verbatim), setting (i.e. population of interest), SARS-CoV-2 infection outcomes (as described verbatim), SARS-CoV-2 infection outcome definition (i.e. how SARS-CoV-2 infection would be measured), whether or not the SARS-CoV-2 infection outcome was specified as being a primary or secondary outcome, and the rationale provided for SARS-CoV-2 infection outcome measurement selection. In addition, the reviewers recorded whether the study record related to a registry entry, trial protocol or the main report. If the registry entry record provided a link to a study protocol, this protocol was also reviewed, and the relevant data extracted.
Extracted data were summarised according to the number of SARS-CoV-2 infection outcomes per trial. For trials that included one SARS-CoV-2 infection outcome, data were summarised according to the single outcome definition specified in each trial (i.e. testing, symptoms, testing or symptoms, testing and symptoms). For trials that included multiple SARS-CoV-2 infection outcomes, data were summarised according to each outcome definition combination. For example, some trials may have measured one SARS-CoV-2 infection outcome using testing alone, a second SARS-CoV-2 infection outcome by symptoms alone and a third SARS-CoV-2 infection outcome by testing and symptoms. In addition to analysing the data by the overarching outcome definition (e.g. testing), we also summarised the data according to the specific underlying definition. For example, a trial would be summarised as having two versions of testing if they specified that one SARS-CoV-2 infection outcome was determined by reverse-transcriptase polymerase chain reaction (RT-PCR), and another SARS-CoV-2 infection outcome was determined by serology (antibodies, seroconversion, seroprevalence). The categorisation of outcome definitions was based on the verbatim text available in the registry records. Analysis was carried out using Stata version 16.1.
Our searching of the Cochrane COVID-19 Study Register returned 430 records that were screened for eligibility, of which 231 were excluded. A total of 199 unique prevention trials were identified (Figure 1).
Of the 199 eligible trials, 176 (88%) had a trial registry record only. The remaining 23 trials had registry records plus a protocol (n = 15, 8%) or journal article (n = 6, 3%) or both a protocol and journal article (n = 2, 1%). Most trials examined drug prophylaxis (n = 92, 46%) or vaccine (n = 54, 27%) interventions. Other interventions included dietary supplements (n=20, 10%), behavioural/social distancing (n = 12, 6%) and PPE or other physical barriers (n = 7, 4%). Fourteen (7%) trials evaluated other interventions that included Ayurveda, homeopathy, traditional Chinese medicine, a medicinal plant, Unani, telehealth and financial support.
The trials were undertaken across eight combinations of setting and population of interest, with almost half (n = 98, 49%) in healthcare workers. Other settings included adults in the community (n = 45, 23%), high risk groups (e.g. elderly, hospitalised, or specific disease; n = 21, 11%), people in close contact with COVID-19 cases (e.g. family members; n = 16, 8%), nursing home residents and/or staff (n = 10, 5%), people with potential exposure to COVID-19 (n = 5, 3%), police (n = 3, 2%) and schools (n = 1, <1%). The characteristics of included trials are described in Table 1.
Number (%) | ||
---|---|---|
Publication sources | ||
Trial registry record only | 176 (88.4) | |
Trial registry record + protocol | 15 (7.5) | |
Trial registry record + journal article | 6 (3) | |
Trial registry record + protocol + journal article | 2 (1) | |
Intervention types | ||
Drug prophylaxis | 92 (46.2) | |
Vaccine | 54 (27.1) | |
Dietary supplement | 20 (10.1) | |
Behavioural/social distancing | 12 (6.0) | |
Personal protective equipment or other physical barrier | 7 (3.5) | |
Other | 14 (7.0) | |
Ayurveda | 5 (2.5) | |
Homeopathy | 4 (2.0) | |
Chinese medicine | 1 (0.5) | |
Medicinal plant | 1 (0.5) | |
Unani | 1 (0.5) | |
Telehealth | 1 (0.5) | |
Financial support | 1 (0.5) | |
Setting | ||
Health care workers1 | 98 (49.3) | |
Community/adults | 45 (22.6) | |
High risk groups (e.g. elderly, hospitalised, or specific disease) | 21 (10.6) | |
In close contact with COVID case (e.g. family members) | 16 (8) | |
Nursing home2 | 10 (5) | |
Potentially exposed to SARS-CoV-2 | 5 (2.5) | |
Police | 3 (1.5) | |
Schools | 1 (0.5) | |
SARS-CoV-2 infection outcomes measured/reported in the trial | ||
No SARS-CoV-2 infection outcomes | 15 (7.5) | |
Single SARS-CoV-2 infection outcome3 | 118 (59.3) | |
Two SARS-CoV-2 infection outcomes | 50 (25.1) | |
Three SARS-CoV-2 infection outcomes | 14 (7.0) | |
Four SARS-CoV-2 infection outcomes | 2 (1.0) |
Of the 199 trials, 15 (8%) did not include any SARS-CoV-2 infection outcomes. These 15 studies predominantly evaluated behavioural (n = 8, where interventions were ‘STOP touching your face' mindfulness online training programme; Brief informational infographic; E-E video about COVID hygiene; Mental Imagery; Digital health literacy intervention; e-learning module; Telepsychoeducation with personalized videos; combined video display and live demonstration as training methods to healthcare providers for donning and doffing PPE) or PPE/physical barrier (n = 3) interventions, with the remaining four trials evaluating social distancing, dietary supplements, a medicinal plant and homeopathy. Examples of outcomes recorded in trials that did not include a SARS-CoV-2 infection outcome included frequency of face-touching behaviour in a behavioural intervention trial12 and contamination of any part of the base clothing or exposed skin of the upper body in a PPE trial.13
A total of 268 SARS-CoV-2 infection outcomes were identified across 184 (92%) trials, with infection measured as a primary outcome in 149 (81%) studies. Table 2 provides the outcome definitions recorded. We categorised outcome definitions using the verbatim text available in the registry records. In 118 (64%) trials, a single infection outcome was included. Fifty (27%) trials included two infection outcomes, 14 (8%) included three infection outcomes and two (1%) included four infection outcomes. Examples of multiple infection outcomes within the same trial are provided in Table 2. Table 3 lists the specific outcome definitions across the 184 trials.
Number of trials (% of 184) | |
---|---|
One SARS-CoV-2 infection outcome per trial1 | 118 (64.1) |
Testing alone | 49 (26.6) |
Non-specific term for infection | 32 (17.4) |
Symptoms alone | 16 (8.7) |
Positive on both testing AND symptoms | 15 (8.2) |
Positive on either testing OR symptoms | 6 (3.3) |
Two SARS-CoV-2 infection outcomes per trial2 | 50 (27.2) |
Testing alone (2 versions) | 7 (3.8) |
Testing alone; Non-specific term for infection | 3 (1.6) |
Testing alone; Symptoms alone | 9 (4.9) |
Testing alone; Testing AND symptoms | 12 (6.5) |
Testing alone; Testing OR symptoms | 3 (1.6) |
Symptoms alone; Non-specific term for infection | 7 (3.8) |
Testing AND symptoms; Non-specific term for infection | 2 (1.1) |
Symptoms alone; Testing AND symptoms | 3 (1.6) |
Symptoms alone; Testing OR symptoms | 2 (1.1) |
Testing AND symptoms (2 versions) | 2 (1.1) |
Three SARS-CoV-2 infection outcomes per trial3 | 14 (7.6) |
Testing alone (3 versions) | 1 (0.5) |
Testing alone (2 versions); Symptoms alone | 2 (1.1) |
Testing alone (2 versions); Testing AND symptoms | 3 (1.6) |
Testing alone; Symptoms alone; Testing AND symptoms | 2 (1.1) |
Testing alone; Symptoms alone; Non-specific term for infection | 1 (0.5) |
Testing alone; Testing AND symptoms; Testing OR symptoms | 1 (0.5) |
Testing alone; Testing AND symptoms (2 versions) | 2 (1.1) |
Symptoms alone; Testing AND symptoms (2 versions) | 2 (1.1) |
Four SARS-CoV-2 infection outcomes per trial | 2 (1.1) |
Testing alone; Symptoms alone; Testing AND symptoms (2 versions) | 1 (0.5) |
Testing AND symptoms (3 versions); Testing OR symptoms | 1 (0.5) |
1 Note that when the same COVID outcome definition was measured at multiple time points or analysed in multiple ways (e.g. as a binary outcome AND a time to event outcome), these repetitions were considered as a single COVID outcome for the purpose of this review.
2 Example of two infection outcomes: ‘Number of people turning symptomatic’ and ‘Number of people turning Covid19 positive’ (see: http://www.ctri.nic.in/Clinicaltrials/pmaindet2.php?trialid=44065).
3 Example of three infection outcomes: ‘positive serology or RT-PCR for COVID-19 up until day 28’, ‘positive serology at day 28’ and ‘symptoms of COVID-19’ (see: https://clinicaltrials.gov/ct2/show/NCT04342156).
4 Example of four infection outcomes: ‘Number of symptomatic infections confirmed by SARS-CoV-2 (COVID-19)’, ‘Number of asymptomatic infections confirmed by SARS-CoV-2 (COVID-19)’, ‘Severity of SARS-CoV-2 (COVID-19) infection’ and ‘Duration of symptoms of COVID-19 coronavirus infection’ (see: https://www.clinicaltrialsregister.eu/ctr-search/trial/2020-001530-35/ES).
Forty-five (25%) trials defined infection in non-specific terms (e.g. “COVID”, “SARS-CoV-2 infection”, “infected”, “confirmed”), without specifying how infection would be/was determined (i.e. via a specific test to be undertaken or symptoms to be recorded). In 32 of these trials (17% of all included trials), the non-specific infection term was the only infection outcome listed. Figure 2 displays the number of trials with non-specific COVID outcomes by month of registration. Of the 45 trials with non-specific COVID outcomes, 23 (51%) trials were registered within the first three months (March to May 2020) of the COVID-19 outbreak being declared a global pandemic.
Testing was the only means employed for determining infection in 57 (31%) trials, whereas relying on symptoms alone to define infection occurred in 16 (9%). All other trials included multiple infection outcomes, defined in different ways (Table 2). Where one of the infection outcomes was to be determined by testing alone (n = 96), this was most commonly by RT-PCR (n = 36, 20%) and serology (n = 22, 12%) tests. Where infection was defined on the basis of symptoms alone (e.g. symptomatic COVID-19) (n = 45), the number of symptoms and definitions varied (Tables 4 and 5). Some trials included outcomes defined by combinations of testing and/or symptoms: 13 (7%) using either testing or symptoms to determine infection; 46 (25%) specifying both testing and symptoms must be positive to define infection (e.g. RT-PCR and symptomatic).
“Symptomatic” definitions |
https://clinicaltrials.gov/ct2/show/NCT04318015 “Symptomatic infection rate by COVID-19 defined as cough, dyspnea, fever, myalgia, arthralgias or rhinorrhea along with a positive COVID-19 real-time polymerase chain reaction test” |
https://pubmed.ncbi.nlm.nih.gov/33068425 “The definition of Covid-19-compatible symptoms was based on guidance from the US Council for State and Territorial Epidemiologists (Appendix)13. Specifically, probable disease was defined as having cough, shortness of breath, or difficulty breathing, OR two or more of the following symptoms: fevers, chills, rigors, myalgia, headache, sore throat, new olfactory and taste disorders. Possible disease was defined as one or more COVID-19-compatible symptoms. Three blinded infectious diseases physicians independently adjudicated cases of symptomatic participants based on the above criteria.” |
http://www.ctri.nic.in/Clinicaltrials/pmaindet2.php?trialid=43105 “defined as self-reported fever or cough, or shortness of breath, or respiratory distress, or runny or blocked nose plus a positive PCR test and/or antibody test” |
https://clinicaltrials.gov/ct2/show/NCT04321174 “fever, cough or other respiratory/systemic symptoms (including but not limited to fatigue, myalgias, arthralgias, shortness of breath, sore throat, headache, chills, coryza, nausea, vomiting, diarrhea) by day 14 in a patient with laboratory confirmed infection, combined with microbiologic confirmation of COVID-19 infection in the participant” |
https://clinicaltrials.gov/show/NCT04377646 “Any COVID-19 related symptoms (cough, fever, headache, vomiting, nausea, dyspnea, diarrhea, smell disorder, conjunctivitis, dizziness)” |
https://clinicaltrials.gov/show/NCT04360122 “any symptoms suggestive of COVID19: fever, plus at least one sign or symptom of respiratory disease including cough, shortness of breath, respiratory distress/failure, runny/blocked nose, plus Laboratory: Sampling of the following will be withdrawn from all participants at beginning and end of the study: COVID 19 IgM (for recent infection) or IgG or (old infection)” |
https://clinicaltrials.gov/show/NCT04445428 “Episodes with self-reported infectious disease morbidity suspected to be caused by COVID (three or more of the following: fever, cough, sore thought, extreme fatigue, loss of smell/taste).” |
https://clinicaltrials.gov/show/NCT04452643 “Incidence of subjects with COVID-19, defined by the presence of: Fever, Any of the respiratory signs and/or symptoms: cough, dyspnea, respiratory failure, runny nose/nasal obstruction, Positive test for SARS-COV-2 (PCR or serology)” |
https://clinicaltrials.gov/show/NCT04461379 “disease defined as positive SARS-Cov-2 test (serology), plus fever (using self-reported questionnaire), or at least one sign or symptom of respiratory disease including cough, shortness of breath, respiratory distress/failure (using self-reported questionnaire)” |
https://clinicaltrials.gov/show/NCT04470427 “The participant must have experienced at least TWO of the following systemic symptoms: Fever (≥ 38°C), chills, myalgia, headache, sore throat, new olfactory and taste disorder(s)” |
https://clinicaltrials.gov/show/NCT04471766 “Self-reported main symptoms of COVID-19 (three or more - fever, cough, fatigue, shortness of breath, loss of smell/taste)” |
https://www.clinicaltrialsregister.eu/ctr-search/trial/2020-002287-31/ES: “Examine the incidence of typical symptoms of SARS-CoV-2 infection: fever, cough, headache, arthromyalgia, pharyngeal pain, dyspnea, diarrhea, vomiting, abdominal pain, anosmia” |
https://clinicaltrials.gov/show/NCT04437693 “COVID-19 related symptoms, that may include: Temperature over 37.8 degrees Celsius, Shortness of Breath, Cough” |
https://clinicaltrials.gov/show/NCT04614948 “Mild COVID-19 includes: Fever, sore throat, malaise, headache, muscle pain, gastrointestinal symptoms, cough, chest congestion, runny nose, wheezing, skin rash, eye irritation or discharge, or chills, without shortness of breath or dyspnea.” |
https://clinicaltrials.gov/show/NCT04614948 “COVID-19 symptoms consistent with those defined by the US FDA harmonized case definition at the time of finalization of this protocol: fever or chills, cough, shortness of breath or difficulty breathing, fatigue, muscle or body aches, headache, new loss of taste or smell, sore throat, congestion or runny nose, nausea or vomiting, diarrhea.” |
https://clinicaltrials.gov/show/NCT04350931 “COVID 19 infection in the form of: fever, dry cough, fatigue, & dyspnea” |
https://pactr.samrc.ac.za/TrialDisplay.aspx?TrialID=12175 “defined as I. fever (using a self-reported questionnaire), plus II. at least one sign or symptom of respiratory disease including cough, runny/blocked nose (using self-reported questionnaire)” |
https://clinicaltrials.gov/show/NCT04534803 “fever (as documented in EHR) or at least one sign or symptom of respiratory disease including cough, shortness of breath, respiratory distress/failure (as documented in EHR)” |
https://clinicaltrials.gov/show/NCT04349228 “The rate of COVID19 infections is defined by the occurrence of the clinical signs below: Cough, Dyspnea, Fever, Myalgia, Arthralgia, Rhinorrhea, Anosmia, Asthenia, Fatigability” |
https://clinicaltrials.gov/show/NCT04452643 “WHO definition” |
https://clinicaltrials.gov/show/NCT04526821 “Severity of SARS-CoV-2 infection (asymptomatic, mild, moderate, severe infection), assessed by the World Health Organization definitions” |
Tables 6 and 7 detail the use of various SARS-CoV-2 infection outcome definitions by setting and intervention type, respectively.
Of the 184 trials that included SARS-CoV-2 infection outcomes, seven provided a rationale for their choice of outcome selection and/or measurement (see Table 8). Of these seven studies, this was often to justify their specific choice of testing, with one drug prophylaxis trial explaining that “testing and criteria for diagnosis COVID-19 is likely to rapidly evolve over the course of this trial and will vary internationally”14 and a school closure trial basing outcome measures on routinely collected testing data to ensure “feasibility, facilitate speed, and limit costs”.15 A drug prophylaxis trial in healthcare workers argued that relying on symptoms consistent with infection without laboratory confirmation raises “concerns of type II error from asymptomatic participants”.16 Conversely, another drug prophylaxis trial justified the decision to use symptoms given issues of access to testing, “Given limited availability of outpatient PCR testing in many jurisdictions during our study period, particularly in April 2020, probable Covid 19 based on Covid-19-compatible symptoms was included in the composite primary endpoint”.17
Rationale |
https://clinicaltrials.gov/show/NCT04346329 “For this, once the informed consent form is signed, the molecular test for the diagnosis of SARS-CoV-2 infection by RT-PCR will be carried out every 4 days in order to determine as closely as possible the moment the participant become positive.” |
https://clinicaltrials.gov/ct2/show/NCT04519125 “To determine the occurrence of infection with the virus the study will use both molecular tests that detect the presence of viral genes in respiratory secretions, and serological tests that detect the response of the immune system to the virus.” |
https://www.clinicaltrialsregister.eu/ctr-search/search?query=eudract_number:2020-001448-24 “Testing and criteria for diagnosis COVID-19 is likely to rapidly evolve over the course of this trial and will vary internationally. Therefore, the following pragmatic definitions will be applied:
|
https://pubmed.ncbi.nlm.nih.gov/33001138 “A recent randomized trial for postexposure COVID-19 prophylaxis with a 5-day course of hydroxychloroquine did not demonstrate clinical benefit. However, the composite primary outcome measure for this study included symptoms consistent with infection without laboratory confirmation; most patients did not have assessment of SARS-CoV-2 infection by reverse-transcriptase polymerase chain reaction (RT-PCR), raising concerns of type II error from asymptomatic participants. We sought to test the hypothesis that administering daily hydroxychloroquine would prevent SARS-CoV-2 infection in hospital-based HCWs over 8 weeks of exposure via RT-PCR testing of nasopharyngeal (NP) swabs and serologic antibody testing from participants at baseline, 4 weeks, and 8 weeks of treatment.” |
https://clinicaltrials.gov/show/NCT04328441 “The aim of our trial is to ensure the continuity of health care, and this is the reason for this pragmatic primary endpoint. Moreover, BCG vaccine is expected to reduce all-cause infections and not only COVID-19. Moreover, this was a non-funded study, precluding intensive personal follow-up, and the measurement of absenteeism as this was considered relevant, and immediately obvious.... One of the secondary endpoints in the HCW trial is documented COVID-19 infection. It is important to keep in mind that health workers in the Netherlands are structurally examined for the presence of COVID-19 in case of any respiratory symptoms or in case of fever. Therefore, we expect a high detection rate.” |
https://pubmed.ncbi.nlm.nih.gov/33068425 “Given limited availability of outpatient PCR testing in many jurisdictions during our study period, particularly in April 2020, probable Covid 19 based on Covid-19-compatible symptoms was included in the composite primary endpoint.” |
https://pubmed.ncbi.nlm.nih.gov/33068425 “To ensure feasibility, facilitate speed, and limit costs, we aim to base our outcome measures on routinely collected data, to the extent possible. Testing for COVID-19 takes place continuously and consistently across the country. All suspect cases have access to testing and laboratories use valid tests meeting quality assurance requirements.” |
We have reviewed COVID-19 prevention trials that were registered in the first nine months of the pandemic and examined in detail their rationale, selection and method of measuring SARS-CoV-2 infection. Of the 199 included trials, 268 SARS-CoV-2 infection outcomes were identified, across 184 studies. Of these, almost one fifth did not define how infection would be measured in the register entry, with the remaining studies specifying a combination of testing alone, symptoms alone, or testing and/or symptoms. Very few trials provided a rationale for the choice of how infection would be measured. The observed heterogeneity of outcome measurement of SARS-CoV-2 infection in COVID-19 prevention studies limits effective evidence synthesis and scientific comparison of interventions, which has enormous consequences for decision-making at an individual and public health level. We recommend that studies provide a rationale for choice of infection measurement and, if possible, that uniformity is agreed internationally on how to measure SARS-CoV-2 infection in trials of prevention interventions.
The heterogeneity of outcome measurement identified in the current study is not uncommon. A recent methodological systematic review revealed significant heterogeneity in the outcome measurement instruments used in trials evaluating therapeutic interventions for COVID-19, thus highlighting the need for greater consistency.18 However, such heterogeneity is not restricted to COVID-19 research studies. A multiplicity of outcome measurement instruments has recently been reported in areas such as delirium, hearing loss, chronic pelvic pain, and breast reconstruction, with studies concluding the need for consensus on outcomes and their associated measures for use in future clinical trials in these disease areas.19–22
To our knowledge, this is the first study to examine how SARS-CoV-2 infection outcomes are measured in registered COVID-19 prevention trials. However, there are some methodological limitations. Only the Cochrane COVID-19 Study Register https://covid-19.cochrane.org/ was searched to identify COVID-19 prevention trials, which may limit the number of relevant studies identified. The Cochrane register, however, is updated regularly and includes studies identified from six data sources, including ClinicalTrials.gov. Furthermore, although we did not specifically search each of the data sources, we used the trial URL provided in the Cochrane Study Register entry to access the original trial registry entry. Another limitation is the lack of detail about infection outcome definitions reported in the trial registry entries. Where readily available via links within the registry, protocols and published papers were also examined. However, due to resource limitations we did not manually search for protocols and full text papers corresponding to the trials included in this review, meaning some detail on outcome definitions may have been missed and may be readily available in trial protocols thus potentially changing the results of this study. Trialists may have a rationale for their choice of outcome measurements, but this was not included in the registry entry.
The COS-COVID-P study recommended SARS-CoV-2 infection be measured in all COVID-19 prevention trials, regardless of the intervention and setting.9 We found 92% of trials included SARS-CoV-2 infection as an outcome; however, very few reported the rationale for SARS-CoV-2 infection outcome selection and measurement. From the limited details we have regarding rationale, decisions often appear to be based on accessibility of testing and feasibility. This is understandable and will likely be due to circumstances specific to the setting and country where the trial is being conducted. Going forward, we advise trialists to be transparent about their rationale for the definition of SARS-CoV-2 infection outcomes used when registering and reporting their trials, to improve understanding of the context of their research. The trial registration should include both the rationale for outcome measurement, along with details on how infection will be analysed. In addition, we understand trialists will collect as much data as they consider relevant, but we recommend they develop a coherent statistical analysis plan (SAP) to ensure transparency and reproducibility.23
In the current review, we found two thirds of the studies evaluating behaviour interventions, such as those intended to increase handwashing or SMS text prompts to wear masks, did not include any SARS-CoV-2 infection outcomes. Trials of simple behavioural interventions may be unlikely to measure infection, as their purpose is to modify an individual’s behaviour rather than to directly prevent an individual from acquiring COVID-19. This is in contrast with those of complex interventions that include a behavioural component (e.g. donning standardised PPE versus specialised PPE; or hand washing with soap versus using alcohol wipes) where the overarching aim is still likely to be prevention of SARS-CoV-2 infection.
Alongside SARS-CoV-2 infection, the COS-COVID-P study also led to the inclusion of intervention-specific harms in the COS for all COVID-19 prevention research studies.9 The current review did not examine this aspect. Determining these harm outcomes requires agreement amongst those with expertise in the specific interventions being tested. It may be that the harms are similar within certain intervention types, e.g. drug prophylaxis and vaccines. However, expert opinion will be necessary to determine the relevance of such an approach, but consistency will be important to avoid research waste and to facilitate evidence synthesis in which the results of studies of similar interventions are compared, contrasted and, where appropriate, combined. There may also be a need for COS for studies in specific settings. For example, a COS evaluating interventions for the prevention of COVID-19 transmission in care homes (COS-COVID-PCARE) is currently under development. Building on the COS-COVID-P study, the COS-COVID-PCARE study is proposing to develop a supplementary specific module for COVID-19 prevention in care homes.24
We have described the variation in the selection and measurement of SARS-CoV-2 infection outcomes, with illustrated examples from on-going trials across various settings and interventions. As knowledge about COVID-19 increases over time, this is leading to improvements in the design of studies and reporting. Nevertheless, our findings highlight the need for clarity, transparency and standardisation of measures of SARS-CoV-2 infection. It is now timely to highlight these issues, to improve the collection of infection data to inform key public health decisions.
University of Liverpool Data Catalogue: SARS-CoV-2 infection outcomes. https://doi.org/10.17638/datacat.liverpool.ac.uk/1279.24
This project contains the following underlying data:
- The dataset consists of COVID-19 prevention trials, with the following data extracted for each trial: intervention type (e.g. drug, vaccine, behavioural), intervention (as described verbatim), setting (i.e. population of interest), SARS-CoV-2 infection outcomes (as described verbatim), SARS-CoV-2 infection outcome definition (i.e. how SARS-CoV-2 infection would be measured), whether or not the SARS-CoV-2 infection outcome was specified as being a primary or secondary outcome, and the rationale provided for SARS-CoV-2 infection outcome measurement selection.
Data are available under the terms of the Creative Commons: Attribution 3.0 license.
In addition to article authors, members of the COS-COVID-P Study Steering Committee include: Elie Akl (American University of Beirut, Lebanon); Aneel Bhangu (University of Birmingham, UK); Deborah Caldwell (University of Bristol, UK); Mike Clarke (Queen's University Belfast, UK); Jonathan Craig (Flinders University, Australia); Liang Du (Cochrane China); Adam Finn (University of Bristol, UK); Ella Flemyng (Cochrane, UK); Atle Fretheim (Norwegian Institute of Public Health, Norway); Claire Goodman (University of Hertfordshire, UK); Lars Hemkens (University of Basel, Switzerland); Tammy Hoffman (Bond University, Australia); John Ioannidis (Stanford University, USA); Martin Knapp (London School of Economics, UK); David Lalloo (Liverpool School of Tropical Medicine, UK); Susan Michie (UCL, UK); Richard Morley (Cochrane, UK); Malcolm G Semple (University of Liverpool, UK); Maureen Smith (Cochrane Consumer Network, Canada); Allison Tong (University of Sydney, Australia); Sean Tunis (Center for Evaluation of Value and Risk in Health, USA); Junhua Zhang (Tianjin University of Traditional Chinese Medicine, China).
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Core outcome set development
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |
---|---|
1 | |
Version 1 28 May 21 |
read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)