Skip to content
Publicly Available Published by De Gruyter June 30, 2021

Using body temperature and variables commonly available in the EHR to predict acute infection: a proof-of-concept study showing improved pretest probability estimates for acute COVID-19 infection among discharged emergency department patients

  • Carl T. Berdahl ORCID logo EMAIL logo , An T. Nguyen , Marcio A. Diniz , Andrew J. Henreid ORCID logo , Teryl K. Nuckols , Christopher P. Libby and Joshua M. Pevnick
From the journal Diagnosis

Abstract

Objectives

Obtaining body temperature is a quick and easy method to screen for acute infection such as COVID-19. Currently, the predictive value of body temperature for acute infection is inhibited by failure to account for other readily available variables that affect temperature values. In this proof-of-concept study, we sought to improve COVID-19 pretest probability estimation by incorporating covariates known to be associated with body temperature, including patient age, sex, comorbidities, month, and time of day.

Methods

For patients discharged from an academic hospital emergency department after testing for COVID-19 in March and April of 2020, we abstracted clinical data. We reviewed physician documentation to retrospectively generate estimates of pretest probability for COVID-19. Using patients’ COVID-19 PCR test results as a gold standard, we compared AUCs of logistic regression models predicting COVID-19 positivity that used: (1) body temperature alone; (2) body temperature and pretest probability; (3) body temperature, pretest probability, and body temperature-relevant covariates. Calibration plots and bootstrap validation were used to assess predictive performance for model #3.

Results

Data from 117 patients were included. The models’ AUCs were: (1) 0.69 (2) 0.72, and (3) 0.76, respectively. The absolute difference in AUC was 0.029 (95% CI −0.057 to 0.114, p=0.25) between model 2 and 1 and 0.038 (95% CI −0.021 to 0.097, p=0.10) between model 3 and 2.

Conclusions

By incorporating covariates known to affect body temperature, we demonstrated improved pretest probability estimates of acute COVID-19 infection. Future work should be undertaken to further develop and validate our model in a larger, multi-institutional sample.

Introduction

Improving the detection of acute COVID-19 infection is critical to minimizing spread of the infection and limiting avoidable morbidity and mortality [1]. However, the ability to diagnose COVID-19 is often limited because of asymptomatic or mildly symptomatic infection [2]; limited testing availability [3]; and slow turnaround time for test results. Given these limitations, improving pretest probability estimation is a crucial way to improve decision-making about when testing is warranted and thus curb community spread of acute infection [4].

Measuring body temperature is a quick, easy, and almost ubiquitously available way to evaluate for acute infection. The Centers for Disease Control and Prevention (CDC) has reported that 96% of patients with symptomatic COVID-19 infection have fever, cough, or shortness of breath [5]. The use of temperature to diagnose COVID-19 has not reached its fullest potential, however, for three reasons. First, clinicians tend to use temperature as a binary variable – often with a cutoff of 100.4 degrees Fahrenheit signaling active infection [6]—despite evidence that there is no absolute cutoff [7], [8], [9]. Thus, many infections can be missed when patients exhibit temperatures that are mildly elevated [10]. Second, failure to account for the effects of other variables on temperature values lead to missed opportunities to detect active infection [7]. Differences in measured body temperature and/or a patient’s ability to mount an elevated temperature in the setting of acute infection occur based on: patient age, sex, baseline body temperature, and comorbidities (e.g. depression, allergic conditions); time of day; ambient temperature; calendar month; and route of body temperature measurement [11], [12], [13]. Third, our clinical experience suggests that, while clinicians are aware of some of these relationships that should alter the interpretation of temperature values, they probably do not account for them effectively when making clinical decisions.

We propose that it may be possible to improve pretest probability estimation for COVID-19 by developing a novel model that includes variables readily available in the electronic health record (EHR). Figure 1 depicts a conceptual model demonstrating how body temperature-related covariates influence pretest probability, including how a clinician should decide to test for COVID-19 and act on results. Estimating pretest probability is critical to determining which patients should be tested [4, 14]. Additionally, because false-negative rates for COVID-19 testing have been found to range between 1 and 27% under different community prevalence scenarios [15], estimating pretest probability is strongly encouraged to determine which patients should get a second test, even after an initial negative test [14]. If we are able to enhance the use of temperature to predict acute COVID-19 infection, future work could use similar methods to improve the use of temperature to predict bacterial infections (e.g. those causing meningitis, neutropenic fever, neonatal fever, and sepsis) in inpatient, outpatient, and even long-term care settings.

Figure 1: 
Conceptual model for use of temperature in diagnosing acute COVID-19 infection.
This conceptual model demonstrates how relevant factors (e.g. antipyretic use, time of day) should influence interpretation of body temperature and the resultant effect on pretest probability. Direct influence is shown in red. In the example shown with a high pretest probability, a positive test result nearly always implies a posttest probability that exceeds the threshold requiring further testing. On the other hand, a negative result may still result in a high enough posttest probability that further testing is warranted.
Figure 1:

Conceptual model for use of temperature in diagnosing acute COVID-19 infection.

This conceptual model demonstrates how relevant factors (e.g. antipyretic use, time of day) should influence interpretation of body temperature and the resultant effect on pretest probability. Direct influence is shown in red. In the example shown with a high pretest probability, a positive test result nearly always implies a posttest probability that exceeds the threshold requiring further testing. On the other hand, a negative result may still result in a high enough posttest probability that further testing is warranted.

In this proof-of-concept study of model development, we sought to evaluate whether pretest probability estimates of COVID-19 could be improved by incorporating covariates known to be associated with body temperature, including demographics, comorbidities, time of day, and month of the year. We hypothesized that adding body temperature-relevant covariates would improve the area under the curve (AUC) for a logistic regression model predicting COVID-19 polymerase chain reaction (PCR) test positivity.

Materials, subjects and methods

Setting and population

This study was conducted at the emergency department (ED) of an academic medical center in Los Angeles, CA. The patients were retrospectively selected based on an electronic health record query of their location during ED evaluation: All patients in the sample were evaluated in a temporary surge tent where triage nurses sent patients who had symptoms suspicious for acute COVID-19 infection between March 12 and April 6, 2020. Additionally, patients were only included in the sample if they underwent PCR testing for COVID-19 and were discharged from the ED without inpatient or observation admission (as determined by manual chart review of the surge tent cases). Patients were excluded from the study if they did not have an oral temperature reading, if they were previously seen in the ED within the prior 7 days, if they did not complete ED evaluation (i.e., left without being seen, left against medical advice, or eloped), or if they reported no symptoms.

Data sources

Study procedures involved accessing an existing operational database to collect variables characterizing: patient age, sex, vital signs (including date, time, and route of measurement), comorbidities, and COVID-19 PCR test results from nasal swab samples [16]. Additionally, the content of physician and physician assistant notes describing patients’ visits was reviewed.

Measures

Outcome variable

The pilot model predicted acute COVID-19 infection at the individual patient level based on results of COVID-19 PCR testing performed during the ED visit.

Predictor variables

Selected predictor variables were covariates known to be associated with individual-level body temperature, including age, sex, month and time of day of temperature measurement, and comorbidities [11], [12], [13, 17]. Body temperature was operationally defined as the highest temperature measurement taken during the ED visit. Time of day of body temperature measurement ranged from 9:00 AM to 9:00 PM and was defined as a continuous variable, derived by squaring the nearest rounded hour of body temperature measurement. Comorbidities associated with individual variation in baseline body temperature and considered for inclusion as predictor variables were cancer, pulmonary disease, hypothyroidism, kidney disease, congestive heart failure, allergies, and depression [11, 13]. Comorbidities were enumerated through study team review of diagnoses listed in the patient’s past medical history.

Justification for using clinicians’ estimates of pretest probability without access to objective temperature data

Because body temperature is a key predictor for acute COVID-19 infection [18, 19], we included it directly in our final model based on structured EHR data rather than asking clinicians to incorporate it in an overall pretest probability. To avoid using body temperature twice, we used clinicians’ estimates of pretest probability without access to objective body temperature data.

Analyses

Derivation of independent variable

Physician pretest probability estimation

Two study team members trained in emergency medicine (CB and CL) independently reviewed clinical documentation (ED notes from physician assistants and physicians) to estimate pretest probability for COVID-19 while blinded to patients’ ED visit temperature measurements. Reviewers rated each case on a 5-point ordinal scale (1 = 20% suspicion or less; 5 = 80% suspicion or more). To determine estimates, reviewers were instructed to consider clinically relevant contextual factors, including local prevalence of COVID-19 at the time of the visit; patient-reported history of recent COVID-19 exposure within the last 14 days [20]; patient-reported symptoms (e.g., fever, chills, cough, shortness of breath, chest tightness, headache, sore throat, fatigue, body aches, diarrhea, abdominal pain, confusion, loss of taste, loss of smell); and objective criteria (e.g., vital signs, excluding redacted temperature). In cases where reviewers’ estimates differed by exactly one point, the estimates were averaged to obtain a mean pretest probability. If estimates differed by more than one point, the two reviewers discussed the case until consensus was achieved. Physician pretest probability estimates were included as a continuous predictor variable in statistical modeling.

Statistical analysis

Descriptive statistics were obtained using SPSS Statistics version 24.0 (IBM Corporation, Armonk, New York, USA) to compare demographics, comorbidities, and clinical characteristics of patients who tested positive and negative for acute COVID-19 infection. Between-group differences in demographics, comorbidities, and clinical characteristics were assessed using independent samples t-tests, Mann Whitney U tests, Pearson’s chi-square tests of independence, or Fisher’s exact tests. Intraclass correlation coefficients (ICCs) were calculated to estimate inter-rater agreement on physician pretest probability estimates [21, 22].

Bivariable and multivariable logistic regression models were fitted to the data to test the research hypotheses that independent variables are predictive of acute COVID-19 infection. Three pilot models were estimated with the following combinations of predictor variables:

  • Model #1: Body temperature

  • Model #2: Clinical probability (Body temperature and pretest probability estimate)

  • Model #3: Enhanced clinical probability (Body temperature, pretest probability estimate, and body temperature-relevant covariates)

Our primary hypothesis was that the predictive performance of Model #3 would exceed that of Model #2 due to the incorporation of body temperature-relevant covariates in Model #3. Predictive performance was evaluated using the area under the curve (AUC) and receiver operating curve (ROC) for discrimination, and calibration-in-large and calibration slope for calibration. Model performance, when evaluated in the same sample that was fitted, is overestimated. Therefore, we calculated the optimism of predictive performance measures using bootstrap to obtain optimism bias-corrected estimates [23, 24]. The significance of predictor main effects in bivariable and multivariable models were assessed using likelihood ratio chi-square tests, calculated as twice the difference of the log-likelihoods between the full model and the constrained model that does not contain the effect. Differences in AUCs between models were tested for statistical significance using Delong’s Test with alpha set to 0.05 and using a one-sided test [25]. Regression analyses were carried out by the logistic procedure in SAS version 9.4 (SAS Institute Inc., Cary, North Carolina, USA). Calibration and optimism bias-corrected estimates were obtained using R version 4.0.3 (The R Foundation for Statistical Computing, Vienna, Austria). All study procedures were reviewed and approved by the Institutional Review Board at Cedars-Sinai Medical Center.

Results

Out of 128 patients evaluated for inclusion in the study, a total of 117 patients met inclusion criteria (see Supplementary Material, Figure 1 for a study flow chart.). 40 out of 117 (34%) patients in the sample tested positive for COVID-19. Compared to patients who tested negative for COVID-19, patients who tested positive had higher maximum oral body temperatures in the emergency department (99.3 vs. 98.5 °F, p<0.05), but demographic and comorbidity characteristics were generally similar to patients who tested negative (Table 1).

Table 1:

Participant demographics.

Characteristics COVID−

(n=77)
COVID+

(n=40)
p-Value
Maximum BT from ED (°F), mean (SD)c 98.5 (0.7) 99.3 (1.3) 0.002a
Pre-test probability estimate without BT, median (range)d 3.5 (3.5) 4.0 (2.5) 0.01a
Age, years, mean, SD 43.2 (13.4) 41.1 (14.7) 0.442
Female 43 (55.8) 19 (47.5) 0.391
English-speaking 73 (94.8) 36 (90.0) 0.443
Non-white 17 (22.1) 10 (25.0) 0.722
Hispanic 11 (14.3) 9 (22.5) 0.425
Number of comorbidities 0.891
 0 44 (57.1) 21 (52.5)
 1 21 (27.3) 12 (30.0)
 ≥2 12 (15.6) 7 (17.5)
Comorbiditiesb
 Pulmonary disease 7 (9.1) 6 (15.0) 0.353
 Cancer 5 (6.5) 6 (15.0) 0.182
 Hypothyroidism 6 (7.8) 1 (2.5) 0.420
 Depression 4 (5.2) 1 (2.5) 0.659
 Kidney disease 2 (2.6) 1 (2.5) 1.000
 Allergies 3 (2.6) 0 (0.0) 0.550
 Congestive heart failure 1 (1.3) 0 (0.0) 1.000
  1. SD, standard deviation; BT, body temperature; ED, emergency department. Values reported as n (%) unless indicated. ap≤0.05. bTwo-sided fisher’s exact test. cWelch’s unequal variances t-test. dMann Whitney U test.

For the 117 patients in our study, COVID-19 pretest probability estimation by dual reviewers yielded identical scores in 55 (43.0%) cases, adjacent scores in 66 (51.6%) cases (i.e. the difference in reviewer scores differed by one point on the ordinal scale, range [1, 2] to [4, 5]); and scores initially differing by two or more points in 7 (5.5%), which were resolved by discussion. The intraclass correlation coefficient (ICC) for pretest probability estimation was 0.75 (95% CI 0.64–0.83).

After pretest probability estimates were determined, we used our data to derive three models of the relationship between body temperature and acute COVID-19 infection (Tables 2 and 3). Model #1, which used a patient’s maximum oral temperature alone, resulted in an AUC of 0.69 (95% CI 0.58–0.79). Model #2 used pretest probability estimates in addition to body temperature, and it achieved an AUC of 0.72 (95% CI 0.62–0.82, absolute difference in AUC between Model two vs. one of 0.029 (95% CI −0.057–0.115, p=0.25)). Model #3, which approximated clinicians’ current method of developing a pretest probability by using the objective temperature data, pretest probability estimates, and body temperature-relevant covariates, yielded an AUC of 0.76 (95% CI 0.67–0.86, absolute difference in AUC between Model three vs. two of 0.038 (95% CI −0.021 to 0.097, p=0.10), and optimism-adjusted AUC estimate of 0.672. See Table 2 for a list of AUCs for the three models, Table 3 for multivariable logistic regression coefficients and odds ratios predicting COVID-19 test result positivity under Model #3. See Figure 2 for a graphical representation of the ROC curves for Models #1–3.

Table 2:

Models derived from pilot data (n=117).

Model Name Data source AUC (95% CI)
#1 BT only (Structured data from EHR) 0.69 (0.58–0.79)
#2 Clinical probability (#1 and pretest probability) 0.72 (0.62–0.82)
#3 Enhanced clinical probability (#2 and BT-related covariates)a 0.76 (0.67–0.86)
  1. AUC, area under the curve; BT, body temperature; CI, confidence interval. aCovariates included age, sex, month, time of day, and comorbidities. None had p<0.05.

Table 3:

Multivariable logistic regression coefficients predicting COVID-19 test positivity under Model #3 (n=117).

Variable β OR (95% CI) SEβ χ2 df p-Value
Maximum BT from ED, °F 1.010 2.75 (1.43–5.25) 0.331 9.325 1 0.002a
Pretest probability estimateb 0.561 1.75 (1.01–3.05) 0.282 3.975 1 0.046a
Age, years 0.008 1.01 (0.98–1.04) 0.017 0.263 1 0.608
Sex (ref: female)
 Male 0.669 1.95 (0.71–5.34) 0.513 1.700 1 0.192
Month of BT measurement (ref: March)
 April −0.397 0.67 (0.15–3.09) 0.778 0.260 1 0.610
Hour of BT measurement, h2 −0.001 1.00 (0.99–1.01) 0.003 0.074 1 0.786
Comorbidities (ref: no disease)
 Cancer 1.450 4.26 (0.79–22.9) 0.858 2.856 1 0.091
 Depression 0.038 1.04 (0.10–11.4) 1.221 0.001 1 0.975
 Hypothyroidism −1.265 0.28 (0.02–3.46) 1.279 0.978 1 0.323
 Kidney disease −3.407 0.03 (0.00–5.46) 2.604 1.711 1 0.191
 Pulmonary disease 1.181 3.26 (0.77–13.7) 0.734 2.587 1 0.108
  1. β, regression coefficient; BT, body temperature; df, degrees of freedom; ED, emergency department; OR, odds ratio; ref, reference category; SE, standard error. Chi-square statistics are Wald based. ap≤0.05. bAverage of two raters’ scores on a 5-point ordinal scale.

Figure 2: 
Receiver operating curves for Models #1–3.
Figure 2:

Receiver operating curves for Models #1–3.

Apparent calibration slope and calibration-in-large for Model #3 were 1.0 and 0.0, respectively, indicating perfect calibration and overfitting. Bootstrap validation of Model #3 yielded an optimism-adjusted calibration slope of 0.478, indicating that the model underestimates low risk of COVID-19 infection and overestimates high risk of COVID-19 infection. Optimism-adjusted calibration-in-large was −0.293, indicating that the model on average overestimates the risk of a COVID-19 infection. See Figure 3 for calibration plots of apparent and optimism-adjusted Model #3.

Figure 3: 
Calibration plot for apparent and optimism-adjusted Model #3.
Figure 3:

Calibration plot for apparent and optimism-adjusted Model #3.

Discussion

Based on our a priori hypothesis that several covariates known to be associated with body temperature in non-infected subjects [11], [12], [13, 17] could improve the performance of body temperature as a predictor of COVID-19 acute infection status, we compared the predictive performance of three models. These models included oral body temperature alone, physicians’ temperature-blinded pretest probability plus objective temperature data, and a proposed enhanced model (which added age, sex, month, time of day, and comorbidities), which increased the AUC from 0.69 to 0.76—a statistically nonsignificant difference but a trend which should undergo investigation in further research. To our knowledge, this study is the first attempt to use a multivariable approach to enhance the predictive performance of temperature for acute infection. While many investigators have sought to predict sepsis and other bacterial infections by incorporating other clinically-relevant variables [26], [27], [28], [29], we are unaware of attempts to use demographic and other readily available variables in the EHR to specifically enhance the use of temperature.

Clinicians have long known that covariates are relevant to the interpretation of body temperature. Until now, however, most consideration of covariates’ effects has been handled qualitatively in clinicians’ minds. To the extent that these associations hold true [11] and can be validated in other datasets, they provide a method for clinicians to quantitatively adjust their interpretation of body temperature which could be used to produce an evidence-based adjusted body temperature for every patient encounter. This could be likened to using a corrected QT interval, which has superseded use of an unadjusted QT interval in nearly all clinical settings [30].

Because there are billions of patient encounters per year in which body temperature is measured, the public health implications of even a tiny improvement in discriminatory ability are large. This means that the impact of recognizing subtle elevations in temperature could be substantial if our approach can be optimized and spread to clinical settings across the globe. Moreover, while we have focused on prediction of acute COVID-19 infection in this proof-of-concept study, we believe that a similar approach could be leveraged to enhance the use of temperature to predict acute bacterial infections such as those causing meningitis, neutropenic fever, neonatal fever, and sepsis. It is also possible that an adjusted body temperature could enhance the predictive power of various clinical decision tools, such as those predicting bacterial infection among children [31], [32], [33].

We recognize that this initial analysis has several limitations. First, we expect that these AUCs are optimistic because they are evaluated in a derivation sample at a single site [34], though we attempted to mitigate this effect by reporting optimism-adjusted performance measures of calibration and discrimination. Second, our data used a subset of patients who were tested for COVID-19 during a time when testing supplies were limited, making selection bias a potential limitation. Third, members of the research team who generated pretest probability estimates were not blinded to the study hypothesis regarding which covariates should affect interpretation of temperature. This may have caused their estimates to differ from estimates that would otherwise by generated in practicing clinicians. Fourth, even when AUC is increased, it must be considered in a clinical context to ensure that yields clinically significant benefit. Fifth, we were unable to include two important classes of variables in our analysis because of gaps in the sample: patients’ baseline temperature values and variables related to women’s hormonal cycles. Accounting for patients’ baseline temperature has been endorsed by the Infectious Disease Society of America [35]. Moreover, use of hormonal contraceptives and estrogen replacement therapy have been demonstrated to impact temperature interpretation [11], which suggests that time since a woman’s last menstrual period may also affect temperature elevation in the setting of acute infection. These variables were infeasible for us to collect in our proof-of-concept study, but we believe that they would likely further enhance predictive performance of our model if we had been able to include them – especially given recent evidence and sex-specific differences have a substantial impact on clinical presentation of COVID-19 infection [36].

In conclusion, we present preliminary evidence that enhancing the interpretation of body temperature with body temperature-relevant covariates can improve discriminatory ability for COVID-19 above and beyond what is currently possible by using clinical suspicion and body temperature alone. If future research can validate these findings in other datasets, use of body-temperature relevant covariates to generate an adjusted temperature could be a promising method to improve detection of COVID-19 and other febrile illnesses, and to curb their spread.


Corresponding author: Carl T. Berdahl, MD, MS, Assistant Professor, Medicine and Emergency Medicine, Cedars-Sinai Medical Center, 8687 Melrose Ave G-562, West Hollywood, CA, 90069, USA, E-mail:

Funding source: Cedars-Sinai Medical Center

Award Identifier / Grant number: Cedars-Sinai Precision Health

Acknowledgments

The authors would like to thank David L. Schriger, MD, MPH, for his thoughtful input on study design, including the protocol for pretest probability scoring.

  1. Research funding: This study was supported by internal funding from Cedars-Sinai Medical Center, including a grant from Precision Health at Cedars-Sinai.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Informed consent: Informed consent was obtained from all individuals included in this study.

  5. Ethical approval: The Cedars-Sinai Institutional Review Board reviewed and approved the study protocol.

References

1. Luo, Z, Ang, MJY, Chan, SY, Yi, Z, Goh, YY, Yan, S, et al.. Combating the coronavirus pandemic: early detection, medical treatment, and a concerted effort by the global community. Research 2020;2020:6925296. https://doi.org/10.34133/2020/6925296.Search in Google Scholar PubMed PubMed Central

2. Kim, GU, Kim, MJ, Ra, SH, Lee, J, Bae, S, Jung, J, et al.. Clinical characteristics of asymptomatic and symptomatic patients with mild COVID-19. Clin Microbiol Infect 2020;26:948 e1–3. https://doi.org/10.1016/j.cmi.2020.04.040.Search in Google Scholar PubMed PubMed Central

3. Cheng, MP, Papenburg, J, Desjardins, M, Kanjilal, S, Quach, C, Libman, M, et al.. Diagnostic esting for evere cute espiratory yndrome-elated oronavirus 2: arrative eview. Ann Intern Med 2020;172:726–34. https://doi.org/10.7326/m20-1301.Search in Google Scholar

4. Watson, J, Whiting, PF, Brush, JE. Interpreting a covid-19 test result. BMJ 2020;369:m1808. https://doi.org/10.1136/bmj.m1808.Search in Google Scholar PubMed

5. Stokes, EK, Zambrano, LD, Anderson, KN, Marder, EP, Raz, KM, El Burai Felix, S, et al.. Coronavirus disease 2019 case surveillance – United States, January 22–May 30, 2020. Morb Mortal Wkly Rep 2020;69:759–65. https://doi.org/10.15585/mmwr.mm6924e2.Search in Google Scholar PubMed PubMed Central

6. Mackowiak, PA, Wasserman, SS, Levine, MM. A critical appraisal of 98.6 degrees F, the upper limit of the normal body temperature, and other legacies of Carl Reinhold August Wunderlich. JAMA 1992;268:1578–80. https://doi.org/10.1001/jama.268.12.1578.Search in Google Scholar

7. Ogoina, D. Fever, fever patterns and diseases called ’fever’ – a review. J Infect Public Health 2011;4:108–24. https://doi.org/10.1016/j.jiph.2011.05.002.Search in Google Scholar PubMed

8. Drewry, AM, Fuller, BM, Bailey, TC, Hotchkiss, RS. Body temperature patterns as a predictor of hospital-acquired sepsis in afebrile adult intensive care unit patients: a case-control study. Crit Care 2013;17:R200. https://doi.org/10.1186/cc12894.Search in Google Scholar PubMed PubMed Central

9. O’Grady, NP, Barie, PS, Bartlett, JG, Bleck, T, Carroll, K, Kalil, AC, et al.. Guidelines for evaluation of new fever in critically ill adult patients: 2008 update from the American College of Critical Care Medicine and the Infectious Diseases Society of America. Crit Care Med 2008;36:1330–49. https://doi.org/10.1097/CCM.0b013e318169eda9.Search in Google Scholar PubMed

10. Liu, K, Fang, YY, Deng, Y, Liu, W, Wang, MF, Ma, JP, et al.. Clinical characteristics of novel coronavirus cases in tertiary hospitals in Hubei Province. Chin Med J 2020;133:1025–31. https://doi.org/10.1097/cm9.0000000000000744.Search in Google Scholar

11. Obermeyer, Z, Samra, JK, Mullainathan, S. Individual differences in normal body temperature: longitudinal big data analysis of patient records. BMJ 2017;359:j5468. https://doi.org/10.1136/bmj.j5468.Search in Google Scholar

12. Kelly, G. Body temperature variability (Part 1): a review of the history of body temperature and its variability due to site selection, biological rhythms, fitness, and aging. Altern Med Rev 2006;11:278–93.Search in Google Scholar

13. Kelly, GS. Body temperature variability (Part 2): masking influences of body temperature variability and a review of body temperature variability in disease. Altern Med Rev 2007;12:49–62.Search in Google Scholar

14. Woloshin, S, Patel, N, Kesselheim, AS. False negative tests for SARS-CoV-2 infection – challenges and implications. N Engl J Med 2020;383:e38. https://doi.org/10.1056/nejmp2015897.Search in Google Scholar

15. Arevalo-Rodriguez, I, Buitrago-Garcia, D, Simancas-Racines, D, Zambrano-Achig, P, Del Campo, R, Ciapponi, A, et al.. False-negative results of initial RT-PCR assays for COVID-19: systematic review. PloS One 2020;15:e0242958. https://doi.org/10.1371/journal.pone.0242958.Search in Google Scholar

16. Berdahl, CT, Glennon, NC, Henreid, AJ, Torbati, SS. The safety of home discharge for low-risk emergency department patients presenting with coronavirus-like symptoms during the COVID-19 pandemic: a retrospective cohort study. J Am Coll Emerg Phys Open 2020 Aug 25. https://doi.org/10.1002/emp2.12230 [Epub ahead of print].Search in Google Scholar

17. Shilaih, M, Goodale, BM, Falco, L, Kübler, F, De Clerck, V, Leeners, B. Modern fertility awareness methods: wrist wearables capture the changes in temperature associated with the menstrual cycle. Biosci Rep 2018;38. https://doi.org/10.1042/BSR20171279.Search in Google Scholar

18. Tharakan, S, Nomoto, K, Miyashita, S, Ishikawa, K. Body temperature correlates with mortality in COVID-19 patients. Crit Care 2020;24:298. https://doi.org/10.1186/s13054-020-03045-8.Search in Google Scholar

19. Mizrahi, B, Shilo, S, Rossman, H, Kalkstein, N, Marcus, K, Barer, Y, et al.. Longitudinal symptom dynamics of COVID-19 infection. Nat Commun 2020;11:6208. https://doi.org/10.1038/s41467-020-20053-y.Search in Google Scholar

20. Zhou, F, Yu, T, Du, R, Fan, G, Liu, Y, Liu, Z, et al.. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 2020;395:1054–62. https://doi.org/10.1016/s0140-6736(20)30566-3.Search in Google Scholar

21. Hallgren, KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol 2012;8:23–34. https://doi.org/10.20982/tqmp.08.1.p023.Search in Google Scholar PubMed PubMed Central

22. Cicchetti, DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 1994;6:284–90. https://doi.org/10.1037/1040-3590.6.4.284.Search in Google Scholar

23. HarrellJrFE. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. New York: Springer; 2015.10.1007/978-3-319-19425-7Search in Google Scholar

24. Huang, Y, Li, W, Macheret, F, Gabriel, RA, Ohno-Machado, L. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inf Assoc 2020;27:621–33. https://doi.org/10.1093/jamia/ocz228.Search in Google Scholar

25. DeLong, ER, DeLong, DM, Clarke-Pearson, DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. https://doi.org/10.2307/2531595.Search in Google Scholar

26. Hamilton, JL, Evans, SG, Bakshi, M. Management of fever in infants and young children. Am Fam Physician 2020;101:721–9.Search in Google Scholar

27. Mayr, FB, Yende, S, Linde-Zwirble, WT, Peck-Palmer, OM, Barnato, AE, Weissfeld, LA, et al.. Infection rate and acute organ dysfunction risk as explanations for racial differences in severe sepsis. JAMA 2010;303:2495–503. https://doi.org/10.1001/jama.2010.851.Search in Google Scholar

28. Angus, DC, Linde-Zwirble, WT, Lidicker, J, Clermont, G, Carcillo, J, Pinsky, MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med 2001;29:1303–10. https://doi.org/10.1097/00003246-200107000-00002.Search in Google Scholar

29. Islam, MM, Nasrin, T, Walther, BA, Wu, C-C, Yang, H-C, Li, Y-C. Prediction of sepsis patients using machine learning approach: meta-analysis. Comput Methods Progr Biomed 2019;170:1–9. https://doi.org/10.1016/j.cmpb.2018.12.027.Search in Google Scholar

30. Vandenberk, B, Vandael, E, Robyns, T, Vandenberghe, J, Garweg, C, Foulon, V, et al.. Which QT correction formulae to use for QT monitoring? J Am Heart Assoc 2016;5. https://doi.org/10.1161/JAHA.116.003264.Search in Google Scholar

31. Torres, FA, Passarelli, I, Cutri, A, Leonardelli, A, Ossorio, MF, Ferrero, F. Safety of a clinical prediction rule for initial management of children with pneumonia in an ambulatory setting. Arch Argent Pediatr 2010;108:511–5. https://doi.org/10.1590/S0325-00752010000600006.Search in Google Scholar

32. Fever in under 5s: assessment and initial management. London (UK): National Institute for Health and Care Excellence; 2019:36 p.Search in Google Scholar

33. Baraff, LJ, Bass, JW, Fleisher, GR, Klein, JO, McCrackenJrGH, Powell, KR, et al.. Practice guideline for the management of infants and children 0 to 36 months of age with fever without source. Agency for Health Care Policy and Research. Ann Emerg Med 1993;22:1198–210. https://doi.org/10.1016/s0196-0644(05)80991-6.Search in Google Scholar

34. Smith, GCS, Seaman, SR, Wood, AM, Royston, P, White, IR. Correcting for optimistic prediction in small data sets. Am J Epidemiol 2014;180:318–24. https://doi.org/10.1093/aje/kwu140.Search in Google Scholar PubMed PubMed Central

35. High, KP, Bradley, SF, Gravenstein, S, Mehr, DR, Quagliarello, VJ, Richards, C, et al.. Clinical practice guideline for the evaluation of fever and infection in older adult residents of long-term care facilities: 2008 update by the infectious diseases society of America. Clin Infect Dis 2009;48:149–71. https://doi.org/10.1086/595683.Search in Google Scholar PubMed

36. Wolfe, J, Safdar, B, Madsen, TE, Sethuraman, KN, Becker, B, Greenberg, MR, et al.. Sex- or gender-specific differences in the clinical presentation, outcome, and treatment of SARS-CoV-2. Clin Ther 2021 Jan 16. https://doi.org/10.1016/j.clinthera.2021.01.015 [Epub ahead of print].Search in Google Scholar PubMed PubMed Central


Supplementary Material

The online version of this article offers supplementary material (https://doi.org/10.1515/dx-2021-0020).


Received: 2021-01-28
Accepted: 2021-05-07
Published Online: 2021-06-30
Published in Print: 2021-11-25

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 28.3.2024 from https://www.degruyter.com/document/doi/10.1515/dx-2021-0020/html
Scroll to top button