J Korean Med Sci. 2020 Sep 28;35(38):e338. English.
Published online Sep 10, 2020.
© 2020 The Korean Academy of Medical Sciences.
letter

Letter to the Editor: The Interpretation of COVID-19 Seroprevalence Study Should Be Cautious

Jaehyeon Lee,1,* So Yeon Kim,2,* Heungsup Sung,3 Young June Choe,4 and Ki Ho Hong5
    • 1Department of Laboratory Medicine, Jeonbuk National University Medical School and Hospital, Jeonju, Korea.
    • 2Department of Laboratory Medicine, National Medical Center, Seoul, Korea.
    • 3Department of Laboratory Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea.
    • 4Department of Social and Preventive Medicine, Hallym University College of Medicine, Chuncheon, Korea.
    • 5Department of Laboratory Medicine, Seoul Medical Center, Seoul, Korea.
Received August 18, 2020; Accepted September 03, 2020.

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

We have read with a great interest the article written by Song and colleagues1 assessing seroprevalence of coronavirus disease 2019 (COVID-19) in Daegu, Korea, aftermath of the largest epicenter in Korea. The study is in line with previous assumptions that there may had been a large transmission of the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) resulting asymptomatic infection in the general population. However, a few points should be clarified prior to drawing conclusions.

First, the performance of the antibody assay that was used in the study should be noted. The clinical specificity of assays should be vigorously evaluated before applying to the large-scale evaluation of a population. The authors described the specificity of the assay as 92%, which the number may not be drawn from 30 specimens. If the specificity is 92%, which means that false-positive rate is 8%, the prevalence of 7.6% can be attributed to false-positive results, not from past-infection. The authors have assumed that moderate performance is acceptable in estimating the seroprevalence in population. However, even if the performance obtained from study is adequate, the application of the assay into general population should be cautious. The positive predictive value of diagnostic assays is affected by the disease prevalence.2 Low prevalence may result in a high false positive rate, or a low positive predictive value, vice versa.

Second, the property of the assay used should be taken into account in drawing the conclusion. The assays employed two antigens (nucleocapsid protein and receptor binding domain of spike protein). There is a report about the SARS-CoV-2 antibody tests using multiple antigens to increase specificity.3 In this report, when a sample was assessed as positive when reacted with either of multiple antigens, the test specificity has decreased. The assay that authors used multiple antigens in single reaction and cannot differentiate between two antigens. Therefore, this property may lower the specificity of assay. Even though authors might declare that the specificity of the assays was 100%, the variation of performance among COVID-19 antibody based on lateral flow immunoassay was reported to be high and the variation among the interpreters was also high.4, 5, 6

Third, although the authors have acknowledged selection bias, they should recognize that the patient group had more of health-related issues compared with general population, resulting limited generalizability of the study. The generalization of the prevalence obtained from patient group should be cautious. A recent systematic review found that 89% of antibody test evaluation results had selection bias.4 In addition, the PCR-negative specimens collected during COVID-19 pandemic period are inadequate for the evaluation of specificity, as the convalescent patients may show positive results in antibody assay, but negative results in PCR.7 If the authors claimed the high seroprevalence in the population, then authors should not use the PCR-negative samples from the population as negative control group. Moreover, given only small specimens have been included (60 positive and 30 negative), providing the confidence intervals for distribution would be more informative. The confidence interval of the specificity seems to be 88.4%–100.0%, based on Clopper-Pearson Interval.

The authors have estimated the “actual number” of SARS-CoV-2 infection in Daegu to be greater than 180,000 persons, which may be misled from the intrinsic limitation of the study. They also have stated several interesting opinions such as the limited value of containment and the limited role of antibody assay in selecting donor for plasma therapy. However, the points that we have mentioned above should be clarified before drawing the conclusion of the study.

Notes

Disclosure:The authors have no potential conflicts of interest to disclose.

Author Contributions:

  • Conceptualization: Lee J, Kim SY, Hong KH, Sung H.

  • Writing - original draft: Lee J, Kim SY, Hong KH.

  • Writing - review & editing: Sung H, Hong KH, Choe YJ.

References

    1. Song SK, Lee DH, Nam JH, Kim KT, Do JS, Kang DW, et al. IgG seroprevalence of COVID-19 among individuals without a history of the coronavirus disease infection in Daegu, Korea. J Korean Med Sci 2020;35(29):e269
    1. Weiss SH, Wormser GP. COVID-19: Understanding the science of antibody testing and lessons from the HIV epidemic. Diagn Microbiol Infect Dis 2020;98(1):115078
    1. den Hartog G, Schepp RM, Kuijer M, GeurtsvanKessel C, van Beek J, Rots N, et al. SARS-CoV-2-specific antibody detection for sero-epidemiology: a multiplex analysis approach accounting for accurate seroprevalence. medRxiv. 2020 [doi: 10.1101/2020.06.18.20133660]
    1. Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020;6(6):CD013652
    1. Lisboa Bastos M, Tavaziva G, Abidi SK, Campbell JR, Haraoui LP, Johnston JC, et al. Diagnostic accuracy of serological tests for covid-19: systematic review and meta-analysis. BMJ. 2020 [doi: 10.1002/14651858.CD013652]
    1. Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Test performance evaluation of SARS-CoV-2 serological assays. medRxiv. 2020 [doi: 10.1101/2020.04.25.20074856]
    1. Favresse J, Eucher C, Elsen M, Tré-Hardy M, Dogné JM, Douxfils. Clinical performance of the Elecsys electrochemiluminescent immunoassay for the detection of SARS-CoV-2 total antibodies. Clin Chem. 2020 [doi: 10.1093/clinchem/hvaa131]

J Korean Med Sci. 2020 Sep;35(38):e338. > Response
J Korean Med Sci. 2020 Sep;35:e338.
doi: https://doi.org/10.3346/jkms.2020.35.e339
© 2020 The Korean Academy of Medical Sciences.
The Author's Response: COVID-19 Antibody Test at Population Level: Why Timing Is the Key
Suk-Kyoon Song,1 Duk-Hee Lee,2 and Myung-Rae Cho1
1Department of Orthopedics, Daegu Catholic University Hospital, Daegu, Korea.
2Department of Preventive Medicine, School of Medicine, Kyungpook National University, Daegu, Korea.

Address for Correspondence: Myung-Rae Cho, MD. Department of Orthopedics, Daegu Catholic University Hospital, 33 Duryugongwon-ro 17-gil, Nam-gu, Daegu 42472, Republic of Korea. Email: cmr0426@cu.ac.kr

Received August 31, 2020; Accepted September 03, 2020.

Dr. Lee and colleagues1 raised several issues related to our study,2 which can be summarized into two categories: 1) the possibility of low specificity of rapid test kit and 2) selection bias of study subjects. First, we appreciate the authors because this letter allowed us to identify a critical typographical error in the published article. They stated that 92% specificity of the assay was a study limitation. However, the truth is that the specificity was 100% but sensitivity was 92%. It was a simple typo which occurred during manuscript revision.

The authors criticized that the seroprevalence of 7.6% of our study can be totally attributed to false-positive cases due to 92% specificity. However, it is more accurate to state that even the value of 7.6% was an underestimation of the true figure because the sensitivity was 92%. In addition, recent findings, which demonstrated the disappearance of antibodies among coronavirus disease 2019 (COVID19) patients in months,3 suggested an additional reason to justify our speculation; the epidemic peak of Daegu was late February, while the collection of blood was performed from late May to early June. In particular, a rapid decline in antibodies was observed in mild cases4; the majority of COVID19 patients in Daegu only had mild symptoms or were asymptomatic.

In fact, many details raised by the authors relate to the accuracy of rapid test kit, especially the possible low specificity. Although 92% specificity is not relevant anymore as explained above, the authors' perspective on accuracy of antibody testing requires further discussion because the importance of this issue differs substantially depending on the purpose of the test.5

If the results of antibody test are applied to individuals, such as immunity passports, the accuracy of the test is crucial to avoid risks due to false-positive and false-negative cases. However, when an antibody test is performed to estimate seroprevalence at the population level, even a serological test with moderate sensitivity and specificity is acceptable5 because of the tradeoff between false-positive and false-negative cases. A recent nationwide population-based study in Spain, reporting 5.0% seroprevalence, clearly demonstrated that the performance of the rapid test was satisfactory compared with that of immunoassay.6

The authors were also skeptical about applying the seroprevalence results of our study to the general population due to selection bias, which was already discussed as a study limitation in our paper. Our study subjects consisted of patients and guardians who visited outpatient clinics, not only a patient group as the authors have described. Due to the similar seroprevalence between patients and guardians, and given that all of the positive cases belonged to different households, the selection bias may not be as serious as the authors have suspected.

Moreover, despite the possibility of selection bias, we believe that the size of missing undiagnosed cases in Daegu should be estimated because it is one of most important reasons underlying any seroprevalence study during the epidemic. This estimation is more important for infectious diseases with a high proportion of asymptomatic or mild cases, such as COVID 19, to determine the optimal public health strategy.

Sometimes, information with limitation would be better than at least no information. The estimated size of missing undiagnosed cases during any epidemics may be a case in point. Of course, it would be best if a seroprevalence survey of representative samples in Daegu could have been performed in March or April. Currently, however, even a population-based seroprevalence survey with a perfect serological assay may not validly estimate the size of missing undiagnosed cases in Daegu because of antibody loss in many previously infected cases.3, 4

Notes

Disclosure:The authors have no potential conflicts of interest to disclose.

Author Contributions:

  • Conceptualization: Song SK, Cho MR.

  • Data curation: Song SK.

  • Supervision: Lee DH, Cho MR.

  • Writing - original draft: Song SK.

  • Writing - review & editing: Lee DH, Cho MR.

References

    1. Lee J, Kim SY, Sung H, Choe YJ, Hong KH. The interpretation of COVID-19 seroprevalence study should be cautious. J Korean Med Sci 2020;35(38):e338
    1. Song SK, Lee DH, Nam JH, Kim KT, Do JS, Kang DW, et al. IgG seroprevalence of COVID-19 among individuals without a history of the coronavirus disease infection in Daegu, Korea. J Korean Med Sci 2020;35(29):e269
    1. Seow J, Graham C, Merrick B, Acors S, Steel K, Hemmings O, et al. Longitudinal evaluation and decline of antibody responses in SARS-CoV-2 infection. medRxiv. 2020 [doi: 10.1101/2020.07.09.20148429]
    1. Ibarrondo FJ, Fulcher JA, Goodman-Meza D, Elliott J, Hofmann C, Hausner MA, et al. Rapid decay of anti-SARS-CoV-2 antibodies in persons with mild Covid-19. N Engl J Med. 2020 [doi: 10.1056/NEJMc2025179]
      Forthcoming.
    1. Bryant JE, Azman AS, Ferrari MJ, Arnold BF, Boni MF, Boum Y, et al. Serology for SARS-CoV-2: apprehensions, opportunities, and the path forward. Sci Immunol 2020;5(47):eabc6347
    1. Pollán M, Pérez-Gómez B, Pastor-Barriuso R, Oteo J, Hernán MA, Pérez-Olmeda M, et al. Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based seroepidemiological study. Lancet 2020;396(10250):535–544.

Metrics
Share
PERMALINK