SARS molecular detection external quality assurance.

Inactivated severe acute respiratory syndrome-associated coronavirus samples were used for an external quality assurance study within the World Health Organization SARS Reference and Verification Network and other reference institutions. Of 58 participants, 51 correctly detected virus in all samples [greater than or equal to] 9,400 RNA copies per milliliter and none in negative samples. Commercial test kits significantly improved the outcome.

**********

Severe acute respiratory syndrome (SARS) is an infectious interstitial pneumonia that causes death in a considerable portion of patients. The first epidemic of SARS began in November 2002 in southern China, spread to all five continents, and was interrupted in July 2003. It caused 774 deaths among the 8,098 cases. Two laboratory-associated infections and four new isolated cases have since occurred (1). SARS is caused by a novel coronavirus (SARS-CoV) that is shed in patients' respiratory secretions after infection (2-5). Immune response to SARS-CoV appears with a latency of up to 4 weeks from infection, and the concentration of virus particles varies greatly between patients or types of clinical samples. Thus achieving a reliable virologic diagnosis early alter disease onset is difficult. Highly sensitive methods for virus detection, such as reverse transcription-polymerase chain reaction (RT-PCR) are required to confirm SARS in the acute phase and prevent transmission.

Molecular detection methods have been developed by several research laboratories, and the first commercial test kits have become available (6,7). The performance of such tests, however, has only been evaluated in pilot feasibility studies. Little data exist about the relative performance of different laboratories and methods. The World Health Organization (WHO) has made the comparing and standardizing of laboratory tests an issue of high priority in SARS research (8). Comparative testing of characterized samples is a direct way to identify weaknesses of single laboratories or certain methods.

The Study

We present the results of the first external quality assurance study on SARS-CoV molecular detection. Ninety-three institutions involved in laboratory diagnostics of SARS were invited to participate in the study. Invitees were members of the international WHO SARS Reference and Verification Laboratory Network (9) or national and regional SARS reference laboratories. The study was announced as an external quality assurance study on diagnostic proficiency, which included certifying and publishing the results in a comparative and anonymous manner. Fifty-eight laboratories from 38 countries (21 European, 9 Austral-Asian, 7 North and South American, and 1 African) eventually enrolled in the study. (1) Four companies that produced commercial diagnostic test systems also participated but were evaluated separately because they do not fulfill public health duties.

Virus material was obtained from supernatants of Vero cell cultures collected one day after infection with SARS-CoV strains Frankfurt 1 and HKU-1. The supernatants were heated to 56[degrees]C for 1 h and [gamma] irradiated with 30 kGy. Residual infectivity was excluded by Veto cell cultures (3 passages). Aliquots of the inactivated virus stock solutions were lyophilized and redissolved, and the virus RNA was quantified by two different noncommercial real-time RT-PCR assays (2,6). Virion integrity was confirmed by morphology by electron microscopy (data not shown). Test samples for the study were generated by diluting the inactivated virus stock solutions in human fresh-frozen plasma testing negative for HIV-1, hepatitis 13 virus, hepatitis C virus, and SARS-CoV by RT-PCR. Aliquots of 100 [micro]L each were then lyophilized and shipped at ambient temperature to the participating laboratories. Each participant received a coded panel of seven positive and three negative samples. Virus-positive samples contained 94-940,000 RNA copies per milliliter after resuspending in 100 [micro]L of water. The participants were asked to analyze the material with the molecular methods they routinely use in suspected cases in humans. Details about the methods were requested, such as the sources of RT-PCR primers and protocols, the type of extraction method used, and suppliers and types of commercial kits, if used. The following two criteria were chosen as minimum requirements for overall proficiency. First, laboratories had to correctly detect the four samples containing [greater than or equal to] 9,400 copies of viral RNA per milliliter, a concentration well above the detection limit of published and commercial nucleic acid amplification tests (NAT) for SARS-CoV, (6,7,10-12). Second, no false-positive results were allowed with the negative samples. Indeterminate results in positive samples were treated as negative and in negative samples were treated as positive since the application of NAT usually does not involve indeterminate endpoints, and laboratories should be able to resolve unclear results by double testing with another amplification assay (13).

Before evaluating the performance of individual laboratories, we determined how many participants managed to detect virus in each sample (Table 1). The concentration-dependent, cumulative positivity rates per sample corresponded exactly with the response rates calculated by a probit regression analysis, which is equivalent to a dose-response model (Figure, p [less than or equal to] 0.0001). The model could predict for the average laboratory that 50% of all test results could be expected to be correctly positive when 158 (95% confidence interval [CI] 76.55-269.15) copies of virus RNA per milliliter of sample were present, and 95% with more than 11,220 (95% CI 5,675-31,988) copies per milliliter. Good compliance with the model furthermore confirmed that all samples contained the expected concentration of RNA upon reception by the participants and that no RNA degradation had occurred even in samples containing low amounts of virus.

[FIGURTE OMITTED]

Applying the proficiency criteria, 51 (88%) of 58 laboratories passed the minimum requirements for successful participation. Failure in three laboratories was due to lack of sensitivity, in three due to false-positive results, and in one due to both. Thirteen of 51 successful laboratories (22.4% of all 58 participants) could also detect the virus in all three weakly positive samples ([less than or equal to] 2,350 copies/mL), and another 17 missed only one positive sample. Ten of the 58 laboratories issued indeterminate results in one or more samples.

Whether common technical factors would influence the performance of laboratories was also assessed. We subjected cumulative results from low concentration samples ([less than or equal to] 2,300 copies/mL) to analysis of variance (ANOVA) analysis. The overall positivity rate in these samples was 65.6% (95% CI 56.1%-75.0%). Seven technical factors (Table 2) were used to characterize the test procedures each laboratory was using. Only use of commercial RT-PCR test kits made a significant difference with regard to total sensitivity. This finding was in concordance with results of the four participating companies who manufacture these kits: all were 100% correct. Fourteen of 58 participants used commercial test kits. For noncommercial tests, whether laboratories developed primers themselves or adapted from other researchers did not make a difference. This finding might be due to availability of well-evaluated primers through a WHO internet resource during the outbreak (14). Forty-two of the 58 participants used at least one procedure listed on this site.

We finally assessed whether laboratories belonging to the international WHO SARS Reference and Verification Network (9) were more proficient in SARS molecular detection than others. In the three samples containing [less than or equal to] 2,350 copies of SARS-CoV RNA per milliliter, the network laboratories achieved a cumulative fraction of correct positive results of 79.5% (95% CI 60.2%-98.9%) as opposed to 61.5% (95% CI 50.6% 72.4%) in the other labs participating in the study. This difference was not significant (p value 0.11, t-test).

Conclusions

The results of this first external quality assurance study on SARS-CoV molecular detection are assuring. Compared to an earlier study on molecular testing for filoviruses, Lassa virus, and orthopoxviruses, using very similar proficiency criteria (15), almost double the portion of participating laboratories completed the study successfully (88% vs. 45.8%). On the other hand, this study only examined paramount issues like sensitivity and control of contamination. Validation of other aspects, like cross-reactivity of primers or control of PCR inhibition, is the responsibility of each diagnostic laboratory.

Commercial tests clearly were the preferred way of achieving good diagnostic performance, possibly because SARS-CoV is a pathogen with which relatively few laboratories have had experience. However, developing and approving commercial tests is a lengthy process and high costs limit their application. Other approaches have to be adopted for efficiently providing good diagnostic tools in immediate response to an infectious disease outbreak. WHO's strategy of disseminating essential information through a public Internet resource before publication has proven successful. Laboratories have willingly shared protocols and positive control material with other institutions, enabling qualified diagnostics within weeks alter the primary description of the new virus. The benefit is proven by good overall results in this study.

International strain collections should be complemented with noninfectious reference material of rare pathogens. Until now, such material has been available only for highly prevalent agents like HIV-1, herpes viruses, or hepatitis viruses. For SARS-CoV, reference material has been created in this study for the first time. All samples described can be obtained for a nonprofit charge through the WHO SARS Reference and Verification Laboratory Network.

Table 1. Characteristics of patients with suspected and confirmed
A/H7N3 cases

Characteristic                Cases (%) (N = 57)

Male sex                           32 (58)
Median age in y (range)           33 (1-68)
Received influenza vaccine         36 (65)
  >2 wk before exposure            12 (22)
Occupation/relationship
  Farm owner                        9 (16)
  Family member                    11 (19)
  Farm employee                    14 (25)
    Farm manager                      3
    Egg collector                     6
    Chicken catcher                   2
    Miscellaneous worker              3
  Federal worker                   12 (22)
    Veterinarian                    4 (a)
    Inspector                         3
    General laborer                 6 (a)
    Decomposition worker              1
Other                               4 (7)
Unknown                             5 (9)

(a) Includes confirmed A/H7N3 case.

Table 2: Clinical profile of suspected and confirmed infections of it
avian influenza H7N3 in humans, Canada

Symptoms                Cases (%) (N = 57)

Conjunctivitis
  Red eye                  12 (21) (a)
  Tearful eye               6 (11) (a)
  Itching eye              13 (23) (a)
  Painful eye                 4 (7)
  Burning eye                 7 (12)
  Discharge from eye        4 (7) (a)
  Photophobia                 5 (9)
ILI symptoms (b)
  Fever                      17 (30)
  Cough                      42 (74)
  Coryza                   35 (61) (a)
  Sore throat                39 (68)
  Myalgia                    26 (46)
  Arthralgia                 16 (28)
  Fatigue                    22 (39)
  Diarrhea                   11 (19)
  Chills                     16 (28)
  Headache                 28 (49) (a)
  Other symptoms             12 (21)

(a) Includes confirmed A/H7N3 cases.

(b) ILI, Influenzalike illness.

(1) University Vienna, Vienna, Austria; University Hospital Leuven, Leuven, Belgium; Statens Serum Institut, Copenhagen, Denmark; Health Protection Agency, London and Salibury. England; University of Helsinki, Helsinki, Finland; Institut Pasteur, Paris, France; Bernhard Nocht Institut, Hamburg, Germany; Philipps Universitat, Marburg, Germany; Robert Koch-Institut, Berlin, Germany; University Frankfurt, Frankfurt, Germany; M & LAT, Berlin, Germany; Artus GmbH, Hamburg, Germany; Euroimmun, Lubeck, Germany; Aristotelian University, Thessaloniki, Greece; University of Athens, Athens, Greece; National Center for Epidemiology, Budapest, Hungary; University Hospital Reykjavik, Reykjavik, Iceland; Chaim Sheba Medical Center, Tel Hashomer, Israel; Army Medical and Veterinary Research Center, Rome, Italy; Istituto Nazionale Malattie Infettive, Rome, Italy; Istituto Superiore di Sanita Rome, Italy; Erasmus MC, Rotterdam, the Netherlands; RIVM, Bilthoven, the Netherlands; Leiden University Medical Center, Leiden, the Netherlands; Norwegian Institute of Public Health, Oslo, Norway.

Table 1. Positive samples in test panel (a)

                           Virus RNA           Fraction of
Sample     SARS-CoV      concentration      laboratories with
code        strain         copies/mL      positive detection (%)

S-CV2     Frankfurt 1       940,000                100
S-CV9     Frankfurt 1        94,000                 98.3
S-CV6        HKU-1           23,500                 98.3
S-CV4     Frankfurt 1         9,400                 94.8
S-CV10       HKU-1            2,350                 87.9
S-CV1     Frankfurt 1           940                 70.7
S-CV5     Frankfurt 1            94                 43.1

(a) SARS-CoV, severe acute respiratory syndrome-associated coronavirus.

Table 2. Factors influencing the performance of laboratories (a)

                                                         Positive
                                                        influence on
                                          No. of        sensitivity
Possible technical influence factors    laboratories      p value

Qiagen viral RNA extraction kit              38             0.9
Roche MagnaPure/HighPure
  extraction kit                              7             0.2
Silica particle-based extraction
  method (Boom)                               9             0.9
Primers originally developed in
  own laboratory                             16             0.5
Any nested PCR assay                         25             0.9
Any real-time PCR assay                      37             0.7
Any commercial test kit                      14             0.03

(a) Analysis of Variance (ANOVA) by factor, eliminating the influence
of other factors; PCR, polymerase chain reaction.

This study was performed by the WHO and the European Network for Diagnostics of Imported Viral Diseases, funded by the European Community DG SANCO under the program AIDS and other communicable diseases, grant no. S12.299717 (2000CVG4-26). Work of the Bernhard-Nocht Institute was funded by the German Ministry of Health under grant No. 325-4539-85/3.

Dr. Drosten heads the molecular diagnostics laboratory group within the Department of Medical Microbiology, Bernhard Nocht Institute for Tropical Medicine, Hamburg, Germany. His research focuses on molecular detection methods for tropical viral and parasitic infections and the evolution and pathogenesis of SARS-CoV.

References

(1.) World Health Organization. New case of laboratory-confirmed SARS in Guangdong, China--update 5. [cited 2004 May 3]. Available from www.who.int/csr/don/2004_01_31/en/

(2.) Drosten C. Gunther S, Preiser W, van der Werf S, Brodt HR, Becket S, et al. Identification of a novel coronavirus in patients with severe acute respiratory syndrome. N Engl J Med. 2003;348:1967-76.

(3.) Ksiazek TG, Erdman D, Goldsmith CS. Zaki SR, Peter T, Emery S, et al. A novel coronavirus associated with severe acute respiratory syndrome. N Engl J Med. 2003:348:1953-66.

(4.) Peiris JSM, Lai ST, Pooh LLM, Guan Y. Yam LYC. Lim W, et al. Coronavirus as a possible cause of severe acute respiratory syndrome. Lancet. 2003:361:1319-25.

(5.) Fouchier RAM, Kuiken T, Schutten M, van Amerongen G, van Doornum GJ, van den Hoogen BG, et al. Aetiology: Koch's postulates fulfilled for SARS virus. Nature. 2003:423:240.

(6.) Drosten C, Chiu LL, Panning M, Leong HN, Preiser W, Tam JS, et al. Evaluation of advanced reverse transcription-PCR assays and an alternative PCR target region for detection of severe acute respiratory syndrome-associated coronavirus. J Clin Microbiol. 2004:42: 2043-7.

(7.) Ng EK. Hut DS, Chan KC, Hung EC, Chiu RW, Lee N, et al. Quantitative analysis and prognostic implication of SARS coronavirus RNA in the plasma and serum of patients with severe acute respiratory syndrome. Clin Chem. 2003:49:1976-80.

(8.) World Health Organization. WHO SARS Scientific Research Advisory Committee concludes its first meeting. [cited 2004 May 3]. Available from www.who.int/csr/sars/archive/research/en/

(9.) World Health Organization. WHO SARS International Reference and Verification Laboratory Network: Policy and procedures in the inter-epidemic period. [cited 2004 May 3]. Available from www.who.int/ csr/sars/resources/en/SARSReferenccLab1.pdf

(10.) Emery SL, Erdman DD, Bowen MD, Newton BR, Winchell JM, Meyer RF, et al. Real-time reverse transcription-polymerase chain reaction assay for SARS-associated coronavirus. Emerg Infect Dis. 2004:10:311-6.

(11.) Zhai J, Briese T. Dai E, Wang X, Pang X. Du Z. et al. Real-time polymerase chain reaction for detecting SARS coronavirus. Beijing, 2003. Emerg Infect Dis. 2004;10:300-3.

(12.) Poon LL, Chan KH, Wong OK, Cheung TK, Ng I, Zheng B, et al. Detection of SARS coronavirus in patients with severe acute respiratory syndrome by conventional and real-time quantitative reverse transcription-PCR assays. Clin Chem. 2004;50:67-72.

(13.) World Health Organization. Alert, verification and public health management of SARS in the post-outbreak period. [cited 2004 May 3]. Available from www.who.int/csr/sars/postoutbreak/en/

(14.) World Health Organization. PCR primers for SARS developed by WHO network laboratories. [cited 2004 May 3]. Available from www.who.int/csr/sars/primers/en/

(15.) Niedrig M, Schmitz H, Becket S, Gunther S, ter Meulen J, Meyer H, et al. First international quality assurance study on the rapid detection of viral agents of bioterrorism. J Clin Microbiol. 2004:42:1753-5.

Christian Drosten, * Hans Wilhelm Doerr, ([dagger]) Wilina Lim, ([double dagger]) Klaus Stohr, ([section]) and Matthias Niedrig ([paragraph])

* Bernhard Nocht Institute for Tropical Medicine, Hamburg, Germany; ([dagger]) University of Frankfurt, Frankfurt, Germany; ([double dagger]) Government Virus Unit Hong Kong, Hong Kong, China; ([section]) World Health Organization, Geneva, Switzerland; and ([paragraph]) Robert Koch Institute, Berlin, Germany

Address for correspondence: Matthias Niedrig, Robert Koch Institut, Nordufer 20, 3353 Berlin, Germany; fax: ++49 30-4547-2321/2309: email: [e-mail]