The limitations of point of care testing for pandemic influenza: what clinicians and public health professionals need to know.
As the world prepares for the next influenza pandemic, governments have made significant funding commitments to vaccine development and antiviral stockpiling. While these are essential components to pandemic response, rapid and accurate diagnostic testing remains an often neglected cornerstone of pandemic influenza preparedness. As outlined in Annex C of the Canadian Pandemic Influenza Plan, (1) accurate laboratory testing is essential to:
* identify the earliest Canadian cases of a novel influenza strain;
* support public health surveillance;
* facilitate clinical management in limited circumstances;
* monitor circulating influenza viruses for anti-viral resistance.
Laboratory and epidemiologic data will also identify triggers that will escalate pandemic phases and dictate appropriate response measures. Recently, the advantages and limitations of currently available influenza testing methods have been reviewed. (2) Understanding the limitations of these tests is essential when interpreting results. This manuscript briefly reviews tissue culture, rapid antigen detection, and nucleic acid-based testing. It also discusses the limitations of these methodologies and the options for pandemic testing and planning from the perspective of the front-line clinician and public health official.
The traditional gold standard for the diagnosis of influenza has been virus isolation using tissue culture. Its turnaround time can be from 1 to 10 days, however, early in the pandemic, this technology will be restricted to a very small number of containment level (CL-3) facilities with approved protocols and trained staff able to safely carry out testing procedures.
Nucleic acid testing (NAT)
Nucleic acid testing utilizes sequence-based amplification methods to detect viral RNA. The method most commonly used is reverse transcriptase polymerase chain reaction (RT-PCR) which can be used to detect or subtype influenza A viruses in 4-8 hours. Data show that the increased yield of viral identification by NAT over culture ranges from 8-800% (Table 1). (3-7) This increased sensitivity and improved turnaround time has made NAT the new diagnostic "gold standard". However, NAT requires specialized technical knowledge and equipment and extensive quality assurance, which often restricts their use to larger virology laboratories. Current guidelines suggest that during seasonal influenza, antiviral therapy should be initiated as soon as possible after symptom onset and within 48 hours. (8,9) The ability to provide rapid and accurate diagnosis while the patient is under observation would be ideal, however even the most rapid methods have a turnaround time of several hours.
Despite these disadvantages, there are many features that make NAT the most effective platform for preparing for the next influenza pandemic:
1. In addition to superior sensitivity and specificity, NAT technologies do not require growth of the virus; thus they are safe to perform in all virology laboratories.
2. Although the initial capital equipment expenditure is high:
* The platform can be used to test for numerous pathogens and can be quickly adapted to novel pathogens;
* Many RT-PCR assays can be manufactured "in house", thus reducing cost and assuring availability in a time of international crisis;
* The majority of these reagents can be stored for years, facilitating stockpiling.
3. The increasing use of NAT in routine clinical circumstances will help reduce wastage, as stockpiled supplies can be put into a rotating inventory system so they can be used for other diagnostic work prior to expiration.
Point of care (POC) testing
The currently available near-patient or POC tests use rapid antigen detection technologies that are relatively simple and can generate results in less than 30 minutes; however the accuracy of these tests is suboptimal when compared to RT-PCR.
A recent survey conducted by the Pandemic Influenza Laboratory Preparedness Network (PILPN), found that 72/90 (80%) of Canadian public health and hospital laboratories responding use POC tests in their diagnostic algorithms. Only 6 of 14 POC tests which are currently licensed in Canada were used (Table 2). The vast majority of surveyed labs use them as the primary method of diagnosis.
A PubMed database search for studies published in English and using human specimens revealed five of the approved kits in Canada do not have any published performance data or peer-reviewed comparative studies (Table 2). As outlined in Table 2, performance characteristics vary greatly for these assays. (10-26) Performance depends on the type of specimen tested, the timing of collection, age of the patient, and the skill with which the specimens are collected and the tests performed. (14,16,23,27,28) Generally, studies reporting better performance are done by trained personnel using optimal specimens such as nasopharyngeal aspirates or swabs. In routine clinical practice, however, POC tests are often performed by per sonnel who lack familiarity with test procedures and interpretation skills, using suboptimal specimens such as throat swabs.
For any diagnostic test, understanding performance characteristics in the context of disease prevalence is essential. Table 3 outlines how positive and negative predictive values are influenced by the prevalence of influenza in the community. For diagnosis of seasonal human influenza, the relatively high specificity of most POC tests allows the clinician to be fairly confident in the accuracy of a positive result from a patient presenting with influenza-like illness (ILI) during the influenza season. However, during periods of low prevalence (i.e., summer months), positive results need to be confirmed with more specific methods such as RT-PCR or culture.27,28 The primary limitation of currently available POC tests is poor sensitivity, which translates into an inability to rule out the diagnosis of influenza. This is important in the context of pandemic influenza planning when optimal sensitivity is required to detect the arrival of the pandemic strain in Canada.
The World Health Organization (WHO) does not recommend the use of rapid POC tests for the diagnosis of novel influenza viruses. (27) The U.S. Food and Drug Administration has also issued a cautionary notice for rapid tests because their performance has not been established for influenza A subtypes, other than A/H3N2 and A/H1N1. (28) Although some assays can identify a wide range of influenza A subtypes including avian influenza viruses, (29) they cannot differentiate between a novel influenza virus or seasonal human influenza. In addition, current POC tests have an analytical sensitivity approximately 1000-fold less sensitive than culture for both avian and human virus strains and only detect H5N1 viruses in high concentrations. (30,31) This may partially explain their poor performance in diagnosing human H5N1 infections. Since 1997, only 8 (17%) of 48 patients with RT-PCR or culture-confirmed H5N1 infection could be diagnosed by POC tests. (32)
In addition to the poor sensitivity, there are other features of POC tests which make them suboptimal for pandemic planning:
* Single use;
* Limited shelf-life (one to two years), leading to significant wastage and excessive cost if stockpiled for pandemic preparedness purposes;
* They must be procured as kits that are manufactured abroad, making them potentially unavailable during a time of international crisis.
The development of a POC test that has the performance characteristics of RT-PCR, that is easy to use and interpret, and has a turnaround time of less than 30 minutes would be the optimal influenza diagnostic test. This test would also have to be reasonably priced, have a prolonged shelf life, and ideally, be able to subtype influenza, or at the very least distinguish novel or potential pandemic isolates from seasonal human influenza. The Centers for Disease Control and Prevention is committing significant resources to develop new and improved POC technologies. Additional attention must be paid to development of new technologies and to optimizing current approaches. Moreover, such tests will need to be rapidly evaluated under field conditions in a pandemic setting, likely against a reference standard including molecular testing, to justify their use for clinical or public health decision-making.
Front-line clinicians need to be aware that the positive and negative predictive values of POC tests depend on the prevalence of influenza. Although there may be some utility in using POC tests during seasonal influenza, their sensitivity is poor and they are not recommended for the diagnosis of novel influenza strains. For public health officials and policy-makers, understanding the limitations of these POC tests and the comparative advantages of molecular platforms is essential when trying to decide how to most wisely invest the limited resources available for diagnostic testing that supports provincial and territorial pandemic preparedness plans. To develop and optimize rapid and clinically relevant POC tests for potential pandemic influenza strains, a significant research investment towards laboratory diagnostics is required. By investing in molecular technologies in preparation for the next pandemic, Canada will not only improve its diagnostic and surveillance capabilities for seasonal influenza, but will create the infrastructure to assess new diagnostic tests under field conditions and rise to the challenge of novel pathogens. Until improvements in POC tests are developed and realized, PILPN believes that the best option for pandemic influenza preparation is the enhancement of nucleic acid-based testing capabilities across Canada (Figure 1).
Received: April 26, 2008
Accepted: November 11, 2008
(1.) The Canadian Pandemic Influenza Plan for the Health Sector. Annex C: Pandemic Influenza Laboratory Preparedness Plan, 2006. Available online at: http://www.phac-aspc.gc.ca/cpip-pclcpi/ann-c-eng.php (Accessed April 15, 2008).
(2.) Petric M, Comanor L, Petti A. The role of the laboratory in diagnosis of influenza during seasonal epidemics and potential pandemics. J Infect Dis 2006;194(Suppl 2):S98-S110.
(3.) Gharabaghi F, Tellier R, Cheung R, Collins C, Broukhanski G, Drews SJ, Richardson SE. Comparison of a commercial qualitative real-time RT-PCR kit with direct immunofluorescence assay (DFA) and cell culture for detection of influenza A and B in children. J Clin Virol 2008;42:190-93.
(4.) van de Pol AC, van Loon AM, Wolfs TF, Jansen NJ, Nijhuis M, Breteler EK, et al. Increased detection of respiratory syncytial virus, influenza viruses, parainfluenza viruses, and adenoviruses with real-time PCR in samples from patients with respiratory symptoms. J Clin Microbiol 2007;45:2260-62.
(5.) Zitterkopf NL, Leekha S, Espy MJ, Wood CM, Sampathkumar P, Smith TF. Relevance of influenza A virus detection by PCR, shell vial assay, and tube cell culture to rapid reporting procedures. J Clin Microbiol 2006;44(9):3366-67.
(6.) Espy MJ, Uhl JR, Sloan LM, Buckwalter SP, Jones MF, Vetter EA, et al. Realtime PCR in clinical microbiology: Applications for routine laboratory testing. Clin Microbiol Rev 2006;19(1):165-256.
(7.) Ellis JS, Fleming DM, Zambon MC. Multiplex reverse transcription-PCR for surveillance of influenza A and B viruses in England and Wales in 1995 and 1996. J Clin Microbiol 1997;35:2076-82.
(8.) Fiore AE, Shay DK, Broder K, Iskander JK, Uyeki TM, Mootrey G, et al. Centers for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP). Prevention and control of influenza: Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 2008;57:1-60.
(9.) Allen UD, Aoki FY, Stiver HG. The use of antiviral drugs for influenza: Recommended guidelines for practitioners. Can J Infect Dis Med Microbiol 2006;17(5):273-84.
(10.) Weinberg A, Walker ML. Evaluation of three immunoassay kits for rapid detection of influenza virus A and B. Clin Diagn Lab Immunol 2005;12(3):36770.
(11.) Hurt AC, Alexander R, Hibbert J, Deed N, Barr IG. Performance of six influenza rapid tests in detecting human influenza in clinical specimens. J Clin Virol 2007;39(2):132-35.
(12.) Waner JL, Todd SJ, Shalaby H, Murphy P, Wall LV. Comparison of Directigen FLU-A with viral isolation and direct immunofluorescence for the rapid detection and identification of influenza A virus. J Clin Microbiol 1991;29(3):47982.
(13.) Johnston SL, Bloy H. Evaluation of a rapid enzyme immunoassay for detection of influenza A virus. J Clin Microbiol 1993;31(1):142-43.
(14.) Kaiser L, Briones MS, Hayden FG . Performance of virus isolation and Directigen Flu A to detect influenza A virus in experimental human infection. J Clin Virol 1999;14(3):191-97.
(15.) Gooskens J, Swaan CM, Claas EC, Kroes AC. Rapid molecular detection of influenza outbreaks in nursing homes. J Clin Virol 2008;41(1):7-12.
(16.) Reina J, Padilla E, Alonso F, Ruiz De Gopegui E, Munar M, Mari M. Evaluation of a new dot blot enzyme immunoassay (Directigen flu A+B) for simultaneous and differential detection of influenza A and B virus antigens from respiratory samples. J Clin Microbiol 2002;40(9):3515-17.
(17.) Hamilton MS, Abel DM, Ballam YJ, Otto MK, Nickell AF, Pence LM, et al. Clinical evaluation of the ZstatFlu-II test: A chemiluminescent rapid diagnostic test for influenza virus. J Clin Microbiol 2002;40(7):2331-34.
(18.) Landry ML, Cohen S, Ferguson D. Comparison of Binax NOW and Directigen for rapid detection of influenza A and B. J Clin Virol 2004;31(2):113-15.
(19.) Booth S, Baleriola C, Rawlinson WD. Comparison of two rapid influenza A/B test kits with reference methods showing high specificity and sensitivity for influenza A infection. J Med Virol 2006;78(5):619-22.
(20.) Rahman M, Kieke BA, Vandermause MF, Mitchell PD, Greenlee RT, Belongia EA. Performance of Directigen flu A+B enzyme immunoassay and direct fluorescent assay for detection of influenza infection during the 2004-2005 season. Diagn Microbiol Infect Dis 2007;58(4):413-18.
(21.) Hindiyeh M, Goulding C, Morgan H, Kenyon B, Langer J, Fox L, et al. Evaluation of BioStar FLU OIA assay for rapid detection of influenza A and B viruses in respiratory specimens. J Clin Virol 2000;17(2):119-26.
(22.) Rodriguez WJ, Schwartz RH, Thorne MM. Evaluation of diagnostic tests for influenza in a pediatric practice. Pediatr Infect Dis J 2002;21(3):193-96.
(23.) Covalciuc KA, Webb KH, Carlson CA. Comparison of four clinical specimen types for detection of influenza A and B viruses by optical immunoassay (FLU OIA test) and cell culture methods. J Clin Microbiol 1999;37(12):3971-74.
(24.) Rashid H, Shafi S, Haworth E, El Bashir H, Ali KA, Memish ZA, Booy R. Value of rapid testing for influenza among Hajj pilgrims. Travel Med Infect Dis 2007;5(5):310-13.
(25.) Weitzel T, Schnabel E, Dieckmann S, Borner U, Schweiger B. Evaluation of a new point-of-care test for influenza A and B virus in travelers with influenzalike symptoms. Clin Microbiol Infect 2007;13(7):665-69.
(26.) Cruz AT, Cazacu AC, Greer JM, Demmler GJ. Rapid assays for the diagnosis of influenza A and B viruses in patients evaluated at a large tertiary care children's hospital during two consecutive winter seasons. J Clin Virol 2008;41(2):143-47.
(27.) WHO recommendations on the use of rapid testing for influenza diagnosis. 2005. Available online at: http://www.who.int/csr/disease/avian_influenza/ guidelines/rapid_testing/en/index.html (Accessed April 15, 2008).
(28.) U.S. Food and Drug Administration, Office of In Vitro Diagnostic Device Evaluation and Safety. 2007. Cautions in Using Rapid Tests for Detecting Influenza A Viruses. Available online at: http://www.fda.gov/cdrh/oivd/tips/ rapidflu.html (Accessed April 15, 2008).
(29.) Chan KH, Maldeis N, Pope W, Yup A, Ozinskas A, Gill J, et al. Evaluation of the Directigen FluA+B test for rapid diagnosis of influenza virus type A and B infections. J Clin Microbiol 2002;40(5):1675-80.
(30.) Fedorko DP, Nelson NA, McAuliffe JM, Subbarao K. Performance of rapid tests for detection of avian influenza A virus types H5N1 and H9N2. J Clin Microbiol 2006;44(4):1596-97.
(31.) Chan KH, Lam SY, Puthavathana P, Nguyen TD, Long HT, Pang CM, et al. Comparative analytical sensitivities of six rapid influenza A antigen detection test kits for detection of influenza A subtypes H1N1, H3N2 and H5N1. J Clin Virol 2007;38(2):169-71.
(32.) Abdel-Ghafar AN, Chotpitayasunondh T, Gao Z, Hayden FG, Nguyen DH, de Jong MD, et al. Writing Committee of the Second World Health Organization Consultation on Clinical Aspects of Human Infection with Avian Influenza A (H5N1) Virus. Update on avian influenza A (H5N1) virus infection in humans. N Engl J Med 2008;358(3):261-73.
Todd F. Hatchette, MD  and Members of the Pandemic Influenza Laboratory Preparedness Network (PILPN) *
[1.] Division of Microbiology, QE II Health Science Centre, Halifax, NS * Members of PILPN: Nathalie Bastien, Jody Berry, Tim F. Booth, Max Chernesky, Michel Couillard, Steven Drews, Anthony Ebsworth, Margaret Fearon, Kevin Fonseca, Julie Fox, Jean-Nicolas Gagnon, Steven Guercio, Greg Horsman, Cathy Jorowski, Theodore Kuschak, Yan Li, Anna Majury, Martin Petric, Sam Ratnam, Marek Smieja, Paul Van Caeseele.
Correspondence: Dr. Todd F. Hatchette, Division of Microbiology, QE II Health Science Centre, 5788 University Avenue, Room 315, Halifax, NS B3H 1V8, Tel: 902473-6885, Fax: 902-473-7971, E-mail: email@example.com
Table 1. Summary of Literature Demonstrating the Increase in Yield Using RT-PCR Compared to Cell Culture for the Detection of Influenza Virus in Clinical Specimens Reference Molecular Method Conventional Influenza-positive Method Specimens Detected by Culture Gharagabhi Real-time RT-PCR DFA + cell 155/169 et al., [Artus influenza culture (RhMK (92%) 2008 (3) RT-PCR Kit MDCK) (QIagen, Hamburg)] Van de Pol Real-time RT-PCR DFA in 8/18 (44%) in et al., (in house) children + children 2007 (4) cell culture 2/18 (11%) in LLC-MK2, adults (using PCR R-HELA, as the reference HEp-2) method) Zitterkoft Real-time RT-PCR R-MIX * Day 0 - 43/50 et al., (in house) (86%) 2006 (5) Day 2 - 22/50 (44%) Espy et al., Real time RT-PCR RMIX 49/557 2006 (6) (in house) Ellis et al., Conventional DFA + cell 200/619 1997 (7) RT-PCR culture (MDCK, RhMK) Reference Influenza-positive Increased Specimens Yield Detected by NAT Gharagabhi 167/169 8% et al., (99% 2008 (3) Van de Pol 18/18 125% et al., 2007 (4) 18/18 800% Zitterkoft Day 0 - 50/50 16% et al., (100% 2006 (5) Day 2 - 34/50 55% (68% Espy et al., 92/557 88% 2006 (6) Ellis et al., 246/619 23% 1997 (7) * If the initial samples were positive, subsequent samples were collected at 48 hours. Abbreviations: DFA (direct fluorescence antibody test); RT-PCR (reverse transcriptase polymerase chain reaction); RhMK (Rhesus monkey kidney cells); MDCK (Madin-Darby Canine Kidney Cells); LLC-MK2 (rhesus monkey kidney cell line), R-HELA (cervical adenocarcinoma); HEp-2 (cervical adenocarcinoma); R-MIX (co-culture of mink lung and MDCK cell line). Table 2. Performance Characteristics of Commercially Available Point of Care Test Approved in Canada Name of Kit Manufacturer Percentage Pub Med of Canadian (English Laboratories language/ Using Kit Human) * (n=90) DIRECTIGEN Becton Dickinson 56 15 FLU A + B and Company BINAX NOW Binax Inc. 18 14 INFLUENZA A & B or NOW FLU A NOW FLU B TEST KIT XPECT FLU A/B Remel 6 1 DIRECTIGEN Becton Dickinson 4 17 FLU A TEST KIT and Company QUICKVUE Quidel Corporation 3 12 INFLUENZA A+B TEST FLU OIA TEST KIT Thermo Biostar Inc. 0 9 BD DIRECTIGEN Becton Dickinson 0 2 EZ FLU A+B and Company IMMUNOCARD STAT Meridian Bioscience 0 2 FLU A & B Inc. ACTIM INFLUENZA Medix Biochemica 0 0 A & B TEST OY AB QUICK S Innovatek Medical 0 0 INFLU A/B TEST Inc. CLEARVIEW Wampole Laboratories 0 0 FLU A/B TEST Inc. INFLU A RESPI Coris Bioconcept 0 0 STRIP INFLU A&B Coris Bioconcept 0 0 TEST KITS Name of Kit Sensitivity Specificity References (%) (%) DIRECTIGEN 22-96 93-100 15-17 FLU A + B BINAX NOW 53-80 93 -100 18-20 INFLUENZA A & B or NOW FLU A NOW FLU B TEST KIT XPECT FLU A/B 48 99 26 DIRECTIGEN 62-100 80-100 12-14 FLU A TEST KIT QUICKVUE 22-95 76-100 21,22,24 INFLUENZA A+B TEST FLU OIA TEST KIT 54-93 73-97 21-23 BD DIRECTIGEN 39-69 94-99 10-11 EZ FLU A+B IMMUNOCARD STAT 67-80 98-99 19,25 FLU A & B ACTIM INFLUENZA N/A N/A N/A A & B TEST QUICK S N/A N/A N/A INFLU A/B TEST CLEARVIEW N/A N/A N/A FLU A/B TEST INFLU A RESPI N/A N/A N/A STRIP INFLU A&B N/A N/A N/A TEST KITS * Indicates the number of articles published examining the performance characteristics of the POC of interest on human specimens in English as listed on PubMed using the search words "influenza" and the name of the specific POC of interest. Note: Data from June 2006. Source: Medical Devices Bureau, Health Canada. Table 3. Predictive Values Depend on the Prevalence of Influenza in the Community Off Season Height of Influenza Season Prevalence of Influenza 1,000/100,000 25,000/100,000 (1%) (25%) Individuals affected 1000 25,000 Individuals not affected 99,000 75,000 Kit sensitivity 80% 80% Kit specificity 97% 97% True positives 800 20,000 False positives 2970 2250 True negatives 96,030 72,750 False negatives 200 5000 Positive predictive value (PPV) 21.2% 89.9% Negative predictive value (NPV) 99.8% 93.6%