The limitations of point of care testing for pandemic influenza: what clinicians and public health professionals need to know.

As the world prepares for the next influenza pandemic, governments have made significant funding commitments to vaccine development and antiviral stockpiling. While these are essential components to pandemic response, rapid and accurate diagnostic testing remains an often neglected cornerstone of pandemic influenza preparedness. As outlined in Annex C of the Canadian Pandemic Influenza Plan, (1) accurate laboratory testing is essential to:

* identify the earliest Canadian cases of a novel influenza strain;

* support public health surveillance;

* facilitate clinical management in limited circumstances;

* monitor circulating influenza viruses for anti-viral resistance.

Laboratory and epidemiologic data will also identify triggers that will escalate pandemic phases and dictate appropriate response measures. Recently, the advantages and limitations of currently available influenza testing methods have been reviewed. (2) Understanding the limitations of these tests is essential when interpreting results. This manuscript briefly reviews tissue culture, rapid antigen detection, and nucleic acid-based testing. It also discusses the limitations of these methodologies and the options for pandemic testing and planning from the perspective of the front-line clinician and public health official.

Tissue culture

The traditional gold standard for the diagnosis of influenza has been virus isolation using tissue culture. Its turnaround time can be from 1 to 10 days, however, early in the pandemic, this technology will be restricted to a very small number of containment level (CL-3) facilities with approved protocols and trained staff able to safely carry out testing procedures.

Nucleic acid testing (NAT)

Nucleic acid testing utilizes sequence-based amplification methods to detect viral RNA. The method most commonly used is reverse transcriptase polymerase chain reaction (RT-PCR) which can be used to detect or subtype influenza A viruses in 4-8 hours. Data show that the increased yield of viral identification by NAT over culture ranges from 8-800% (Table 1). (3-7) This increased sensitivity and improved turnaround time has made NAT the new diagnostic "gold standard". However, NAT requires specialized technical knowledge and equipment and extensive quality assurance, which often restricts their use to larger virology laboratories. Current guidelines suggest that during seasonal influenza, antiviral therapy should be initiated as soon as possible after symptom onset and within 48 hours. (8,9) The ability to provide rapid and accurate diagnosis while the patient is under observation would be ideal, however even the most rapid methods have a turnaround time of several hours.

Despite these disadvantages, there are many features that make NAT the most effective platform for preparing for the next influenza pandemic:

1. In addition to superior sensitivity and specificity, NAT technologies do not require growth of the virus; thus they are safe to perform in all virology laboratories.

2. Although the initial capital equipment expenditure is high:

* The platform can be used to test for numerous pathogens and can be quickly adapted to novel pathogens;

* Many RT-PCR assays can be manufactured "in house", thus reducing cost and assuring availability in a time of international crisis;

* The majority of these reagents can be stored for years, facilitating stockpiling.

3. The increasing use of NAT in routine clinical circumstances will help reduce wastage, as stockpiled supplies can be put into a rotating inventory system so they can be used for other diagnostic work prior to expiration.

Point of care (POC) testing

The currently available near-patient or POC tests use rapid antigen detection technologies that are relatively simple and can generate results in less than 30 minutes; however the accuracy of these tests is suboptimal when compared to RT-PCR.

A recent survey conducted by the Pandemic Influenza Laboratory Preparedness Network (PILPN), found that 72/90 (80%) of Canadian public health and hospital laboratories responding use POC tests in their diagnostic algorithms. Only 6 of 14 POC tests which are currently licensed in Canada were used (Table 2). The vast majority of surveyed labs use them as the primary method of diagnosis.

A PubMed database search for studies published in English and using human specimens revealed five of the approved kits in Canada do not have any published performance data or peer-reviewed comparative studies (Table 2). As outlined in Table 2, performance characteristics vary greatly for these assays. (10-26) Performance depends on the type of specimen tested, the timing of collection, age of the patient, and the skill with which the specimens are collected and the tests performed. (14,16,23,27,28) Generally, studies reporting better performance are done by trained personnel using optimal specimens such as nasopharyngeal aspirates or swabs. In routine clinical practice, however, POC tests are often performed by per sonnel who lack familiarity with test procedures and interpretation skills, using suboptimal specimens such as throat swabs.

For any diagnostic test, understanding performance characteristics in the context of disease prevalence is essential. Table 3 outlines how positive and negative predictive values are influenced by the prevalence of influenza in the community. For diagnosis of seasonal human influenza, the relatively high specificity of most POC tests allows the clinician to be fairly confident in the accuracy of a positive result from a patient presenting with influenza-like illness (ILI) during the influenza season. However, during periods of low prevalence (i.e., summer months), positive results need to be confirmed with more specific methods such as RT-PCR or culture.27,28 The primary limitation of currently available POC tests is poor sensitivity, which translates into an inability to rule out the diagnosis of influenza. This is important in the context of pandemic influenza planning when optimal sensitivity is required to detect the arrival of the pandemic strain in Canada.

The World Health Organization (WHO) does not recommend the use of rapid POC tests for the diagnosis of novel influenza viruses. (27) The U.S. Food and Drug Administration has also issued a cautionary notice for rapid tests because their performance has not been established for influenza A subtypes, other than A/H3N2 and A/H1N1. (28) Although some assays can identify a wide range of influenza A subtypes including avian influenza viruses, (29) they cannot differentiate between a novel influenza virus or seasonal human influenza. In addition, current POC tests have an analytical sensitivity approximately 1000-fold less sensitive than culture for both avian and human virus strains and only detect H5N1 viruses in high concentrations. (30,31) This may partially explain their poor performance in diagnosing human H5N1 infections. Since 1997, only 8 (17%) of 48 patients with RT-PCR or culture-confirmed H5N1 infection could be diagnosed by POC tests. (32)

In addition to the poor sensitivity, there are other features of POC tests which make them suboptimal for pandemic planning:

* Single use;

* Limited shelf-life (one to two years), leading to significant wastage and excessive cost if stockpiled for pandemic preparedness purposes;

* They must be procured as kits that are manufactured abroad, making them potentially unavailable during a time of international crisis.

The development of a POC test that has the performance characteristics of RT-PCR, that is easy to use and interpret, and has a turnaround time of less than 30 minutes would be the optimal influenza diagnostic test. This test would also have to be reasonably priced, have a prolonged shelf life, and ideally, be able to subtype influenza, or at the very least distinguish novel or potential pandemic isolates from seasonal human influenza. The Centers for Disease Control and Prevention is committing significant resources to develop new and improved POC technologies. Additional attention must be paid to development of new technologies and to optimizing current approaches. Moreover, such tests will need to be rapidly evaluated under field conditions in a pandemic setting, likely against a reference standard including molecular testing, to justify their use for clinical or public health decision-making.

CONCLUSION

Front-line clinicians need to be aware that the positive and negative predictive values of POC tests depend on the prevalence of influenza. Although there may be some utility in using POC tests during seasonal influenza, their sensitivity is poor and they are not recommended for the diagnosis of novel influenza strains. For public health officials and policy-makers, understanding the limitations of these POC tests and the comparative advantages of molecular platforms is essential when trying to decide how to most wisely invest the limited resources available for diagnostic testing that supports provincial and territorial pandemic preparedness plans. To develop and optimize rapid and clinically relevant POC tests for potential pandemic influenza strains, a significant research investment towards laboratory diagnostics is required. By investing in molecular technologies in preparation for the next pandemic, Canada will not only improve its diagnostic and surveillance capabilities for seasonal influenza, but will create the infrastructure to assess new diagnostic tests under field conditions and rise to the challenge of novel pathogens. Until improvements in POC tests are developed and realized, PILPN believes that the best option for pandemic influenza preparation is the enhancement of nucleic acid-based testing capabilities across Canada (Figure 1).

Received: April 26, 2008

Accepted: November 11, 2008

REFERENCES

(1.) The Canadian Pandemic Influenza Plan for the Health Sector. Annex C: Pandemic Influenza Laboratory Preparedness Plan, 2006. Available online at: http://www.phac-aspc.gc.ca/cpip-pclcpi/ann-c-eng.php (Accessed April 15, 2008).

(2.) Petric M, Comanor L, Petti A. The role of the laboratory in diagnosis of influenza during seasonal epidemics and potential pandemics. J Infect Dis 2006;194(Suppl 2):S98-S110.

(3.) Gharabaghi F, Tellier R, Cheung R, Collins C, Broukhanski G, Drews SJ, Richardson SE. Comparison of a commercial qualitative real-time RT-PCR kit with direct immunofluorescence assay (DFA) and cell culture for detection of influenza A and B in children. J Clin Virol 2008;42:190-93.

(4.) van de Pol AC, van Loon AM, Wolfs TF, Jansen NJ, Nijhuis M, Breteler EK, et al. Increased detection of respiratory syncytial virus, influenza viruses, parainfluenza viruses, and adenoviruses with real-time PCR in samples from patients with respiratory symptoms. J Clin Microbiol 2007;45:2260-62.

(5.) Zitterkopf NL, Leekha S, Espy MJ, Wood CM, Sampathkumar P, Smith TF. Relevance of influenza A virus detection by PCR, shell vial assay, and tube cell culture to rapid reporting procedures. J Clin Microbiol 2006;44(9):3366-67.

(6.) Espy MJ, Uhl JR, Sloan LM, Buckwalter SP, Jones MF, Vetter EA, et al. Realtime PCR in clinical microbiology: Applications for routine laboratory testing. Clin Microbiol Rev 2006;19(1):165-256.

(7.) Ellis JS, Fleming DM, Zambon MC. Multiplex reverse transcription-PCR for surveillance of influenza A and B viruses in England and Wales in 1995 and 1996. J Clin Microbiol 1997;35:2076-82.

(8.) Fiore AE, Shay DK, Broder K, Iskander JK, Uyeki TM, Mootrey G, et al. Centers for Disease Control and Prevention (CDC); Advisory Committee on Immunization Practices (ACIP). Prevention and control of influenza: Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 2008;57:1-60.

(9.) Allen UD, Aoki FY, Stiver HG. The use of antiviral drugs for influenza: Recommended guidelines for practitioners. Can J Infect Dis Med Microbiol 2006;17(5):273-84.

(10.) Weinberg A, Walker ML. Evaluation of three immunoassay kits for rapid detection of influenza virus A and B. Clin Diagn Lab Immunol 2005;12(3):36770.

(11.) Hurt AC, Alexander R, Hibbert J, Deed N, Barr IG. Performance of six influenza rapid tests in detecting human influenza in clinical specimens. J Clin Virol 2007;39(2):132-35.

(12.) Waner JL, Todd SJ, Shalaby H, Murphy P, Wall LV. Comparison of Directigen FLU-A with viral isolation and direct immunofluorescence for the rapid detection and identification of influenza A virus. J Clin Microbiol 1991;29(3):47982.

(13.) Johnston SL, Bloy H. Evaluation of a rapid enzyme immunoassay for detection of influenza A virus. J Clin Microbiol 1993;31(1):142-43.

(14.) Kaiser L, Briones MS, Hayden FG . Performance of virus isolation and Directigen Flu A to detect influenza A virus in experimental human infection. J Clin Virol 1999;14(3):191-97.

(15.) Gooskens J, Swaan CM, Claas EC, Kroes AC. Rapid molecular detection of influenza outbreaks in nursing homes. J Clin Virol 2008;41(1):7-12.

(16.) Reina J, Padilla E, Alonso F, Ruiz De Gopegui E, Munar M, Mari M. Evaluation of a new dot blot enzyme immunoassay (Directigen flu A+B) for simultaneous and differential detection of influenza A and B virus antigens from respiratory samples. J Clin Microbiol 2002;40(9):3515-17.

(17.) Hamilton MS, Abel DM, Ballam YJ, Otto MK, Nickell AF, Pence LM, et al. Clinical evaluation of the ZstatFlu-II test: A chemiluminescent rapid diagnostic test for influenza virus. J Clin Microbiol 2002;40(7):2331-34.

(18.) Landry ML, Cohen S, Ferguson D. Comparison of Binax NOW and Directigen for rapid detection of influenza A and B. J Clin Virol 2004;31(2):113-15.

(19.) Booth S, Baleriola C, Rawlinson WD. Comparison of two rapid influenza A/B test kits with reference methods showing high specificity and sensitivity for influenza A infection. J Med Virol 2006;78(5):619-22.

(20.) Rahman M, Kieke BA, Vandermause MF, Mitchell PD, Greenlee RT, Belongia EA. Performance of Directigen flu A+B enzyme immunoassay and direct fluorescent assay for detection of influenza infection during the 2004-2005 season. Diagn Microbiol Infect Dis 2007;58(4):413-18.

(21.) Hindiyeh M, Goulding C, Morgan H, Kenyon B, Langer J, Fox L, et al. Evaluation of BioStar FLU OIA assay for rapid detection of influenza A and B viruses in respiratory specimens. J Clin Virol 2000;17(2):119-26.

(22.) Rodriguez WJ, Schwartz RH, Thorne MM. Evaluation of diagnostic tests for influenza in a pediatric practice. Pediatr Infect Dis J 2002;21(3):193-96.

(23.) Covalciuc KA, Webb KH, Carlson CA. Comparison of four clinical specimen types for detection of influenza A and B viruses by optical immunoassay (FLU OIA test) and cell culture methods. J Clin Microbiol 1999;37(12):3971-74.

(24.) Rashid H, Shafi S, Haworth E, El Bashir H, Ali KA, Memish ZA, Booy R. Value of rapid testing for influenza among Hajj pilgrims. Travel Med Infect Dis 2007;5(5):310-13.

(25.) Weitzel T, Schnabel E, Dieckmann S, Borner U, Schweiger B. Evaluation of a new point-of-care test for influenza A and B virus in travelers with influenzalike symptoms. Clin Microbiol Infect 2007;13(7):665-69.

(26.) Cruz AT, Cazacu AC, Greer JM, Demmler GJ. Rapid assays for the diagnosis of influenza A and B viruses in patients evaluated at a large tertiary care children's hospital during two consecutive winter seasons. J Clin Virol 2008;41(2):143-47.

(27.) WHO recommendations on the use of rapid testing for influenza diagnosis. 2005. Available online at: http://www.who.int/csr/disease/avian_influenza/ guidelines/rapid_testing/en/index.html (Accessed April 15, 2008).

(28.) U.S. Food and Drug Administration, Office of In Vitro Diagnostic Device Evaluation and Safety. 2007. Cautions in Using Rapid Tests for Detecting Influenza A Viruses. Available online at: http://www.fda.gov/cdrh/oivd/tips/ rapidflu.html (Accessed April 15, 2008).

(29.) Chan KH, Maldeis N, Pope W, Yup A, Ozinskas A, Gill J, et al. Evaluation of the Directigen FluA+B test for rapid diagnosis of influenza virus type A and B infections. J Clin Microbiol 2002;40(5):1675-80.

(30.) Fedorko DP, Nelson NA, McAuliffe JM, Subbarao K. Performance of rapid tests for detection of avian influenza A virus types H5N1 and H9N2. J Clin Microbiol 2006;44(4):1596-97.

(31.) Chan KH, Lam SY, Puthavathana P, Nguyen TD, Long HT, Pang CM, et al. Comparative analytical sensitivities of six rapid influenza A antigen detection test kits for detection of influenza A subtypes H1N1, H3N2 and H5N1. J Clin Virol 2007;38(2):169-71.

(32.) Abdel-Ghafar AN, Chotpitayasunondh T, Gao Z, Hayden FG, Nguyen DH, de Jong MD, et al. Writing Committee of the Second World Health Organization Consultation on Clinical Aspects of Human Infection with Avian Influenza A (H5N1) Virus. Update on avian influenza A (H5N1) virus infection in humans. N Engl J Med 2008;358(3):261-73.

Todd F. Hatchette, MD [1] and Members of the Pandemic Influenza Laboratory Preparedness Network (PILPN) *

Author Affiliations

[1.] Division of Microbiology, QE II Health Science Centre, Halifax, NS * Members of PILPN: Nathalie Bastien, Jody Berry, Tim F. Booth, Max Chernesky, Michel Couillard, Steven Drews, Anthony Ebsworth, Margaret Fearon, Kevin Fonseca, Julie Fox, Jean-Nicolas Gagnon, Steven Guercio, Greg Horsman, Cathy Jorowski, Theodore Kuschak, Yan Li, Anna Majury, Martin Petric, Sam Ratnam, Marek Smieja, Paul Van Caeseele.

Correspondence: Dr. Todd F. Hatchette, Division of Microbiology, QE II Health Science Centre, 5788 University Avenue, Room 315, Halifax, NS B3H 1V8, Tel: [phone], Fax: [phone], E-mail: [e-mail]

Table 1. Summary of Literature Demonstrating the Increase in Yield
Using RT-PCR Compared to Cell Culture for the Detection of
Influenza Virus in Clinical Specimens

Reference       Molecular Method    Conventional    Influenza-positive
                                    Method          Specimens
                                                    Detected by
                                                    Culture

Gharagabhi      Real-time RT-PCR    DFA + cell      155/169
et al.,         [Artus influenza    culture (RhMK   (92%)
 2008 (3)       RT-PCR Kit          MDCK)
                (QIagen,
                Hamburg)]

Van de Pol      Real-time RT-PCR    DFA in          8/18 (44%) in
et al.,         (in house)          children +      children
2007 (4)                            cell culture    2/18 (11%) in
                                    LLC-MK2,        adults (using PCR
                                    R-HELA,         as the reference
                                    HEp-2)          method)

Zitterkoft      Real-time RT-PCR    R-MIX           * Day 0 - 43/50
et al.,         (in house)                          (86%)
2006 (5)                                            Day 2 - 22/50
                                                    (44%)

Espy et al.,    Real time RT-PCR    RMIX            49/557
2006 (6)        (in house)

Ellis et al.,   Conventional        DFA + cell      200/619
1997 (7)        RT-PCR              culture
                                    (MDCK, RhMK)

Reference       Influenza-positive   Increased
                Specimens            Yield
                Detected by
                NAT

Gharagabhi      167/169              8%
et al.,         (99%
2008 (3)

Van de Pol      18/18                125%
et al.,
2007 (4)        18/18                800%

Zitterkoft      Day 0 - 50/50        16%
et al.,         (100%
2006 (5)        Day 2 - 34/50        55%
                (68%

Espy et al.,    92/557               88%
2006 (6)

Ellis et al.,   246/619              23%
1997 (7)

* If the initial samples were positive, subsequent samples were
collected at 48 hours. Abbreviations: DFA (direct fluorescence antibody
test); RT-PCR (reverse transcriptase polymerase chain reaction); RhMK
(Rhesus monkey kidney cells); MDCK (Madin-Darby Canine Kidney Cells);
LLC-MK2 (rhesus monkey kidney cell line), R-HELA (cervical
adenocarcinoma); HEp-2 (cervical adenocarcinoma); R-MIX (co-culture of
mink lung and MDCK cell line).

Table 2. Performance Characteristics of Commercially
Available Point of Care Test Approved in Canada

Name of Kit          Manufacturer           Percentage     Pub Med
                                            of Canadian    (English
                                            Laboratories   language/
                                            Using Kit      Human) *
                                            (n=90)

DIRECTIGEN           Becton Dickinson       56             15
FLU A + B            and Company
BINAX NOW            Binax Inc.             18             14
INFLUENZA A & B
or NOW FLU A
NOW FLU B TEST KIT
XPECT FLU A/B        Remel                  6              1
DIRECTIGEN           Becton Dickinson       4              17
FLU A TEST KIT       and Company
QUICKVUE             Quidel Corporation     3              12
INFLUENZA A+B TEST
FLU OIA TEST KIT     Thermo Biostar Inc.    0              9
BD DIRECTIGEN        Becton Dickinson       0              2
EZ FLU A+B           and Company
IMMUNOCARD STAT      Meridian Bioscience    0              2
FLU A & B            Inc.
ACTIM INFLUENZA      Medix Biochemica       0              0
A & B TEST           OY AB
QUICK S              Innovatek Medical      0              0
INFLU A/B TEST       Inc.
CLEARVIEW            Wampole Laboratories   0              0
FLU A/B TEST         Inc.
INFLU A RESPI        Coris Bioconcept       0              0
STRIP
INFLU A&B            Coris Bioconcept       0              0
TEST KITS

Name of Kit          Sensitivity            Specificity    References
                     (%)                    (%)

DIRECTIGEN           22-96                  93-100         15-17
FLU A + B
BINAX NOW            53-80                  93 -100        18-20
INFLUENZA A & B
or NOW FLU A
NOW FLU B TEST KIT
XPECT FLU A/B        48                     99             26
DIRECTIGEN           62-100                 80-100         12-14
FLU A TEST KIT
QUICKVUE             22-95                  76-100         21,22,24
INFLUENZA A+B TEST
FLU OIA TEST KIT     54-93                  73-97          21-23
BD DIRECTIGEN        39-69                  94-99          10-11
EZ FLU A+B
IMMUNOCARD STAT      67-80                  98-99          19,25
FLU A & B
ACTIM INFLUENZA      N/A                    N/A            N/A
A & B TEST
QUICK S              N/A                    N/A            N/A
INFLU A/B TEST
CLEARVIEW            N/A                    N/A            N/A
FLU A/B TEST
INFLU A RESPI        N/A                    N/A            N/A
STRIP
INFLU A&B            N/A                    N/A            N/A
TEST KITS

* Indicates the number of articles published examining
the performance characteristics of the POC of interest on human
specimens in English
as listed on PubMed using the search words "influenza" and the name
of the specific POC of interest.

Note: Data from June 2006.

Source: Medical Devices Bureau, Health Canada.

Table 3. Predictive Values Depend on the Prevalence of Influenza in the
Community

                                    Off Season        Height of
                                                   Influenza Season

Prevalence of Influenza           1,000/100,000     25,000/100,000
                                       (1%)             (25%)
Individuals affected                   1000             25,000
Individuals not affected              99,000            75,000
Kit sensitivity                        80%               80%
Kit specificity                        97%               97%
True positives                         800              20,000
False positives                        2970              2250
True negatives                        96,030            72,750
False negatives                        200               5000
Positive predictive value (PPV)       21.2%             89.9%
Negative predictive value (NPV)       99.8%             93.6%