Update on the College of American Pathologists Experience With High-Risk Human Papillomavirus Proficiency Testing for Cytology.
Moriarty et al (2) reported on the College of American Pathologists' first (3) years (2008-2010) of experience with the CHPV program. They identified statistically significant differences in the overall performance of various sample media but also noticed an upward trend in the proficiency of laboratories performing HPV testing during the study period. In this study, we report the performance of laboratories participating in the CHPV program during the 4-year period immediately after that initial report.
MATERIALS AND METHODS
Responses of laboratories participating in the CHPV program from 2011 through 2014 were summarized and analyzed. There were 3 mailings a year with 5 challenges/mailing. Participating laboratories could choose from 1 of 4 modules based on the medium most commonly used for HPV testing in their laboratories: Digene (CHPVD), SurePath (CHPVK), ThinPrep (CHPVM), or mixed. The mixed module comprised all 3 different media types. A correct response was defined as one that matched the specimen's target value as determined by preconfigured cell lines containing HPV. For this analysis, the mixed module samples were reclassified into their appropriate medium types.
To analyze which factors were associated with performance, a nonlinear mixed model was fit using the method and specimen-target type. The interaction terms for these factors were also included in the model. A significance level of .05 was used for this analysis. All analyses were performed with SAS 9.3 (SAS Institute, Cary, North Carolina).
There were 818 laboratories that participated in the program at least once during the 4 years of the study. Overall participation in the CHPV program increased during the study period, with 502 laboratories participating in 2011, 571 laboratories participating in 2012, 630 laboratories participating in 2013, and 695 laboratories participating in 2014. Not all laboratories participated in all challenges during the 4-year study period.
In terms of testing platform, 323 laboratories used Hybrid Capture 2 (HC2; Qiagen, Gaithersburg, Maryland), 279 laboratories used Cervista (Hologic, Bedford, Massachusetts), 187 laboratories used Aptima (Hologic), 183 laboratories used cobas (Roche, Pleasanton, California), 35 laboratories used laboratory-developed tests, 176 laboratories used unspecified commercial kits (assays that are usually not available in the United States and may represent responses from international subscribers), and 80 laboratories used other (noncommercial) methods (a mixture of methods that were either not specified or illegible). Several laboratories used more than one platform. The Figure shows trends in using different testing platforms for the CHPV program during the study period.
There was participant consensus (defined as >80% agreement with the target value) for all specimens in the defined time frame. The overall concordance of participant responses with the target value was 98.4% (38 280 of 38 892). The concordance rates by year were 98.5% (8098 of 8218) in 2011, 98.2% (8977 of 9140) in 2012, 98.1% (9981 of 10 177) in 2013, and 98.8% (11 224 of 11 357) in 2014. Program year was not included in the statistical analyses because there was no significant trend over time (Cochran-Armitage trend test; P = .09).
The mixed-model results are displayed in Tables 1 through 3. Testing performance was statistically equivalent for samples with negative and positive results, and there was no significant difference in overall testing performance among the 3 media types (ThinPrep, SurePath, or Digene) (Table 1). However, when performance of the 3 media types was compared within each category of testing platform (Table 2), only HC2, laboratory-developed tests, and other (noncommercial) methods were shown to perform equally well in all 3 media, whereas there were significant performance differences for Cervista and unspecified commercial kits, depending on the medium used: Cervista underperformed in SurePath and Digene media compared with ThinPrep (both P < .001), and unspecified commercial kits underperformed in ThinPrep compared with SurePath (P < .001). Because detailed information regarding the specific commercial kit used was not provided in this latter category, it was not possible to perform more detailed analyses on the unspecified commercial kit group. Medium performance comparisons could not be made for Aptima because of the few responses in SurePath and Digene media or for cobas because SurePath and Digene media were not offered as proficiency testing options during the study period. Although the survey includes testing for HPV genotyping performance, statistical analyses were not performed because of material issues for HPV genotype 18 and limited specimens for HPV genotype 31.
When performance of testing platforms was compared while controlling for medium type (Table 3), all 4 US Food and Drug Administration (FDA)-approved methods--HC2, Cervista, Aptima, and cobas--outperformed laboratory-developed tests, unspecified commercial kits, and other (noncommercial) methods in ThinPrep medium (P < .001). In SurePath medium, HC2 performed better than unspecified commercial kits (P = .01) and other (noncommercial) methods (P = .02), whereas Cervista underperformed compared with all other methods that had a sufficient number of cases (P < .001). In Digene medium, HC2 performed better than Cervista and laboratory-developed tests (P < .001); Cervista also underperformed compared with unspecified commercial kits (P < .001) and laboratory-developed tests (P = .004).
Participation in the College of American Pathologists CHPV proficiency testing program continued to increase during the 4-year study period, from 502 laboratories in 2011 to 695 laboratories in 2014, reflecting the widespread use of HPV testing in cervical cancer screening and management of cervical dysplasia. Compared with the previous study, (2) which showed a peak of 414 laboratories participating in 2010, the number of laboratories participating in 2014 represents a 68% increase in laboratory participation, which is concordant with results of a separate survey published by the College of American Pathologists Cytopathology Committee showing an increase in the annual volume of HPV testing and an associated shift toward in-house testing. (3)
Currently, 4 HPV testing platforms are approved by the FDA for triage of a Papanicolaou test diagnosis of atypical squamous cells of undetermined significance (reflex testing) and as an adjunct to the Papanicolaou test (cotesting) in women 30 years and older: HC2 was approved for these purposes in 2003, Cervista in 2009, and Aptima and cobas in 2011 (cobas was further approved for primary HPV screening of women 25 years and older in 2014). (4,5) Of these 4 tests, Aptima is an messenger RNA-based test, whereas the other 3 tests detect HPV DNA. (6) In the current study, there was an overall decrease in the number of laboratories using HC2 and Cervista and a rapid increase in laboratories using cobas. We could not document trends in Aptima usage because Aptima was offered in the CHPV program as a separately identified testing platform beginning in 2014, the last year of the current study period.
In terms of testing media, ThinPrep is the only medium that is currently approved for use on all 4 FDA-approved testing platforms. Digene Specimen Transport Medium is also approved by the FDA for use on HC2. At the time of this writing, SurePath medium is not approved by the FDA for HPV testing on any platform, and laboratories offering HPV testing of SurePath specimens must document laboratory validation for off-label use.
During the first 3 years of the CHPV program, participating laboratories demonstrated proficiency with all tested platforms and media, but significant differences were noted in the performance of various testing platforms. (2) Similarly, in this study, laboratories demonstrated proficiency in HPV testing during years 4 through 7, regardless of survey year and target response, and although the testing platforms available during the initial and current study period were different, we also observed significant variation in the performance of certain HPV testing platforms in various testing media. The HC2, laboratory-developed tests, and other (noncommercial) methods performed equally well in all 3 media. The robustness of HC2 may be due, at least in part, to laboratories having had the most time to validate its off-label use. Several other studies have also showed statistically equivalent performance of HC2 in different media. (7-9) On the other hand, there were significant medium-specific performance differences for Cervista and unspecified commercial kits, with Cervista significantly underperforming in SurePath and Digene media compared with ThinPrep and unspecified commercial kits underperforming in ThinPrep compared with SurePath. To our knowledge, there are no published studies evaluating the accuracy of off-label use of Cervista in media other than ThinPrep. For example, one study (10) comparing the performance of HC2 and Cervista in 875 split samples demonstrated slightly lower sensitivity and specificity for Cervista in ThinPrep medium compared with HC2, but these differences did not reach statistical significance, and performance in media other than ThinPrep was not measured. In the current study, medium-specific performance comparisons could not be made for the more-recent Aptima and cobas platforms because of the few laboratories using Aptima in SurePath and Digene and because cobas was not being offered for these 2 media during the study period.
During the first 3 years of the program, Moriarty et al (2) identified significant differences in the overall performance of different media as well, with SurePath medium underperforming compared with Digene and ThinPrep, although SurePath performance improved during the course of that study. In the current study, overall testing performance among the 3 media types, regardless of testing platform, was no longer statistically different, and all 3 media performed equally well. However, when the performance of the 3 different media was subcategorized by testing platform, all 4 FDA-approved methods--HC2, Cervista, Aptima, and cobas--outperformed laboratory-developed tests, unspecified commercial kits, and other (noncommercial) methods in ThinPrep medium. With SurePath and Digene, Cervista significantly underperformed compared with several other methods that had a sufficient number of cases. These observations may be relevant in light of the ongoing discussions about potential FDA oversight of laboratory-developed tests.
In conclusion, this report provides an update on the performance of laboratories in the College of American Pathologists CHPV proficiency testing program. The number of laboratories participating in the CHPV program has continuously increased since it began in 2008. Although laboratories demonstrated acceptable proficiency in various combinations of testing platform and testing medium, statistically significant differences in performance were noted. Specifically, HC2 demonstrated robust performance regardless of testing medium, whereas Cervista underperformed in media other than ThinPrep. More data are needed to compare the performance of Aptima and cobas in media other than ThinPrep.
Mohiedean Ghofrani, MD, MBA; Chengquan Zhao, MD; Diane D. Davey, MD; Fang Fan, MD, PhD; Mujtaba Husain, MD; Alice Laser, MD; Idris T. Ocal, MD; Rulong Z. Shen, MD; Kelly Goodrich, CT(ASCP); Rhona J. Souers, MS; Barbara A. Crothers, DO
Accepted for publication May 27, 2016.
From the Department of Pathology, PeaceHealth Laboratories, Vancouver, Washington (Dr Ghofrani); the Department of Pathology, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania (Dr Zhao);University of Central Florida College of Medicine, Orlando (Drs Davey and Husain);the Department of Pathology, University of Kansas Medical Center, Kansas City (Dr Fan); the Department of Pathology, Northwell Health, New Hyde Park, New York (Dr Laser); the Department of Pathology, Mayo Clinic, Scottsdale, Arizona (Dr Ocal); Ohio State University Wexner Medical Center, Columbus (Dr Shen); the Cytopathology Resource Committee (Ms Goodrich) and Biostatistics (Ms Souers), College of American Pathologists, Northfield, Illinois; and the Department of Pathology, Walter Reed National Military Medical Center, Bethesda, Maryland (Dr Crothers).
The authors have no relevant financial interest in the products or companies described in this article.
Corresponding author: Mohiedean Ghofrani, MD, MBA, Department of Pathology, PeaceHealth Laboratories, PO Box 873097, Vancouver, WA 98687-3097 (email: email@example.com).
(1.) Department of Health and Human Services. Clinical laboratory improvement amendments of 1988: final rule--standard;virology. Fed Regist. 1992; 57(40):7001-7186. Codified at 42 CFR [section]493.831.
(2.) Moriarty AT, Bentz JS, Winkler B, et al. The College of American Pathologists' first 3 years' experience with high-risk human papillomavirus proficiency testing for cytology and other laboratories. Arch Pathol Lab Med. 2013;137(5):606-609.
(3.) Zhao C, Moriarty AT, Ghofrani M, et al. Human papillomavirus testing and reporting rates in 2012: results of a College of American Pathologists national survey. Arch Pathol Lab Med. 2015;139(6):757-761.
(4.) Kurtycz DF, Smith M, He R, Miyazaki K, Shalkham J. Comparison of methods trial for high-risk HPV. Diagn Cytopathol. 2010;38(2):104-108.
(5.) Binnicker MJ, Pritt BS, Duresko BJ, et al. Comparative evaluation of three commercial systems for detection of high-risk human papillomavirus in cervical and vaginal ThinPrep PreservCyt samples and correlation with biopsy results. / Clin Microbiol. 2014;52(10):3763-3768.
(6.) Arbyn M, Snijders PJ, Meijer CJ, et al. Which high-risk HPV assays fulfill criteria for use in primary cervical cancer screening? Clin Microbiol Infect. 2015; 21(9):817-826.
(7.) Ko V, Tambouret RH, Kuebler DL, Black-Schaffer WS, Wilbur DC. Human papillomavirus testing using hybrid capture II with SurePath collection: initial evaluation and longitudinal data provide clinical validation for this method. Cancer. 2006;108(6):468-474.
(8.) Zhou F, Pulinthanathu R, Elgert P, Cangiarella J, Simsir A. Sensitivity of high-risk HPV Hybrid Capture II (hrHPV HC2) test using SurePath[TM] specimens in the prediction of cervical high-grade squamous lesions. Diagn Cytopathol. 2015; 43(5):381-387.
(9.) Levi AW, Bernstein JI, Hui P, Duch K, Schofield K, Chhieng DC. A comparison of the Roche Cobas HPV Test with the Hybrid Capture 2 Test for the detection of high-risk human papillomavirus genotypes. Arch Pathol Lab Med. 2016;140(2):153-157.
(10.) Alameda F, Garrote L, Mojal S, et al. Cervista HPV HR test for cervical cancer screening: a comparative study in the Catalonian population. Arch Pathol Lab Med. 2015;139(2):241-244.
Please Note: Illustration(s) are not available due to copyright restrictions.
Caption: Human papillomavirus for cytology laboratories testing volumes by platform.
Table 1. Nonlinear, Mixed-Model Results of Proficiency Testing Module and Target Response Total Concordance Responses, to Target, Factor No. No. (%) P Value Module (a) .51 ThinPrep 25 742 25 409 (98.7) SurePath 7717 7505 (97.3) Digene 5433 5366 (98.8) Specimen target .48 Positive 23 354 22 982 (98.4) Negative 15 538 15 298 (98.5) (a) ThinPrep (Hologic, Bedford, Massachusetts); SurePath (Becton, Dickinson and Company, Franklin Lakes, New Jersey); Digene (Qiagen, Gaithersburg, Maryland). Table 2. Nonlinear, Mixed-Model Results Comparing Medium Performance by Testing Platform Total Concordance Responses, to Target, Factor (a) No. No. (%) P Value (b) Method x media <.001 Qiagen Digene HC2 NS ThinPrep 7735 7677 (99.3) SurePath 2924 2905 (99.4) Digene 4662 4626 (99.2) Hologic Cervista + ThinPrep 8842 8761 (99.1) SurePath 1901 1751 (92.1) Digene 140 125 (89.3) Roche Cobas 4800 Not tested ThinPrep 3883 3847 (99.1) SurePath Not offered Digene Not offered Gen-Probe Aptima Not tested ThinPrep 2183 2161 (99.0) SurePath 26 26 (100.0) Digene 8 7 (87.5) Commercial kit ++ ThinPrep 1833 1749 (95.4) SurePath 1972 1945 (98.6) Digene 234 230 (98.3) LDT NS ThinPrep 605 575 (95.0) SurePath 383 377 (98.4) Digene 272 263 (96.7) Other NS ThinPrep 642 621 (96.7) SurePath 454 446 (98.2) Digene 113 113 (100.0) Abbreviations: LDT, laboratory-developed test; NS, No significant differences among media. (a) ThinPrep (Hologic, Bedford, Massachusetts); SurePath (Becton, Dickinson and Company, Franklin Lakes, New Jersey); Digene (Qiagen, Gaithersburg, Maryland); Digene Hybrid Capture 2 (HC2; Qiagen, Gaithersburg, Maryland); Cervista (Hologic, Bedford, Massachusetts); cobas 4800 (Roche, Pleasanton, California); Gen-Probe Aptima (Hologic, Bedford, Massachusetts). (b) +, Hologic Cervista: ThinPrep-SurePath (P < .001), ThinPrep-Digene (P < .001); ++, commercial kit: ThinPrep-SurePath (P < .001); Qiagen Digene: ThinPrep-SurePath (P = .60), ThinPrep-Digene (P = .88), SurePath-Digene (P = .54); LDT: ThinPrep-SurePath (P = .09), ThinPrep-Digene (P = .28), SurePath-Digene (P = .15); Other: ThinPrep-SurePath (P = .13), ThinPrep-Digene (P = .99), SurePath-Digene (P = .99). Table 3. Nonlinear, Mixed-Model Results Comparing Platform Performance by Testing Medium Total Concordance Responses, to Target, Factor (a) No. No. (%) P Value ThinPrep <.001 Qiagen Digene HC2 7735 7677 (99.3) Hologic Cervista 8842 8761 (99.1) Roche Cobas 4800 3883 3847 (99.1) Gen-Probe Aptima 2183 2161 (99.0) Commercial kit 1833 1749 (95.4) LDT 605 575 (95.0) Other 642 621 (96.7) SurePath <.001 Qiagen Digene HC2 2924 2905 (99.4) Hologic Cervista 1901 1751 (92.1) Roche Cobas 4800 Not offered Gen-Probe Aptima 26 26 (100.0) Commercial kit 1972 1945 (98.6) LDT 383 377 (98.4) Other 454 446 (98.2) Digene <.001 Qiagen Digene HC2 4662 4626 (99.2) Hologic Cervista 140 125 (89.3) Roche Cobas 4800 Not offered Gen-Probe Aptima 8 7 (87.5) Commercial kit 234 230 (98.3) LDT 272 263 (96.7) Other 113 113 (100.0) Abbreviation: LDT, laboratory-developed test. (a) ThinPrep (Hologic, Bedford, Massachusetts); SurePath (Becton, Dickinson and Company, Franklin Lakes, New Jersey); Digene (Qiagen, Gaithersburg, Maryland); Digene Hybrid Capture 2 (HC2; Qiagen, Gaithersburg, Maryland); Cervista (Hologic, Bedford, Massachusetts); cobas 4800 (Roche, Pleasanton, California); Gen-Probe Aptima (Hologic, Bedford, Massachusetts).
|Printer friendly Cite/link Email Feedback|
|Author:||Ghofrani, Mohiedean; Zhao, Chengquan; Davey, Diane D.; Fan, Fang; Husain, Mujtaba; Laser, Alice; Oca|
|Publication:||Archives of Pathology & Laboratory Medicine|
|Date:||Dec 1, 2016|
|Previous Article:||The Pathology of Chronic Obstructive Pulmonary Disease: Progress in the 20th and 21st Centuries.|
|Next Article:||Development and Validation of an ERCC1 Immunohistochemistry Assay for Solid Tumors.|