Printer Friendly

Analytic Validation of Immunohistochemistry Assays: New Benchmark Data From a Survey of 1085 Laboratories.

In 2015, the College of American Pathologists (CAP) conducted a survey of laboratory practices to examine the current state of immunohistochemical assay (IHC) validation procedures, assess general awareness of the CAP's 2014 evidence-based laboratory practice guideline (LPG) on IHC validation, (1) and determine what changes, if any, occurred following its publication. The survey was developed as part of a cooperative agreement between the CAP and the US Centers for Disease Control and Prevention (CDC). As part of the collaboration between CAP and CDC, a Guideline Metrics Expert Panel was formed to oversee surveys regarding awareness and implementation of new and existing guidelines.

Also see p. 1247.

The 2015 survey included many of the same questions as those used in a 2010 survey of validation practices by Hardy et al, (2) but also included new questions to collect data specifically related to the IHC validation guideline recommendations. As the IHC validation LPG was developed, gaps were identified in the literature when data were not available to describe current IHC validation practices; this included the issue of what circumstances required assay revalidation. Analysis of the survey results led to a before and after comparison of laboratory practices, which is addressed in a companion paper by Fitzgibbons et al, (3) as well as new benchmark data from current laboratory practices that are described in this article. This article covers 3 broad aspects of IHC validation: laboratory policies on initial assay validation; revalidation practices; and the laboratory's actual practice during its most recent validation.

METHODS

In the latter half of 2015, the CAP distributed a survey of IHC validation practices and procedures to laboratories enrolled in 1 or more of the following CAP proficiency testing (PT) programs: the CAP/National Society for Histotechnology HistoQIP (HQIP-B), the Performance Improvement Program in Surgical Pathology (PIP-C), and the HER2 Immunohistochemistry Program (HER2-B). Not all subscribers of these programs perform IHC staining. Laboratories that interpret IHC slides that have been stained in another laboratory also participate in the programs and were therefore included in the survey distribution, but their responses were excluded from the analysis. The same survey was also mailed to a selection of laboratories identified by Centers for Medicare & Medicaid Services (CMS) Part B reimbursement claims that indicated they perform IHC testing; these CMS-identified laboratories were not enrolled in any of the abovementioned CAP PT programs. Laboratory accreditation status was not a factor in distribution of the survey or analysis of the results. The 2010 survey was sent to subscribers of CAP's HER2 program. Thus, some of the laboratories included in the current survey had also participated in the earlier survey.

The survey development, analysis, and this publication were supported by the Cooperative Agreement No. NU47OE000057-04, funded by the US CDC. The cooperative agreement with CDC required preapproval of the survey instrument by the US Office of Management and Budget (OMB No. 0920-1067).

The survey contained 21 questions that encompassed specific questions on validation policies and practices, guideline awareness and adoption status, and demographic factors. The survey questions specifically excluded HER2 and hormone receptor assays, as separate guidelines for those markers had already been established. (4-7) Survey answers included yes/no/unsure format, multiple choice, and numerical responses. In addition to questions included in the earlier survey, the 2015 survey included a number of new questions prompted during the development of the IHC validation LPG.

Since many of the laboratories participated in more than 1 of the CAP PT programs, duplicate surveys were received and only the single most complete survey from each laboratory was included in the study. Results were also excluded from 74 laboratories that returned incomplete surveys. For some questions, the results were stratified by laboratory size (measured by surgical pathology accession volume) and marker type.

Differences between marker types for qualitative questions were analyzed by using [chi square] and Fisher exact tests, and the Wilcoxon rank sum test was used for quantitative test differences. A significance level of .05 was used for the analysis. The survey results were summarized and analyzed with SAS 9.3 (SAS Institute Inc, Cary, North Carolina).

RESULTS

Of the 3512 survey mailings, a total of 1624 completed surveys were available for analysis; this included 1539 of 3064 responses (50%) from laboratories participating in the CAP PT programs and 85 of 448 responses (19%) from laboratories not enrolled in CAP PT. Most of the responses received were from US laboratories (domestic laboratories) but 181 non-US laboratories (international laboratories) also participated in the survey. Analysis was conducted on surveys from the 1085 respondents who indicated they perform IHC staining.

A range of personnel completed the survey with the most common being IHC supervisors (44.8%; 671), followed by MD or DO immunohistochemical laboratory directors (14.2%; 148), histotechnologists (11.4%; 119), staff pathologists (10.7%; 112), and department chair/laboratory medical directors (8.1%; 84); all others accounted for 10.8% (113 of the 1043) of respondents. Table 1 includes distribution volumes of the total number of antibodies, new antibodies introduced in 2014, and 2014 surgical pathology accessions from laboratories participating in the survey. The median number of IHC antibodies used by 1036 laboratories was 78 (5th-95th percentile range, 15-200); the median number of new antibodies introduced in 2014 by 1022 laboratories was 3 (5th-95th percentile range, 0-20); and the median surgical pathology accessions for 983 laboratories was 15 000 (5th-95th percentile range, 2800-79 568).

General Practices Reported for Initial Validation Procedures

Table 2 details the extent of written validation procedures for new IHC assays as reported by survey participants. More than 95% (809 of 844) reported always documenting the results of validation. Sixty percent (648 of 1078) reported having separate written procedures for predictive and non-predictive assays, while 34% (366) did not. A lower proportion of laboratories had separate written procedures for verifying validation of unmodified US Food and Drug Administration (FDA)-approved assays (54.2%; 404 of 746) and laboratory-developed or laboratory-modified tests (42.7%; 220 of 515). A new question in this survey asked whether laboratories have separate validation procedures for decalcified and cytologic specimens, and approximately half of laboratories reported having validation procedures specific for these specimen types. For cytologic specimens, there was a significant shift in frequency from 37% (179 of 486) of laboratories responding affirmatively in 2010 to 47% (328 of 697) in 2015, but only for nonpredictive markers; there was no change for predictive markers over that same period (42% [126 of 298] in 2010 versus 40% [275 of 697] in 2015).

Laboratories also reported minimum requirements for initial validation of assays, as summarized in Tables 3 and 4. A minimum number of validation cases was specified by most laboratories (85.6% [720 of 838] for non-predictive markers; 76.0% [584 of 768] for predictive markers) and most specified a minimum required number of positive and negative cases. However, only slightly more than half stipulated minimum concordance rates for either assay type. As reported by 714 laboratories, the median number of cases required for nonpredictive assays was 20 (5th-95th percentile range, 5-40), which is stated in the 2014 LPG for this assay type. The median number of cases required for predictive assays was 25 (5th-95th percentile range, 8-50; n = 579), which is below the LPG recommendation of 40. Table 4 also demonstrates the percentage of respondents that adopted the recommendation for initial IHC validations in terms of concordance rates ([greater than or equal to] 90% for positive concordance, negative concordance, and overall concordance) as specified in their procedures (93.2%, 93.7%, and 96.9%, respectively, for nonpredictive markers and 94.9%, 98.1%, and 97.0%, respectively, for predictive markers). In addition (data not shown in Table 4) roughly one-third of laboratories (32.2%, 35.4%, and 30.5% for positive, negative, and overall concordance, respectively) required 100% concordance for nonpredictive markers and a smaller proportion of laboratories (27.8%, 29.9%, and 23.8%, for positive, negative and overall concordance, respectively) required 100% concordance for predictive markers.

Testing Condition Changes Requiring Revalidation of Previously Validated Assays

For an existing, previously validated IHC assay, the LPG recommends having a written procedure that specifies when to reassess assay performance if there is a change in testing conditions. (1) Having written procedures for assay revalidation was reported by 61.4% (607 of 988) of laboratories for both nonpredictive and predictive markers; 3.7% (37) reported having procedures for predictive markers only and 8.4% (83) for nonpredictive markers only. Sixteen percent (158) had no such procedure, and 10.4% (103) were unsure. Table 5 details which changes in testing conditions necessitate assay revalidation. The most common reasons included in the procedure for revalidating nonpredictive assays were change in antibody clone (92.0%; 613 of 666), followed by change in antigen detection system (86.4%; 572 of 662), change in antigen retrieval method (86.2%; 568 of 659), introduction of a new antibody lot (83.6%; 559 of 669), and change in antibody vendor (83.2%; 553 of 665). Roughly two-thirds of laboratories specified minimum numbers of cases needed for assay revalidation for specific changes (Table 6). Change in antibody clone required the most cases, with a median of 20 (5th-95th percentile range, 2-20; n = 388), while introduction of a new antibody lot required the fewest, with a median of 2 (5th-95th percentile range, 1-20; n = 342).

Table 7 lists the changes in test conditions requiring revalidation of previously validated predictive markers. The most common conditions specified in the procedure for predictive markers were change in antibody clone (86.0%; 533 of 620), change in antigen detection system (82.6%; 511 of 619), change in antigen retrieval method (81.7%; 505 of 618), introduction of a new antibody lot (78.3%; 486 of 621), and change in antibody vendor (77.5%; 478 of 617). Table 8 summarizes the minimum numbers of cases needed for assay revalidation for specific changes, with most requirements listed as roughly double what was required for nonpredictive markers.

Initial Validations Reported From Most Recently Introduced Assays

Respondents also reported data from their most recently introduced IHC assay. The chronology of the most recent validations from 1019 laboratories was as follows: 4.1% (42) before 2013; 6.0% (61) in 2013; 21.6% (220) in 2014; 57.0% (581) in 2015; and 11.3% (115) were unsure. A validation study for the most recently introduced IHC assay was reported by 87.7% (902 of 1029) of laboratories; 3.5% (36) reported not having performed a validation study and 8.8% (91) were unsure. Table 9 demonstrates the methods laboratories used for their most recent validations. Correlation with morphology and expected results was the most common method for both 725 nonpredictive and 101 predictive markers (61.1% [443] and 46.5% [47], respectively). Direct comparison to previously validated assay results, performed either in-house or at an outside laboratory, was more common for predictive markers (50.4% [51] versus 34.7% [251]). The method validation differences between nonpredictive and predictive assay validation were statistically significant (P = .002, Fisher exact test). Table 10 presents the number of cases used in the most recent assay initial validation. The median case number for nonpredictive markers was 20 (5th-95th percentile range, 4-40; n = 685) and the median case number for predictive markers was 31 (5th-95th percentile range, 5-65; n = 97). The case number differences were statistically significant (P < .001, Wilcoxon rank sum test).

DISCUSSION

In conducting this survey, we gathered current practice data regarding laboratory policies and procedures on IHC assay validation. Information regarding changes in laboratory policies since 2010 is discussed in the companion article. (3) This article describes issues covered in the 2015 survey that were not addressed in the earlier survey. (2) Nearly 96% of laboratories documented validation procedures and 85% had written policies, 60% have separate written procedures for predictive and nonpredictive assays with 54.2% specifically addressing unmodified FDA-approved assays. Only 42.7% of laboratories also specifically address laboratory-developed or laboratory-modified tests. While this demonstrates increased attention to analytic validation of antibodies, some confusion may still remain regarding the need to have written procedures that distinguish between predictive and nonpredictive markers as well as FDA-approved/cleared tests and laboratory-developed tests. While it is not necessary to have separate procedures, differences in assay types must be addressed.

The responses to specific questions surrounding validation of IHC procedures for cytologic specimens were notable. In response to the question about having written procedures and specifications for validation of IHC tests for cytologic specimens, significantly more laboratories responded affirmatively as compared with 2010, but only for nonpredictive markers; there was no change for predictive markers over that same period. In view of the increasing utilization of predictive immunohistochemical markers for informing treatment options and the increasing reliance on cytologic specimens, one might anticipate growing motivation to validate these assays for cytologic specimens. The issue is somewhat obscured by what specimens are considered "cytologic." Practically, in some departments, core biopsies are classified as cytology specimens despite being preserved and processed identically to tissue specimens. The protocols for "cell block" preparations are variable, encompassing cellular samples entrapped in clotted blood or centrifugally concentrated cellular specimens sometimes entrapped in a fibrin clot, all subsequently processed and embedded like tissue blocks. The matter is further complicated by the inclusion of various smear preparation methods (direct versus liquid based), alcohol fixation versus air-drying, and different preservatives and fixatives. It cannot be assumed that a procedure established and validated for one type of specimen would be equally valid for any other. Again, the issue becomes particularly important in the context of predictive markers.

The 2014 IHC validation LPG identified a number of changes in testing conditions that necessitate a reassessment of assay performance. When there is a change in antibody dilution, antibody vendor (same clone), or incubation or retrieval times, the guideline recommends confirming assay performance with at least 2 known positive and 2 known negative cases. If there is a change in fixative type, antigen retrieval method, antigen detection system, tissue processing or testing equipment, environmental conditions of testing, or laboratory water supply, the laboratory medical director is responsible for determining how many predictive and nonpredictive markers and how many positive and negative cases to test. Last, when an antibody clone is changed, the guideline recommends a full revalidation (equivalent to initial analytic validation). At the time of guideline development, there were insufficient data and evidence to support a defined recommendation and therefore the numbers offered to confirm performance were based on expert opinion and feedback from the open comment period. In this article, we present actual numbers of cases used to confirm performance for changes in testing condition including 6 situations not previously considered.

For predictive and nonpredictive antibodies, the median number of cases specified for revalidation met or exceeded the guideline statements in all scenarios, with the exception of a change in antibody clone for predictive markers. While the guideline recommends confirming predictive marker performance with 40 validation cases, the median number of reported cases was 25. Laboratories tended to test a higher number of cases when there was a change in antibody vendor for predictive markers (median = 10). Similar trends were identified for predictive markers when there was a change in antigen detection system, fixative type, or testing equipment. Although the guideline did not specify a minimum number of cases for these scenarios, laboratories reported using a substantially higher number of cases for predictive markers versus nonpredictive markers. These findings suggest that respondents generally recognize the increased clinical significance of predictive markers and design revalidation procedures commensurate with the increased importance of predictive assays.

The 2014 IHC LPG suggested methods used to validate tissues. In the current survey, 61.0% and 46.5% of laboratories correlated the new test's result with the morphology and expected results for nonpredictive and predictive assays, respectively. Seventeen percent and 26.7% of laboratories compared the new test's results with another laboratory for nonpredictive and predictive assays, respectively. Beyond HER2 assays, little was known regarding which methods were used to validate antibody assays. It is significant that a higher proportion of laboratories compare to another laboratory in the case of a predictive marker. This may be a more trusted method and implies greater rigor. This method is, however, dependent on the accuracy of the reference laboratory.

One-third (539) of laboratories responding to the 2015 survey indicated that they do not perform immunohistochemical staining. Pathologists in these laboratories interpret IHC slides that have been stained in another laboratory. This is a common practice, but its extent has not been well documented. It is important to recognize that the 2014 LPG focused exclusively on the technical aspects of immunohistochemical validation procedures and not on interpretation.

While uniform, quality technical performance is an essential foundation for overall test performance, it does not, in itself, address interpretive elements. The need for appropriate and uniform interpretation of IHC assays, often in a quantitative manner, will only increase with the continued introduction of predictive biomarkers linked to specific therapies and appropriate education thereof.

In summary, this article offers benchmark data on current procedures and practices of IHC validation assays that were previously unknown. This information may help laboratories understand how other laboratories address issues of revalidation and also may help shape further refinement of LPG recommendations on IHC validation.

Please Note: Illustration(s) are not available due to copyright restrictions.

This work was supported by the Cooperative Agreement No. 1U47OE000057 from the Centers for Disease Control and Prevention (CDC).

References

(1.) Fitzgibbons PL, Bradley LA, Fatheree LA, et al. Principles of analytic validation of immunohistochemical assays: Guideline from the CAP Pathology and Laboratory Quality Center. Arch Pathol Lab Med. 2014; 138(11):1432-1443.

(2.) Hardy LB, Fitzgibbons PL, Goldsmith JD, et al. Immunohistochemistry validation procedures and practices: a College of American Pathologists survey of 727 laboratories. Arch Pathol Lab Med. 2013; 137(1):19-25.

(3.) Fitzgibbons PL, Goldsmith JD, Souers RJ, et al. Analytic validation of immunohistochemical assays: a comparison of laboratory practices before and after introduction of an evidence-based guideline. Arch Pathol Lab Med. 2017; 141:1247-1254.

(4.) Wolff AC, Hammond MEH, Schwartz JN, et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. Arch Pathol Lab Med. 2007; 131(1):18-43.

(5.) Wolff AC, Hammond MEH, Hicks DG, et al. Recommendations for human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology/College of American Pathologists clinical practice guideline update. Arch Pathol Lab Med. 2014; 138(2):241-256.

(6.) Hammond MEH, Hayes DF, Dowsett M, et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for immunohistochemical testing of estrogen and progesterone receptors in breast cancer. Arch Pathol Lab Med. 2010; 134(6):907-922.

(7.) Fitzgibbons PL, Murphy DA, Hammond MEH, Allred DC, Valenstein PN. Recommendations for validating estrogen and progesterone receptor immunohistochemistry assays. Arch Pathol Lab Med. 2010; 134(6):930-935.

Lauren N. Stuart, MD, MBA; Keith E. Volmar, MD; Jan A. Nowak, MD, PhD; Lisa A. Fatheree, BS, SCT(ASCP); Rhona J. Souers, MS; Patrick L. Fitzgibbons, MD; Jeffrey D. Goldsmith, MD; J. Rex Astles, PhD; Raouf E. Nakhleh, MD

Accepted for publication December 6, 2016.

Published as an Early Online Release May 30, 2017.

From the Department of Pathology, Treasure Coast Pathology, East Ocean, Florida (Dr Stuart); the Department of Pathology, UNC REX Healthcare, Raleigh, North Carolina (Dr Volmar); the Department of Pathology, Roswell Park Cancer Institute, Buffalo, New York (Dr Nowak); Pathology and Laboratory Quality Center (Ms Fatheree) and Biostatistics Department (Ms Souers), College of American Pathologists, Northfield, Illinois; the Department of Pathology, St. Jude Medical Center, Fullerton, California (Dr Fitzgibbons); the Department of Pathology, Boston Children's Hospital, Boston, Massachusetts (Dr Goldsmith); Office of Laboratory Systems Development, Division of Laboratory Systems, US Centers for Disease Control & Prevention, Atlanta, Georgia (Dr Astles); and the Department of Pathology, Mayo Clinic Jacksonville, Jacksonville, Florida (Dr Nakhleh).

Dr Goldsmith is a member of the advisory board for Roche Diagnostics Corporation. Ms Fatheree is the Principle Investigator on grant OE13-1304 CDC "Improving the Impact of Laboratory Practice Guidelines: A New Paradigm for Metrics." The other authors have no relevant financial interest in the products or companies described in this article.

The manuscript was written by a subgroup of the College of American Pathologists Guideline Metric Expert Panel. Presented in part as an abstract at the annual meeting of the United States and Canadian Academy of Pathology; March 4-10, 2017; San Antonio, Texas.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Reprints: Patrick L. Fitzgibbons, MD, Department of Pathology, St. Jude Medical Center, 101 E Valencia Mesa Dr, Fullerton, CA 92835 (email: Patrick.Fitzgibbons@stjoe.org).
Table 1. Assay and Surgical Pathology Accession Volumes

                                 Percentile Distribution

                                  n     5th   10th   25th

Total No. of antibodies          1036     15     25     48
  used in IHC laboratory
Total No. of new antibodies     1022      0      0      1
  introduced in 2014
Total No. of 2014                983   2800   4700   8294
  surgical pathology
  accessions

                                   Percentile Distribution

                                Median    75th     90th     95th

Total No. of antibodies              78      120      166      200
  used in IHC laboratory
Total No. of new antibodies          3        6       11       20
  introduced in 2014
Total No. of 2014               15 000   30 315   60 000   79 568
  surgical pathology
  accessions

Abbreviation: IHC, immunohistochemistry.

Table 2. Written Procedures for Initial Validation of
Immunohistochemistry (IHC) Assays

                                            No.   Percentage

Laboratory documents validations and        844
verifications of IHC assays

  Yes, always                               809         95.9
  Yes, sometimes                             19          2.3
  No                                          6          0.7
  Unsure                                     10          1.2

Laboratory has separate written             1078
procedures for validation of IHC
predictive and nonpredictive markers

  Yes                                       648         60.1
  No                                        366         34.0
  Unsure                                     64          5.9

Laboratory has a written procedure          1077
outlining the steps needed for analytic
validation of new IHC assays

  Yes, for both predictive and              742         68.9
     nonpredictive markers
  Yes, for predictive markers only           53          4.9
  Yes, for nonpredictive markers only       124         11.5
  No                                        101          9.4
  Unsure                                     57          5.3

Written procedure includes specification    746
for verifying unmodified FDA-approved
assays

Yes   No   Unsure   Not applicable          404         54.2
                                            222         29.8
                                            120         16.1
                                            162

Written procedure includes specification    515
for validation of LDTor LMTassays

Yes, for both predictive and                175         34.0
nonpredictive LDTs or LMTs   Yes, for        18          3.5
predictive LDTs or LMTs only   Yes, for      27          5.2
nonpredictive LDTs or LMTs only   No        154         29.9
Unsure   Not applicable                     141         27.4
                                            389

Written procedure includes                  697
specification(s) for validating IHC
tests performed on cytologic specimens

Yes, for both predictive and                254         36.4
nonpredictive markers   Yes, for             21          3.0
predictive markers only   Yes, for           74         10.6
nonpredictive markers only   No   Unsure    320         45.9
Not applicable                               28          4.0
                                            219

Written procedure includes                  785
specification(s) for validating IHC
tests performed on decalcified specimens

  Yes, for both predictive and              248         31.6
    nonpredictive markers
  Yes, for predictive markers only           38          4.8
  Yes, for nonpredictive markers only        77          9.8
  No                                        387         49.3
  Unsure                                     35          4.5
  Not applicable                            112

Table 3. Initial Assay Validation: What
Elements Laboratories Specify for
Validation Set Requirements

                                        Total No.      Yes
                                        Responses    No. (%)
Nonpredictive marker
assay specifications
  Minimum No. of cases                     838      720 (85.9)
  No. of positive and                      838      673 (80.3)
  negative cases
  Minimum positive and                     814      435 (53.4)
  negative concordance rates
  Minimum overall concordance rate         818      462 (56.5)
Predictive marker
assay specifications
  Minimum No. of cases                     768      584 (76.0)
  No. of positive and negative cases       767      560 (73.0)
  Minimum positive and                     747      408 (54.6)
  negative concordance rates
  Minimum overall concordance rate         756      424 (56.1)

                                            No       Unsure/NA
                                         No. (%)      No. (%)
Nonpredictive marker
assay specifications
  Minimum No. of cases                   60 (7.2)     58 (6.9)
  No. of positive and                   102 (12.2)    63 (7.5)
  negative cases
  Minimum positive and                  227 (27.9)   152 (18.7)
  negative concordance rates
  Minimum overall concordance rate      200 (24.4)   156 (19.1)
Predictive marker
assay specifications
  Minimum No. of cases                   41 (5.3)    143 (18.6)
  No. of positive and negative cases     64 (8.3)    143 (18.6)
  Minimum positive and                  133 (17.8)   206 (27.6)
  negative concordance rates
  Minimum overall concordance rate      125 (16.5)   207 (27.4)

Abbreviation: N/A, not applicable.

Abbreviations: FDA, United States Food
and Drug Administration;LDT, laboratory
developed test;LMT, laboratory-modified
test.

Table 4. Initial Assay Validation: Minimum Case and Concordance
Requirements Specified in Procedure

                                           n    Median   5th-95th Pctl
                                                             Range

Nonpredictive marker
assay specifications

  Total minimum No. of cases              714     20         5-40
  No. of positive cases specified         663     10         3-20
  No. of negative cases specified         660     10         2-20
  Minimum positive concordance rate, %    427     95     80-100
  Minimum negative concordance rate, %    413     95     80-100
  Minimum overall concordance rate, %     452     95     90-100

Predictive marker
assay specifications

  Total minimum No. of cases              579     25         8-50
  No. of positive cases specified         554     15         5-30
  No. of negative cases specified         547     10         2-25
  Minimum positive concordance rate, %    396     95     89-100
  Minimum negative concordance rate, %    385     95     90-100
  Minimum overall concordance rate, %     369     95     90-100

                                             Minimum
                                            Guideline
                                          Specification1

Nonpredictive marker
assay specifications

  Total minimum No. of cases              20 cases
  No. of positive cases specified         10 cases
  No. of negative cases specified         10 cases
  Minimum positive concordance rate, %    90%
  Minimum negative concordance rate, %    90%
  Minimum overall concordance rate, %     90%

Predictive marker
assay specifications

  Total minimum No. of cases              40 cases
  No. of positive cases specified         20 cases
  No. of negative cases specified         20 cases
  Minimum positive concordance rate, %    90%
  Minimum negative concordance rate, %    90%
  Minimum overall concordance rate, %     90%

                                          No. (%) Meeting
                                             Guideline
                                           Specification

Nonpredictive marker
assay specifications

  Total minimum No. of cases                538 (75.4)
  No. of positive cases specified           533 (80.4)
  No. of negative cases specified           508 (77.0)
  Minimum positive concordance rate, %      398 (93.2)
  Minimum negative concordance rate, %      387 (93.7)
  Minimum overall concordance rate, %       448 (96.9)

Predictive marker
assay specifications

  Total minimum No. of cases                266 (45.9)
  No. of positive cases specified           274 (49.5)
  No. of negative cases specified           261 (47.7)
  Minimum positive concordance rate, %      376 (94.9)
  Minimum negative concordance rate, %      366 (95.1)
  Minimum overall concordance rate, %       358 (97.0)

Abbreviation: Pctl, percentile.

(a) Fitzgibbons et al,1 2014.

Table 5. Revalidation of Previously Validated Assays: Changes
Specified in Revalidation Procedures for Nonpredictive
Markers

                                             Change Specified
                               Total No.           Yes
Changes                       of Responses       No. (%)

Introduction of                   669           559 (83.6)
  a new antibody lot
Change in antibody dilution       653           521 (79.8)
Change in antibody vendor         665           553 (83.2)
Change in antibody clone          666           613 (92.0)
Introduction or change            659           568 (86.2)
  in antigen
  retrieval method
Change in incubation/             658           532 (80.9)
  retrieval times
Change in antigen                 662           572 (86.4)
  detection system
Change in fixative type           650           445 (68.5)
Change in tissue                  655           411 (62.7)
  -processing equipment
Change in testing equipment       658           525 (79.8)
Change in environmental           653           379 (58.0)
  conditions
Change in water supply            649           278 (42.8)

                               If Yes, Are Cases Specified?

                               Total No.     Yes, Specified
Changes                       of Responses      No. (%)

Introduction of                   527          342 (64.9)
  a new antibody lot
Change in antibody dilution       493          310 (62.9)
Change in antibody vendor         519          332 (64.0)
Change in antibody clone          579          388 (67.0)
Introduction or change            536          322 (60.1)
  in antigen
  retrieval method
Change in incubation/             500          311 (62.2)
  retrieval times
Change in antigen                 538          321 (59.7)
  detection system
Change in fixative type           415          231 (55.7)
Change in tissue                  377          194 (51.5)
  -processing equipment
Change in testing equipment       490          270 (55.1)
Change in environmental           355          183 (51.5)
  conditions
Change in water supply            258          134 (51.9)

                             If Yes, Are Cases Specified?

                              Yes, Variable      No
Changes                          No. (%)       No. (%)

Introduction of                115 (21.8)     70 (13.3)
  a new antibody lot
Change in antibody dilution    130 (26.4)     53 (10.8)
Change in antibody vendor      137 (26.4)     50 (9.6)
Change in antibody clone       138 (23.8)     53 (9.2)
Introduction or change         161 (30.0)     53 (9.9)
  in antigen
  retrieval method
Change in incubation/          143 (28.6)     46 (9.2)
  retrieval times
Change in antigen              163 (30.3)     54 (10.0)
  detection system
Change in fixative type        138 (33.3)     46 (11.1)
Change in tissue               139 (36.9)     44 (11.7)
  -processing equipment
Change in testing equipment    172 (35.1)     48 (9.8)
Change in environmental        138 (38.9)     34 (9.6)
  conditions
Change in water supply          98 (38.0)     26 (10.1)

Table 6. Revalidation of Previously Validated Assays:
Minimum Number of Cases Specified for
Nonpredictive Markers

                               n    Median (a)   5th-95th
                                                 Pctl Range

Introduction of a new         342       2           1-20
  antibody lot
Change in antibody dilution   310       4           1-20
Change in antibody vendor     332       5           2-20
Change in antibody clone      388       20          2-20
Introduction or change in     322       10          1-20
  antigen retrieval method
Change in incubation or       311       5           1-20
  retrieval times
Change in antigen detection   321       10          1-20
  system
Change in fixative type       231       10          2-20
Change in tissue-processing   194       10          1-20
  equipment
Change in testing equipment   270       10          2-25
Change in environmental       183       5           1-20
  conditions
Change in water supply        134       5           1-20

Abbreviation: Pctl, percentile.

(a) Median number of test cases specified for revalidation.

Table 7. Revalidation of Previously Validated Assays: Changes
Specified in Revalidation Procedures for Predictive
Markers

                                               Change
                                             Specified?

                               Total No.     Yes No. (%)
                              of Responses

Introduction of a new             621        486 (78.3)
  antibody lot
Change in antibody dilution       609        467 (76.7)
Change in antibody vendor         617        478 (77.5)
Change in antibody clone          620        533 (86.0)
Introduction or change            618        505 (81.7)
  in antigen
  retrieval method
Change in incubation/             617        474 (76.8)
  retrieval times
Change in antigen                 619        511 (82.6)
  detection system
Change in fixative type           602        400 (66.4)
Change in tissue-                 606        372 (61.4)
  processing equipment
Change in testing equipment       613        462 (75.4)
Change in environmental           610        346 (56.7)
  conditions
Change in water supply            606        250 (41.3)

                               If Yes, Are Cases Specified?

                               Total No.     Yes, Specified
                              of Responses       No. (%)

Introduction of a new             449          296 (65.9)
  antibody lot
Change in antibody dilution       430          270 (62.8)
Change in antibody vendor         442          284 (64.3)
Change in antibody clone          497          330 (66.4)
Introduction or change            469          285 (60.8)
  in antigen
  retrieval method
Change in incubation/             439          270 (61.5)
  retrieval times
Change in antigen                 472          285 (60.4)
  detection system
Change in fixative type           369          206 (55.8)
Change in tissue-                 338          181 (53.6)
  processing equipment
Change in testing equipment       425          238 (56.0)
Change in environmental           319          165 (51.7)
  conditions
Change in water supply            226          118 (52.2)

                              If Yes, Are Cases Specified?

                              Yes, Variable       No
                                 No. (%)        No. (%)

Introduction of a new           101 (22.5)     52 (11.6)
  antibody lot
Change in antibody dilution     113 (26.3)     47 (10.9)
Change in antibody vendor       119 (26.9)     39 (8.8)
Change in antibody clone        122 (24.5)     45 (9.1)
Introduction or change          140 (29.9)     44 (9.4)
  in antigen
  retrieval method
Change in incubation/           126 (28.7)     43 (9.8)
  retrieval times
Change in antigen               146 (30.9)     41 (8.7)
  detection system
Change in fixative type         123 (33.3)     40 (10.8)
Change in tissue-               118 (34.9)     39 (11.5)
  processing equipment
Change in testing equipment     143 (33.6)     44 (10.4)
Change in environmental         120 (37.6)     34 (10.7)
  conditions
Change in water supply          83 (36.7)      25 (11.1)

Table 8. Revalidation of Previously Validated Assays:
Minimum Number of Cases Specified for Predictive
Markers

                               n    Median (a)   5th-95th
                                                 Pctl Range

Introduction of a new         296       2           1-40
  antibody lot
Change in antibody dilution   270       5           1-40
Change in antibody vendor     284       10          2-40
Change in antibody clone      330       20          3-40
Introduction or change in     285       10          1-40
  antigen retrieval method
Change in incubation or       270       5           1-40
  retrieval times
Change in antigen detection   285       20          2-40
  system
Change in fixative type       206       20          2-50
Change in tissue-processing   181       10          1-50
  equipment
Change in testing equipment   238       20          2-40
Change in environmental       165       7           1-40
  conditions
Change in water supply        118       5           1-50

Abbreviation: Pctl, percentile.

(a) Median number of test cases specified for revalidation.

Table 9. Most Recent Initial Validation Procedure: Method Used
for Validation (a)

                                 Nonpredictive         Predictive
                                (n = 725) No. (%)   (n = 101) No. (%)

Correlated the new test's          443 (61.1)           47 (46.5)
results with the morphology
and expected results

Compared the new test's            123 (17.0)           27 (26.7)
results with the results of
testing the same tissue
validation set in another
laboratory using a validated
assay

Compared the new test's            118 (16.3)           17 (16.8)
results with the results of
prior testing of the same
tissues with a validated
assay in the same laboratory

Compared the new test's             10 (1.4)             7 (6.9)
results with previously
validated
nonimmunohistochemical tests

Tested previously graded             6 (0.8)             2 (2.0)
tissue challenges from a
formal proficiency testing
program (if available) and
compared the results with
the graded responses

Other                               16 (2.2)             1 (1.0)

Unsure                               9 (1.2)             0 (0.0)

(a) Fisher exact test; P = .002 for difference between
nonpredictive and predictive biomarkers.

Table 10. Most Recent Initial Validation Procedure: Cases Used
for Validation

                                       Percentile Distribution

Volumes                      Marker Type       n      5th    10th

Total No. of cases           Nonpredictive    685      4      7
included in validation set   Predictive        97      5      10

No. of known positive        Nonpredictive    652      2      4
cases included               Predictive        93      3      5

No. of known negative        Nonpredictive    635      0      2
cases included               Predictive        91      1      4

                                       Percentile Distribution

Volumes                       25th     Median     75th     90th

Total No. of cases             10        20        20       33
included in validation set     20        31        42       55

No. of known positive           6        10        10       20
cases included                 10        15        20       25

No. of known negative           5        10        10       17
cases included                 10        15        22       28

                             Percentile Distribution

Volumes                       95th    P Value (a)

Total No. of cases             40        <.001
included in validation set     65

No. of known positive          20        <.001
cases included                 30

No. of known negative          20        <.001
cases included                 38

(a) Wilcoxon rank sum test.
COPYRIGHT 2017 College of American Pathologists
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2017 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Stuart, Lauren N.; Volmar, Keith E.; Nowak, Jan A.; Fatheree, Lisa A.; Souers, Rhona J.; Fitzgibbons
Publication:Archives of Pathology & Laboratory Medicine
Geographic Code:1USA
Date:Sep 1, 2017
Words:5726
Previous Article:Analytic Validation of Immunohistochemical Assays: A Comparison of Laboratory Practices Before and After Introduction of an Evidence-Based Guideline.
Next Article:Challenges in Diagnosing Medication Resins in Surgical Pathology Specimens: A Crystal-Clear Review Guide.
Topics:

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters