Recommendations for validating estrogen and progesterone receptor immunohistochemistry assays.
Validation of a clinical laboratory test means confirmation, through a defined process, that the test performs as intended or claimed. Proper validation provides reasonable, but not absolute, assurance that a test is performing as anticipated. There is no single, universally acceptable procedure for validating clinical laboratory tests. The design of a validation protocol requires professional judgment, and validation schemes must take into account the test's intended use, other claims made about the test, and risks that may prevent the test from meeting performance claims.
This article provides guidance on analytic (technical) validation procedures that we believe should be used by laboratories offering estrogen receptor (ER) and progesterone receptor (PgR) assays by immunohistochemical (IHC) methods. We describe minimal procedures for initially validating the tests before they are placed in clinical service. We also discuss labeling requirements (language) applicable to reporting and to claims a laboratory may choose to make about its assays. A separate guideline (1) describes required elements of an ongoing quality management program for hormone-receptor testing by IHC methods, including daily quality control testing, external proficiency testing, and general controls applied to laboratory personnel, equipment, reagents, and other aspects of laboratory service.
USE OF IHC HORMONE-RECEPTOR TESTING
Estrogen receptor and PgR status is assessed in all newly diagnosed, invasive breast carcinomas and in recurrences to determine patient eligibility for adjuvant hormonal therapy. (2) There is a substantial survival benefit from tamoxifen and aromatase inhibitors, but only among patients with ER-positive tumors. (3-5) Accurate classification of hormone-receptor status is, therefore, critical to ensure patients receive appropriate therapy.
Immunohistochemistry is currently the most commonly used method for determining ER and PgR status because of its relatively low cost, its applicability to routinely processed and archival tissue samples, and importantly, its use in evaluating small cancers to ensure that only invasive tumor cells are assessed. This guideline describes validation procedures for ER and PgR IHC assays that are used to predict response to tamoxifen and aromatase inhibitors (predictive markers). Validation procedures are designed to reasonably confirm that a new test performs this task as well as existing validated assays. The procedures described in this guideline are not adequate to demonstrate that a new assay is superior to existing assays; such claims require additional validation procedures.
Risks of IHC Hormone-Receptor Testing
Patients with breast cancer who are misclassified as having ER-negative tumors are denied the potential benefit of hormonal treatment, whereas those who are misclassified as having ER-positive tumors will be exposed unnecessarily to the risks and costs of ineffectual treatment and potentially being denied the benefit of other treatments. Other risks from hormonal treatment include a decrease in bone density with an increased fracture risk, an increase in the risk of thromboembolic events, and an increase in the risk of uterine cancer. (6)
Although IHC test methods have improved with the widespread adoption of automated staining platforms, the development of more sensitive antibody clones and detection systems, and the use of US Food and Drug Administration (FDA)-approved test kits, (7) results are still affected by factors such as delayed or inadequate fixation, nonoptimized antigen retrieval, and nonstandardized interpretation of findings or reporting of results. (8-11) Initial validation and periodic or ongoing reassessment of IHC ER assays provide reasonable assurance that manageable factors that affect test accuracy and clinical usefulness are properly controlled.
Special Considerations in IHC Hormone-Receptor Testing
The test-validation procedures described in this guideline are designed to provide reasonable assurance that false-positive and false-negative test results are minimized. Because patients with even low levels of hormone-receptor expression (ie, 1%) may respond to hormonal therapy, (12-20) these procedures are intended to ensure that ER assays can accurately identify tumors with weak ER expression.
The validation of any clinical assay requires that results be compared with a standard. Some clinical assays may be validated with certified reference material with known reactivity that is traceable to an authoritative standard, but such materials are not currently available for ER IHC. For ER and PgR assays, which are primarily used as predictive markers, the ideal standard is a clinical demonstration that the test accurately identifies patients who will benefit from hormonal therapy; however, few laboratories have the resources available to validate their assay with reference to clinical outcome. This validation protocol was designed for assays that are used to guide therapeutic decision making but are not being evaluated directly against a clinical standard. Some of the procedures described in this protocol may be unnecessary if direct clinical validation is done, but additional requirements for clinical validation may be applicable.
The approach to validation used in this guideline relies on comparing the assay's results with results obtained by another laboratory using a testing method that has been validated against clinical outcome or against proficiency-testing material that has been validated by showing consensus results among multiple laboratories in a peer group (which must include laboratories with validated assays). Acceptable approaches are listed in Table 1. Assay "validation" that relies on comparison with another unvalidated assay is not sufficient. (21,22)
The level of agreement required for validating a new assay (90% agreement for positive specimens and 95% agreement for negative specimens) has been achieved in several studies that compared local laboratory and central laboratory ER results. (16,17) If the positive samples in the validation set are enriched with weakly positive specimens, as we propose, we believe that laboratories with at least 90% positive agreement at the time of assay validation under these conditions are likely to detect considerably more than 90% of ER-positive specimens in clinical practice.
The laboratory director is responsible for ensuring that all validation steps have been performed according to these guidelines.
Initial validation or verification of ER and PgR IHC assays must be successfully completed before the tests can be placed into clinical service.
An IHC predictive-marker assay includes a defined set of test conditions (reagents, equipment, specimen types, and standard operating procedures) and an ongoing quality management regimen that includes, as applicable, routine quality control, periodic assay recalibration, employee-competency testing, external proficiency testing, and laboratory inspection. During the interval when a test is being validated, the test conditions and ongoing quality-management regimen should be the same as the conditions and quality-management regimen that will be used once the assay is placed in clinical service. Laboratories that intend to use specialized techniques for scoring, such as image analysis, must use those same techniques in the validation study.
To ensure that the validation study assesses between-run variation, the laboratory validating its assay should not test all specimens in the validation set on the same day. These specimens should be run in batches on multiple days, with multiple testing personnel, when possible.
FDA-Cleared Assays.--Laboratories with assays cleared or approved by the FDA must verify the performance specifications stated by the manufacturer. Manufacturers of FDA-approved or cleared test kits may provide the user with FDA-approved or cleared recommendations and directions for verifying that the kit is performing according to the manufacturer's specification. Usually this is performed by testing known positive and negative samples that either are supplied by the manufacturer or have been tested by a validated reference-laboratory method. (22) Recommended verification procedures for these assays are described in Table 2.
For laboratories with unmodified FDA-cleared or FDA-approved ER or PgR assays that were in clinical service before the publication of the American Society of Clinical Oncology/College of American Pathologists Guideline Recommendations for Immunohistochemical Estrogen/Progesterone Receptor Testing in Breast Cancer1 and that have been used on at least 200 clinical specimens, the verification procedure may include the results of previously analyzed cases; however, laboratories introducing an assay after publication of the guidelines must complete the verification study before the test is placed in service.
A verification study does not need to be repeated if a previous study conforming to the manufacturer's FDA-cleared recommendations or with this guideline has been completed and the records are available for review by external inspectors. Laboratories that cannot document initial test verification of an FDA-cleared or FDA-approved assay must complete a new verification study to show that the test performs as intended.
Laboratory-Developed and Laboratory-Modified Assays.--Laboratories must validate laboratory-developed tests (LDTs) and any FDA-cleared or FDA-approved laboratory-modified tests (LMTs). Recommended validation procedures for ER or PgR LDTs or LMTs are described in Table 3. For laboratories with assays that were in clinical service before the publication of the American Society of Clinical Oncology/College of American Pathologists testing guidelines1 and that have been applied to at least 200 clinical specimens, the validation set may include the results of previously analyzed cases provided that the previously analyzed cases have been tested using one of the methods listed in Table 1; however, laboratories introducing an LDT or LMT after publication of the guidelines must complete the validation study before the test is placed in service.
A validation study does not need to be repeated if a previous validation study conforming to this guideline has been completed and the records are available for review by external inspectors. Laboratories that cannot document initial validation of an LDT or LMT must complete a new validation study to show that the test performs as intended.
Changes in Test Methods
All assays must be revalidated whenever there is a significant change to the test system, such as a change in the primary antibody clone, introduction of new antigen-retrieval or immunohistochemistry detection systems, or a significant relaxation of ongoing quality-management procedures. Assay revalidation after significant changes should meet the requirements specified in Table 3.
Regardless of the methodology used, all laboratories with validated ER and PgR IHC assays must periodically reassess the assays to ensure that their analytic sensitivity has not drifted. Required ongoing assay assessment procedures are described in Table 4. Ongoing assay reassessment does not require repeating the same procedures used for initial test validation.
LABELING AND REPORTING
Laboratories using LDT and LMT assays must append a statement to each IHC result indicating that the assay was developed and its performance characteristics determined by [name of laboratory]. Laboratories should not make claims that their ER or PgR assays are superior to other assays, unless such claims have been specifically validated by comparison with clinical outcome.
Records of method validation must be maintained while the test is in service and for at least 2 years after the method is no longer used for clinical testing. Records of microscopist validation must be maintained for the same time period.
* Analyte-Specific Reagent.--Antibodies, both polyclonal and monoclonal, specific-receptor proteins, ligands, nucleic acid sequences, and similar reagents that, through specific binding or chemical reaction with substances in a specimen, are intended for use in a diagnostic application for identification and quantification of an individual chemical substance or ligand in biological specimens [21CFR864.4020(a)].
* FDA-Cleared Test.--A test that has been cleared by the FDA after analysis of data showing substantial performance equivalence to other tests being marketed for the same purpose. Such tests typically follow the 510(k) approval route [21CFR807].
* FDA-Approved Test.--A test that is classified as a class III medical device and that has been approved by the FDA through the premarket approval process [21CFR814.3].
* Laboratory-Modified Test.--An FDA-cleared or FDA-approved test that is modified by a clinical laboratory but not to a degree that changes the stated purpose of the test, the approved test population, the specimen type, the specimen handling, or claims related to the interpretation of results.
* Laboratory-Developed Test.--A test developed within a clinical laboratory that is performed by the laboratory in which the test was developed and is neither FDA-cleared nor FDA-approved.
** Note.--All LMTs are, by definition, LDTs. An LDT may or may not employ analyte-specific reagents, research-use only reagents, or investigationaluse only reagents; the types of reagents and devices employed does not affect whether a test is classified as an LDT. A laboratory is considered to have developed a test if the test procedure or implementation of the test was created by the laboratory performing the testing, irrespective of whether fundamental research underlying the test was developed elsewhere or whether reagents, equipment, or technology integral to the test was purchased, adopted, or licensed from another entity.
* Test Validation.--Confirmation through a defined process that a test performs as intended or claimed.
** Note.--There is no single universally accepted procedure for validating tests. The process for validating tests must take into account the purpose for which a test is intended, claims made about the test, and the risks that may prevent the test from serving its intended purpose or meeting performance claims. Even FDA-approved and FDA-cleared tests require limited revalidation in clinical laboratories (a process often referred to as verification) to establish that local implementation of the test can reproduce a manufacturer's validated claims. Tests that use reagents or equipment that have not been validated typically pose increased risks that require more extensive validation, as do tests used in more loosely controlled settings. The determination of whether a test has been adequately validated requires professional judgment.
* Test Verification.--An abbreviated process through which a clinical laboratory establishes that its implementation of an FDA-approved and FDA-cleared test performs in substantial conformance to a manufacturer's stated claims.
* Analytic Validity.--A test's ability to accurately and reliably measure the analyte (the measurand) of interest. The elements of analytic validity include the following, as applicable:
** Accuracy.--The closeness of agreement between the average value obtained from a large series of measurements and the true value of the analyte.
** Precision.--The closeness of agreement between independent results of measurements obtained under stipulated conditions.
** Analytic Sensitivity.--For quantitative and semiquantitative tests, analytic sensitivity is the lowest amount of an analyte in a sample that can be detected with (stated) probability. With respect to hormone-receptor testing, analytic sensitivity refers to the lowest amount of the receptor protein that can be detected by the assay.
** Analytic Specificity.--The ability of a test to measure solely the analyte.
** Note.--Analytic validity is expressed in the context of a defined set of test conditions (including standard operating procedures and permissible specimen types) and an ongoing quality-management regimen (including, as applicable, routine quality control, periodic assay recalibration, and external proficiency testing or alternative external testing). If the test conditions or quality-management regimen changes, the analytic validity of a test may change.
* Clinical Validity.--A test's ability to detect or predict a disorder, a prognostic risk, or another condition or to assist in the management of patients. The elements of clinical validity include the following, as applicable:
** Clinical Sensitivity (Clinical Detection Rate).--The proportion of individuals with a disorder, prognostic risk, or condition that is detected by the test. For hormone-receptor testing, clinical sensitivity refers to the ability of the assay to correctly identify patients who are eligible for hormonal therapy.
** Clinical Specificity.--The proportion of individuals without a disorder, prognostic risk, or condition that is excluded by the test. For ER and PgR testing, clinical specificity refers to the ability of the assay to correctly identify patients who are not likely to benefit from hormonal therapy.
** Reference Limits.--A value or range of values for an analyte that assists in clinical decision making. Reference values are generally of 2 types: reference intervals and clinical decision limits. A reference interval (or reference range) is the range of test values expected for a designated population of individuals. This may be the central 95% interval of the distribution of values from individuals who are presumed to be healthy (or have normal results). For some analytes that reflect high-prevalence conditions (such as cholesterol), significantly fewer than 95% of the population may be healthy. In such a case, the reference interval may be something other than the central 95% of values. A clinical decision limit represents the lower or upper limit of a test value at which a specific clinical diagnosis is indicated or a specified course of action is recommended.
* Clinical Utility.--The clinical usefulness of the test. The clinical utility is the net balance of risks and benefits associated with using a test in a specific clinical setting. Clinical utility does not take into consideration the economic cost or economic benefit of testing and is to be distinguished from cost-benefit and cost-effectiveness analysis. Clinical utility focuses entirely on the probabilities and on the magnitude of the clinical benefit and clinical harm that result from using a test in a particular clinical context.
** Note 1.--The qualities listed above represent the primary performance measurements that are used to describe the clinical capabilities of a test. Other measures of clinical validity may be applicable in particular circumstances.
** Note 2.--Clinical validity is expressed in the context of a defined test population and a defined testing procedure. If the test population changes (eg, a change in the prevalence of disease) or the testing procedure changes, the clinical validity of a test may change.
* Percent Agreement.--The proportion of specimens that produce the same result when tested twice (eg, results that are either both positive or both negative). The term is generally used when the reference method is acknowledged to be imperfect and the true result is not known with high confidence. In this situation the terms sensitivity and specificity are not appropriate to describe the comparative results.
** Positive Percent Agreement.--Refers to the percentage of agreement among specimens that test positive with the reference assay.
** Negative Percent Agreement.--Refers to percentage of agreement among specimens that test negative with the reference assay. (23)
An example of a concordance study illustrating calculations with positive and negative percentages of agreement is provided in Table 5.
(1.) Hammond MEH, Hayes D, Dowsett M, et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for immunohistochemical testing of estrogen/progesterone receptor testing in breast cancer. Arch Pathol Lab Med. 2010;134(6):907-922.
(2.) Harris L, Fritsche H, Mennel R, et al. American Society of Clinical Oncology 2007 update of recommendations for the use of tumor markers in breast cancer. J Clin Oncol. 2007;25(33):5287-5312.
(3.) Early Breast Cancer Trialists' Collaborative Group (EBCTCG). Effects of chemotherapy and hormonal therapy for early breast cancer on recurrence and 15-year survival: An overview of the randomised trials. Lancet. 2005;365(9472): 1687-1717.
(4.) Clark GM, McGuire WL, Hubay CA, Pearson OH, Carter AC: The importance of estrogen and progesterone receptor in primary breast cancer. Prog Clin Biol Res. 1983;132E:183-190.
(5.) Ravdin PM, Green S, Dorr TM, et al: Prognostic significance of progesterone receptor levels in estrogen receptor-positive patients with metastatic breast cancer treated with tamoxifen: results of a prospective Southwest Oncology Group study. J Clin Oncol. 1992;10(8):1284-1291.
(6.) Visvanathan K, Chlebowski RT, Hurley P, et al. American Society of Clinical Oncology clinical practice guideline update on the use of pharmacologic interventions including tamoxifen, raloxifene, and aromatase inhibition for breast cancer risk reduction. J Clin Oncol. 2009;27(19):3235-3258.
(7.) Gown AM. Current issues in ER and HER2 testing by IHC in breast cancer. Mod Pathol. 2008;21(suppl 2):S8-S15.
(8.) Rhodes A, Jasani B, Balaton AJ, Miller KD. Immunohistochemical demonstration of oestrogen and progesterone receptors: correlation of standards achieved on in house tumours with that achieved on external quality assessment material in over 150 laboratories from 26 countries. J Clin Pathol. 2000;53(4):292-301.
(9.) Rhodes A, Jasani B, Barnes DM, Bobrow LG, Miller KD. Reliability of immunohistochemical demonstration of oestrogen receptors in routine practice: interlaboratory variance in the sensitivity of detection and evaluation of scoring systems. J Clin Pathol. 2000;53(2):125-130.
(10.) Rhodes A, Jasani B, Balaton AJ, et al. Study of interlaboratory reliability and reproducibility of estrogen and progesterone receptor assays in Europe: documentation of poor reliability and identification of insufficient microwave antigen retrieval time as a major contributory element of unreliable assays. Am J Clin Pathol. 2001;115(1): 44-58.
(11.) Leake R, Barnes D, Pinder S, et al. Immunohistochemical detection of steroid receptors in breast cancer: a working protocol. J Clin Pathol. 2000;53(8): 634-635.
(12.) Harvey JM, Clark GM, Osborne CK, et al. Estrogen receptor status by immunohistochemistry is superior to the ligand binding assay for predicting response to adjuvant endocrine therapy in breast cancer. J Clin Oncol. 1999; 17(5):1474-1481.
(13.) Cheang MC, Treaba DO, Speers CH, et al. Immunohistochemical detection using the new rabbit monoclonal antibody SP1 of estrogen receptor in breast cancer is superior to mouse monoclonal antibody 1D5 in predicting survival. J Clin Oncol. 2006;24(36):5637-5644.
(14.) Dowsett M, Allred C, Knox J, et al. Relationship between quantitative estrogen and progesterone receptor expression and human epidermal growth factor receptor 2 (HER-2) status with recurrence in the Arimidex, Tamoxifen, Alone or in Combination trial. J Clin Oncol. 2008;26(7):1059-1065.
(15.) Mohsin SK, Weiss H, Havighurst T, et al. Progesterone receptor by immunohistochemistry and clinical outcome in breast cancer: a validation study. Mod Pathol. 2004;17(12):1545-1554.
(16.) Phillips T, Murray G, Wakamiya K, et al. Development of standard estrogen and progesterone receptor immunohistochemical assays for selection of patients for antihormonal therapy. Appl Immunohistochem Mol Morphol. 2007; 15(3):325-331.
(17.) Badve SS, Baehner FL, Gray RP, et al. Estrogen- and progesterone-receptor status in ECOG 2197: comparison of immunohistochemistry by local and central laboratories and quantitative reverse transcription polymerase chain reaction by central laboratory. J Clin Oncol. 2008;26(15):2473-2481.
(18.) Regan MM, Viale G, Mastropasqua MG, et al. Re-evaluating adjuvant breast cancer trials: assessing hormone receptor status by immunohistochemical versus extraction assays. J Natl Cancer Inst. 2006;98(21):1571-1581.
(19.) Viale G, Regan MM, Maiorano E, et al. Prognostic and predictive value of centrally reviewed expression of estrogen and progesterone receptors in a randomized trial comparing letrozole and tamoxifen adjuvant therapy for postmenopausal early breast cancer: BIG 1-98. J Clin Oncol. 2007;25(25): 3846-3852.
(20.) Viale G, Regan MM, Maiorano E, et al. Chemoendocrine compared with endocrine adjuvant therapies for node-negative breast cancer: predictive value of centrally reviewed expression of estrogen and progesterone receptors. J Clin Oncol. 2008;26(9):1404-1410.
(21.) Goldstein NS, Hewitt SM, Taylor CR, et al. Recommendations for improved standardization of immunohistochemistry. Appl Immunohistochem Mol Morphol. 2007;15(2):124-133.
(22.) Immunology Branch, Division of Clinical Laboratory Devices, Office of Device Evaluation. 3.9: manufacturers' recommendations for verification of IHC performance by the user. In: Guidance for Submission of Immunohistochemistry Applications to the FDA. Center for Devices and Radiological Health, US Food and Drug Administration; 1998. http://www.fda.gov/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/ucm094002.htm. Accessed February 1, 2010.
(23.) Diagnostic Devices Branch, Division of Biostatistics, Office of Surveillance and Biometrics 4.1: benchmark and study population recommendations. In: Statistical Guidance on Reporting Results From Studies Evaluating Diagnostic Tests. Center for Devices and Radiological Health, US Food and Drug Administration; 2007. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/ GuidanceDocuments/ucm071148.htm. Accessed February 1, 2010.
(24.) Allred DC, Carlson RW, Berry DA, et al. NCCN Task Force report: estrogen receptor and progesterone receptor testing in breast cancer by immunohistochemistry. J Natl Compr Canc Netw. 2009;7(suppl 6):S1-S21.
(25.) Dunnwald LK, Rossing MA, Li CI. Hormone receptor status, tumor characteristics, and prognosis: a prospective cohort of breast cancer patients. Breast Cancer Res 2007;9(1):R6.
Patrick L. Fitzgibbons, MD; Douglas A. Murphy, MT; M. Elizabeth H. Hammond, MD; D. Craig Allred, MD; Paul N. Valenstein, MD
Accepted for publication February 2, 2010.
From the Department of Pathology, St Jude Medical Center, Fullerton, California (Dr Fitzgibbons); the Surveys Department, College of American Pathologists, Northfield, Illinois (Mr Murphy); the Department of Pathology, Intermountain Healthcare, University of Utah School of Medicine, Salt Lake City (Dr Hammond); the Department of Pathology and Immunology, Washington University School of Medicine, St Louis, Missouri (Dr Allred); and the Department of Pathology, St Joseph Mercy Hospital, Ann Arbor, Michigan (Dr Valenstein).
The authors have no relevant financial interest in the products or companies described in this article.
Reprints: Patrick L. Fitzgibbons, MD, Department of Pathology, St Jude Medical Center, 101 E Valencia Mesa Dr, Fullerton, CA 92835 (e-mail: email@example.com).
Table 1. Acceptable Validation of Hormone-Receptor Assaysa ER and PgR IHC assays not subjected to direct clinical validation may be validated by showing 90% agreement for positive results and 95% agreement for negative results with any of the following: 1. Testing performed on the same blocks in another laboratory that has directly validated its assay against clinical outcome 2. Testing performed on the same blocks using a previously validated ligand binding assay 3. Testing performed on the same blocks in another laboratory that provides written attestation that it is in conformance with ASCO/CAP testing requirements and is using one of the following: a. An FDA-approved assay that has been fully validated using an 80-specimen challenge set as described in Table 3, or b. An LDT or LMT that has been validated according to all other requirements set forth in this document 4. Testing performed on the same blocks in another laboratory that uses an alternative, clinically validated method for measuring hormone-receptor expression (eg, a gene-expression assay)17 5. Testing performed on one of the following: a. Tissue challenges used in a formal PT program, provided that each case used in the validation study was graded by the PT vendor and [greater than or equal to] 50 laboratories are included in the participant's peer group, or b. Validation tissues provided by an organization such as the CAP or the NIST, with established ER and PgR status determined through IHC testing using a technically validated assay (24) Abbreviations: ASCO, American Society of Clinical Oncology; CAP, College of American Pathologists; ER, estrogen receptor; FDA, US Food and Drug Administration; IHC, immunohistochemical; LDT, laboratory-developed test; LMT, laboratory-modified test; NIST, National Institute of Standards and Technology; PgR, progesterone receptor; PT, proficiency testing. (a) In the case of unmodified, FDA-cleared or FDA-approved ER and PgR IHC assays, an alternative verification procedure may be specified in the FDA-approved or FDA-cleared package insert. Table 2. Recommendations for Initial Test Verification of US Food and Drug Administration (FDA)-Cleared Assays (a) Procedure Acceptable Outcome The laboratory should compare the Agreement must be [greater than results from testing [greater or equal to] 90% for positive than or equal to] 20 positive and results and [greater than or [greater than or equal to] 20 equal to] 95% for negative negative specimens using one of results; positive results are the methods described in Table 1 defined as [greater than or equal (b); [greater than or equal to] 5 to] 1% immunoreactive cells of the positive specimens should be weakly positive (1%-10%), and [less than or equal to] 10 specimens should be tested in any one run or or Any verification procedure Acceptable outcome described described in an FDA- in an FDA-cleared or FDA- cleared or FDA-approved approved package insert for package insert for the test the test system system Any pathologist who interprets Each pathologist must ER and PgR IHC results but did demonstrate [less than or equal not participate in the to] 2 incorrect assessments in validation procedure described the 40-slide challenge set; an above must have his or her skill incorrect assessment is any validated by examining [greater specimen with [greater than or than or equal to] 20 positive and equal to] 1% immunoreactive [greater than or equal to] 20 cells reported as negative, or negative specimens; at least any specimen with <1% some of the positive specimens immunoreactive cells reported as must be weakly positive; slides positive from the initial assay validation may be used for this skill- validating procedure. Abbreviations: ER, estrogen receptor; IHC, immunohistochemical; PgR, progesterone receptor; PT, proficiency testing. (a) Applies to unmodified, FDA-cleared or FDA-approved, ER and PgR IHC assays. Validation specimens for FDA-cleared or FDA-approved assays may be obtained fromthe assaymanufacturer if the assay's FDA-approved package insert indicates that manufacturer-supplied specimens may be relied on to verify performance of the assay. A verification study does not need to be repeated if a previous study conforming to the manufacturer's FDA-cleared recommendations or to this guideline has been completed and the records are available for review by external inspectors. (b) Laboratories with assays already in clinical service (and used on [greater than or equal to] 200 specimens) may use the results of previously analyzed cases in their validation study, provided that the previously analyzed cases have been tested using one of the methods listed in Table 1. Those using historic results of PT challenges should include the same number of cases as specified above. If the PT program has, for example, 20 annual challenges, results for [greater than or equal to] 2-years continuous participation (40 challenges) would be required. Table 3. Recommendations for Initial Test Validation of all Laboratory-Developed and Laboratory- Modified Assays (a) Procedure Acceptable Outcome The laboratory should compare Agreement must be [greater than the results of testing [greater or equal to] 90% for positive than or equal to] 40 positive results and [greater than or and [greater than or equal equal to] 95% for negative to]40 negative specimens using results; positive results are one of the methods described in defined as [greater than or Table 1 (b); [greater than or equal to] 1% immunoreactive equal to] 10 of the positive cells cases should be weakly positive (1%-10%), and [less than or equal to] 20 specimens should be tested in any one run Any pathologist who interprets Each pathologist must ER and PgR IHC results but did demonstrate [less than or equal not participate in the to] 2 incorrect assessments in validation procedure described the 40-slide challenge set; an above must have his or her incorrect assessment is any skill validated by examining specimen with [greater than or [greater than or equal to] 20 equal to] 1% immunoreactive positive and [greater than or cells reported as negative, or equal to] 20 negative specimens; any specimen with <1% at least some of the positive immunoreactive cells reported specimens must be weakly as positive positive; slides from the initial assay validation may be used for this procedure Abbreviations: ER, estrogen receptor; IHC, immunohistochemical; PgR, progesterone receptor; PT, proficiency testing. (a) Applies to all laboratory-developed or laboratory-modified assays and to any assay in which there is a major change to the test system, such as a change in the primary antibody, in the antigen-retrieval system, or in the antigen-detection system (but does not apply to new reagent lots or other minor modifications). A validation study does not need to be repeated if a previous validation study conforming to this guideline has been completed and the records are available for review by external inspectors. (b) Laboratories with assays already in clinical service (and used on [greater than or equal to] 200 specimens) may use the results of previously analyzed cases in their validation study, provided that the previously analyzed cases have been tested using one of the methods listed in Table 1. Those using historic results of PT challenges should include the same number of cases as specified above. If the PT program has, for example, 20 annual challenges, results for [greater than or equal to] 4-years continuous participation (80 challenges) would be required. Table 4. Recommendations for Ongoing (Periodic) Reassessment for All Assays Procedure Acceptable Outcome Monitor overall positive and Overall ER2 rate should be negative ER rates (trend <30%. If [greater than or equal analysis), calculated at to] 30%, correlate ER results least twice annually with age and histologic parameters (ie, grade and histologic type) * About 80% of invasive carcinomas in women older than 65 y should be [ER.sup.+12]; if [ER.sup.-] rate among patients older than 65 y is >20%, repeat the validation procedure described in Table 3 * Nearly all low-grade breast carcinomas should be [ER.sup.+25]; if [ER.sup.-] rate among low-grade carcinomas is [greater than or equal to] 5%, repeat the validation procedure described in Table 3 Monitor concordance Concordance should be [greater between ER and PgR than or equal to] 95% for both results and those of gene ER and PgR results expression analyses (if gene expression analysis performed) Demonstrate successful Laboratories must achieve 90% performance in an external correct responses on graded PT program for each PT challenges marker Monitor ER and PgR results Acceptable variation among by pathologist (calculated pathologists should be at least semiannually) established by the laboratory director Abbreviations: ER, estrogen receptor; PgR, progesterone receptor; PT, proficiency testing. Table 5. Example of a Concordance Study Reference Assay, No. Test Result Positive Negative (a) (b) Positive 38 1 New assay Negative 2 39 Total 40 40 (a) Positive percentage of agreement (new assay/validated assay) = 38/ (38 + 2) = 38/40 x 100 = 95.0%. b Negative percentage of agreement (new assay/validated assay) = 39/ (39 + 1) = 39/40 = 100 = 97.5%.