Test verification and validation for molecular diagnostic assays.
Before introducing a molecular assay into the clinical laboratory, the laboratory director must be familiar with current federal and state regulations and with the FDA approval status of the assay. The articles in this special section do not address these critical issues in depth. Therefore, it is important to briefly review the FDA status of assays and other regulations affecting molecular diagnostics.
Some molecular assays have been commercially produced and have gone through an FDA 510(k) clearance or PMA (premarket approval) process. These assays are referred to as "in vitro diagnostics" or "IVDs." In contrast, most new molecular diagnostic assays are developed within individual clinical laboratories and are referred to as "laboratory-developed tests" or "LDTs." Many of these clinically important LDTs are unlikely to ever become FDA cleared or approved because low test volumes make it economically unfeasible to go through the current FDA 510(k) or PMA approval process.
In vitro diagnostics that have gone through and passed a 510(k) trial are considered FDA cleared. 510(k) trials are designed to show that an IVD is substantially equivalent to a legally marketed predicate device. 510(k) trials are less involved and less expensive than PMA trials. An IVD that has gone through and passed a PMA trial is considered FDA approved. Premarket approval trials are required for class III medical devices or if there is no predicate device available for comparison. They are significantly more expensive to perform than 510(k) trials.
Molecular diagnostic assays are classified by Clinical Laboratory Improvement Amendments of 1988 (CLIA) as nonwaived testing. As such, the quality system requirements defined by CLIA state that the clinical laboratory performing the FDA-cleared or FDA-approved test must "verify" that the manufacturer's performance characteristics, which were established
during the FDA trial, can be obtained or exceeded by the laboratory. Specifically, for FDA-cleared/approved tests, a laboratory must document its verification of the following 4 performance characteristics: accuracy, precision, reference range, and reportable range. These performance characteristics are published in the manufacturer's package insert.
CLIA requires that laboratories performing LDTs, and modified FDA-cleared/approved tests, "establish" the same 4 performance characteristics that are required for FDA-cleared/approved tests, as well as determine analytic sensitivity, analytic specificity, and any additional performance characteristics that may be important to establish (eg, specimen stability, linearity). Verification of FDA-cleared/approved tests generally entails smaller studies with fewer samples than are required to validate an LDT or a modified FDA-cleared/approved test because the manufacturer has already established the performance characteristics of the FDA-cleared/approved tests. A laboratory must document that it has adequately verified or validated test performance characteristics for all clinically offered tests. It should be noted that the term validation is widely used by laboratorians to mean establishing the performance characteristics of an LDT, but the term validation never appears anywhere within the CLIA regulations.
A useful mnemonic for remembering the performance characteristics that need to be verified for FDA-cleared/ approved tests is PARR (precision, accuracy, reportable range, and reference range). For LDTs or modified FDA-cleared/approved tests it is PARR+AS+AS (precision, accuracy, reportable range, reference range, analytic sensitivity, and analytic specificity). It should be noted that there may be additional parameters that a laboratory needs to address beyond the 4 or 6 respective performance characteristics. For example, the laboratory may find it important to assess specimen stability, linearity, and carryover.
Although not required by CLIA, knowledge of the clinical validity (ie, clinical sensitivity and specificity) and clinical utility of these assays is also good laboratory practice. It is often difficult for an individual laboratory to thoroughly assess these parameters (eg, clinical sensitivity) because there may not be enough patients with an uncommon disease at an institution to do a large study. However, patients without disease can easily be identified to determine clinical specificity. It is currently considered permissible to cite scientific literature that established clinical sensitivity and specificity. The CAP and various laboratory organizations are beginning to work with the FDA to discuss what will be required by individual laboratories for evidence of clinical validity and utility.
While a laboratory must, at a minimum, address all the performance characteristics required by CLIA, the specifics of how to verify or establish these is left by CLIA to the judgment of the laboratory director. The scientific approach used to verify or validate the performance characteristics of a test is one of the more challenging aspects of being a laboratory director. CLIA '67 was revised and expanded in 1988, well before genetic testing became widely used. For this reason, the meaning and relevance of some performance characteristics, such as reportable range, reference range, and analytic specificity for molecular diagnostic tests, is still being discussed and refined. Organizations such as the Clinical and Laboratory Standards Institute (CLSI) provide excellent guidance on how to approach test verification and validation for a wide variety of test types. In addition to CLSI, other organizations such as the Centers for Disease Control and Prevention and the American College of Medical Genetics have published recommendations on approaches to validation for certain types of molecular pathology testing. The CAP Molecular Pathology Checklist also provides some guidance on test validation.
Nonetheless, laboratories across the country are looking for further guidance and specific examples on how to verify FDA-approved tests and especially how to validate LDTs. The goal of the set of special section articles included in this issue of Archives of Pathology & Laboratory Medicine is to provide examples of how individual laboratories have verified or validated the analytical performance characteristics for certain FDA-approved/ cleared or laboratory-developed molecular diagnostic assays. Two examples of validating and implementing FDA-cleared or FDA-approved tests (cystic fibrosis and Clostridium difficile cytotoxin) are provided, as well as 4 examples of validating and implementing LDTs. The LDT examples include validation of (1) a KRAS test for colorectal cancer treatment decisions, (2) a BCR/ABL1 minimal residual disease assay for guiding treatment of chronic myelogenous leukemia patients, (3) a sequencing assay for germline PTEN mutations, and (4) a fluorescence in situ hybridization assay for the detection of MLL rearrangements in leukemia. Although the primary goal of these articles is to discuss verification and validation, the articles also address other aspects of the total process of introducing a new molecular assay in the clinical laboratory, starting from familiarization/characterization and proceeding to validation and test implementation.
(1.) Jennings L, Van Deerlin VM, Gulley ML; for the College of American Pathologists Molecular Pathology Resource Committee. Recommended principles and practices for validating clinical molecular pathology tests. Arch Pathol Lab Med. 2009; 133(5):743-755.
Kevin C. Halling, MD, PhD; Iris Schrijver, MD; Diane L. Persons, MD
Accepted for publication May 6, 2011.
From the Department of Laboratory Medicine, Mayo Foundation, Rochester, Minnesota (Dr Halling); the Department of Pathology, Stanford University School of Medicine, Stanford, California (Dr Schrijver); and the Department of Pathology and Laboratory Medicine, University of Kansas Medical Center, Kansas City (Dr Persons).
The authors have no relevant financial interest in the products or companies described in this article.
Reprints: Diane L. Persons, MD, Department of Pathology and Laboratory Medicine, 1213 KU Hospital University of Kansas Medical Center, 3901 Rainbow Blvd, Kansas City, KS 66160 (e-mail: dpersons@ kumc.edu).
Kevin C. Halling, MD, PhD, received his medical degree and doctorate degree in biochemistry from the University of Kansas Medical Center in Kansas City. He completed a residency in anatomic and clinical pathology and a fellowship in clinical molecular genetics at the Mayo Clinic, Rochester, Minnesota. He is board certified in anatomic and clinical pathology, clinical molecular genetics, and molecular genetic pathology. Dr Halling currently is associate professor of pathology. He codirects the clinical molecular genetics and molecular cytology laboratories and is the vice chair for research and development for the Department of Laboratory Medicine and Pathology at Mayo Clinic. Dr Halling is a member of the College of American Pathologists Molecular Oncology Committee and previously chaired that committee.
Iris Schrijver, MD, received her medical degree from the University of Utrecht in The Netherlands and completed a postdoctoral research fellowship at Stanford University in Palo Alto, California. She is a diplomate of the American Board of Medical Genetics and holds specialty certification in clinical molecular genetics. In addition, she trained in clinical pathology at Stanford University and is a diplomate of the American Board of Pathology. Dr Schrijver currently is associate professor of pathology and (by courtesy) pediatrics. She directs the diagnostic molecular pathology laboratory as well as the Molecular Genetic Pathology Fellowship Program and is associate program director for residency training in clinical pathology. She is chair of the Biochemical and Molecular Genetics Resource Committee of the College of American Pathologists (CAP)/American College of Medical Genetics and chair of the Molecular Pathology and Genetics Cluster of the CAP.
Diane L. Persons, MD, received her medical degree from the University of Kansas Medical Center (KUMC) in Kansas City, where she also completed her residency in anatomic pathology. She completed a clinical cytogenetics fellowship at the Mayo Clinic and is board certified in both anatomic pathology (American Board of Pathology) and in clinical cytogenetics (American Board of Medical Genetics). She is currently a professor of pathology and laboratory medicine, director of the cytogenetics laboratory, and director of the Pathology Residency Program at KUMC. Dr Persons is a past chair of the Cytogenetics Resource Committee of the College of American Pathologists (CAP)/American College of Medical Genetics, a past chair of the Molecular Pathology and Genetics Cluster, and currently serves on the Molecular Oncology Resource Committee of the CAP.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Special Section--Contributions on Validation From the College of American Pathologists Resource Committees|
|Author:||Halling, Kevin C.; Schrijver, Iris; Persons, Diane L.|
|Publication:||Archives of Pathology & Laboratory Medicine|
|Date:||Jan 1, 2012|
|Previous Article:||The practice of pathology in Canada: decreasing pathologist supply and uncertain outcomes.|
|Next Article:||Pathologic diagnostic correlation with breast imaging findings: a college of American pathologists Q-Probes study of 48 institutions.|