CLSI evaluation protocols.
This article details the relationship between CLSI EP documents and a laboratory's needs for conducting different types of studies to evaluate the performance of instruments and clinical assays, as outlined in EP19-R--A Framework for NCCLS Evaluation Protocols; A Report. This report categorizes each EP guideline with respect to its evaluation type, which takes place when an assay is considered for use or placed in use by a laboratory.
CLSI EP documents provide recommendations for the assessment of laboratory test procedures in a way that defines laboratory best practices grounded on scientific and statistical theory. CLSI evaluation protocols are primarily intended to be practical in their use in the clinical laboratory since all clinical laboratories must perform method-evaluation studies prior to using a new or revised test system for reporting patient test results. Manufacturers can also use these protocols as a means of generating performance data for their in vitro diagnostic (IVD) devices and/or assays in a form to which the laboratory can relate. In essence such documents provide a framework for evaluation studies that should result in greater consensus and standardization of practice.
For user verification of performance
Accreditation requirements (particularly CLIA '88) include the need to verify the performance of a method prior to use on patient samples. EP15-A2--User Verification of Performance for Precision and Trueness; Approved Guideline--Second Edition and EP12-A--User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline have been designed for these purposes. Basic procedures to meet accreditation requirements are detailed but can be modified to meet laboratories' particular circumstances. All accredited clinical laboratories should use such documents routinely.
EP10-A2--Preliminary Evaluation of Quantitative Clinical Laboratory Methods; Approved Guideline--Second Edition provides a simple but powerful protocol for an early evaluation of a new instrument or for checking performance after major maintenance.
EP15-A2 includes a five-day testing protocol and simplified worksheets for all data gathering, statistical calculations, and tests of observed precision and trueness. A computer spreadsheet is provided to simplify implementation. This document is primarily intended for use when an established method is initially set up in the laboratory; however, it may also be used to verify method performance after corrective action following a failed proficiency-testing event.
EP12-A provides a protocol designed to optimize the experimental design for the evaluation of qualitative tests; to better measure performance; and to provide a structured data analysis. This guideline provides evaluation protocols for the demonstration of qualitative test performance. EP12 is written for clinical laboratory personnel who are the end-users of such tests. Demonstration of test performance by the user can satisfy internal (as well as external) expectations that the test performs acceptably in meeting the user's clinical and analytical goals.
EP10-A2 gives a minimal procedure that can be used as preliminary indication of performance, and includes sample data sheets that simplify analysis of the data and assist in determining the possible causes of imprecision. This might not satisfy accreditation requirements for verification but can be useful as a troubleshooting tool, for a quick check when considering adoption of a method, or when modifying methods.
For more detailed understanding
Laboratories may need to know more about their methods for specific applications or may need to meet more detailed accreditation requirements. These EP documents can also be used for describing performance for laboratories that develop or modify methods. They include the following:
EP6-A--Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach; Approved Guideline for verifying calibration or checking linearity across the measuring range. This guideline describes the required conditions, materials, and easy-to-follow instructions for evaluating linearity using the polynomial method, which is statistically rigorous and can be easily implemented by spreadsheet programs. For ease of use, the linearity protocol is illustrated in a flowchart in which each task has a parenthetical note referring to the appropriate section in the document (see Figure 1); references to appropriate sections in the text are in parentheses.
[FIGURE 1 OMITTED]
EP17-A--Protocols for Determination of Limits of Detection and Limits of Quantitation; Approved Guideline for determining or verifying the lower limit of detection or LoD and lower limit of quantitation or LoQ of a method. The limits of detection and quantitation are critical because detecting extremely small amounts of an analyte can be necessary to define disease states, screen for disease, identify significant exposure, or to reveal the presence or absence of toxins, pollutants, carcinogens, contaminants, infectious agents, and illicit drugs. These two critical performance characteristics are also important in laboratory examinations for tumor markers, hormones, agents of infectious diseases, therapeutic drugs, and other tests where low values separate subjects into different disease or exposure categories. The EP17 protocols are practical for use with common clinical laboratory devices and include easy-to-follow flowcharts showing the steps involved in determining and/or verifying limits of detection and quantitation (see Figure 2).
[FIGURE 2 OMITTED]
EP9-A2--Method Comparison and Bias Estimation Using Patient Samples; Approved Guideline--Second Edition for a more rigorous verification of trueness, of bias relative to a reference procedure. This would also be useful when migrating from one method to another, or for aligning two different devices for the same measurement. For carrying out method-comparison evaluations, an overview of the experiment, sample data recording and calculation sheets, and an overview flowchart and a detailed flowchart for preliminary data examination are included. As an additional aid, a sample scatter plot and bias plot are introduced for those who are unfamiliar with these procedures (see Figure 3).
[FIGURE 3 OMITTED]
EP21-A2--Estimation of Total Analytical Error for Clinical Laboratory Methods; Approved Guideline for determining the total error of measurement, to check against laboratory goals for error. These protocols can be used to judge the clinical-laboratory acceptability of new quantitative analytical methods by using patient specimens or to monitor an assay's total analytical error by using quality-control samples, and are applicable in settings with only one instrument, as well as to those with multiple laboratories and multiple instruments.
EP18-A--Quality Management for Unit-Use Testing; Approved Guideline for quality control of unit-use devices. This is a comprehensive and flexible guideline that describes quality-management models to identify potential sources of errors in unit-use test systems related to specimen collection through reporting of results. The recommendations in this guideline are applicable to various devices and settings and are practical to implement, so that sources of error (potential failure modes) are identified, understood, and managed. This document complements the other CLSI document for quality control, C24-A2--Statistical Quality Control for Quantitative Measurements: Principles and Definitions; Approved Guideline--Second Edition.
For detailed descriptions of performance
There are a number of other EP documents designed for detailed insight into method performance. This can be necessary for laboratories that modify methods or develop in-house methods. These documents are also used extensively by IVD manufacturers and by the U.S. Food and Drug Administration to review applications for approval or clearance of new methods.
EP5-A2--Evaluation of Precision Performance of Quantitative Measurement Methods; Approved Guideline--Second Edition gives a rigorous protocol for determining the precision of quantitative methods across the entire measuring range. Included are guidelines for the duration, procedures, materials, data summaries, and interpretation techniques that are adaptable for the widest possible range of analytes and device complexity. A balance is created in the document between complexity of design and formulae, and simplicity of operation.
EP7-A--Interference Testing in Clinical Chemistry; Approved Guideline and EP14-A2--Evaluation of Matrix Effects; Approved Guideline--Second Edition both provide protocols to help identify sources of error that can affect patient care and/or assess the suitability of a method. EP7-A gives a protocol for detecting interfering substances and describing the effect of those substances. This guideline contains background information on interference testing concepts; tables of recommended test concentrations for analytes and potential interference; data collection and analysis worksheets; and detailed examples. There is a balance between consistency of structured protocols and flexibility to accommodate the technology being evaluated.
EP14-A2 is of use for evaluating reference materials (or proficiency-test samples) and determining whether they perform the same as patient samples. The document is informative about the impact of matrix effects on the assessment of the quality of laboratory performance, and provides guidance in evaluating the bias in analyte measurements that is due to the sample matrix (physiological or artificial) when two measurement procedures are compared.
In summary, the CLSI EP series is applicable for most laboratory tests and measurements and provides invaluable insight into method performance. The use of these protocols, and comments and participation in the ongoing efforts to improve them, are encouraged.
CLSI welcomes comments and questions about the documents; this feedback serves as the basis for updated document editions. All comments and responses are formally addressed and published in the next edition of the document. For more information about Clinical and Laboratory Standards Institute references and best practices, visit www.clsi.org or call 610-688-0100.
Dan Tholen, MS, Traverse City, MI, is a consultant and trainer in statistical methods for laboratory quality. His clients include those in the commercial, professional, and governmental areas, in all fields of laboratory measurements. He is a member of the CLSI Area Committee on Evaluation Protocols.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||LAB MANAGEMENT; Clinical and Laboratory Standards Institute|
|Publication:||Medical Laboratory Observer|
|Date:||Aug 1, 2006|
|Previous Article:||Efficient laboratory design.|
|Next Article:||At bioMerieux, a family's history melds with microbiology's future: the Merieux mantle is again passed.|