Current landscape and new paradigms of proficiency testing and external quality assessment for molecular genetics.
Quality management is defined as an ongoing effort that includes policies and procedures established and implemented for the purpose of providing accurate laboratory test results. (2) Quality management of the analytic component encompasses a variety of quality assurance processes designed to ensure the performance of a test in the clinical laboratory. (3,4) This assurance requires a system that includes both internal and external procedures that are described in national and international guidance and regulatory documents (4-15) Internal quality assurance processes include measures to maintain analytic accuracy, such as quality control and personnel competency. External quality assessment measures include examination of laboratory procedures by a third party accreditation process and participation in proficiency testing (PT) or external quality assessment (EQA) programs. Most of the current quality assurance practices commonly used in genetic testing laboratories are designed for well-established technologies, such as targeted mutation analysis that detect limited sequence variations in one or a few genes associated with a particular disorder or condition. Newer technologies, such as next-generation DNA sequencing and chromosomal microarrays, allow detection of more sequence variations or gene expression levels. These new tests exhibit a higher level of complexity owing to the sophistication of chemistry, hardware, and software innovations. (16,17) Therefore, it is unclear whether traditional methods of quality assurance and assessment will suffice for this higher level of complexity or whether new paradigms must be developed.
Proficiency testing is defined in International Organization for Standardization (ISO) 17043 as: evaluation of participant performance against preestablished criteria by means of interlaboratory comparisons.'' (18) This can be achieved through participation in a formal PT program. Proficiency testing programs usually focus on the analytic results. External quality assessment also provides evaluation of laboratory performance on examination of external samples but focuses more on the preanalytic (preexamination) and postanalytic (postexamination) activities than PT does. (18) In the absence of a formal PT or EQA program, laboratories can assess their performance through alternative assessment activities, such as exchanging samples with another laboratory performing similar tests, or by internal assessment. (19)
Proficiency testing/EQA is an important component of clinical laboratory quality assurance. It provides a mechanism to compare analytic test performance among different laboratories, which is important for determining the consistency of test results for a common analyte. Participants in formal PT/EQA programs periodically receive specimens (with a genotype unknown to them), which are tested in a manner similar to procedures used for their regular clinical samples. Participating laboratories return the results of the requisite analyses to the PT/EQA program, which then compiles the data and provides summarized results and educational insights to the participants. These programs provide an independent measure of laboratory performance in comparison with an external standard or a mean value obtained by other participating laboratories. Participation in PT/EQA allows laboratories to recognize analytic and interpretive errors that may indicate internal problems with quality control, calibration, assay design, or test interpretation. This is important because most clinical molecular genetic tests are developed by individual laboratories and are not available as commercial test kits that are manufactured and evaluated in a consistent manner. Also, unlike molecular tests for infectious diseases, such as HIV and tumor markers, which are typically used for patient monitoring and may be performed repeatedly, most molecular genetic tests for inherited disease are performed only once in a patient's lifetime. In this situation, errors may not be noticed through discrepancy with subsequent testing.
An international survey of molecular genetic testing laboratories determined that 74% of responding laboratories participate in PT/EQA, which was suggested to correlate with higher quality assurance scores. (20,21) The most common reason cited by laboratories for failure to participate in PT was a lack of programs relevant to the clinical laboratory services offered. (21) In the United States, formal molecular genetic PT programs are available from the College of American Pathologists (CAP) (22) for 27 tests for inherited diseases, representing only a small fraction of the 1739 (US) or 2247 (worldwide) molecular genetic tests currently available. (1) However, the 27 CAP PT surveys are for tests with the greatest frequency of use, including factor V Leiden, prothrombin 20210A variant, cystic fibrosis, and fragile X syndrome. The CAP and other PT/EQA providers also offer schemes for many other molecular genetic tests, including cytogenomic microarray analysis, pharmacogenetics, paternity testing, and human leukocyte antigen typing.
Another survey of clinical genetic laboratories in the United States indicated that increased participation in PT correlated directly with fewer PT failures and fewer incorrect patient test reports, as reported by laboratories. (23) Published perspectives about the ability of PT to accurately measure routine laboratory performance are conflicting; therefore, this correlation may be difficult to prove. (24-30) Formal PT/ EQA provides interlaboratory comparison of specific samples and does not always examine the entire testing process or day-to-day quality management issues that may affect laboratory performance. Proficiency testing represents a snapshot'' and is not intended to provide a comprehensive evaluation of a laboratory's quality assurance processes. Nonetheless, it has documented value for identifying problems that compromise the quality of laboratory test results.
Regulatory authorities and professional organizations recognize that PT/EQA is an essential component of quality assurance and have developed policies and recommendations for inclusion of PT/EQA, where feasible, into laboratory practice. Several international organizations, such as the Clinical and Laboratory Standards Institute and the ISO, have published guidelines and standards related to PT for molecular diagnostic methods. The Clinical and Laboratory Standards Institute protocol documents MM14-A, Proficiency Testing (External Quality Assessment) for Molecular Methods, (31) GP27-A2, Using Proficiency Testing to Improve the Clinical Laboratory, (32) and GP29-A2, Assessment of Laboratory Tests When Proficiency Testing is Not Available, (19) offer guidance for the management and operation of PT/ EQA for PT providers as well as molecular genetic diagnostic laboratories. The ISO/International Electrotechnical Commission standard 17 043:2010, Conformity Assessment--General Requirements for Proficiency Testing, (18) specifies the requirements for the competence of PT providers and for the development and operational aspects of providing PT schemes. This guidance also describes methods of PT testing. The ISO document 15189, Medical Laboratories--Particular Requirements for Quality and Competence, (3) recommends that laboratories participate in EQA and that these EQA schemes should provide clinically relevant challenges that mimic clinical samples and encompass all steps of the testing process, including preanalytic and postanalytic components.
In the United States, the Secretary's Advisory Committee on Genetics, Health, and Society was mandated by the Department of Health and Human Services to report on the adequacy of genetic testing oversight and regulation and to identify gaps that could affect patient safety. In their 2008 report, U.S. System of Oversight of Genetic Testing: A Response to the Charge of the Secretary of Health and Human Services, (33) the Secretary's Advisory Committee on Genetics, Health, and Society formulated recommendations to address the main gaps in genetic testing oversight. One recommendation addressed the absence of formal PT programs for all genetic tests, particularly rare genetic disorders, and suggested that the Department of Health and Human Services promote the development of new PT products by investigating other performance assessment approaches, including method-based processes.
In this article, we review PT/EQA as it applies to molecular genetic testing for inherited conditions in the United States, Europe, and Australasia. We consider the successes and demonstrated value of available PT/EQA programs and examine the challenges posed by evolving laboratory practices and testing technologies. Proficiency testing/EQA programs are often unable to assess the total laboratory testing process, focusing primarily on the analytic phase of testing (often neglecting the preanalytic and postanalytic phases) nor are they able to provide PT/EQA for all available tests. This critical evaluation suggests the need for research and development of targeted efforts to meet future PT/EQA needs.
PT/EQA--THE CURRENT SITUATION
Resources Describing the Scope of Genetic Testing
No formal mechanism exists either to capture information, such as disorders and genes tested, or analytic methods about all genetic tests offered in the United States or worldwide or to identify all laboratories that perform these tests. Data are limited to voluntary registries, such as the GeneTests (34) and Orphanet. (35) These resources do not obtain information from laboratories that either decline to analyze samples submitted from outside of their own institutions or that decline to register for other reasons. Thus, the true scope of molecular genetic testing and the laboratories that perform such testing is unknown. The National Institutes of Health in the United States has recently developed another registry (36) to collect information about genetic tests offered in the United States, although it may not be more comprehensive than the registries that currently exist.
According to the GeneTests Web site, the number of disorders with available genetic testing worldwide has increased more than 20-fold between 1993 and 2012 (from 100-2300 tests). (34) Nucleic acid-based testing is available for about 2000 of those genetic disorders; other diagnoses use biochemical, chromosomal, or other genetic tests. (1)
The Orphanet database, also a voluntary registry, has catalogued 5954 rare diseases, but only some of those diseases have diagnostic testing available. The database lists 5424 laboratories that offer testing (both research and clinical), mostly located in Europe. (35)
Molecular Methods Used for Genetic Testing
Many molecular methods can be used to detect mutations, depending on the disorder and the associated molecular defects. For example, many common mutations cause cystic fibrosis; testing for that disorder is usually performed by targeted mutation analysis using a panel of the more frequent mutations (single or a few nucleotides) associated with severe disease phenotype. Duchenne muscular dystrophy is most often caused by deletions and duplications in the DMD gene; therefore, testing for that disorder includes deletion/duplication analysis. Other commonly used molecular testing techniques include DNA sequence analysis, mutation scanning, and methylation analysis. To estimate the percentage of tests using each method, we chose a random sample (10%) of diseases with available molecular genetics diagnostic methods (February 26, 2009, report of ~970 diseases obtained from GeneTests) and tabulated the method or methods used to detect mutations for each. Whole-gene or targeted-exon DNA sequence analysis was used by the laboratories to analyze 93% (90 of 97) of the disorders in our sample (Table 1). Testing for 49% (48 of 97) of the disorders in our random sample was performed using only DNA sequencing techniques. That fraction may increase as next-generation DNA sequencing technologies transition to clinical laboratories. Testing for approximately 23% (22 of 97) of the disorders in our sample used targeted mutation analysis, and testing for 27% (26 of 97) of the disorders used deletion/ duplication analysis, usually as a follow-up to sequencing assays, which may miss large deletions and duplications (Table 1).
Molecular Genetic Tests Evaluated by Formal PT/EQA Programs
In the past 2 decades, formal PT and EQA programs have become an integrated part of clinical genetics laboratory practice. There are many regional, national, and international PT/EQA programs. We describe here some of the larger programs that serve national and international participants; however, smaller programs that serve a more-limited base or programs without publicly available information were not included. A comprehensive list of available molecular genetic PT/EQA programs can be found on the Eurogentest Web site (http://www.eurogentest.org/ laboratories). (14) In the United States, CAP is the largest provider of molecular genetic PT challenges. (22) In partnership with the American College of Medical Genetics, CAP offers PT to laboratories worldwide for 27 inherited genetic disorders, with 5 pharmacogenetic loci and methods-based PT for cytogenomic microarray analysis and for post-analytic Sanger DNA sequence analysis (Table 2). The samples distributed in the PT challenges are typically highly purified nucleic acids extracted from human cell lines. This program provides participants with 3 samples per disorder twice per year. The methods-based challenge for Sanger DNA sequencing assays is currently focused on interpretation of electronic data files but will evolve to include a ''wet'' challenge using extracted DNA and including DNA sequence analyses. The Centers for Disease Control and Prevention's Newborn Screening Quality Assurance Program (37) offers PT for molecular cystic fibrosis screening. The Newborn Screening Quality Assurance Program requires laboratories to extract DNA from blood spotted on filter-paper collection devices and sends 5 blood spots to participating laboratories on a quarterly basis, which allows laboratories to perform all phases of the testing process, including DNA extraction from the appropriate matrix, within the PT context.
European PT/EQA challenges are provided by both national and international organizations (Table 2) with additional participation by laboratories outside Europe. The European Molecular Genetics Quality Network (EMQN), (38) which is supported by user subscriptions, offers disease-specific EQA for a variety of genetic disorders as well as methods-based EQA for Sanger DNA sequencing, molecular cytogenomic analysis, and mutation scanning. The EMQN EQA programs provide 3 lyophilized DNA samples per disease to participants once each year. Participants are assessed on their ability to correctly genotype, interpret, and report the results using their usual laboratory report format. The Cystic Fibrosis (CF) European Network (39) offers an external quality assessment scheme for CF molecular genetic testing to more than 200 laboratories worldwide. This program ''aims to evaluate the entire analytical process, from DNA sample receipt and genotyping up to the written report with the final interpretation of the data as it is normally being sent to the clinician who requested the genetic test.'' The United Kingdom National External Quality Assessment Service (UKNEQAS) (40) provides external quality assessment for a range of inherited diseases currently tested in diagnostic molecular genetic testing laboratories by providing challenges using lyophilized DNA or dried blood spots. This program examines the analytic and postanalytic stages, including evaluation of laboratory reports of testing and provides 3 samples per disorder per year to participants worldwide. The Italy-based Istituto Superiore di Sanita addresses current practice, problems, and future directions of interlaboratory comparisons. Their focus is PT/EQA for molecular genetic testing of a few diseases. The PT materials used are DNA samples extracted from lymphoblastoid cell lines. This program covers both the analytic and postanalytic phases of testing and provides 6 samples per disease annually. (41) The Deutsche Vereinte Gesellschaft fur Klinische Chemie und Laboratoriumsmedizin eV (42) in Germany offers a variety of test modules as well as methods-based PT for DNA sequence analysis and DNA isolation for factor V Leiden genotyping. Lyophilized DNA is provided for all its PT challenges, except the DNA isolation scheme, for which, whole-blood samples are sent.
Since 2008, the Human Genetics Society of Australasia Molecular Genetics Quality Assurance Program (43) has offered 23 disease-specific PT modules in collaboration with EMQN and the CF European Network. This program also offers 1 to 3 generic modules, per annum, which are based on methods and techniques to enable wider participation in the program. In particular, the generic modules are aimed at laboratories that do not participate in any of the disease-specific modules but use the same methods or technologies in their testing. One such module was an audit-based assessment of result reporting, for which laboratories were asked to provide deidentified copies of reports for one positive, one negative, and one not-tested result received for analysis within a specific period. Another module was ''Mutation Detection and Biological Interpretation,'' for which, laboratories were provided with amplified products and sequencing primers and were required to genotype the fragment, detect the variation, and report a biologic interpretation of the results. Laboratories were assessed for measures of the quality of data provided and result interpretation. Laboratories from Australia, Hong Kong, Indonesia, and Saudi Arabia participate in this program, which has amalgamated with the Royal College of Pathologists of Australasia quality assurance programs (RCPA QAP Pty/Ltd, Surry Hills, New South Wales, Australia) in 2010. There is also a PT program for cytogenetic testing offered through the Human Genetics Society of Australasia.
These formal PT/EQA programs include some of the more-common genetic disorders that are tested in many laboratories (Table 2). However, these tests represent only a small fraction of the more than 2300 disorders with available genetic tests. Proficiency testing/EQA providers select disorders, such as CF, which are tested by multiple laboratories to make the programs economically feasible and usually perform customer surveys to assess needs before developing a new disease challenge. In 2003, EQA was available for 8.4% and 4.2% of available genetic tests in the United Kingdom and the Netherlands, respectively. However when these data are considered in the context of how frequently a particular test was performed, that represented 63% of the total cases reported in the United Kingdom and 48% of the cases reported in the Netherlands at that time. (44)
The availability of appropriate and diverse materials, such as blood from affected patients or characterized cell lines from which PT/EQA materials are derived, also affects the availability of PT/EQA programs. It is not logistically or economically possible to provide a formal PT/EQA challenge for disorders that are tested in only one or a few laboratories or for those without a supply of available materials.
In the absence of formal PT/EQA schemes, laboratories in the United States and elsewhere must evaluate the performance of their assays using alternative methods (alternative assessment). This can be accomplished by blinded retesting of previously tested samples, by sample exchanges with laboratories performing similar tests, or by internal evaluation of data. Methods for alternative assessment are described in numerous guidance documents. (3,19,31,45,46) Proficiency testing/EQA providers, such as CAP and UKNEQAS, facilitate sample exchanges among laboratories for tests without formal PT/EQA schemes. The effect of alternative assessment on laboratory quality has not been evaluated.
Phases of Testing and Their Evaluation by PT/EQA
To have the greatest value, PT/EQA challenges should evaluate performance in the preanalytic, analytic, and postanalytic (preexamination, examination, and postexamination) stages of the testing process. Laboratories should process and analyze PT/EQA samples in the same manner as routine clinical patient specimens to the fullest extent possible. Successful PT/EQA participation should demonstrate proficiency in all examined phases of testing.
Preanalytic Phase.--The preanalytic testing phase includes the receiving, accessioning, labeling, and initial processing of the sample. When PT/EQA results are evaluated across laboratories, a common type of error is a sample switch or mislabeling that is probably caused by a clerical error in the preanalytic phase. Even though the analytic process may be performed correctly, the clerical mistake made earlier in the testing process produces incorrect test results and reporting (Table 3). This type of error is of particular concern because laboratory workers are frequently aware that the sample was provided for PT/EQA purposes. (47) However, because PT samples do not enter the laboratory workflow in the same way that routine patient samples do, there may be hidden, uncontrollable factors in the acquisition of PT specimens that could make such errors more likely.
The next step in the testing process is nucleic acid isolation, which can be performed using laboratory-developed methods or a variety of commercially available reagent kits and instruments. The isolated DNA or RNA obtained through these extractions may be derived from blood; bone marrow; formalin-fixed, paraffin-embedded tissue; or fresh tissues. The quality of the isolated nucleic acids is central to the success of molecular genetic testing methods.
The ideal PT/EQA sample would be a clinical specimen because it would most closely represent what is actually tested in a clinical laboratory and would permit the evaluation of all phases of the testing process, including the DNA extraction step. (47) It is, however, often difficult or impossible for formal PT programs to obtain sufficient quantities of appropriate, high-quality, safe, homogeneous, and stable clinical samples to supply all of the laboratories participating in the PT event with the exact same specimen material. (44,47) For many genetic disorders, it is difficult to recruit enough patients to reflect the variety of mutations that would be required for a comprehensive PT/EQA program, especially for rare diseases or mutations. Institutional review board and informed consent requirements may also present an obstacle to the use of clinical specimens. Clinical samples are often incompletely characterized and may yield unexpected genotypes, including patterns of mosaicism, minor clonal populations, (48) or previously unidentified alleles. (49,50) It is also difficult to transport potentially infectious whole blood across international borders. (51) The DNA extracted from cell lines, which is available in almost unlimited supply, is often substituted as PT/EQA samples. The disadvantage of this practice is that the nucleic acid isolation phase is not performed by the participating laboratory and cannot be evaluated as part of the PT process. In addition, sources of potential PT materials are limited in the range of diseases and mutations represented. For these reasons, most PT/EQA samples are not directly comparable to the samples usually received and cannot be processed in the same way as actual clinical samples. (47) Anecdotal evidence also indicates that some analytic techniques, such as next-generation sequencing and cytogenomic microarrays, often do not perform optimally with DNA isolated by some methods used by external sources.
Analytic Phase.--The analytic phase encompasses the actual testing of the analyte. Each participating laboratory performs the testing using its own validated method or methods. In surveys with many subscribers, such as for CF, it is not uncommon for participants to use a wide variety of commercially available assays as well as laboratory-developed tests. Although PT/EQA performance is usually excellent for molecular genetic tests, (52,53) analytic errors do occur. Some errors may be associated with the design of the assay. For example, previously unrecognized polymorphic variants located in the binding sites of the amplification primer may preclude effective amplification and detection of the actual mutation. One PT/EQA scheme for CF determined that a particular laboratory-developed test could not accurately detect the 621+1G>T mutation in the PT sample. The laboratory subsequently removed this mutation from its clinical assay. (54) In a similar case, a method-specific artifact produced a false result in a hereditary hemochromatosis PT challenge. (55) Poor performance on a PT/EQA challenge alerted another laboratory that the primer binding site in a BRCA assay was too close to the target mutation nucleotide, which hindered the detection of the variant. (44) Proficiency testing/EQA testing is one mechanism to alert laboratories to such problems and to indicate changes to avoid future errors. The results of PT/EQA can also be used to compare laboratory performance with different assay methods among laboratories. (54) This has been especially useful when assessing the accuracy of trinucleotide repeat sizing. For example, it is very important to accurately determine the size of fragile X premutation expansions to correctly predict the risk of allelic expansion, premature ovarian failure, and fragile X-associated tremor/ataxia syndrome. Inaccurate sizing, by even a few triplet repeats, could affect prenatal diagnosis and the risk of expansion estimates or cause conflicting reports among different family members with similar repeat sizes tested in different laboratories. If laboratories cannot accurately determine the size of the fragile X triplet repeat, when compared with results from the other PT/EQA survey participants or from the previously measured repeat size of the sample, additional calibration of the assay is warranted. Poor performance by several laboratories in the 2002 and 2003 UKNEQAS fragile X PT/ EQA scheme resulted in the development of consensus testing and reporting guidelines in the United Kingdom. (44) The American College of Medical Genetics assesses results from a subset of the CAP PT surveys; if a particular problem occurs at a high frequency, disease-specific testing practice guidelines are established and published. (47) American College of Medical Genetics guidelines for fragile X testing (56) were written in response to suboptimal performance on the CAP fragile X proficiency survey.
Postanalytic Phase.--During the postanalytic phase of testing, the test results are reviewed and interpreted. Most of the molecular genetic PT modules offered by the CAP have an analytic, as well as a clinical interpretation, component, which are graded separately. (44) Participants provide an interpretation of the detected genotype within the context of a described clinical scenario (presentation of the patient), such as whether the identified mutations are consistent with the diagnosis of the disorder or whether the genotype indicates a genetic carrier. However, other components of the laboratory report, such as compliance with existing guidance or recommendations (15) describing required information elements, accuracy about the indications for testing, test performed, results obtained, and appropriateness of follow-up guidance, are not evaluated. In European EQA programs, such as UKNEQAS, the CF European Network, and the EMQN, the participants are required to submit results in their usual clinical reporting format. Those reports are assessed for accuracy of genotyping, the appropriateness of the interpretation, and clerical accuracy. (38-40,44,51,57) That requirement allows the laboratory's interpretation of the analytic results to be evaluated in the context of the mock clinical data supplied with the DNA samples and permits assessment of other important elements in the laboratory report, such as residual risk calculations, recommendations for further testing, use of proper genetic nomenclature, and accurate inclusion of patient identification such as name, gender, and birth date. Correct interpretation of the analytic result is essential because patient management will, in many cases, be based on a combination of the analytic result and the final interpretation. This is especially important for genetic testing, where the test may only be performed once in a patient's lifetime. Because a genotype is not informative, per se, proper result interpretation requires integration with other information that may include family history, ancestry, and knowledge of genotype/phenotype associations, which is important in clinical genetics because insights about the causes of disease and the effects of various mutations or combinations changes rapidly and may affect the clinical management of the patient.
Another important component of the postanalytic analysis is laboratory reporting of the identified mutation using the appropriate gene mutation nomenclature, which makes clear to the physician interacting with the patient which sequence change has been identified. This may also include a reference to ''common'' nomenclature, such as ''factor V Leiden,'' whichisnotconsensusnomenclature but may be helpful in the report because it is most familiar to clinicians. (44,51) Use of incorrect or ambiguous nomenclature can lead to errors in interpretation, treatment selection, and testing other family members or their result interpretations, especially if testing is performed in different laboratories.
Methods of grading proficiency test results vary with the provider, the specific scheme, and the analyte. Some providers, such as the CAP, grade survey results on the accuracy of the genotypic result and the interpretation. For example, the CAP has recently started to grade the sizing of Huntington disease and myotonic dystrophy repeats in all size categories but has graded interpretation at 80% consensus for many years. Many of the CAP surveys are graded based on the consensus of 80% of participants; if that level of consensus is not reached, the challenge remains ungraded and ''educational.'' Other providers, such as EMQN, provide numeric scores based on genotyping, interpretation, and reporting. (38) Each EMQN participant receives an individualized report with their scores and comments from the evaluators, including areas of their test report that need improvement. Scoring of EMQN schemes depends on the analyte and the scheme. For example, there are 2 EQA schemes for BRCA gene testing (breast cancer risk). The BRCA Full scheme assesses genotyping, biologic, and clinical interpretation, whereas the EMQN BRCA Geno scheme assesses only genotyping and biologic interpretation. Other EMQN schemes, such as congenital adrenal hyperplasia and hereditary deafness (GJB2/GJB6 mutations), score genotyping only. These differences reflect the reporting practices of the participating laboratories. Finally, it is often challenging for PT programs to compare results because of the use of different methods, different calibration standards, and different cutoff values by participants. These issues do not usually affect molecular genetic testing for inherited genetic disease because most results are qualitative, but they are important for tests such as BCR/ABL1 or viral load, where the results are quantitative. Proficiency testing/EQA programs cannot effectively grade some quantitative surveys because of the lack of calibration materials, standardized values for reference ranges, and cutoff values.
The Role of Laboratory PT in Quality Management
Participation in PT/EQA allows laboratories to compare their performance against the range of responses provided by a group of peer laboratories. These comparisons are most effectively used to influence laboratory practice when they are systematically integrated into a Quality Management System (QMS; defined by the ISO as a management system to direct and control an organization with regard to quality''). (3) As part of QMS, a laboratory must participate in relevant PT/EQA schemes and ensure that the cost is integrated into the budget with sufficient time and staff resources assigned to support participation. In addition, the laboratory must ensure that the results of PT/EQA are properly considered, disseminated, and implemented to improve laboratory testing. Participation in appropriate PT/ EQA and/or alternative assessment is a requirement within the CAP, the ISO, and the Australian accreditation processes.
The quality policy implemented through a QMS defines the overall approach to PT/EQA and how PT/EQA challenges are appropriately handled in the laboratory. The QMS should include clearly defined laboratory procedures for receipt of performance results from a PT/EQA agency or other assessment protocols, such as alternative assessment. This procedure will normally follow 3 phases. First, the laboratory director is required to disseminate the data to staff immediately. This conveys to staff the importance of participating in PT/EQA schemes and enhances the educational content of the exercise. The performance data must be reviewed for any indication of deficient performance that requires immediate corrective action to avoid errors. Proficiency testing/EQA data should be routinely discussed in the most appropriate meetings of management, the quality team, and staff so that lessons learned can be discussed and, if necessary, changes in standard operating procedures can be quickly integrated into the QMS. Proficiency testing/EQA data provides an opportunity for a laboratory to compare its performance with peers as well as to celebrate and praise the staff when performance is exemplary. Second, PT performance data should be presented during management review meetings to examine any recurrent deficiencies that require correction or improvement. Third, PT/EQA records should be stored and formatted for external audit by an accrediting agency.
Laboratories may experience a lapse or error in PT/EQA performance. Single occurrences of poor performance should be logged as an incident and used as an opportunity to review procedures and make improvements. A careful evaluation of the error may determine whether there is a system failure that may require redesign of a test, more-frequent instrument calibration, or adjustments to training procedures. However, PT/EQA may detect serial or persistent failures that the laboratory is obligated to address with a more fundamental review. Some PT/EQA providers report that laboratories have discontinued testing services following instances of poor performance in PT/EQA. (44) That indicates the importance that laboratory directors assign to this external comparison and emphasizes the role of PT/ EQA as an educational tool and mechanism to improve performance in clinical laboratories.
In some countries, PT/EQA providers are required to report poor and recurrent poor performance to an official monitoring agency. That agency may have a role in ensuring that corrective actions are designed to address poor PT/EQA performance. Ultimately, they may have the power to escalate their intervention to involve the host institution of the laboratory concerned as well as regulatory agencies.
Evidence of the Value of PT/EQA in Ensuring Good Laboratory Performance
Although few studies have addressed whether participation in PT/EQA programs directly improved laboratory performance, there is empirical evidence showing that the educational aspect of PT/EQA does help laboratories detect errors in their testing protocols and identify problems associated with their assays. (2) One study reviewed 3 rounds of PT data from 2002 (any provider) from approximately 6300 CAP-accredited laboratories. (58) The study evaluated whether laboratories corrected deficiencies identified by PT or whether they continued to have unsuccessful PT performance. The analysis indicated that about 90% of the PT problems were resolved after the first round of PT and 99% were resolved by the third round, suggesting that the laboratories had successfully corrected mistakes identified by PT performance. Other studies have shown that participation in the CAP ''Calibration Verification/Linearity Surveys,'' which examine calibration verification and analytic measurement range of several nongenetic analytes (chemistry, immunology, hematology, among others), is associated with fewer PT failures. (59,60) A UKNEQAS PT/EQA scheme for hemophilia A from 2003 failed 4 laboratories based on poor performance in the evaluation of their clinical report. In subsequent surveys, only 1 laboratory failed to report properly, suggesting that the educational value and improvement of testing practices were derived from that exercise. (51) Another value of PT/EQA is the opportunity for laboratories to analyze unusual samples not often encountered in their service, such as, mosaicism of variants in fragile X syndrome.
The UKNEQAS PT/EQA program emphasizes interpretation of data and their implications for both the patient and the family. One summary of data indicates that poor PT/ EQA performance because of interpretation errors has decreased between 1997 and 2006. (44) This may be derived from continued participation in PT/EQA.
Analysis of PT/EQA results has also revealed inconsistent results stemming from the lack of uniformity in practices among laboratories. Once these issues were identified, steps were implemented to harmonize practices among the laboratories. Poor and inconsistent performance of laboratories participating in a PT/EQA scheme for fragile X syndrome led to consensus testing and reporting guidelines development in the United Kingdom (44) and the United States. (56) Because of ambiguities in the ways laboratories report sequence variations, PT schemes (such as UKNEQAS) strongly suggested that genotypic PT/EQA results should be reported using the Human Genome Variation Society standard nomenclature. (44,51) Participants in the CAP's disease-specific surveys typically report results using the common allele names; however, CAP requires the use of the Human Genome Variation Society nomenclature for participants in its PT scheme on interpretation of sequence variants in rare disorders and is moving toward integration of the consensus nomenclature for all schemes.
Limitations of PT
To assess a given test, PT/EQA should evaluate the capability of a particular assay to identify a range of possible test results or alleles, either during an individual challenge or during the course of several challenges. Considering that, PT for a given disease or genetic variant may be limited by the following:
1. Issues related to the limited availability of resources, including appropriate samples and reference materials;
2. Logistical and practical difficulties encountered, including the inability to offer PT/EQA for all genetic disorders, especially those tests for rare disorders or those performed by a few laboratories only; and
3. Limitations in the PT process for evaluation of laboratory performance.
Ideally, PT/EQA should evaluate the ability of the laboratory to identify all genotypes of interest for a particular disorder, which requires the availability of appropriate samples. For simple tests, such as factor V Leiden or prothrombin 20210A, very few alleles are tested in a clinical setting; thus, relatively few PT/EQA samples are required to represent the population variant affecting patients. For other disorders, such as CF, there is a large variation in the number and composition of alleles included in clinical assays. Some assays only examine the 23 alleles recommended for carrier screening by the American College of Medical Genetics and the American College of Obstetricians and Gynecologists, (61) whereas other laboratories offer assays that test more than 100 alleles, some of which are unique to particular patient populations. Many of the alleles included in the more comprehensive CF assays are very rare, and it is often difficult to obtain samples from patients or cell lines for PT/EQA. Another difficulty is the lack of highly characterized reference materials or calibrators with which to evaluate potential PT/EQA materials. This is especially important for DNA fragment-sizing assays, such as triplet repeat testing but may also be a factor for qualitative assays.
Proficiency testing programs also encounter logistic and practical problems. Most of the 2300 clinical genetic tests are offered in only one or a few laboratories, making provision of a formal proficiency survey logistically and economically difficult. Many multiplex genetic tests may simultaneously detect dozens of alleles (eg, CF or pharmacogenetic loci). It is logistically impossible for PT/EQA programs distributing 3 samples 1 or 2 times a year to provide enough different samples to adequately challenge the variety of genotypes represented in all patients or all assays for a particular genetic disorder. The DNA sequence analysis identifies mutations in any part of a gene and can pose a unique challenge in interpretation. In addition, PT/EQA typically targets well-characterized conditions and mutations. However, new disease associations are discovered regularly and genotype-phenotype correlations are often not fully elucidated until years after the initial discovery. Allelic heterogeneity and the clinical effect of some sequence variants, such as missense mutations and complex variants, pose considerable challenges for interpretation when functional studies have not yet been performed or were inconclusive. An example of this situation is pharmacogenetic testing for CYP2D6 variants. Many tests for this gene include polymorphisms with undefined effects on CYP2D6 enzyme activity. These less-well-characterized variants are challenging for developing of reference materials, reporting, and interpreting PT/EQA results. Formal PT/EQA might become available for more inherited conditions in the future, but it will probably not be feasible for such programs to cover all conditions and available tests.
Approaches to Address Limitations of Proficiency Testing: Methods-Based PT/EQA
Proficiency testing that is designed to evaluate the performance of analytic methods, rather than to evaluate specific genotypic assays, has been proposed as a complmentary mechanism to more-broadly assess laboratory performance. (45,62-64) Methods-based PT/EQA examines technologies common to many genetic tests, such as DNA purification, polymerase chain reaction (PCR) amplification, or DNA sequencing, and allows evaluation of critical analytic steps for individual assays that are not directly assessed by traditional PT/EQA schemes. It also permits interlaboratory comparisons and can highlight analytic practices, such as DNA quantification, which could be optimized for improved performance. (62)
The European Commission funded a project (EQUAL) (Full program title: ''Multinational External Quality Assay [EQA] Programmes in Clinical Molecular Diagnostics Based on Performance and Interpretation of PCR Assay Methods Including Dissemination and Training'') to develop and evaluate the utility of methods-based EQA to address methodologic procedures and analytic proficiency in molecular diagnostic test performance independent of the target. Three EQUAL pilot projects for qualitative analysis, quantitative PCR, and DNA sequencing (EQUAL-Qual, EQUAL-Quant, and EQUAL-Seq) were initiated. (62,65,66) The results from those pilot studies identified many areas of laboratory performance that varied considerably between participants (Table 4) and suggested areas that could be targeted for improvement.
Because of relatively poor laboratory performance in the EQUAL-Seq project, specific training highlighting analytic and methodologic skills were subsequently offered to the participating laboratories. A significant improvement of technical and interpretative skills was demonstrated in a confirmatory second round of EQA. (67)
Currently, methods-based PT is available for a variety of assays (Table 4). The EMQN offers a methods-based scheme for Sanger sequencing. Participants receive amplicons to characterize, identify, and report the sequence variants using the proper nomenclature. They also provide their raw sequence data, which is evaluated by EMQN for quality scores, quality read length, and quality read overlap. (63) In the United States, the CAP has launched an electronic DNA sequence-analysis survey. In 2012, that survey will send a set of primers and 3 DNA specimens to participants. Participants will identify all variants and report on them with accepted nomenclature. In the future, both the CAP and the EMQN plan to offer methods-based surveys to address the performance of next-generation DNA sequencing.
Synthetic DNA samples can be used for PT/EQA. These samples may be composed of plasmid DNA containing specific sequences or PCR amplicons. The CF Network tested the usefulness of a synthetic reference material containing 6 homozygous mutations in the cystic fibrosis gene (CFTR) and one polymorphism for suitability as PT material. (68) Most of the laboratories in the study successfully identified the mutations, although some technical difficulties, such as incorrect genotyping (10 of 197 participants; 5%), or absent, incorrect, or insufficient interpretation (33 of 197 participants; 17%) regardless of genotype, were reported. This synthetic sample was supplied in a bloodlike matrix, which also permitted evaluation of the DNA extraction step. In the United States, artificially constructed CF mutation samples designed to mimic extracted human genomic DNA produced similar results. (69)
Proficiency testing has demonstrated value as an important laboratory quality assurance tool and has helped laboratories identify issues related to test design and performance. In addition, the ability to compare laboratory performance with others using the same or different methods on identical samples can highlight issues related to test methodology or interpretation or may inform development of best-practice guidelines and standard policy.
Adoption of new and complex testing technologies, such as next-generation sequencing assays, will require modifications to PT/EQA design and provision. In contrast to traditional genetic tests, which identify only a few mutations or perhaps the whole sequence of a well-characterized gene, next-generation sequencing has the capacity to examine the sequence of large gene panels, the exome, or the entire human genome, with an almost infinite variety of possible variants. Proficiency testing programs for next-generation sequence assays need to monitor the ability of the laboratory to detect mutations in any part of the genome included in their validated test. In addition to the analytic phase of the testing process, the data analysis and interpretation of next-generation DNA sequencing is considerably more complicated than that of sequencing tests with smaller scopes. Proficiency testing/EQA can be used to compare performance among laboratories during all phases of next-generation DNA sequencing testing and may provide important indicators of which steps of the testing process are problematic. That information may not be readily discernible through daily quality-control practices, and PT/EQA could be an important tool to assess whether the testing algorithm is sufficient to detect a loss of sensitivity or specificity for the detection of sequence variations that may only be evident when comparing results among laboratories. Interpretation of the analytic test result of such large-scale analyses is difficult and uncertain because the effect of individual mutations, the function of each gene and its interaction with other genes in the genome has not yet been determined. Proficiency testing/ EQA schemes to assess the ability of the laboratory to interpret and report complex data could be quite informative. Novel and innovative PT/EQA challenges will need to be developed to ensure the quality of these new tests.
Proficiency testing or EQA should include sufficient analytes to provide a reasonable estimate of interlaboratory comparability. For example, PT for factor V Leiden testing assesses the laboratory's capacity to identify 1 of 3 possible genotypes per sample (wild type, factor V Leiden heterozygote, and factor V Leiden homozygote). Developing a PT program to gauge interlaboratory comparability in this example is fairly straightforward. Proficiency testing for disorders with many disease-associated alleles, such as CF, or a next-generation sequencing test presents additional challenges. Current PT can only assess a subset of possible mutations per challenge. Is the current process sufficient for achieving a credible interlaboratory comparison for very complex tests? The limited availability of characterized DNA materials and the cost associated with increasing the number of PT samples can be significant barriers for a PT/ EQA program with a goal to offer comprehensive challenges.
Significant research, needs assessment, and pilot testing should be performed to ensure that PT addresses the changing needs of genetic testing technology and an evolving knowledge base and continues to be a relevant and useful quality assurance tool. Some of the research topics may include ways to provide PT/EQA for the increasing number of new tests, many of which are offered in only one or a few laboratories; to include all phases of the testing process, rather than just the analytic phase; and to develop novel approaches for effective multiplex genotype-testing challenges, as well as PT/EQA strategies to assess new technologies, such as next-generation sequencing. Additional research will define PT/EQA improvements to promote the quality of laboratory testing, interpreting, and reporting.
Most genetic tests are done in only one or a few laboratories. Many of these tests use the same technology, such as Sanger or next-generation sequencing to examine a particular gene or set of genes. Many other tests share similar methodology, such as DNA and RNA purification, PCR amplification, or multiplex ligation-dependent probe amplification. Other aspects, such as result reporting, are also common across tests. Research to develop and evaluate novel method- or technology-based PT/EQA schemes may simultaneously assess the performance of many tests. Schemes that evaluate the quality of laboratory reports, for example, those of the CF Network, UKNEQAS, and the EMQN, could also be developed more broadly. In addition, research should be conducted to assess the effectiveness and relationship of the commonly used methods of alternative assessment, such as sample exchange or blinded retesting of previously tested specimens, to the accuracy of routine laboratory test results. Information gleaned from such studies might provide guidance to laboratories on effective methods of alternative assessment.
Much work needs to be done to ensure that PT/EQA programs can meet the needs of new and evolving genetic tests and technologies. An important first step would be to design a scientific approach to collecting data on the effect of PT/EQA on laboratory testing quality. Such studies could measure the relationship between PT performance and the accuracy of routine test results in that same laboratory and could be expanded to include PT/EQA for the newer testing technologies, including microarrays and next-generation sequencing. Information from these studies might inform the development of improved PT programs, professional guidelines, and regulations.
A variety of research and development projects will address some of the current limitations of PT/EQA, including the shortage of characterized reference materials, both naturally occurring and synthetic, that can be used as PT/EQA samples. Cell lines can be created from patients with genetic disorders who are currently part of PT/EQA programs and also from those with disorders for which PT/ EQA would be useful but for which programs do not yet exist. Consideration should be given to each disorder to ensure that a comprehensive set of reference materials, containing as many clinically significant alleles as possible, will be developed. In addition, development of synthetic reference materials containing many alleles for a given disorder simultaneously should also be considered. These materials must be evaluated by testing in several clinical laboratories using a variety of assays and technologies before they are used as PT/EQA samples.
Proficiency testing and external quality assessment have been and should remain an integral part of laboratory quality assurance. In the next few years, we hope to conduct and facilitate these and other research projects to evaluate and improve the quality of PT for molecular genetic testing.
The authors would like to thank John Beilby, PhD, Head, Diagnostic Molecular Genetics Department (PathWest, Nedlands, Western Australia, Australia), for his comments on this work.
(1.) National Center for Biotechnology Information. GeneTests. http://www. ncbi.nlm.nih.gov/projects/GeneTests/static/abou^dat^summary.shtml. Updated March 22, 2012. Accessed May 14, 2012.
(2.) Dequeker E, Ramsden S, Grody WW, Stenzel TT, Barton DE. Quality control in molecular genetic testing. Nat Rev Genet. 2001;2(9):717-723.
(3.) International Organization for Standardization. Medical Laboratories--Particular Requirements for Quality And Competence. Geneva, Switzerland: International Organization for Standardization; 2007. Report No.: ISO 15189.
(4.) Clinical & Laboratory Standards Institute. Quality Management System: A Model for Laboratory Services; Approved Guideline--Fourth Edition. Wayne, PA: Clinical and Laboratory Standards Institute; 2011. Report No.: GP26-A4.
(5.) American College of Medical Genetics. ACMG Standards & Guidelines for Clinical Genetics Laboratories. http://www.acmg.net/AM/Template. cfm?Section=Laboratory_Standards_and_Guidelines&Template=/CM/ HTMLDisplay.cfm&ContentID=6958MD. Accessed May 15, 2012.
(6.) Clinical and Laboratory Standards Institute. Molecular Diagnostics Methods for Genetic Disease; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2006. Report No.: MM01-A2.
(7.) New York State Department of Health. Clinical Laboratory Evaluation Program. http://www.wadsworth.org/labcert/clep/clep.html. Accessed May 15, 2012.
(8.) Department of Health and Human Services Health Care Financing Administration. Clinical Laboratory improvement amendments of 1998-- laboratory requirements. Fed Regist. 1992;57(40):7146. Codified at 42 CFR [section]493. 2004. http://wwwn.cdc.gov/clia/regs/toc.aspx. Accessed May 15, 2012.
(9.) Clinical and Laboratory Standards Institute. Nucleic Acid Sequencing Methods in Diagnostic Laboratory Medicine; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2004. Report No.: MM9-A.
(10.) Clinical and Laboratory Standards Institute. Diagnostic Nucleic Acid Microarrays; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2006. Report No.: MM12-A.
(11.) Clinical and Laboratory Standards Institute. Verification and Validation of Multiplex Nucleic Acid Assays; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2008. Report No.: MM17-A.
(12.) Clinical and Laboratory Standards Institute. Establishing Molecular Testing in Clinical Laboratory Environments; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2011. Report No.: MM19-A.
(13.) Mattocks CJ, Morris MA, Matthijs G, et al; EuroGentest Validation Group. A standardized framework for the validation and verification of clinical molecular genetic tests. Eur J Hum Genet. 2010;18(12):1276-1288.
(14.) EuroGentest 2 Coordination Action. Eurogentest Web site: Laboratories. http://www.eurogentest.org/laboratories/. Updated April17, 2011. Accessed May 14, 2012.
(15.) Chen B, Gagnon M, Shahangian S, et al; Centers for Disease Control and Prevention. Good laboratory practices for molecular genetic testing for heritable diseases and conditions. MMWR Recomm Rep. 2009;58(RR-6):1-35. http:// www.cdc.gov/mmwr/pdf/rr/rr5806.pdf. Accessed May 10, 2012.
(16.) Anderson MW, Schrijver I. Next generation DNA sequencing and the future of genomic medicine. Genes. 2010;1(1):38-69.
(17.) Kearney HM, South ST, Wolff DJ, et al; Working Group of the American College of Medical Genetics. American College of Medical Genetics recommendations for the design and performance expectations for clinical genomic copy number microarrays intended for use in the postnatal setting for detection of constitutional abnormalities. Genet Med. 2011;13(7):676-679.
(18.) International Organization for Standardization-International Electrotechnical Commission. Conformity Assessment--General Requirements for Proficiency Testing. Geneva, Switzerland: International Organization for Standardization; 2010. ISO/IEC 17043.
(19.) Clinicaland Laboratory Standards Institute. Assessment of Laboratory Tests When Proficiency Testing is Not Available; Approved Guideline. Wayne, PA:. Clinical and Laboratory Standards Institute 2008. Report No.: GP29-A2.
(20.) McGovern MM, Benach MO, Wallenstein S, Desnick RJ, Keenlyside R. Quality assurance in molecular genetic testing laboratories. JAMA. 1999;281(9): 835-840.
(21.) McGovern MM, Elles R, Beretta I, et al. Reportofan international survey of molecular genetic testing laboratories. Community Genet. 2007;10(3):123-131.
(22.) College of American Pathologists. CAP Web site: Genetics. http://www. cap.org. Updated July 27, 2012. Accessed May 14, 2012.
(23.) Hudson KL, Murphy JA, Kaufman DJ, Javitt GH, Katsanis SH, Scott J. Oversight of US genetic testing laboratories. Nat Biotechnol. 2006;24(9):1083 1090.
(24.) Libeer JC. Role of external quality assurance schemes in assessing and improving quality in medical laboratories. Clin Chim Acta. 2001;309(2):173 177.
(25.) Ehrmeyer SS, Laessig RH. Has compliance with CLIA requirements really improved quality in US clinical laboratories? Clin Chim Acta. 2004;3 46(1):37-43.
(26.) Hofgartner WT, Tait JF. Frequency of problems during clinical molecular-genetic testing. Am J Clin Pathol. 1999;112(1):14-21.
(27.) Keenlyside RA, Collins CL, Hancock JS, et al. Do proficiency test results correlate with the work performance of screeners who screen Papanicolaou smears? Am J Clin Pathol. 1999;112(6):769-776.
(28.) LaMotte LC, Guerrant GO, Lewis DS, Hall CT. Comparison of laboratory performance with blind and mail-distributed proficiency testing samples. Public Health Rep. 1977;92(6):554-550.
(29.) Jenny RW, Jackson KY. Proficiency test performance as a predictor of accuracy of routine patient testing for theophylline. Clin Chem. 1993;39(1):76 81.
(30.) Howerton D, Krolak JM, Manasterski A, Handsfield JH. Proficiency testing performance in US laboratories: results reported to the Centers for Medicare & Medicaid Services, 1994 through 2006. Arch Pathol Lab Med. 2010;134(5):751758.
(31.) Clinical and Laboratory Standards Institute. Proficiency Testing (External Quality Assessment) For Molecular Methods; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2005. Report No.: MM14-A.
(32.) Clinical and Laboratory Standards Institute. Using Proficiency Testing to Improve the Clinical Laboratory; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2007. Report No.: GP27-A2.
(33.) Secretary's Advisory Committee on Genetics, Health, and Society. U.S. System of Oversight of Genetic Testing: A Response to the Charge of the Secretary of Health and Human Services. Bethesda, MD: Department of Health & Human Services; 2008. http://oba.od.nih.gov/oba/SACGHS/reports/SACGHS_oversight_report.pdf. Accessed May 10, 2012.
(34.) National Center for Biotechnology. GeneTests Web site. http://www.ncbi. nlm.nih.gov/sites/GeneTests/. Accessed May 14, 2012.
(35.) INSERM. Orphanet Web site. http://www.orpha.net/consor/cgi-bin/index. php. Last updated: May 13, 2012. Accessed May 14, 2012.
(36.) National Center for Biotechnology. Genetic Testing Registry Web site. http://www.ncbi.nlm.nih.gov/gtr/. Accessed May 15, 2012.
(37.) The Centers for Disease Control and Prevention. Newborn screening quality assurance program Web site. http://www.cdc.gov/labstandards/nsqap. html. Last updated May 9, 2012. Accessed May 14, 2012.
(38.) European Molecular Genetics Quality Network. The European Molecular Genetics Quality Network Web site. http://www.emqn.org/emqn/Home. Accessed May 14, 2012.
(39.) CF Network. Cystic Fibrosis European Network Web site. http://cf. eqascheme.org/. Accessed May 14, 2012.
(40.) TheUnitedKingdomNationalExternalQualityAssessmentServiceWebsite.http:// www.ukneqas.org.uk/content/PageServer.asp?S=99359106&C=1252&CID=1&type=G. Updated July 6, 2010. Accessed May 14, 2012.
(41.) EuroGentest. External Quality Assessment. http://www.eurogentest.org/ laboratories/qau/eqa/. Updated November 23, 2011. Accessed May 14, 2012.
(42.) The Deutsche Vereinte Gesellschaft fur Klinische Chemie und Laboratoriumsmedizin e.V. (DGKL) Web site. http://www.dgkl-rfb.de/index_E.shtml. Accessed May 14, 2012.
(43.) Human Genetics Society of Australasia Molecular Genetics Quality Assurance Program (HGSA MGQAP) Web site. http://www.hgsa.org.au/ resources/quality-assessment/. Updated January 10, 2012. Accessed May 14, 2012.
(44.) Ramsden SC, Deans Z, Robinson DO, et al. Monitoring standards for molecular genetic testing in the United Kingdom, The Netherlands and Ireland. Genetic Test. 2006;10(3):147-156.
(45.) Richards CS, Grody WW. Alternative approaches to proficiency testing in molecular genetics. Clin Chem. 2003;49(5):717-718.
(46.) Organization for Economic Cooperation and Development. OECD Guidelines for Quality Assurance in Molecular Genetic Testing. Paris, France: OECD, 2007:1-38.
(47.) Bellissimo DB: Practice guidelines and proficiency testing for molecular assays. Transfusion. 2007;47(1)(suppl):79S-84S.
(48.) Korn K, Reil H, Walter H, Schmidt B. Quality control trial for human immunodeficiency virus type 1 drug resistance testing using clinical samples reveals problems with detecting minority species and interpretation of test results. J. Clin. Microbiol. 2003;41(8):3559-3565.
(49.) Fae I, Lau M, Voorter C, Mayr WR, Fischer GF. HLA-B*8102, a new allele found in an external proficiency testing scheme. Tissue Antigens. 2004;64(5): 608-610.
(50.) Longhi E, Frison S, Colombini I, Malagoli A, Crespiatico L, Poli F. HLA-A*2626, a new allele identified through external proficiency-testing exercise. Tissue Antigens. 2005;66(4):325-326.
(51.) Perry DJ, Goodeve A, Hill M, Jennings I, Kitchen S, Walker I; UK NEQAS for Blood Coagulation. The UK National External Quality Assessment Scheme (UK NEQAS) for molecular genetic testing in haemophilia. Thromb Haemost. 2006;96(5):597-601.
(52.) Weck KE, Zehnbauer B, DattoM, Schrijver I; CAP/ACMG Biochemical and Molecular Genetics Resource Committee. Molecular genetic testing for fragile X syndrome: laboratory performance on the College of American Pathologists proficiency surveys (2001-2009). Genet Med. 2012;14(3)306-312. doi:10.1038/ gim.2011.11.
(53.) Palomaki GE, Richards CS. Assessing the analytic validity of molecular testing for Huntington disease using data from an external proficiency testing survey. Genet Med. 2012;14(1):69-75. doi: 10.1038/gim.0b013e3182310bb5.
(54.) Dequeker E, Cassiman JJ. Evaluation of CFTR gene mutation testing methods in 136 diagnostic laboratories: report of a large European external quality assessment. Eur J Hum Genet. 1998;6(2):165-175.
(55.) Noll, WW, Belloni, DR, Stenzel T, Grody. WW. Polymorphism in intron 4 of HFE does not compromise haemochromatosis mutation results [correspondence]. Nature Genet. 1999;23(3):271-272. doi:10.1038/15722.
(56.) Spector EB, Kathryn Kronquist; FragileX Working Group of the Laboratory Quality Assurance Committee. Technical Standards and Guidelines for Fragile X Testing: A Revision to the Disease-Specific Supplements to the Standards and Guidelines for Clinical Genetics Laboratories of the American College of Medical Genetics. Standards and Guidelines for Clinical Genetics Laboratories 2006 Edition. Bethesda, MD: American College of Medical Genetics; 2006. http:// www.acmg.net/Pages/ACMG_Activities/stds-2002/fx.htm. Accessed May 14, 2012.
(57.) Mueller CR, Kristoffersson U, Stoppa-Lyonnet D. External quality assessment for mutation detection in the BRCA1 and BRCA2 genes: EMQN's experience of 3 years. Ann Oncol. 2004; 15(suppl 1): i14-i17. doi:10.1093/ annonc/mdh652.
(58.) Hoeltge FA, Phillips MG, Styer PE, Mockridge P. Detection and correction of systematic laboratory problems by analysis of clustered proficiency testing failures. Arch Pathol Lab Med. 2005;129(2):186-189.
(59.) Lum G, Tholen DW, Floering DA. The usefulness of calibration verification and linearity surveys in predicting acceptable performance in graded proficiency tests. Arch Pathol Lab Med. 1995;119(5):401-408.
(60.) Kroll MH, Styer PE, Vasquez DA. Calibration verification performance relates to proficiency testing performance. Arch Pathol Lab Med. 2004;128(5): 544-548.
(61.) Watson MS, Cutting GR, Desnick RJ, et al. Cystic fibrosis population carrier screening: 2004 revision of American College of Medical Genetics mutation panel. Genet Med. 2004;6(5):387-391. Erratum Genet Med. 2005;7(4): 286; Genet Med. 2004;6(6):548.
(62.) Ahmad-Nejad P, Dorn-Beineke A, Pfeiffer U, et al. Methodologic European external quality assurance for DNA sequencing: the EQUALseq program. Clin Chem. 2006;52(4):716-727.
(63.) Patton SJ, Wallace AJ, Elles R. Benchmark for evaluating the quality of DNA sequencing: proposal from an international external quality assessment scheme. Clin Chem. 2006;52(4):728-736.
(64.) Raggi CC, Pizani R, Paradiso A, Pazzagli M, Orlando C. External quality assurance program for PCR amplification of genomic DNA: an Italian experience. Clin Chem. 2003;49(5):782-791.
(65.) Orlando C, Verderio P, Maatmanet R, et al. EQUAL-qual: a European program for external quality assessment of genomic DNA extraction and PCR amplification. Clin Chem. 2007;53(7):1349-1357.
(66.) Ramsden SC, Daly S, Geilenkeuser WJ, et al. EQUAL-quant: an international external quality assessment scheme for real-time PCR. Clin Chem. 2006;52(8):1584-1591.
(67.) Dorn-Beineke A, Ahmad-Nejad P, Pfeiffer U, et al. Improvement of technical and analytical performance in DNA sequencing by external quality assessment-based molecular training. Clin Chem. 2006;52(11):2072-2078.
(68.) Berwouts S, Gordon JT, Rundell CA, Barton DE, Dequeker E. Evaluation and use of a synthetic quality control material included in the European external quality assessment scheme for cystic fibrosis. Hum Mutat. 2008;29(8):1063 1070.
(69.) Jarvis M, Iyer RK, Williams LO, NollWW, Thomas K, Grody WW. Anovel method for creating artificial mutant samples for performance evaluation and quality control in clinical molecular genetics. J. Mol Diagn. 2005;7(2):247-251.
(70.) Seneca S, Morris MA, Patton S, Elles R, Sequeiros J. Experience and outcome of 3 years of a European EQA scheme for genetic testing of the spinocerebellar ataxias. Eur J Hum Genet. 2008;16(8):913-920.
(71.) Falbo V, Floridia G, Tosto F, et al. The Italian external quality assessment scheme for fragile xsyndrome: the results of a 5-year survey. Genetic Test. 2008; 12(2):279-288.
(72.) Taruscio D, Falbo V, Floridia G, et al. Quality assessment in cytogenetic and molecular genetic testing: the experience of the Italian Project on Standardisation and Quality Assurance. Clin Chem Lab Med. 2004;42(8):915-921.
(73.) Tosto F, Salvatore M, Falbo V, et al. The Italian scheme of external quality assessment for p-thalassemia: genotyping and reporting results and testing strategies in a 5-year survey. Genet Test Mol Biomarkers. 2009;13(1):31-36.
(74.) Hertzberg M, Neville S, McDonald D. External quality assurance of molecular analysis of haemochromatosis gene mutations. J Clin Pathol. 2006; 59(7):744-747.
(75.) Brothman AR, Dolan MM, Goodman BK, et al. College of American Pathologists/American College of Medical Genetics proficiency testing for constitutional cytogenomic microarray analysis. Genet Med. 2011;13(9):765 769.
(76.) Birch L, English CA, Burns M, Keer JT. Generic scheme for independent performance assessment in the molecular biology laboratory. Clin Chem. 2004; 50(9):1553-1559.
(77.) Raggi CC, Verderi P, Pazzagli M, et al. An Italian program of external quality control for quantitative assays based on real-time PCR with Taq-Man" probes. Clin Chem Lab Med. 2005;43(5):542-548.
Lisa V. Kalman, PhD; Ira M. Lubin, PhD; Shannon Barker, PhD; Desiree du Sart, PhD; Rob Elles, PhD; Wayne W. Grody, MD, PhD; Mario Pazzagli, PhD; Sue Richards, PhD; Iris Schrijver, MD; Barbara Zehnbauer, PhD
Accepted for publication August 28, 2012.
From the Laboratory Research and Evaluation Branch, Division of Laboratory Science and Standards, Office of Surveillance, Epidemiology, and Laboratory Services, Centers for Disease Control and Prevention, Atlanta, Georgia (Drs Kalman, Lubin, Barker, and Zehnbauer); the Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, Georgia (Dr Barker); the Victorian Clinical Genetics Services, Murdoch Children's Research Institute, Parkville, Victoria Australia (Dr du Sart);the Manchester Academic Health Science Centre, Genetic Medicine, St Mary's Hospital, Manchester, Greater Manchester, United Kingdom (Dr Elles); the Departments of Pathology & Laboratory Medicine, Pediatrics, and Human Genetics, University of California, Los Angeles School of Medicine, Los Angeles, California (Dr Grody); the Department of Clinical Physiopathology, University of Florence, Florence, Italy (Dr Pazzagli); the Department of Molecular and Medical Genetics, Oregon Health and Science University, Portland, Oregon (Dr Richards);and the Departments of Pathology and Pediatrics, Stanford University School of Medicine, Stanford, California (Dr Schrijver).
The authors have no relevant financial interest in the products or companies described in this article.
The findings and conclusions in this report are those of the author and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the Agency for Toxic Substances and Disease Registry.
Reprints: Lisa V. Kalman, PhD, Laboratory Research and Evaluation Branch, Division of Laboratory Science and Standards, Office of Surveillance, Epidemiology, and Laboratory Services, Centers for Disease Control and Prevention, 1600 Clifton Rd, Mailstop G23, Atlanta, GA 30333 (e-mail: LKalman@cdc.gov).
Table 1. Methods Used to Test for Inherited Disorders Method Used % of Diseases Tested Sequencing (a) 93 (90 of 97) Transcription-mediated 23 (22 of 97) amplification Deletion/duplication 27 (26 of 97) analysis Mutation scanning 18 (18 of 97) Methylation analysis 2 (2 of 97) (a) Analysis of a random sample (~10%) of diseases for which molecular genetic testing methods are used (February 26, 2009 report of ~970 diseases obtained from GeneTests), determined that testing for ~93% (90 of 97) of these diseases used DNA sequencing methods in at least some of the laboratories that offered testing;49% (48 of 97) of the diseases in the sample were tested using only sequencing techniques. Table 2. Proficiency Testing/External Quality Assessment (PT/EQA) Provider Schemes for Inherited Disorders PT/EQA Provider Source, y Analytes Covered CAP, (22) 2012 Disease-specific schemes: factor V Leiden, fragile X syndrome, hemochromatosis, MTHFR, Prader-Willi /Angelman syndrome, prothrombin, CF, Huntington disease, Friedreich ataxia, hemoglobin SC, Duchenne muscular dystrophy, myotonic dystrophy, RhD, spinal muscular atrophy, spinocerebellar ataxia, BRCA1 and BRCA2, connexin 26, multiple endocrine neoplasia type 2, Canavan disease, familial dysautonomia, Tay-Sachs disease, mucolipidosis IV, Bloom syndrome, Fanconi anemia, Gaucher disease, glycogen storage disease type 1A, Niemann-Pick disease type A. Pharmacogenetic markers. Methods-based schemes: DNA sequencing, microarray genomic copy number assay. EMQN,(38) 2012 Disease-specific schemes: Y-chromosome microdeletions, BRCA1 and BRCA2, CAH, CMT, Familial Adenomatous Polyposis Colon Cancer, GJB2 and GJB6, HNPCC, monogenic diabetes, Marfan syndrome, porphyria, hereditary recurrent fevers, myotonic dystrophy, Duchenne muscular dystrophy, fragile X syndrome, Friedreich ataxia, Huntington disease, hemochromatosis, multiple endocrine neoplasia type 2A, PKU, Prader-Willi/Angelman syndromes, retinoblastoma, short stature homeobox gene testing, spinocerebellar ataxia, spinal muscular atrophy, Von Hippel Lindau disease, Wilson disease. Methods-based schemes: Sanger DNA sequencing, array-CGH, next-generation sequencing (pilot). UKNEQAS, (40) Molecular genetics: Angelman syndrome, 2012 Becker/Duchenne muscular dystrophy, BRCA1 and BRCA2, CF, factor V Leiden, familial adenomatous polyposis, fragile X syndrome, hereditary and motor sensory neuropathy, Huntington disease, HNPCC, hereditary neuropathy with liability to pressure palsies, MCADD, mitochondrial disorders, molecular rapid aneuploidy testing, myotonic dystrophy, Prader-Willi syndrome, spinal muscular atrophy, spinocerebellar ataxia. Molecular testing on dried blood spots: CF, MCADD. Microarray CGH. HGSA,(43) 2012 23 disease-specific PT modules in collaboration with EMQN and the CF Network. Y-chromosome deletions, familial breast cancer (BRCA), CAH, CF, CMT, connexin 26, myotonic dystrophy, Becker/Duchenne muscular dystrophies (BMD/DMD), familial adenomatous polyposis coli, fragile X syndrome, Friedreich ataxia, Huntington disease, HNPCC, hereditary recurrent fevers (pilot), multiple endocrine neoplasia Type 2, mitochondrial myopathy, monogenic diabetes, PKU, porphyria, Prader-Willi/Angelman syndromes, retinoblastoma, spinocerebellar ataxias, spinal muscular atrophy, Von Hippel Lindau syndrome, Wilson disease. Also methods-based DNA sequence and analysis scheme and cytogenetics are also offered. ISS, (41) 2012 Cystic fibrosis, b thalassemia, DGKL, (42) 2012 fragile X syndrome, adenomatous polyposis of colon. Molecular Biology schemes: Factor V-Leiden, prothrombin, MTHFR, PAII (SERPINE1), factor XIII (F13A1), GPIIIa (1TGB3), [3Fib (FGB), VKORC1, Factor XII (F12), a1 PI, APOE, APOB, ACE, CETP, TPMT, CYP2C19, CYP2D6, CYP2C8, CYP2C9, UGT1A1 , DPD (DPYD), BCHE, ALDOB, HFE, LCT, NOD2, ATP7B, FSAP (HABP2), ITGA2, KRAS. Method schemes: DNA sequencing, DNA isolation + Factor V genotyping. CDC NSQAP, (37) Dried blood-spot testing 2012 for CF mutations. PT/EQA Provider Molecular Genetic Sample Types Source, y Challenges/y CAP, (22) 2012 Disease-specific Disease-specific, schemes: 3 DNA pharmacogenetic, and samples 2 times/y; microarray genomic cystic fibrosis: copy number assay 2 samples 2 times/y schemes: extracted Pharmacogenetic DNA. Postanalytic markers: 2 DNA DNA sequencing samples 2 times/y. scheme:1 CD-ROM Postanalytic DNA containing sequencing scheme: DNA sequence 3 electronic electropherogram challenges 2 times/y. files Microarray genomic copy number assay: 2 DNA samples + challenge 2 times/y EMQN,(38) 2012 3 DNA samples + Lyophilized DNA mock clinical data 1 time/y UKNEQAS, (40) Includes most Lyophilized DNA, 2012 disease-specific whole blood schemes: 3 samples spotted on 1 time/y; molecular filter paper testing on blood and dried spots: 3 samples 4 times/y HGSA,(43) 2012 3 samples are Extracted DNA sent out or RNA, lyophilized per disease DNA(through module, 1 times/y EMQN); amplified DNA products and sequencing primers ISS, (41) 2012 6 samples/disease DNA from DGKL, (42) 2012 Molecular biology lymphoblastoid cells schemes: 2 times/y. Molecular Sequencing-based biology schemes: scheme: 2 times/y. lyophilized DNA. DNA isolation: Sequencing-based 2 times/y. scheme: 2 lyophilized DNA samples, DNA isolation: 2 tubes whole blood CDC NSQAP, (37) 5 dried blood spots Blood from patient 2012 4 times/y with CF spotted on filter paper and dried PT/EQA Provider Phases of Source, y Testing Covered CAP, (22) 2012 Disease-specific, pharmacogenetic and microarray genomic copy number assay schemes: genotyping and interpretation Postanalytic DNA sequencing scheme: Interpretation EMQN,(38) 2012 Genotyping, interpreting and reporting UKNEQAS, (40) Genotyping, 2012 interpreting and reporting HGSA,(43) 2012 Analytic, postanalytic ISS, (41) 2012 Analytic and DGKL, (42) 2012 postanalytic Molecular biology schemes: analytic. Method-based schemes: sequence result and interpretation. DNA isolation: determination of concentration of DNA, ratio 260:280, method of identification, defined genotypes CDC NSQAP, (37) Preanalytic, 2012 analytic, interpretation Abbreviations: b-Fib, Fibrinogen-Beta polypeptide chain; CAH, congenital adrenal hyperplasia; CAP, College of American Pathologists; CDC, Centers for Disease Control and Prevention;CD-ROM, compact disk-read-only memory;CF, cystic fibrosis;CGH, comparative genomic hybridization; CMT, Charcot-Marie-Tooth disease; DGKL, Reference Institute for Bioanalytics; DPD, Dihydropyrimidine dehydrogenase; EMQN, European Molecular Genetics Quality Network; GPIIIa, platelet glycoprptein IIIa; FASP, Factor VII-Activating Protease; HGSA, Human Genetics Society of Australasia; HNPCC, hereditary nonpolyposis colorectal cancer (Lynch syndrome); ISS, Istituto Superiore di Sanita; MCADD, medium chain acyl-coenzyme A dehydrogenase deficiency; MTHFR, methylenetetrahydrofolate reductase; NSQAP, Newborn Screening Quality Assurance Program; PAII, Plasminogen Activator Inhibitor I PI, proteinase inhibitor; PKU, phenylketonuria; RhD, Rhesus (hemolytic) disease; SC, sickle cell; UKNEQAS, United Kingdom National External Quality Assessment Service. Table 3. Disease-Specific Proficiency Testing Performance (as Assessed From Select, Published Literature) Source, y Disease Participation Palomaki and Huntington 33 US and 23 Richards disease international (CAP), (53) laboratories 2012 Weck et al Fragile X Average of 90 (CAP), (52) syndrome laboratories 2012 during study period; US and international Seneca et al Spinocerebellar Only 28.8% (EMQN), (70) ataxia (15 of 52) 2008 laboratories participated in all consecutive y Mueller et al BRCA1 and BRCA2 (EMQN), (57) 2004 Dequeker, Cystic fibrosis 135 European Cassiman laboratories, (European 1 Australia Concerted laboratory Action on Cystic Fibrosis framework), (54) 1998 Ramsden et al, Several diseases (44) 2006; Perry et al (UK NEQAS), (51) 2006 Falbo et al Fragile X syndrome 65% (22 of 34) (ISS), (71) 2008 of all Italian laboratories that perform testing participated. 15 laboratories participated in each of the 5 y Taruscio et al Cystic fibrosis, (ISS), (72) fragile X 2004 syndrome, others Tosto et al b thalassemia 11-18 laboratories (ISS), (73) 2009 during 5y Hertzberg et HFE 37 laboratories al (RCPA), (74) currently 2006 Source, y General Performance Analytic Errors Identified Palomaki and Analytic sensitivity, Repeat length errors Richards 99.5%, during 6 occurred in 2.6% (CAP), (53) surveys (28 of 1060) allelic 2012 challenges. Most errors were minor and occurred in a small subset of the laboratories. Non-US participants had the higher error rate (17.5%, (107 of 612)). Weck et al Analytic Measurement of repeat (CAP), (52) sensitivity for length was more 2012 detection of full accurate with smaller mutations 99% samples. (males), 96% (females); premutations 98% Seneca et al 97.5% of reports Large interlaboratory (EMQN), (70) correctly identified variation seen in 2008 genotype as allele sizing. normal or pathogenic Gross diagnostic errors occurred every year (false-negatives, false-positives, and sample misidentification). Not all laboratories reported allele size. Mueller et al Error rate Missed mutations (53% (EMQN), (57) during 3 y, (9 of 17) of 2004 7.4% (9 of 121)- false results). 3.7% (4 of 108) Problems with incorrect nomenclature. Dequeker, 35% (48 of 136) of Mistyping, Cassiman laboratories misinterpretation (European incorrectly of data, erroneous Concerted genotyped technical results. Action on Cystic >1of12 Fibrosis CFTR alleles framework), (54) in study 1998 Ramsden et al, Poor performance Technical errors (44) 2006; was sporadic and wrong Perry et al (UK nomenclature. NEQAS), (51) 2006 Falbo et al Overall success Overall, 5% (30 of 601) (ISS), (71) 2008 rate, of samples tested were 76% (458 of incorrectly genotyped, 601). and 1.5% (9 of 601) of that was due to 1 sample, a male mosaic normal/ full mutation, which was incorrectly genotyped 9 times (3 by the same laboratory). Some technical failures caused by prescreening methods. Taruscio et al Interpretive 7% (2 of 28) of (ISS), (72) performance laboratories incorrectly 2004 varied by disease genotyped 2 mutations, although their commercial kit detects those mutations. Many laboratories did not include clinically relevant cystic fibrosis mutations in their panels. Tosto et al 98.8% (889 of 900) (ISS), (73) 2009 of alleles correctly identified Hertzberg et 99.47% Most errors were al (RCPA), (74) (3000 of 3016) due to sample 2006 success during handling or result 10 surveys transcription. A few errors caused by technical issues. Source, y Interpretation Errors Identified Palomaki and 3 interpretation Richards errors were made (CAP), (53) in US laboratories 2012 during the 6 surveys. Weck et al In samples with (CAP), (52) repeat sizes of 2012 20-33, during 18 surveys, 3 interpretation errors made. Many laboratories incorrectly identified samples with 42 repeats as gray-zone or premutation. Seneca et al Wide range of allele (EMQN), (70) sizing but 2008 without incorrect diagnoses. Clerical errors observed. Mueller et al Misinterpretation of (EMQN), (57) observed 2004 sequence changes. Dequeker, Cassiman (European Concerted Action on Cystic Fibrosis framework), (54) 1998 Ramsden et al, Incomplete reports; (44) 2006; wrong clinical Perry et al (UK interpretations. NEQAS), (51) 2006 Falbo et al Reports showed (ISS), (71) 2008 considerable variation. Starting in 2004, a model was issued for reports. Taruscio et al Written reports (ISS), (72) were inadequate 2004 for .50% of laboratories. The most common error was lack of risk calculation when a mutation was not detected. Tosto et al (ISS), (73) 2009 Hertzberg et Incorrect results al (RCPA), (74) because of 2006 transcriptional Source, y Lessons Learned Palomaki and Performance was very Richards good. An analytic (CAP), (53) method with a high 2012 error rate was identified. Weck et al Accuracy of sizing (CAP), (52) and interpretation 2012 of larger repeats improved over time. Seneca et al (1) One-third (5 of 15) (EMQN), (70) of laboratories with 2008 inaccurate sizing did not participate the following year, and for those that did, >50% (6 of 10) continued to report wrong sizes, showing importance of participation in PT every year; and (2) always be reported: it is a measure of technical quality and may give clinically important information. Mueller et al Horizontal comparison (EMQN), (57) among laboratories 2004 allows for better understanding of the state of the field and for detection of systematic technical failures. Dequeker, Regular participation in Cassiman PT schemes would (European be beneficial for Concerted laboratories. Laboratory Action on Cystic personnel should Fibrosis participate in regular framework), (54) training sessions. 1998 Need consensus strategies for diagnostic testing. Centralized mutation analysis facilities would be useful for identified rare mutations. Ramsden et al, (1) Any poor performance (44) 2006; causes immediate Perry et al (UK correspondence between NEQAS), (51) 2006 the organizer and the laboratory; and (2) scheme located test-validity problems as well as clinical practice problems, with wider implications for the genetic testing community. Falbo et al (1) Importance of reports (ISS), (71) 2008 for patient care; and (2) importance of secondary screening methods. Taruscio et al (1) Even commercial kits (ISS), (72) must be validated in 2004 the laboratory before use for clinical testing; (2) laboratories need guidance on the kinds of mutations to include in their panels; (3) complete reports are crucial for patient care; and (4) Tosto et al laboratory errors were (ISS), (73) 2009 disease specific and were not indicative of overall laboratory analytic performance. job of reporting some aspects of the test, such as diagnostic sensitivity; however, many reports did not identify the mutations tested, indication for genetic counseling, or interpretation of results. Hertzberg et Performance was very good, al (RCPA), (74) but laboratories 2006 should continue to participate in large quality assurance programs to maintain high testing quality. Abbreviations: CAP, College of American Pathologists;EMQN, European Molecular Genetics Quality Network;ISS, Istituto Superiore di Sanita;PT, proficiency training;QAP, quality assurance program; RCPA, Royal College of Pathologists of Australasia;UK NEQAS, United Kingdom National Table 4. Methods-Based Proficiency Testing (PT) Schemes and Performance Source, y Technology Laboratories Assessed Provided With Brothman aCGH Extracted DNA et al from cell (CAP), lines derived (75) 2011 from individuals (ongoing PT with a scheme) constitutional abnormality. (CAP), (22) Postanalytic One CD-ROM (ongoing PT DNA containing DNA scheme) sequencing sequence electropherogram files. Orlando DNA extraction (1) Two et al and preextracted (EQUAL), amplification DNA samples, (2) (65) 2007 whole blood (pilot) samples, and (3) primer sets. Ramsden Real-time (1) ABL gene et al PCR primers, (EQUAL), (2) 5 -FAM/3 - (66) 2006 TAMRA-labeled (pilot) probes, (3) 5 ABL standard plasmids (10E2-10E5), (4) 3 test cDNA samples, and (5) 2 cell samples. Ahmad-Nejad Sequencing (1) Primer sets, et al (2) 2 plasmid (EQUAL), DNA (62) 2006 samples (1, a (pilot) vector; 1, a mixture of 2 different plasmids generated to provoke comments about template quality), (3) a purified PCR product, and (4) a finished sequencing reaction to be purified and analyzed. Patton Sequencing (1) 450-bp, PCR- et al amplified fragments (EMQN), of CFTR covering all (63) 2006 main types of (ongoing EQA sequence changes, scheme) (2) a wild-type control, (3) primers (with location in sequence), (4) reference sequence, and (5) protein translation Raggi et al DNA (1) Three primer (ISS),(64) extraction, PCR pairs, 2003 (pilot) performance, (2) 1 reference interpretation DNA, (3) 3 DNA calibrators, and (4) 1 blood sample (HGSA-ASOC), Microarray Extracted DNA (43) (ongoing analysis from an scheme) individual tested for a constitutional abnormality (HGSA-MGSA), DNA sequencing (1) Amplified (43) (ongoing and products scheme) interpretation and (2) sequencing primers Birch et al, DNA extraction, (1) Tubes (76) 2004 PCR containing high, amplification medium, and low concentrations of bacteria in buffer; (2) control DNA in buffer; (3) PCR reagents (dNTPs, primers, Taq polymerase, reaction buffer);and (4) 100-bp DNA ladder, gel loading dye. (DGKL), DNA sequencing Lyophilized DNA (42) (ongoing and scheme) interpretation (DGKL), DNA isolation, Whole blood (42) (ongoing quantitation scheme) (and (and factor V genotyping) Raggi et al TaqMan (1) Standard cDNA (ISS), (77) real-time solution 2005 (pilot) PCR obtained by in vitro transcription of a fragment of hTERT cloned into a plasmid vector, (2) mix of primers, (3) 3 unknown Cdna samples, and (4) PCR conditions. Source, y PT protocol Laboratory Performance Brothman Laboratories must (1) 95.7% (472 of 493) et al analyze specimens and of responses (CAP), identify and interpret were concordant for an (75) 2011 results for clinically abnormality involving a (ongoing PT significant copy number particular chromosomal scheme) abnormalities, and (2) location. Errors identify and characterize included designating an gains or losses and the opposite change, cytogenetic location of reporting a clinically any abnormalities insignificant copy detected using standard number change, and nomenclature. failure to report a clinically significant abnormality. (CAP), (22) Laboratories must No published data (ongoing PT interpret available. scheme) and report DNA sequence variants for inherited disease using standard nomenclature. Orlando Laboratories must (1) 25% (42 of 165) of et al perform DNA extraction laboratories (EQUAL), by routine procedures, performed poorly in (65) 2007 (2) estimate DNA quality quantification of >1 (pilot) and quantity of both of the 2 preextracted preextracted and samples; 27% (46 laboratory-extracted of 166) of laboratories DNA, (3) perform PCR had questionable with 100 ng of DNA results for quality from all samples, (4) and/or quantity of submit raw data from blood sample extractions. DNA quantification and High degree of post-PCR interpretation, variability seen and (5) send aliquot of with PCR DNA extracted from performance of whole blood back to all samples. EQUAL. Ramsden Laboratories must (1) For the cDNA samples, et al construct a calibration 80% (74 of 93) of (EQUAL), curve; (2) estimate laboratories provided (66) 2006 cDNA copy numbers in accurate values for (pilot) 3 cDNA samples; (3) all 3. For cell perform RNA extraction, samples, approximately real-time PCR, and 35% (27 of 75, 25 cDNA quantification of of 74) of laboratories the 2 cell samples; and provided 95% CI (4) provide Ct values for limits that fell NTC, calibrators, and outside the range unknowns, plus details of standard of the testing platforms dilutions. Wide used. variations in laboratory results were observed. Ahmad-Nejad Laboratories must (1) For the cloning vector et al perform sequencing and plasmid, the (EQUAL), provide longest possible mean sequence (62) 2006 sequence without any stretch (of the (pilot) mistakes (primer-walking 2355-bp insert) not allowed) for DNA was 537 bases plasmids and purified and only 14 of 43 PCR product and (2) participants (33%) purify and analyze the identified the insert finished sequencing correctly. For the reaction. mixture of 2 different plasmids, 21% (9 of 43) of laboratories did not register any kind of difficulty; however, some excellent specific comments were received. For the finished PCR product, 57% (24 of 43) of laboratories made no effort at identification. Patton Laboratories must 59% (36 of 61 et al (1) return laboratories) scored (EMQN), form with genotyping the maximum (12.0 (63) 2006 results and (2) return marks); 19 genotyping (ongoing EQA color copies of errors were made scheme) electropherograms and from 346 genotypes electronic copies of analyzed (5%); 10 sequence data files. of 19 (53%) were false-negative results, and 9 of 19 (47%) were false-positive results; 59% (27 of 46) of errors were made in naming mutations. Most laboratories produced data with acceptable diagnostic quality. Raggi et al Laboratories must (1) Variability was (ISS),(64) extract DNA from blood high for DNA 2003 (pilot) sample and measure quantification quality and quantity, (range, 0.012-0.54 (2) measure ig/|iL;CV, 82%). The quantity and 260:280 ratios varied quality of 3 standard (range, 0.8-2.5;CV, DNA samples, (3) 21%). Large perform PCR, and (4) variability seen in return PCR results, yield and extracted DNA aliquot, quality of PCR and PCR product performance. aliquots. (HGSA-ASOC), Assessment of No published data (43) (ongoing laboratory available. scheme) processing and/or analytic skills, as well as interpretive comments. (HGSA-MGSA), Laboratories must (1) No published data (43) (ongoing perform Sanger available. scheme) sequencing using PCR products and primers provided, and (2) analyze and report results using standard nomenclature and provide biologic interpretation. Birch et al, Laboratories must (1) Identified analytic (76) 2004 extract DNA from issues such suspended bacteria, (2) as inappropriate PCR-amplify extracted extraction or DNA according to amplification procedures, protocol provided, and PCR inhibition or (3) analyze using contamination, poor agarose gel labeling electrophoresis. or poor-quality gel photography, and failure to record results correctly. (DGKL), DNA sequencing and No published data (42) (ongoing corresponding available. scheme) diagnostic interpretation (DGKL), Laboratories must (1) No published data (42) (ongoing isolate DNA, (2) available. scheme) determine the concentration of DNA, (3) report 260:280 ratio, and (4) identify the factor V genotype of the sample. Raggi et al Laboratories must (1) Only 12 of 42 (ISS), (77) prepare dilutions laboratories (29%) 2005 (pilot) of the gave results that provided standard, (2) were both concise perform PCR and and accurate for all analyze results, (3) samples tested; report 17 of 42 Ct values for each laboratories well, (40%) reported and (4) report inaccurate data concentrations of for >1 result. unknown samples. Inaccuracy showed an inverse dose-dependent trend; 12 of 42 laboratories were unable to measure a sample with low concentration. Abbreviations: aCGH, array-comparative genomic hybridization; ASOC, Australasian Society of Cytogeneticists; bp, base pair; CAP, College of American Pathologists; cDNA, complementary DNA; CI, confidence interval; Ct, cycle threshold; CV, coefficient of variation; dNTP, deoxynucleotide triphosphate;EMQN, European Molecular Genetics Quality Network;EQA, external quality assessment;HGSA, Human Genetics Society of Australasia; hTERT, human telomerase reverse transcriptase; ISS, Istituto Superiore di Sanita; MGQAP, Molecular Genetics Quality Assurance Program; NTC, no template control; PCR, polymerase chain reaction.
|Printer friendly Cite/link Email Feedback|
|Author:||Kalman, Lisa V.; Lubin, Ira M.; Barker, Shannon; du Sart, Desiree; Elles, Rob; Grody, Wayne W.; Pazz|
|Publication:||Archives of Pathology & Laboratory Medicine|
|Date:||Jul 1, 2013|
|Previous Article:||Cytology and human papillomavirus screening test results associated with 2827 histopathologic diagnoses of cervical intraepithelial neoplasia 2/3.|
|Next Article:||Screening for Down syndrome in the United States: results of surveys in 2011 and 2012.|