Printer Friendly

Current landscape and new paradigms of proficiency testing and external quality assessment for molecular genetics.

Recent advances in genetic testing technologies and an increased understanding of the role of DNA variations in health and disease have produced expansion of molecular diagnostics and led to an increased role for clinical genetic testing in patient management. Currently, molecular genetic, biochemical, and cytogenetic tests for approximately 2300 inherited genetic diseases are offered in clinical laboratories for diagnosing disease, screening carriers, predicting clinical disease susceptibility, assessing risk, and prognosticating on the course of disease. (1) Because of the rapid growth and the potential effect of genetic testing results on clinical management or reproductive decisions, quality management practices are essential at all stages of the testing process to ensure the accuracy and utility of these tests.

Quality management is defined as an ongoing effort that includes policies and procedures established and implemented for the purpose of providing accurate laboratory test results. (2) Quality management of the analytic component encompasses a variety of quality assurance processes designed to ensure the performance of a test in the clinical laboratory. (3,4) This assurance requires a system that includes both internal and external procedures that are described in national and international guidance and regulatory documents (4-15) Internal quality assurance processes include measures to maintain analytic accuracy, such as quality control and personnel competency. External quality assessment measures include examination of laboratory procedures by a third party accreditation process and participation in proficiency testing (PT) or external quality assessment (EQA) programs. Most of the current quality assurance practices commonly used in genetic testing laboratories are designed for well-established technologies, such as targeted mutation analysis that detect limited sequence variations in one or a few genes associated with a particular disorder or condition. Newer technologies, such as next-generation DNA sequencing and chromosomal microarrays, allow detection of more sequence variations or gene expression levels. These new tests exhibit a higher level of complexity owing to the sophistication of chemistry, hardware, and software innovations. (16,17) Therefore, it is unclear whether traditional methods of quality assurance and assessment will suffice for this higher level of complexity or whether new paradigms must be developed.

Proficiency testing is defined in International Organization for Standardization (ISO) 17043 as: evaluation of participant performance against preestablished criteria by means of interlaboratory comparisons.'' (18) This can be achieved through participation in a formal PT program. Proficiency testing programs usually focus on the analytic results. External quality assessment also provides evaluation of laboratory performance on examination of external samples but focuses more on the preanalytic (preexamination) and postanalytic (postexamination) activities than PT does. (18) In the absence of a formal PT or EQA program, laboratories can assess their performance through alternative assessment activities, such as exchanging samples with another laboratory performing similar tests, or by internal assessment. (19)

Proficiency testing/EQA is an important component of clinical laboratory quality assurance. It provides a mechanism to compare analytic test performance among different laboratories, which is important for determining the consistency of test results for a common analyte. Participants in formal PT/EQA programs periodically receive specimens (with a genotype unknown to them), which are tested in a manner similar to procedures used for their regular clinical samples. Participating laboratories return the results of the requisite analyses to the PT/EQA program, which then compiles the data and provides summarized results and educational insights to the participants. These programs provide an independent measure of laboratory performance in comparison with an external standard or a mean value obtained by other participating laboratories. Participation in PT/EQA allows laboratories to recognize analytic and interpretive errors that may indicate internal problems with quality control, calibration, assay design, or test interpretation. This is important because most clinical molecular genetic tests are developed by individual laboratories and are not available as commercial test kits that are manufactured and evaluated in a consistent manner. Also, unlike molecular tests for infectious diseases, such as HIV and tumor markers, which are typically used for patient monitoring and may be performed repeatedly, most molecular genetic tests for inherited disease are performed only once in a patient's lifetime. In this situation, errors may not be noticed through discrepancy with subsequent testing.

An international survey of molecular genetic testing laboratories determined that 74% of responding laboratories participate in PT/EQA, which was suggested to correlate with higher quality assurance scores. (20,21) The most common reason cited by laboratories for failure to participate in PT was a lack of programs relevant to the clinical laboratory services offered. (21) In the United States, formal molecular genetic PT programs are available from the College of American Pathologists (CAP) (22) for 27 tests for inherited diseases, representing only a small fraction of the 1739 (US) or 2247 (worldwide) molecular genetic tests currently available. (1) However, the 27 CAP PT surveys are for tests with the greatest frequency of use, including factor V Leiden, prothrombin 20210A variant, cystic fibrosis, and fragile X syndrome. The CAP and other PT/EQA providers also offer schemes for many other molecular genetic tests, including cytogenomic microarray analysis, pharmacogenetics, paternity testing, and human leukocyte antigen typing.

Another survey of clinical genetic laboratories in the United States indicated that increased participation in PT correlated directly with fewer PT failures and fewer incorrect patient test reports, as reported by laboratories. (23) Published perspectives about the ability of PT to accurately measure routine laboratory performance are conflicting; therefore, this correlation may be difficult to prove. (24-30) Formal PT/ EQA provides interlaboratory comparison of specific samples and does not always examine the entire testing process or day-to-day quality management issues that may affect laboratory performance. Proficiency testing represents a snapshot'' and is not intended to provide a comprehensive evaluation of a laboratory's quality assurance processes. Nonetheless, it has documented value for identifying problems that compromise the quality of laboratory test results.

Regulatory authorities and professional organizations recognize that PT/EQA is an essential component of quality assurance and have developed policies and recommendations for inclusion of PT/EQA, where feasible, into laboratory practice. Several international organizations, such as the Clinical and Laboratory Standards Institute and the ISO, have published guidelines and standards related to PT for molecular diagnostic methods. The Clinical and Laboratory Standards Institute protocol documents MM14-A, Proficiency Testing (External Quality Assessment) for Molecular Methods, (31) GP27-A2, Using Proficiency Testing to Improve the Clinical Laboratory, (32) and GP29-A2, Assessment of Laboratory Tests When Proficiency Testing is Not Available, (19) offer guidance for the management and operation of PT/ EQA for PT providers as well as molecular genetic diagnostic laboratories. The ISO/International Electrotechnical Commission standard 17 043:2010, Conformity Assessment--General Requirements for Proficiency Testing, (18) specifies the requirements for the competence of PT providers and for the development and operational aspects of providing PT schemes. This guidance also describes methods of PT testing. The ISO document 15189, Medical Laboratories--Particular Requirements for Quality and Competence, (3) recommends that laboratories participate in EQA and that these EQA schemes should provide clinically relevant challenges that mimic clinical samples and encompass all steps of the testing process, including preanalytic and postanalytic components.

In the United States, the Secretary's Advisory Committee on Genetics, Health, and Society was mandated by the Department of Health and Human Services to report on the adequacy of genetic testing oversight and regulation and to identify gaps that could affect patient safety. In their 2008 report, U.S. System of Oversight of Genetic Testing: A Response to the Charge of the Secretary of Health and Human Services, (33) the Secretary's Advisory Committee on Genetics, Health, and Society formulated recommendations to address the main gaps in genetic testing oversight. One recommendation addressed the absence of formal PT programs for all genetic tests, particularly rare genetic disorders, and suggested that the Department of Health and Human Services promote the development of new PT products by investigating other performance assessment approaches, including method-based processes.

In this article, we review PT/EQA as it applies to molecular genetic testing for inherited conditions in the United States, Europe, and Australasia. We consider the successes and demonstrated value of available PT/EQA programs and examine the challenges posed by evolving laboratory practices and testing technologies. Proficiency testing/EQA programs are often unable to assess the total laboratory testing process, focusing primarily on the analytic phase of testing (often neglecting the preanalytic and postanalytic phases) nor are they able to provide PT/EQA for all available tests. This critical evaluation suggests the need for research and development of targeted efforts to meet future PT/EQA needs.

PT/EQA--THE CURRENT SITUATION

Resources Describing the Scope of Genetic Testing

No formal mechanism exists either to capture information, such as disorders and genes tested, or analytic methods about all genetic tests offered in the United States or worldwide or to identify all laboratories that perform these tests. Data are limited to voluntary registries, such as the GeneTests (34) and Orphanet. (35) These resources do not obtain information from laboratories that either decline to analyze samples submitted from outside of their own institutions or that decline to register for other reasons. Thus, the true scope of molecular genetic testing and the laboratories that perform such testing is unknown. The National Institutes of Health in the United States has recently developed another registry (36) to collect information about genetic tests offered in the United States, although it may not be more comprehensive than the registries that currently exist.

According to the GeneTests Web site, the number of disorders with available genetic testing worldwide has increased more than 20-fold between 1993 and 2012 (from 100-2300 tests). (34) Nucleic acid-based testing is available for about 2000 of those genetic disorders; other diagnoses use biochemical, chromosomal, or other genetic tests. (1)

The Orphanet database, also a voluntary registry, has catalogued 5954 rare diseases, but only some of those diseases have diagnostic testing available. The database lists 5424 laboratories that offer testing (both research and clinical), mostly located in Europe. (35)

Molecular Methods Used for Genetic Testing

Many molecular methods can be used to detect mutations, depending on the disorder and the associated molecular defects. For example, many common mutations cause cystic fibrosis; testing for that disorder is usually performed by targeted mutation analysis using a panel of the more frequent mutations (single or a few nucleotides) associated with severe disease phenotype. Duchenne muscular dystrophy is most often caused by deletions and duplications in the DMD gene; therefore, testing for that disorder includes deletion/duplication analysis. Other commonly used molecular testing techniques include DNA sequence analysis, mutation scanning, and methylation analysis. To estimate the percentage of tests using each method, we chose a random sample (10%) of diseases with available molecular genetics diagnostic methods (February 26, 2009, report of ~970 diseases obtained from GeneTests) and tabulated the method or methods used to detect mutations for each. Whole-gene or targeted-exon DNA sequence analysis was used by the laboratories to analyze 93% (90 of 97) of the disorders in our sample (Table 1). Testing for 49% (48 of 97) of the disorders in our random sample was performed using only DNA sequencing techniques. That fraction may increase as next-generation DNA sequencing technologies transition to clinical laboratories. Testing for approximately 23% (22 of 97) of the disorders in our sample used targeted mutation analysis, and testing for 27% (26 of 97) of the disorders used deletion/ duplication analysis, usually as a follow-up to sequencing assays, which may miss large deletions and duplications (Table 1).

Molecular Genetic Tests Evaluated by Formal PT/EQA Programs

In the past 2 decades, formal PT and EQA programs have become an integrated part of clinical genetics laboratory practice. There are many regional, national, and international PT/EQA programs. We describe here some of the larger programs that serve national and international participants; however, smaller programs that serve a more-limited base or programs without publicly available information were not included. A comprehensive list of available molecular genetic PT/EQA programs can be found on the Eurogentest Web site (http://www.eurogentest.org/ laboratories). (14) In the United States, CAP is the largest provider of molecular genetic PT challenges. (22) In partnership with the American College of Medical Genetics, CAP offers PT to laboratories worldwide for 27 inherited genetic disorders, with 5 pharmacogenetic loci and methods-based PT for cytogenomic microarray analysis and for post-analytic Sanger DNA sequence analysis (Table 2). The samples distributed in the PT challenges are typically highly purified nucleic acids extracted from human cell lines. This program provides participants with 3 samples per disorder twice per year. The methods-based challenge for Sanger DNA sequencing assays is currently focused on interpretation of electronic data files but will evolve to include a ''wet'' challenge using extracted DNA and including DNA sequence analyses. The Centers for Disease Control and Prevention's Newborn Screening Quality Assurance Program (37) offers PT for molecular cystic fibrosis screening. The Newborn Screening Quality Assurance Program requires laboratories to extract DNA from blood spotted on filter-paper collection devices and sends 5 blood spots to participating laboratories on a quarterly basis, which allows laboratories to perform all phases of the testing process, including DNA extraction from the appropriate matrix, within the PT context.

European PT/EQA challenges are provided by both national and international organizations (Table 2) with additional participation by laboratories outside Europe. The European Molecular Genetics Quality Network (EMQN), (38) which is supported by user subscriptions, offers disease-specific EQA for a variety of genetic disorders as well as methods-based EQA for Sanger DNA sequencing, molecular cytogenomic analysis, and mutation scanning. The EMQN EQA programs provide 3 lyophilized DNA samples per disease to participants once each year. Participants are assessed on their ability to correctly genotype, interpret, and report the results using their usual laboratory report format. The Cystic Fibrosis (CF) European Network (39) offers an external quality assessment scheme for CF molecular genetic testing to more than 200 laboratories worldwide. This program ''aims to evaluate the entire analytical process, from DNA sample receipt and genotyping up to the written report with the final interpretation of the data as it is normally being sent to the clinician who requested the genetic test.'' The United Kingdom National External Quality Assessment Service (UKNEQAS) (40) provides external quality assessment for a range of inherited diseases currently tested in diagnostic molecular genetic testing laboratories by providing challenges using lyophilized DNA or dried blood spots. This program examines the analytic and postanalytic stages, including evaluation of laboratory reports of testing and provides 3 samples per disorder per year to participants worldwide. The Italy-based Istituto Superiore di Sanita addresses current practice, problems, and future directions of interlaboratory comparisons. Their focus is PT/EQA for molecular genetic testing of a few diseases. The PT materials used are DNA samples extracted from lymphoblastoid cell lines. This program covers both the analytic and postanalytic phases of testing and provides 6 samples per disease annually. (41) The Deutsche Vereinte Gesellschaft fur Klinische Chemie und Laboratoriumsmedizin eV (42) in Germany offers a variety of test modules as well as methods-based PT for DNA sequence analysis and DNA isolation for factor V Leiden genotyping. Lyophilized DNA is provided for all its PT challenges, except the DNA isolation scheme, for which, whole-blood samples are sent.

Since 2008, the Human Genetics Society of Australasia Molecular Genetics Quality Assurance Program (43) has offered 23 disease-specific PT modules in collaboration with EMQN and the CF European Network. This program also offers 1 to 3 generic modules, per annum, which are based on methods and techniques to enable wider participation in the program. In particular, the generic modules are aimed at laboratories that do not participate in any of the disease-specific modules but use the same methods or technologies in their testing. One such module was an audit-based assessment of result reporting, for which laboratories were asked to provide deidentified copies of reports for one positive, one negative, and one not-tested result received for analysis within a specific period. Another module was ''Mutation Detection and Biological Interpretation,'' for which, laboratories were provided with amplified products and sequencing primers and were required to genotype the fragment, detect the variation, and report a biologic interpretation of the results. Laboratories were assessed for measures of the quality of data provided and result interpretation. Laboratories from Australia, Hong Kong, Indonesia, and Saudi Arabia participate in this program, which has amalgamated with the Royal College of Pathologists of Australasia quality assurance programs (RCPA QAP Pty/Ltd, Surry Hills, New South Wales, Australia) in 2010. There is also a PT program for cytogenetic testing offered through the Human Genetics Society of Australasia.

These formal PT/EQA programs include some of the more-common genetic disorders that are tested in many laboratories (Table 2). However, these tests represent only a small fraction of the more than 2300 disorders with available genetic tests. Proficiency testing/EQA providers select disorders, such as CF, which are tested by multiple laboratories to make the programs economically feasible and usually perform customer surveys to assess needs before developing a new disease challenge. In 2003, EQA was available for 8.4% and 4.2% of available genetic tests in the United Kingdom and the Netherlands, respectively. However when these data are considered in the context of how frequently a particular test was performed, that represented 63% of the total cases reported in the United Kingdom and 48% of the cases reported in the Netherlands at that time. (44)

The availability of appropriate and diverse materials, such as blood from affected patients or characterized cell lines from which PT/EQA materials are derived, also affects the availability of PT/EQA programs. It is not logistically or economically possible to provide a formal PT/EQA challenge for disorders that are tested in only one or a few laboratories or for those without a supply of available materials.

In the absence of formal PT/EQA schemes, laboratories in the United States and elsewhere must evaluate the performance of their assays using alternative methods (alternative assessment). This can be accomplished by blinded retesting of previously tested samples, by sample exchanges with laboratories performing similar tests, or by internal evaluation of data. Methods for alternative assessment are described in numerous guidance documents. (3,19,31,45,46) Proficiency testing/EQA providers, such as CAP and UKNEQAS, facilitate sample exchanges among laboratories for tests without formal PT/EQA schemes. The effect of alternative assessment on laboratory quality has not been evaluated.

Phases of Testing and Their Evaluation by PT/EQA

To have the greatest value, PT/EQA challenges should evaluate performance in the preanalytic, analytic, and postanalytic (preexamination, examination, and postexamination) stages of the testing process. Laboratories should process and analyze PT/EQA samples in the same manner as routine clinical patient specimens to the fullest extent possible. Successful PT/EQA participation should demonstrate proficiency in all examined phases of testing.

Preanalytic Phase.--The preanalytic testing phase includes the receiving, accessioning, labeling, and initial processing of the sample. When PT/EQA results are evaluated across laboratories, a common type of error is a sample switch or mislabeling that is probably caused by a clerical error in the preanalytic phase. Even though the analytic process may be performed correctly, the clerical mistake made earlier in the testing process produces incorrect test results and reporting (Table 3). This type of error is of particular concern because laboratory workers are frequently aware that the sample was provided for PT/EQA purposes. (47) However, because PT samples do not enter the laboratory workflow in the same way that routine patient samples do, there may be hidden, uncontrollable factors in the acquisition of PT specimens that could make such errors more likely.

The next step in the testing process is nucleic acid isolation, which can be performed using laboratory-developed methods or a variety of commercially available reagent kits and instruments. The isolated DNA or RNA obtained through these extractions may be derived from blood; bone marrow; formalin-fixed, paraffin-embedded tissue; or fresh tissues. The quality of the isolated nucleic acids is central to the success of molecular genetic testing methods.

The ideal PT/EQA sample would be a clinical specimen because it would most closely represent what is actually tested in a clinical laboratory and would permit the evaluation of all phases of the testing process, including the DNA extraction step. (47) It is, however, often difficult or impossible for formal PT programs to obtain sufficient quantities of appropriate, high-quality, safe, homogeneous, and stable clinical samples to supply all of the laboratories participating in the PT event with the exact same specimen material. (44,47) For many genetic disorders, it is difficult to recruit enough patients to reflect the variety of mutations that would be required for a comprehensive PT/EQA program, especially for rare diseases or mutations. Institutional review board and informed consent requirements may also present an obstacle to the use of clinical specimens. Clinical samples are often incompletely characterized and may yield unexpected genotypes, including patterns of mosaicism, minor clonal populations, (48) or previously unidentified alleles. (49,50) It is also difficult to transport potentially infectious whole blood across international borders. (51) The DNA extracted from cell lines, which is available in almost unlimited supply, is often substituted as PT/EQA samples. The disadvantage of this practice is that the nucleic acid isolation phase is not performed by the participating laboratory and cannot be evaluated as part of the PT process. In addition, sources of potential PT materials are limited in the range of diseases and mutations represented. For these reasons, most PT/EQA samples are not directly comparable to the samples usually received and cannot be processed in the same way as actual clinical samples. (47) Anecdotal evidence also indicates that some analytic techniques, such as next-generation sequencing and cytogenomic microarrays, often do not perform optimally with DNA isolated by some methods used by external sources.

Analytic Phase.--The analytic phase encompasses the actual testing of the analyte. Each participating laboratory performs the testing using its own validated method or methods. In surveys with many subscribers, such as for CF, it is not uncommon for participants to use a wide variety of commercially available assays as well as laboratory-developed tests. Although PT/EQA performance is usually excellent for molecular genetic tests, (52,53) analytic errors do occur. Some errors may be associated with the design of the assay. For example, previously unrecognized polymorphic variants located in the binding sites of the amplification primer may preclude effective amplification and detection of the actual mutation. One PT/EQA scheme for CF determined that a particular laboratory-developed test could not accurately detect the 621+1G>T mutation in the PT sample. The laboratory subsequently removed this mutation from its clinical assay. (54) In a similar case, a method-specific artifact produced a false result in a hereditary hemochromatosis PT challenge. (55) Poor performance on a PT/EQA challenge alerted another laboratory that the primer binding site in a BRCA assay was too close to the target mutation nucleotide, which hindered the detection of the variant. (44) Proficiency testing/EQA testing is one mechanism to alert laboratories to such problems and to indicate changes to avoid future errors. The results of PT/EQA can also be used to compare laboratory performance with different assay methods among laboratories. (54) This has been especially useful when assessing the accuracy of trinucleotide repeat sizing. For example, it is very important to accurately determine the size of fragile X premutation expansions to correctly predict the risk of allelic expansion, premature ovarian failure, and fragile X-associated tremor/ataxia syndrome. Inaccurate sizing, by even a few triplet repeats, could affect prenatal diagnosis and the risk of expansion estimates or cause conflicting reports among different family members with similar repeat sizes tested in different laboratories. If laboratories cannot accurately determine the size of the fragile X triplet repeat, when compared with results from the other PT/EQA survey participants or from the previously measured repeat size of the sample, additional calibration of the assay is warranted. Poor performance by several laboratories in the 2002 and 2003 UKNEQAS fragile X PT/ EQA scheme resulted in the development of consensus testing and reporting guidelines in the United Kingdom. (44) The American College of Medical Genetics assesses results from a subset of the CAP PT surveys; if a particular problem occurs at a high frequency, disease-specific testing practice guidelines are established and published. (47) American College of Medical Genetics guidelines for fragile X testing (56) were written in response to suboptimal performance on the CAP fragile X proficiency survey.

Postanalytic Phase.--During the postanalytic phase of testing, the test results are reviewed and interpreted. Most of the molecular genetic PT modules offered by the CAP have an analytic, as well as a clinical interpretation, component, which are graded separately. (44) Participants provide an interpretation of the detected genotype within the context of a described clinical scenario (presentation of the patient), such as whether the identified mutations are consistent with the diagnosis of the disorder or whether the genotype indicates a genetic carrier. However, other components of the laboratory report, such as compliance with existing guidance or recommendations (15) describing required information elements, accuracy about the indications for testing, test performed, results obtained, and appropriateness of follow-up guidance, are not evaluated. In European EQA programs, such as UKNEQAS, the CF European Network, and the EMQN, the participants are required to submit results in their usual clinical reporting format. Those reports are assessed for accuracy of genotyping, the appropriateness of the interpretation, and clerical accuracy. (38-40,44,51,57) That requirement allows the laboratory's interpretation of the analytic results to be evaluated in the context of the mock clinical data supplied with the DNA samples and permits assessment of other important elements in the laboratory report, such as residual risk calculations, recommendations for further testing, use of proper genetic nomenclature, and accurate inclusion of patient identification such as name, gender, and birth date. Correct interpretation of the analytic result is essential because patient management will, in many cases, be based on a combination of the analytic result and the final interpretation. This is especially important for genetic testing, where the test may only be performed once in a patient's lifetime. Because a genotype is not informative, per se, proper result interpretation requires integration with other information that may include family history, ancestry, and knowledge of genotype/phenotype associations, which is important in clinical genetics because insights about the causes of disease and the effects of various mutations or combinations changes rapidly and may affect the clinical management of the patient.

Another important component of the postanalytic analysis is laboratory reporting of the identified mutation using the appropriate gene mutation nomenclature, which makes clear to the physician interacting with the patient which sequence change has been identified. This may also include a reference to ''common'' nomenclature, such as ''factor V Leiden,'' whichisnotconsensusnomenclature but may be helpful in the report because it is most familiar to clinicians. (44,51) Use of incorrect or ambiguous nomenclature can lead to errors in interpretation, treatment selection, and testing other family members or their result interpretations, especially if testing is performed in different laboratories.

Grading

Methods of grading proficiency test results vary with the provider, the specific scheme, and the analyte. Some providers, such as the CAP, grade survey results on the accuracy of the genotypic result and the interpretation. For example, the CAP has recently started to grade the sizing of Huntington disease and myotonic dystrophy repeats in all size categories but has graded interpretation at 80% consensus for many years. Many of the CAP surveys are graded based on the consensus of 80% of participants; if that level of consensus is not reached, the challenge remains ungraded and ''educational.'' Other providers, such as EMQN, provide numeric scores based on genotyping, interpretation, and reporting. (38) Each EMQN participant receives an individualized report with their scores and comments from the evaluators, including areas of their test report that need improvement. Scoring of EMQN schemes depends on the analyte and the scheme. For example, there are 2 EQA schemes for BRCA gene testing (breast cancer risk). The BRCA Full scheme assesses genotyping, biologic, and clinical interpretation, whereas the EMQN BRCA Geno scheme assesses only genotyping and biologic interpretation. Other EMQN schemes, such as congenital adrenal hyperplasia and hereditary deafness (GJB2/GJB6 mutations), score genotyping only. These differences reflect the reporting practices of the participating laboratories. Finally, it is often challenging for PT programs to compare results because of the use of different methods, different calibration standards, and different cutoff values by participants. These issues do not usually affect molecular genetic testing for inherited genetic disease because most results are qualitative, but they are important for tests such as BCR/ABL1 or viral load, where the results are quantitative. Proficiency testing/EQA programs cannot effectively grade some quantitative surveys because of the lack of calibration materials, standardized values for reference ranges, and cutoff values.

The Role of Laboratory PT in Quality Management

Participation in PT/EQA allows laboratories to compare their performance against the range of responses provided by a group of peer laboratories. These comparisons are most effectively used to influence laboratory practice when they are systematically integrated into a Quality Management System (QMS; defined by the ISO as a management system to direct and control an organization with regard to quality''). (3) As part of QMS, a laboratory must participate in relevant PT/EQA schemes and ensure that the cost is integrated into the budget with sufficient time and staff resources assigned to support participation. In addition, the laboratory must ensure that the results of PT/EQA are properly considered, disseminated, and implemented to improve laboratory testing. Participation in appropriate PT/ EQA and/or alternative assessment is a requirement within the CAP, the ISO, and the Australian accreditation processes.

The quality policy implemented through a QMS defines the overall approach to PT/EQA and how PT/EQA challenges are appropriately handled in the laboratory. The QMS should include clearly defined laboratory procedures for receipt of performance results from a PT/EQA agency or other assessment protocols, such as alternative assessment. This procedure will normally follow 3 phases. First, the laboratory director is required to disseminate the data to staff immediately. This conveys to staff the importance of participating in PT/EQA schemes and enhances the educational content of the exercise. The performance data must be reviewed for any indication of deficient performance that requires immediate corrective action to avoid errors. Proficiency testing/EQA data should be routinely discussed in the most appropriate meetings of management, the quality team, and staff so that lessons learned can be discussed and, if necessary, changes in standard operating procedures can be quickly integrated into the QMS. Proficiency testing/EQA data provides an opportunity for a laboratory to compare its performance with peers as well as to celebrate and praise the staff when performance is exemplary. Second, PT performance data should be presented during management review meetings to examine any recurrent deficiencies that require correction or improvement. Third, PT/EQA records should be stored and formatted for external audit by an accrediting agency.

Laboratories may experience a lapse or error in PT/EQA performance. Single occurrences of poor performance should be logged as an incident and used as an opportunity to review procedures and make improvements. A careful evaluation of the error may determine whether there is a system failure that may require redesign of a test, more-frequent instrument calibration, or adjustments to training procedures. However, PT/EQA may detect serial or persistent failures that the laboratory is obligated to address with a more fundamental review. Some PT/EQA providers report that laboratories have discontinued testing services following instances of poor performance in PT/EQA. (44) That indicates the importance that laboratory directors assign to this external comparison and emphasizes the role of PT/ EQA as an educational tool and mechanism to improve performance in clinical laboratories.

In some countries, PT/EQA providers are required to report poor and recurrent poor performance to an official monitoring agency. That agency may have a role in ensuring that corrective actions are designed to address poor PT/EQA performance. Ultimately, they may have the power to escalate their intervention to involve the host institution of the laboratory concerned as well as regulatory agencies.

Evidence of the Value of PT/EQA in Ensuring Good Laboratory Performance

Although few studies have addressed whether participation in PT/EQA programs directly improved laboratory performance, there is empirical evidence showing that the educational aspect of PT/EQA does help laboratories detect errors in their testing protocols and identify problems associated with their assays. (2) One study reviewed 3 rounds of PT data from 2002 (any provider) from approximately 6300 CAP-accredited laboratories. (58) The study evaluated whether laboratories corrected deficiencies identified by PT or whether they continued to have unsuccessful PT performance. The analysis indicated that about 90% of the PT problems were resolved after the first round of PT and 99% were resolved by the third round, suggesting that the laboratories had successfully corrected mistakes identified by PT performance. Other studies have shown that participation in the CAP ''Calibration Verification/Linearity Surveys,'' which examine calibration verification and analytic measurement range of several nongenetic analytes (chemistry, immunology, hematology, among others), is associated with fewer PT failures. (59,60) A UKNEQAS PT/EQA scheme for hemophilia A from 2003 failed 4 laboratories based on poor performance in the evaluation of their clinical report. In subsequent surveys, only 1 laboratory failed to report properly, suggesting that the educational value and improvement of testing practices were derived from that exercise. (51) Another value of PT/EQA is the opportunity for laboratories to analyze unusual samples not often encountered in their service, such as, mosaicism of variants in fragile X syndrome.

The UKNEQAS PT/EQA program emphasizes interpretation of data and their implications for both the patient and the family. One summary of data indicates that poor PT/ EQA performance because of interpretation errors has decreased between 1997 and 2006. (44) This may be derived from continued participation in PT/EQA.

Analysis of PT/EQA results has also revealed inconsistent results stemming from the lack of uniformity in practices among laboratories. Once these issues were identified, steps were implemented to harmonize practices among the laboratories. Poor and inconsistent performance of laboratories participating in a PT/EQA scheme for fragile X syndrome led to consensus testing and reporting guidelines development in the United Kingdom (44) and the United States. (56) Because of ambiguities in the ways laboratories report sequence variations, PT schemes (such as UKNEQAS) strongly suggested that genotypic PT/EQA results should be reported using the Human Genome Variation Society standard nomenclature. (44,51) Participants in the CAP's disease-specific surveys typically report results using the common allele names; however, CAP requires the use of the Human Genome Variation Society nomenclature for participants in its PT scheme on interpretation of sequence variants in rare disorders and is moving toward integration of the consensus nomenclature for all schemes.

Limitations of PT

To assess a given test, PT/EQA should evaluate the capability of a particular assay to identify a range of possible test results or alleles, either during an individual challenge or during the course of several challenges. Considering that, PT for a given disease or genetic variant may be limited by the following:

1. Issues related to the limited availability of resources, including appropriate samples and reference materials;

2. Logistical and practical difficulties encountered, including the inability to offer PT/EQA for all genetic disorders, especially those tests for rare disorders or those performed by a few laboratories only; and

3. Limitations in the PT process for evaluation of laboratory performance.

Ideally, PT/EQA should evaluate the ability of the laboratory to identify all genotypes of interest for a particular disorder, which requires the availability of appropriate samples. For simple tests, such as factor V Leiden or prothrombin 20210A, very few alleles are tested in a clinical setting; thus, relatively few PT/EQA samples are required to represent the population variant affecting patients. For other disorders, such as CF, there is a large variation in the number and composition of alleles included in clinical assays. Some assays only examine the 23 alleles recommended for carrier screening by the American College of Medical Genetics and the American College of Obstetricians and Gynecologists, (61) whereas other laboratories offer assays that test more than 100 alleles, some of which are unique to particular patient populations. Many of the alleles included in the more comprehensive CF assays are very rare, and it is often difficult to obtain samples from patients or cell lines for PT/EQA. Another difficulty is the lack of highly characterized reference materials or calibrators with which to evaluate potential PT/EQA materials. This is especially important for DNA fragment-sizing assays, such as triplet repeat testing but may also be a factor for qualitative assays.

Proficiency testing programs also encounter logistic and practical problems. Most of the 2300 clinical genetic tests are offered in only one or a few laboratories, making provision of a formal proficiency survey logistically and economically difficult. Many multiplex genetic tests may simultaneously detect dozens of alleles (eg, CF or pharmacogenetic loci). It is logistically impossible for PT/EQA programs distributing 3 samples 1 or 2 times a year to provide enough different samples to adequately challenge the variety of genotypes represented in all patients or all assays for a particular genetic disorder. The DNA sequence analysis identifies mutations in any part of a gene and can pose a unique challenge in interpretation. In addition, PT/EQA typically targets well-characterized conditions and mutations. However, new disease associations are discovered regularly and genotype-phenotype correlations are often not fully elucidated until years after the initial discovery. Allelic heterogeneity and the clinical effect of some sequence variants, such as missense mutations and complex variants, pose considerable challenges for interpretation when functional studies have not yet been performed or were inconclusive. An example of this situation is pharmacogenetic testing for CYP2D6 variants. Many tests for this gene include polymorphisms with undefined effects on CYP2D6 enzyme activity. These less-well-characterized variants are challenging for developing of reference materials, reporting, and interpreting PT/EQA results. Formal PT/EQA might become available for more inherited conditions in the future, but it will probably not be feasible for such programs to cover all conditions and available tests.

Approaches to Address Limitations of Proficiency Testing: Methods-Based PT/EQA

Proficiency testing that is designed to evaluate the performance of analytic methods, rather than to evaluate specific genotypic assays, has been proposed as a complmentary mechanism to more-broadly assess laboratory performance. (45,62-64) Methods-based PT/EQA examines technologies common to many genetic tests, such as DNA purification, polymerase chain reaction (PCR) amplification, or DNA sequencing, and allows evaluation of critical analytic steps for individual assays that are not directly assessed by traditional PT/EQA schemes. It also permits interlaboratory comparisons and can highlight analytic practices, such as DNA quantification, which could be optimized for improved performance. (62)

The European Commission funded a project (EQUAL) (Full program title: ''Multinational External Quality Assay [EQA] Programmes in Clinical Molecular Diagnostics Based on Performance and Interpretation of PCR Assay Methods Including Dissemination and Training'') to develop and evaluate the utility of methods-based EQA to address methodologic procedures and analytic proficiency in molecular diagnostic test performance independent of the target. Three EQUAL pilot projects for qualitative analysis, quantitative PCR, and DNA sequencing (EQUAL-Qual, EQUAL-Quant, and EQUAL-Seq) were initiated. (62,65,66) The results from those pilot studies identified many areas of laboratory performance that varied considerably between participants (Table 4) and suggested areas that could be targeted for improvement.

Because of relatively poor laboratory performance in the EQUAL-Seq project, specific training highlighting analytic and methodologic skills were subsequently offered to the participating laboratories. A significant improvement of technical and interpretative skills was demonstrated in a confirmatory second round of EQA. (67)

Currently, methods-based PT is available for a variety of assays (Table 4). The EMQN offers a methods-based scheme for Sanger sequencing. Participants receive amplicons to characterize, identify, and report the sequence variants using the proper nomenclature. They also provide their raw sequence data, which is evaluated by EMQN for quality scores, quality read length, and quality read overlap. (63) In the United States, the CAP has launched an electronic DNA sequence-analysis survey. In 2012, that survey will send a set of primers and 3 DNA specimens to participants. Participants will identify all variants and report on them with accepted nomenclature. In the future, both the CAP and the EMQN plan to offer methods-based surveys to address the performance of next-generation DNA sequencing.

Synthetic Samples

Synthetic DNA samples can be used for PT/EQA. These samples may be composed of plasmid DNA containing specific sequences or PCR amplicons. The CF Network tested the usefulness of a synthetic reference material containing 6 homozygous mutations in the cystic fibrosis gene (CFTR) and one polymorphism for suitability as PT material. (68) Most of the laboratories in the study successfully identified the mutations, although some technical difficulties, such as incorrect genotyping (10 of 197 participants; 5%), or absent, incorrect, or insufficient interpretation (33 of 197 participants; 17%) regardless of genotype, were reported. This synthetic sample was supplied in a bloodlike matrix, which also permitted evaluation of the DNA extraction step. In the United States, artificially constructed CF mutation samples designed to mimic extracted human genomic DNA produced similar results. (69)

COMMENT

Proficiency testing has demonstrated value as an important laboratory quality assurance tool and has helped laboratories identify issues related to test design and performance. In addition, the ability to compare laboratory performance with others using the same or different methods on identical samples can highlight issues related to test methodology or interpretation or may inform development of best-practice guidelines and standard policy.

Adoption of new and complex testing technologies, such as next-generation sequencing assays, will require modifications to PT/EQA design and provision. In contrast to traditional genetic tests, which identify only a few mutations or perhaps the whole sequence of a well-characterized gene, next-generation sequencing has the capacity to examine the sequence of large gene panels, the exome, or the entire human genome, with an almost infinite variety of possible variants. Proficiency testing programs for next-generation sequence assays need to monitor the ability of the laboratory to detect mutations in any part of the genome included in their validated test. In addition to the analytic phase of the testing process, the data analysis and interpretation of next-generation DNA sequencing is considerably more complicated than that of sequencing tests with smaller scopes. Proficiency testing/EQA can be used to compare performance among laboratories during all phases of next-generation DNA sequencing testing and may provide important indicators of which steps of the testing process are problematic. That information may not be readily discernible through daily quality-control practices, and PT/EQA could be an important tool to assess whether the testing algorithm is sufficient to detect a loss of sensitivity or specificity for the detection of sequence variations that may only be evident when comparing results among laboratories. Interpretation of the analytic test result of such large-scale analyses is difficult and uncertain because the effect of individual mutations, the function of each gene and its interaction with other genes in the genome has not yet been determined. Proficiency testing/ EQA schemes to assess the ability of the laboratory to interpret and report complex data could be quite informative. Novel and innovative PT/EQA challenges will need to be developed to ensure the quality of these new tests.

Proficiency testing or EQA should include sufficient analytes to provide a reasonable estimate of interlaboratory comparability. For example, PT for factor V Leiden testing assesses the laboratory's capacity to identify 1 of 3 possible genotypes per sample (wild type, factor V Leiden heterozygote, and factor V Leiden homozygote). Developing a PT program to gauge interlaboratory comparability in this example is fairly straightforward. Proficiency testing for disorders with many disease-associated alleles, such as CF, or a next-generation sequencing test presents additional challenges. Current PT can only assess a subset of possible mutations per challenge. Is the current process sufficient for achieving a credible interlaboratory comparison for very complex tests? The limited availability of characterized DNA materials and the cost associated with increasing the number of PT samples can be significant barriers for a PT/ EQA program with a goal to offer comprehensive challenges.

Significant research, needs assessment, and pilot testing should be performed to ensure that PT addresses the changing needs of genetic testing technology and an evolving knowledge base and continues to be a relevant and useful quality assurance tool. Some of the research topics may include ways to provide PT/EQA for the increasing number of new tests, many of which are offered in only one or a few laboratories; to include all phases of the testing process, rather than just the analytic phase; and to develop novel approaches for effective multiplex genotype-testing challenges, as well as PT/EQA strategies to assess new technologies, such as next-generation sequencing. Additional research will define PT/EQA improvements to promote the quality of laboratory testing, interpreting, and reporting.

Most genetic tests are done in only one or a few laboratories. Many of these tests use the same technology, such as Sanger or next-generation sequencing to examine a particular gene or set of genes. Many other tests share similar methodology, such as DNA and RNA purification, PCR amplification, or multiplex ligation-dependent probe amplification. Other aspects, such as result reporting, are also common across tests. Research to develop and evaluate novel method- or technology-based PT/EQA schemes may simultaneously assess the performance of many tests. Schemes that evaluate the quality of laboratory reports, for example, those of the CF Network, UKNEQAS, and the EMQN, could also be developed more broadly. In addition, research should be conducted to assess the effectiveness and relationship of the commonly used methods of alternative assessment, such as sample exchange or blinded retesting of previously tested specimens, to the accuracy of routine laboratory test results. Information gleaned from such studies might provide guidance to laboratories on effective methods of alternative assessment.

Much work needs to be done to ensure that PT/EQA programs can meet the needs of new and evolving genetic tests and technologies. An important first step would be to design a scientific approach to collecting data on the effect of PT/EQA on laboratory testing quality. Such studies could measure the relationship between PT performance and the accuracy of routine test results in that same laboratory and could be expanded to include PT/EQA for the newer testing technologies, including microarrays and next-generation sequencing. Information from these studies might inform the development of improved PT programs, professional guidelines, and regulations.

A variety of research and development projects will address some of the current limitations of PT/EQA, including the shortage of characterized reference materials, both naturally occurring and synthetic, that can be used as PT/EQA samples. Cell lines can be created from patients with genetic disorders who are currently part of PT/EQA programs and also from those with disorders for which PT/ EQA would be useful but for which programs do not yet exist. Consideration should be given to each disorder to ensure that a comprehensive set of reference materials, containing as many clinically significant alleles as possible, will be developed. In addition, development of synthetic reference materials containing many alleles for a given disorder simultaneously should also be considered. These materials must be evaluated by testing in several clinical laboratories using a variety of assays and technologies before they are used as PT/EQA samples.

Proficiency testing and external quality assessment have been and should remain an integral part of laboratory quality assurance. In the next few years, we hope to conduct and facilitate these and other research projects to evaluate and improve the quality of PT for molecular genetic testing.

The authors would like to thank John Beilby, PhD, Head, Diagnostic Molecular Genetics Department (PathWest, Nedlands, Western Australia, Australia), for his comments on this work.

References

(1.) National Center for Biotechnology Information. GeneTests. http://www. ncbi.nlm.nih.gov/projects/GeneTests/static/abou^dat^summary.shtml. Updated March 22, 2012. Accessed May 14, 2012.

(2.) Dequeker E, Ramsden S, Grody WW, Stenzel TT, Barton DE. Quality control in molecular genetic testing. Nat Rev Genet. 2001;2(9):717-723.

(3.) International Organization for Standardization. Medical Laboratories--Particular Requirements for Quality And Competence. Geneva, Switzerland: International Organization for Standardization; 2007. Report No.: ISO 15189.

(4.) Clinical & Laboratory Standards Institute. Quality Management System: A Model for Laboratory Services; Approved Guideline--Fourth Edition. Wayne, PA: Clinical and Laboratory Standards Institute; 2011. Report No.: GP26-A4.

(5.) American College of Medical Genetics. ACMG Standards & Guidelines for Clinical Genetics Laboratories. http://www.acmg.net/AM/Template. cfm?Section=Laboratory_Standards_and_Guidelines&Template=/CM/ HTMLDisplay.cfm&ContentID=6958MD. Accessed May 15, 2012.

(6.) Clinical and Laboratory Standards Institute. Molecular Diagnostics Methods for Genetic Disease; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2006. Report No.: MM01-A2.

(7.) New York State Department of Health. Clinical Laboratory Evaluation Program. http://www.wadsworth.org/labcert/clep/clep.html. Accessed May 15, 2012.

(8.) Department of Health and Human Services Health Care Financing Administration. Clinical Laboratory improvement amendments of 1998-- laboratory requirements. Fed Regist. 1992;57(40):7146. Codified at 42 CFR [section]493. 2004. http://wwwn.cdc.gov/clia/regs/toc.aspx. Accessed May 15, 2012.

(9.) Clinical and Laboratory Standards Institute. Nucleic Acid Sequencing Methods in Diagnostic Laboratory Medicine; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2004. Report No.: MM9-A.

(10.) Clinical and Laboratory Standards Institute. Diagnostic Nucleic Acid Microarrays; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2006. Report No.: MM12-A.

(11.) Clinical and Laboratory Standards Institute. Verification and Validation of Multiplex Nucleic Acid Assays; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2008. Report No.: MM17-A.

(12.) Clinical and Laboratory Standards Institute. Establishing Molecular Testing in Clinical Laboratory Environments; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2011. Report No.: MM19-A.

(13.) Mattocks CJ, Morris MA, Matthijs G, et al; EuroGentest Validation Group. A standardized framework for the validation and verification of clinical molecular genetic tests. Eur J Hum Genet. 2010;18(12):1276-1288.

(14.) EuroGentest 2 Coordination Action. Eurogentest Web site: Laboratories. http://www.eurogentest.org/laboratories/. Updated April17, 2011. Accessed May 14, 2012.

(15.) Chen B, Gagnon M, Shahangian S, et al; Centers for Disease Control and Prevention. Good laboratory practices for molecular genetic testing for heritable diseases and conditions. MMWR Recomm Rep. 2009;58(RR-6):1-35. http:// www.cdc.gov/mmwr/pdf/rr/rr5806.pdf. Accessed May 10, 2012.

(16.) Anderson MW, Schrijver I. Next generation DNA sequencing and the future of genomic medicine. Genes. 2010;1(1):38-69.

(17.) Kearney HM, South ST, Wolff DJ, et al; Working Group of the American College of Medical Genetics. American College of Medical Genetics recommendations for the design and performance expectations for clinical genomic copy number microarrays intended for use in the postnatal setting for detection of constitutional abnormalities. Genet Med. 2011;13(7):676-679.

(18.) International Organization for Standardization-International Electrotechnical Commission. Conformity Assessment--General Requirements for Proficiency Testing. Geneva, Switzerland: International Organization for Standardization; 2010. ISO/IEC 17043.

(19.) Clinicaland Laboratory Standards Institute. Assessment of Laboratory Tests When Proficiency Testing is Not Available; Approved Guideline. Wayne, PA:. Clinical and Laboratory Standards Institute 2008. Report No.: GP29-A2.

(20.) McGovern MM, Benach MO, Wallenstein S, Desnick RJ, Keenlyside R. Quality assurance in molecular genetic testing laboratories. JAMA. 1999;281(9): 835-840.

(21.) McGovern MM, Elles R, Beretta I, et al. Reportofan international survey of molecular genetic testing laboratories. Community Genet. 2007;10(3):123-131.

(22.) College of American Pathologists. CAP Web site: Genetics. http://www. cap.org. Updated July 27, 2012. Accessed May 14, 2012.

(23.) Hudson KL, Murphy JA, Kaufman DJ, Javitt GH, Katsanis SH, Scott J. Oversight of US genetic testing laboratories. Nat Biotechnol. 2006;24(9):1083 1090.

(24.) Libeer JC. Role of external quality assurance schemes in assessing and improving quality in medical laboratories. Clin Chim Acta. 2001;309(2):173 177.

(25.) Ehrmeyer SS, Laessig RH. Has compliance with CLIA requirements really improved quality in US clinical laboratories? Clin Chim Acta. 2004;3 46(1):37-43.

(26.) Hofgartner WT, Tait JF. Frequency of problems during clinical molecular-genetic testing. Am J Clin Pathol. 1999;112(1):14-21.

(27.) Keenlyside RA, Collins CL, Hancock JS, et al. Do proficiency test results correlate with the work performance of screeners who screen Papanicolaou smears? Am J Clin Pathol. 1999;112(6):769-776.

(28.) LaMotte LC, Guerrant GO, Lewis DS, Hall CT. Comparison of laboratory performance with blind and mail-distributed proficiency testing samples. Public Health Rep. 1977;92(6):554-550.

(29.) Jenny RW, Jackson KY. Proficiency test performance as a predictor of accuracy of routine patient testing for theophylline. Clin Chem. 1993;39(1):76 81.

(30.) Howerton D, Krolak JM, Manasterski A, Handsfield JH. Proficiency testing performance in US laboratories: results reported to the Centers for Medicare & Medicaid Services, 1994 through 2006. Arch Pathol Lab Med. 2010;134(5):751758.

(31.) Clinical and Laboratory Standards Institute. Proficiency Testing (External Quality Assessment) For Molecular Methods; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2005. Report No.: MM14-A.

(32.) Clinical and Laboratory Standards Institute. Using Proficiency Testing to Improve the Clinical Laboratory; Approved Guideline. Wayne, PA: Clinical and Laboratory Standards Institute; 2007. Report No.: GP27-A2.

(33.) Secretary's Advisory Committee on Genetics, Health, and Society. U.S. System of Oversight of Genetic Testing: A Response to the Charge of the Secretary of Health and Human Services. Bethesda, MD: Department of Health & Human Services; 2008. http://oba.od.nih.gov/oba/SACGHS/reports/SACGHS_oversight_report.pdf. Accessed May 10, 2012.

(34.) National Center for Biotechnology. GeneTests Web site. http://www.ncbi. nlm.nih.gov/sites/GeneTests/. Accessed May 14, 2012.

(35.) INSERM. Orphanet Web site. http://www.orpha.net/consor/cgi-bin/index. php. Last updated: May 13, 2012. Accessed May 14, 2012.

(36.) National Center for Biotechnology. Genetic Testing Registry Web site. http://www.ncbi.nlm.nih.gov/gtr/. Accessed May 15, 2012.

(37.) The Centers for Disease Control and Prevention. Newborn screening quality assurance program Web site. http://www.cdc.gov/labstandards/nsqap. html. Last updated May 9, 2012. Accessed May 14, 2012.

(38.) European Molecular Genetics Quality Network. The European Molecular Genetics Quality Network Web site. http://www.emqn.org/emqn/Home. Accessed May 14, 2012.

(39.) CF Network. Cystic Fibrosis European Network Web site. http://cf. eqascheme.org/. Accessed May 14, 2012.

(40.) TheUnitedKingdomNationalExternalQualityAssessmentServiceWebsite.http:// www.ukneqas.org.uk/content/PageServer.asp?S=99359106&C=1252&CID=1&type=G. Updated July 6, 2010. Accessed May 14, 2012.

(41.) EuroGentest. External Quality Assessment. http://www.eurogentest.org/ laboratories/qau/eqa/. Updated November 23, 2011. Accessed May 14, 2012.

(42.) The Deutsche Vereinte Gesellschaft fur Klinische Chemie und Laboratoriumsmedizin e.V. (DGKL) Web site. http://www.dgkl-rfb.de/index_E.shtml. Accessed May 14, 2012.

(43.) Human Genetics Society of Australasia Molecular Genetics Quality Assurance Program (HGSA MGQAP) Web site. http://www.hgsa.org.au/ resources/quality-assessment/. Updated January 10, 2012. Accessed May 14, 2012.

(44.) Ramsden SC, Deans Z, Robinson DO, et al. Monitoring standards for molecular genetic testing in the United Kingdom, The Netherlands and Ireland. Genetic Test. 2006;10(3):147-156.

(45.) Richards CS, Grody WW. Alternative approaches to proficiency testing in molecular genetics. Clin Chem. 2003;49(5):717-718.

(46.) Organization for Economic Cooperation and Development. OECD Guidelines for Quality Assurance in Molecular Genetic Testing. Paris, France: OECD, 2007:1-38.

(47.) Bellissimo DB: Practice guidelines and proficiency testing for molecular assays. Transfusion. 2007;47(1)(suppl):79S-84S.

(48.) Korn K, Reil H, Walter H, Schmidt B. Quality control trial for human immunodeficiency virus type 1 drug resistance testing using clinical samples reveals problems with detecting minority species and interpretation of test results. J. Clin. Microbiol. 2003;41(8):3559-3565.

(49.) Fae I, Lau M, Voorter C, Mayr WR, Fischer GF. HLA-B*8102, a new allele found in an external proficiency testing scheme. Tissue Antigens. 2004;64(5): 608-610.

(50.) Longhi E, Frison S, Colombini I, Malagoli A, Crespiatico L, Poli F. HLA-A*2626, a new allele identified through external proficiency-testing exercise. Tissue Antigens. 2005;66(4):325-326.

(51.) Perry DJ, Goodeve A, Hill M, Jennings I, Kitchen S, Walker I; UK NEQAS for Blood Coagulation. The UK National External Quality Assessment Scheme (UK NEQAS) for molecular genetic testing in haemophilia. Thromb Haemost. 2006;96(5):597-601.

(52.) Weck KE, Zehnbauer B, DattoM, Schrijver I; CAP/ACMG Biochemical and Molecular Genetics Resource Committee. Molecular genetic testing for fragile X syndrome: laboratory performance on the College of American Pathologists proficiency surveys (2001-2009). Genet Med. 2012;14(3)306-312. doi:10.1038/ gim.2011.11.

(53.) Palomaki GE, Richards CS. Assessing the analytic validity of molecular testing for Huntington disease using data from an external proficiency testing survey. Genet Med. 2012;14(1):69-75. doi: 10.1038/gim.0b013e3182310bb5.

(54.) Dequeker E, Cassiman JJ. Evaluation of CFTR gene mutation testing methods in 136 diagnostic laboratories: report of a large European external quality assessment. Eur J Hum Genet. 1998;6(2):165-175.

(55.) Noll, WW, Belloni, DR, Stenzel T, Grody. WW. Polymorphism in intron 4 of HFE does not compromise haemochromatosis mutation results [correspondence]. Nature Genet. 1999;23(3):271-272. doi:10.1038/15722.

(56.) Spector EB, Kathryn Kronquist; FragileX Working Group of the Laboratory Quality Assurance Committee. Technical Standards and Guidelines for Fragile X Testing: A Revision to the Disease-Specific Supplements to the Standards and Guidelines for Clinical Genetics Laboratories of the American College of Medical Genetics. Standards and Guidelines for Clinical Genetics Laboratories 2006 Edition. Bethesda, MD: American College of Medical Genetics; 2006. http:// www.acmg.net/Pages/ACMG_Activities/stds-2002/fx.htm. Accessed May 14, 2012.

(57.) Mueller CR, Kristoffersson U, Stoppa-Lyonnet D. External quality assessment for mutation detection in the BRCA1 and BRCA2 genes: EMQN's experience of 3 years. Ann Oncol. 2004; 15(suppl 1): i14-i17. doi:10.1093/ annonc/mdh652.

(58.) Hoeltge FA, Phillips MG, Styer PE, Mockridge P. Detection and correction of systematic laboratory problems by analysis of clustered proficiency testing failures. Arch Pathol Lab Med. 2005;129(2):186-189.

(59.) Lum G, Tholen DW, Floering DA. The usefulness of calibration verification and linearity surveys in predicting acceptable performance in graded proficiency tests. Arch Pathol Lab Med. 1995;119(5):401-408.

(60.) Kroll MH, Styer PE, Vasquez DA. Calibration verification performance relates to proficiency testing performance. Arch Pathol Lab Med. 2004;128(5): 544-548.

(61.) Watson MS, Cutting GR, Desnick RJ, et al. Cystic fibrosis population carrier screening: 2004 revision of American College of Medical Genetics mutation panel. Genet Med. 2004;6(5):387-391. Erratum Genet Med. 2005;7(4): 286; Genet Med. 2004;6(6):548.

(62.) Ahmad-Nejad P, Dorn-Beineke A, Pfeiffer U, et al. Methodologic European external quality assurance for DNA sequencing: the EQUALseq program. Clin Chem. 2006;52(4):716-727.

(63.) Patton SJ, Wallace AJ, Elles R. Benchmark for evaluating the quality of DNA sequencing: proposal from an international external quality assessment scheme. Clin Chem. 2006;52(4):728-736.

(64.) Raggi CC, Pizani R, Paradiso A, Pazzagli M, Orlando C. External quality assurance program for PCR amplification of genomic DNA: an Italian experience. Clin Chem. 2003;49(5):782-791.

(65.) Orlando C, Verderio P, Maatmanet R, et al. EQUAL-qual: a European program for external quality assessment of genomic DNA extraction and PCR amplification. Clin Chem. 2007;53(7):1349-1357.

(66.) Ramsden SC, Daly S, Geilenkeuser WJ, et al. EQUAL-quant: an international external quality assessment scheme for real-time PCR. Clin Chem. 2006;52(8):1584-1591.

(67.) Dorn-Beineke A, Ahmad-Nejad P, Pfeiffer U, et al. Improvement of technical and analytical performance in DNA sequencing by external quality assessment-based molecular training. Clin Chem. 2006;52(11):2072-2078.

(68.) Berwouts S, Gordon JT, Rundell CA, Barton DE, Dequeker E. Evaluation and use of a synthetic quality control material included in the European external quality assessment scheme for cystic fibrosis. Hum Mutat. 2008;29(8):1063 1070.

(69.) Jarvis M, Iyer RK, Williams LO, NollWW, Thomas K, Grody WW. Anovel method for creating artificial mutant samples for performance evaluation and quality control in clinical molecular genetics. J. Mol Diagn. 2005;7(2):247-251.

(70.) Seneca S, Morris MA, Patton S, Elles R, Sequeiros J. Experience and outcome of 3 years of a European EQA scheme for genetic testing of the spinocerebellar ataxias. Eur J Hum Genet. 2008;16(8):913-920.

(71.) Falbo V, Floridia G, Tosto F, et al. The Italian external quality assessment scheme for fragile xsyndrome: the results of a 5-year survey. Genetic Test. 2008; 12(2):279-288.

(72.) Taruscio D, Falbo V, Floridia G, et al. Quality assessment in cytogenetic and molecular genetic testing: the experience of the Italian Project on Standardisation and Quality Assurance. Clin Chem Lab Med. 2004;42(8):915-921.

(73.) Tosto F, Salvatore M, Falbo V, et al. The Italian scheme of external quality assessment for p-thalassemia: genotyping and reporting results and testing strategies in a 5-year survey. Genet Test Mol Biomarkers. 2009;13(1):31-36.

(74.) Hertzberg M, Neville S, McDonald D. External quality assurance of molecular analysis of haemochromatosis gene mutations. J Clin Pathol. 2006; 59(7):744-747.

(75.) Brothman AR, Dolan MM, Goodman BK, et al. College of American Pathologists/American College of Medical Genetics proficiency testing for constitutional cytogenomic microarray analysis. Genet Med. 2011;13(9):765 769.

(76.) Birch L, English CA, Burns M, Keer JT. Generic scheme for independent performance assessment in the molecular biology laboratory. Clin Chem. 2004; 50(9):1553-1559.

(77.) Raggi CC, Verderi P, Pazzagli M, et al. An Italian program of external quality control for quantitative assays based on real-time PCR with Taq-Man" probes. Clin Chem Lab Med. 2005;43(5):542-548.

Lisa V. Kalman, PhD; Ira M. Lubin, PhD; Shannon Barker, PhD; Desiree du Sart, PhD; Rob Elles, PhD; Wayne W. Grody, MD, PhD; Mario Pazzagli, PhD; Sue Richards, PhD; Iris Schrijver, MD; Barbara Zehnbauer, PhD

Accepted for publication August 28, 2012.

From the Laboratory Research and Evaluation Branch, Division of Laboratory Science and Standards, Office of Surveillance, Epidemiology, and Laboratory Services, Centers for Disease Control and Prevention, Atlanta, Georgia (Drs Kalman, Lubin, Barker, and Zehnbauer); the Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, Georgia (Dr Barker); the Victorian Clinical Genetics Services, Murdoch Children's Research Institute, Parkville, Victoria Australia (Dr du Sart);the Manchester Academic Health Science Centre, Genetic Medicine, St Mary's Hospital, Manchester, Greater Manchester, United Kingdom (Dr Elles); the Departments of Pathology & Laboratory Medicine, Pediatrics, and Human Genetics, University of California, Los Angeles School of Medicine, Los Angeles, California (Dr Grody); the Department of Clinical Physiopathology, University of Florence, Florence, Italy (Dr Pazzagli); the Department of Molecular and Medical Genetics, Oregon Health and Science University, Portland, Oregon (Dr Richards);and the Departments of Pathology and Pediatrics, Stanford University School of Medicine, Stanford, California (Dr Schrijver).

The authors have no relevant financial interest in the products or companies described in this article.

The findings and conclusions in this report are those of the author and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the Agency for Toxic Substances and Disease Registry.

Reprints: Lisa V. Kalman, PhD, Laboratory Research and Evaluation Branch, Division of Laboratory Science and Standards, Office of Surveillance, Epidemiology, and Laboratory Services, Centers for Disease Control and Prevention, 1600 Clifton Rd, Mailstop G23, Atlanta, GA 30333 (e-mail: LKalman@cdc.gov).
Table 1. Methods Used to Test for Inherited
Disorders

Method Used               % of Diseases
                          Tested

Sequencing (a)            93 (90 of 97)

Transcription-mediated    23 (22 of 97)
amplification

Deletion/duplication      27 (26 of 97)
analysis

Mutation scanning         18 (18 of 97)

Methylation analysis      2 (2 of 97)

(a) Analysis of a random sample (~10%) of
diseases for which molecular
genetic testing methods are used (February
26, 2009 report of ~970
diseases obtained from GeneTests),
determined that testing for ~93%
(90 of 97) of these diseases used DNA
sequencing methods in at least
some of the laboratories that offered
testing;49% (48 of 97) of the
diseases in the sample were tested using
only sequencing techniques.

Table 2. Proficiency Testing/External Quality Assessment (PT/EQA)
Provider Schemes for Inherited Disorders

PT/EQA Provider
   Source, y                    Analytes Covered

CAP, (22) 2012     Disease-specific schemes: factor V
                   Leiden, fragile X syndrome,
                   hemochromatosis, MTHFR, Prader-Willi
                   /Angelman syndrome, prothrombin,
                   CF, Huntington disease,
                   Friedreich ataxia, hemoglobin
                   SC, Duchenne muscular dystrophy,
                   myotonic dystrophy, RhD,
                   spinal muscular atrophy,
                   spinocerebellar ataxia, BRCA1 and
                   BRCA2, connexin 26, multiple
                   endocrine neoplasia type 2,
                   Canavan disease, familial
                   dysautonomia, Tay-Sachs disease,
                   mucolipidosis IV, Bloom syndrome,
                   Fanconi anemia, Gaucher disease,
                    glycogen storage disease type 1A,
                   Niemann-Pick disease type
                   A. Pharmacogenetic markers.
                    Methods-based schemes: DNA
                   sequencing, microarray genomic
                   copy number assay.

EMQN,(38) 2012     Disease-specific schemes: Y-chromosome
                   microdeletions, BRCA1 and
                   BRCA2, CAH, CMT, Familial
                   Adenomatous Polyposis Colon
                   Cancer, GJB2 and GJB6, HNPCC,
                   monogenic diabetes, Marfan
                   syndrome, porphyria, hereditary
                   recurrent fevers, myotonic
                   dystrophy, Duchenne muscular
                   dystrophy, fragile X syndrome, Friedreich
                   ataxia, Huntington disease,
                   hemochromatosis, multiple
                   endocrine neoplasia type 2A,
                   PKU, Prader-Willi/Angelman
                   syndromes, retinoblastoma, short
                   stature homeobox gene
                   testing, spinocerebellar ataxia,
                   spinal muscular atrophy, Von Hippel
                   Lindau disease, Wilson
                   disease. Methods-based schemes:
                   Sanger DNA sequencing, array-CGH,
                   next-generation sequencing (pilot).

UKNEQAS, (40)      Molecular genetics: Angelman syndrome,
2012               Becker/Duchenne muscular
                   dystrophy, BRCA1 and BRCA2,
                   CF, factor V Leiden, familial
                   adenomatous polyposis, fragile X
                   syndrome, hereditary and motor
                   sensory neuropathy, Huntington disease,
                   HNPCC, hereditary neuropathy with
                   liability to pressure
                   palsies, MCADD, mitochondrial
                   disorders, molecular rapid
                   aneuploidy testing, myotonic
                   dystrophy, Prader-Willi syndrome,
                    spinal muscular atrophy,
                   spinocerebellar ataxia. Molecular
                   testing on dried blood spots: CF, MCADD.
                   Microarray CGH.

HGSA,(43) 2012     23 disease-specific PT modules in
                   collaboration with EMQN and the
                   CF Network. Y-chromosome
                   deletions, familial breast cancer
                   (BRCA), CAH, CF, CMT, connexin
                   26, myotonic dystrophy,
                   Becker/Duchenne muscular
                   dystrophies (BMD/DMD), familial
                   adenomatous polyposis coli, fragile
                   X syndrome, Friedreich ataxia,
                    Huntington disease, HNPCC,
                   hereditary recurrent fevers (pilot),
                   multiple endocrine neoplasia
                   Type 2, mitochondrial
                   myopathy, monogenic diabetes, PKU,
                   porphyria, Prader-Willi/Angelman
                   syndromes, retinoblastoma,
                   spinocerebellar ataxias, spinal
                   muscular atrophy, Von Hippel
                   Lindau syndrome, Wilson disease.
                   Also methods-based DNA
                   sequence and analysis scheme and
                   cytogenetics are also offered.

ISS, (41) 2012     Cystic fibrosis, b thalassemia,
DGKL, (42) 2012    fragile X syndrome, adenomatous
                   polyposis of colon.
                   Molecular Biology schemes: Factor
                   V-Leiden, prothrombin, MTHFR, PAII
                   (SERPINE1), factor XIII
                   (F13A1), GPIIIa (1TGB3),
                   [3Fib (FGB), VKORC1, Factor XII (F12), a1
                   PI, APOE, APOB, ACE, CETP,
                   TPMT, CYP2C19, CYP2D6, CYP2C8,
                   CYP2C9, UGT1A1 , DPD
                   (DPYD), BCHE, ALDOB, HFE, LCT,
                   NOD2, ATP7B, FSAP (HABP2),
                   ITGA2, KRAS. Method schemes: DNA
                   sequencing, DNA isolation
                   + Factor V genotyping.

CDC NSQAP, (37)    Dried blood-spot testing
2012               for CF mutations.

PT/EQA Provider     Molecular Genetic          Sample Types
   Source, y           Challenges/y

CAP, (22) 2012     Disease-specific        Disease-specific,
                   schemes: 3 DNA          pharmacogenetic, and
                   samples 2 times/y;      microarray genomic
                   cystic fibrosis:        copy number assay
                   2 samples 2 times/y     schemes: extracted
                   Pharmacogenetic         DNA. Postanalytic
                   markers: 2 DNA          DNA sequencing
                   samples 2 times/y.      scheme:1 CD-ROM
                   Postanalytic DNA        containing
                   sequencing scheme:      DNA sequence
                   3 electronic            electropherogram
                   challenges 2 times/y.   files
                   Microarray genomic
                   copy number assay:
                   2 DNA samples +
                   challenge 2 times/y

EMQN,(38) 2012     3 DNA samples +         Lyophilized DNA
                   mock clinical data 1
                   time/y

UKNEQAS, (40)      Includes most           Lyophilized DNA,
2012               disease-specific        whole blood
                   schemes: 3 samples      spotted on
                   1 time/y; molecular     filter paper
                   testing on blood        and dried
                   spots: 3 samples
                   4 times/y

HGSA,(43) 2012     3 samples are           Extracted DNA
                   sent out                or RNA, lyophilized
                   per disease             DNA(through
                   module, 1 times/y       EMQN); amplified
                                           DNA products and
                                           sequencing primers

ISS, (41) 2012     6 samples/disease       DNA from
DGKL, (42) 2012    Molecular biology       lymphoblastoid cells
                   schemes: 2 times/y.     Molecular
                   Sequencing-based        biology schemes:
                   scheme: 2 times/y.      lyophilized DNA.
                   DNA isolation:          Sequencing-based
                   2 times/y.              scheme: 2
                                           lyophilized
                                           DNA samples,
                                            DNA isolation:
                                           2 tubes whole blood

CDC NSQAP, (37)    5 dried blood spots     Blood from patient
2012               4 times/y               with CF spotted
                                           on filter
                                           paper and dried

PT/EQA Provider        Phases of
   Source, y        Testing Covered

CAP, (22) 2012     Disease-specific,
                   pharmacogenetic
                   and microarray
                   genomic copy
                   number assay
                   schemes:
                   genotyping and
                   interpretation
                   Postanalytic DNA
                   sequencing
                   scheme:
                   Interpretation

EMQN,(38) 2012     Genotyping,
                   interpreting and
                   reporting

UKNEQAS, (40)      Genotyping,
2012               interpreting
                   and reporting

HGSA,(43) 2012     Analytic,
                   postanalytic

ISS, (41) 2012     Analytic and
DGKL, (42) 2012    postanalytic
                   Molecular biology
                   schemes: analytic.
                   Method-based
                   schemes: sequence
                   result and
                   interpretation.
                   DNA isolation:
                   determination of
                   concentration of
                   DNA, ratio
                   260:280, method of
                   identification,
                   defined genotypes

CDC NSQAP, (37)    Preanalytic,
2012               analytic,
                   interpretation

Abbreviations: b-Fib, Fibrinogen-Beta polypeptide chain; CAH,
congenital adrenal hyperplasia; CAP, College of American Pathologists;
CDC, Centers for Disease Control and Prevention;CD-ROM, compact
disk-read-only memory;CF, cystic fibrosis;CGH, comparative genomic
hybridization; CMT, Charcot-Marie-Tooth disease; DGKL, Reference
Institute for Bioanalytics; DPD, Dihydropyrimidine dehydrogenase;
EMQN, European Molecular Genetics Quality Network; GPIIIa, platelet
glycoprptein IIIa; FASP, Factor VII-Activating Protease; HGSA, Human
Genetics Society of Australasia; HNPCC, hereditary nonpolyposis
colorectal cancer (Lynch syndrome); ISS, Istituto Superiore
di Sanita; MCADD, medium chain acyl-coenzyme A dehydrogenase
deficiency; MTHFR, methylenetetrahydrofolate reductase; NSQAP,
Newborn Screening Quality Assurance Program; PAII, Plasminogen
Activator Inhibitor I PI, proteinase inhibitor; PKU, phenylketonuria;
RhD, Rhesus (hemolytic) disease; SC, sickle cell;
UKNEQAS, United Kingdom National External Quality Assessment Service.

Table 3. Disease-Specific Proficiency Testing Performance
(as Assessed From Select, Published Literature)

Source, y           Disease              Participation

Palomaki and        Huntington           33 US and 23
Richards            disease              international
(CAP), (53)                              laboratories
2012

Weck et al          Fragile X            Average of 90
(CAP), (52)         syndrome             laboratories
2012                                     during study
                                         period; US
                                         and international

Seneca et al        Spinocerebellar      Only 28.8%
(EMQN), (70)         ataxia              (15 of 52)
2008                                     laboratories
                                         participated in
                                         all consecutive
                                         y

Mueller et al       BRCA1 and BRCA2
(EMQN), (57)
2004

Dequeker,           Cystic fibrosis      135 European
Cassiman                                 laboratories,
(European                                1 Australia
Concerted                                laboratory
Action on Cystic
Fibrosis
framework), (54)
1998

Ramsden et al,      Several diseases
(44) 2006;
Perry et al (UK
NEQAS), (51) 2006

Falbo et al         Fragile X syndrome   65% (22 of 34)
(ISS), (71) 2008                         of all Italian
                                         laboratories that
                                         perform testing
                                         participated. 15
                                         laboratories
                                         participated in
                                         each of the 5 y

Taruscio et al      Cystic fibrosis,
(ISS), (72)         fragile X
2004                syndrome, others

Tosto et al         b thalassemia        11-18 laboratories
(ISS), (73) 2009                         during 5y

Hertzberg et        HFE                  37 laboratories
al (RCPA), (74)                          currently
2006

Source, y           General Performance      Analytic Errors
                                             Identified

Palomaki and        Analytic sensitivity,    Repeat length errors
Richards            99.5%, during 6          occurred in 2.6%
(CAP), (53)         surveys                  (28 of 1060) allelic
2012                                         challenges. Most
                                             errors were minor
                                             and occurred in a
                                             small subset of
                                             the laboratories.
                                             Non-US participants
                                             had the higher
                                             error rate (17.5%,
                                             (107 of 612)).

Weck et al          Analytic                 Measurement of repeat
(CAP), (52)         sensitivity for          length was more
2012                detection of full        accurate with smaller
                    mutations 99%            samples.
                    (males), 96%
                    (females);
                    premutations 98%

Seneca et al        97.5% of reports         Large interlaboratory
(EMQN), (70)        correctly identified     variation seen in
2008                genotype as              allele sizing.
                    normal or pathogenic     Gross diagnostic errors
                                             occurred every
                                             year (false-negatives,
                                             false-positives,
                                             and sample
                                             misidentification).
                                             Not all laboratories
                                             reported allele size.

Mueller et al       Error rate               Missed mutations (53%
(EMQN), (57)        during 3 y,              (9 of 17) of
2004                7.4% (9 of 121)-         false results).
                    3.7% (4 of 108)          Problems with
                                             incorrect nomenclature.

Dequeker,           35% (48 of 136) of       Mistyping,
Cassiman            laboratories             misinterpretation
(European           incorrectly              of data, erroneous
Concerted           genotyped                technical results.
Action on Cystic    >1of12
Fibrosis            CFTR alleles
framework), (54)    in study
1998

Ramsden et al,      Poor performance         Technical errors
(44) 2006;          was sporadic             and wrong
Perry et al (UK                              nomenclature.
NEQAS), (51) 2006

Falbo et al         Overall success          Overall, 5% (30 of 601)
(ISS), (71) 2008    rate,                    of samples tested were
                    76% (458 of              incorrectly genotyped,
                    601).                    and 1.5% (9 of 601)
                                             of that was due
                                             to 1 sample, a
                                             male mosaic normal/
                                             full mutation, which
                                             was incorrectly
                                             genotyped 9 times
                                             (3 by the same
                                             laboratory).
                                             Some technical
                                             failures caused by
                                             prescreening methods.

Taruscio et al      Interpretive             7% (2 of 28) of
(ISS), (72)         performance              laboratories incorrectly
2004                varied by disease        genotyped 2 mutations,
                                             although their
                                             commercial kit
                                             detects those
                                             mutations. Many
                                             laboratories did
                                             not include clinically
                                             relevant cystic
                                             fibrosis mutations
                                             in their panels.

Tosto et al         98.8% (889 of 900)
(ISS), (73) 2009    of alleles
                    correctly
                    identified

Hertzberg et        99.47%                   Most errors were
al (RCPA), (74)     (3000 of 3016)           due to sample
2006                success during           handling or result
                    10 surveys               transcription. A
                                             few errors caused
                                             by technical
                                             issues.

Source, y           Interpretation
                    Errors Identified

Palomaki and        3 interpretation
Richards            errors were made
(CAP), (53)         in US laboratories
2012                during the 6
                    surveys.

Weck et al          In samples with
(CAP), (52)         repeat sizes of
2012                20-33, during 18
                    surveys, 3 interpretation
                    errors made. Many
                    laboratories incorrectly
                    identified samples
                    with 42 repeats
                    as gray-zone or
                    premutation.

Seneca et al        Wide range of allele
(EMQN), (70)        sizing but
2008                without incorrect
                    diagnoses. Clerical
                    errors observed.

Mueller et al       Misinterpretation of
(EMQN), (57)        observed
2004                sequence changes.

Dequeker,
Cassiman
(European
Concerted
Action on Cystic
Fibrosis
framework), (54)
1998

Ramsden et al,      Incomplete reports;
(44) 2006;          wrong clinical
Perry et al (UK     interpretations.
NEQAS), (51) 2006

Falbo et al         Reports showed
(ISS), (71) 2008    considerable
                    variation.
                    Starting in 2004,
                    a model was
                    issued for reports.

Taruscio et al      Written reports
(ISS), (72)         were inadequate
2004                for .50% of
                    laboratories. The
                    most common error
                    was lack of
                    risk calculation
                    when a
                    mutation was
                    not detected.

Tosto et al
(ISS), (73) 2009

Hertzberg et        Incorrect results
al (RCPA), (74)     because of
2006                transcriptional

Source, y           Lessons Learned

Palomaki and        Performance was very
Richards            good. An analytic
(CAP), (53)         method with a high
2012                error rate was
                    identified.

Weck et al          Accuracy of sizing
(CAP), (52)         and interpretation
2012                of larger
                    repeats improved
                    over time.

Seneca et al        (1) One-third (5 of 15)
(EMQN), (70)        of laboratories with
2008                inaccurate sizing did
                    not participate the
                    following year, and
                    for those that did,
                    >50% (6 of 10) continued
                    to report wrong sizes,
                    showing importance of
                    participation in PT
                    every year; and (2)
                    always be reported: it
                    is a measure of
                    technical quality and
                    may give clinically
                    important information.

Mueller et al       Horizontal comparison
(EMQN), (57)        among laboratories
2004                allows for better
                    understanding of the state
                    of the field and for
                    detection of systematic
                    technical failures.

Dequeker,           Regular participation in
Cassiman            PT schemes would
(European           be beneficial for
Concerted           laboratories. Laboratory
Action on Cystic    personnel should
Fibrosis            participate in regular
framework), (54)    training sessions.
1998                Need consensus strategies
                    for diagnostic testing.
                    Centralized mutation
                    analysis facilities would
                    be useful for
                    identified rare mutations.

Ramsden et al,      (1) Any poor performance
(44) 2006;          causes immediate
Perry et al (UK     correspondence between
NEQAS), (51) 2006   the organizer and
                    the laboratory; and
                    (2) scheme located
                    test-validity problems
                    as well as clinical
                    practice problems,
                    with wider implications
                    for the genetic
                    testing community.

Falbo et al         (1) Importance of reports
(ISS), (71) 2008    for patient care; and
                    (2) importance of secondary
                    screening methods.

Taruscio et al      (1) Even commercial kits
(ISS), (72)         must be validated in
2004                the laboratory before
                    use for clinical
                    testing; (2) laboratories
                    need guidance on
                    the kinds of mutations to
                    include in their
                    panels; (3) complete reports
                    are crucial for
                    patient care; and (4)

Tosto et al         laboratory errors were
(ISS), (73) 2009    disease specific and
                    were not indicative of
                    overall laboratory
                    analytic performance.
                    job of reporting some
                    aspects of the test,
                    such as diagnostic
                    sensitivity; however,
                    many reports did not
                    identify the mutations
                    tested, indication for
                    genetic counseling, or
                    interpretation of
                    results.

Hertzberg et        Performance was very good,
al (RCPA), (74)     but laboratories
2006                should continue to
                    participate in large
                    quality assurance programs
                    to maintain
                    high testing quality.

Abbreviations: CAP, College of American Pathologists;EMQN, European
Molecular Genetics Quality Network;ISS, Istituto Superiore di
Sanita;PT, proficiency training;QAP, quality assurance program;
RCPA, Royal College of Pathologists of Australasia;UK NEQAS,
United Kingdom National

Table 4. Methods-Based Proficiency Testing
(PT) Schemes and Performance

Source, y       Technology        Laboratories
                Assessed          Provided
                                  With

Brothman        aCGH              Extracted DNA
et al                             from cell
(CAP),                            lines derived
(75) 2011                         from individuals
(ongoing PT                       with a
scheme)                           constitutional
                                  abnormality.

(CAP), (22)     Postanalytic      One CD-ROM
(ongoing PT     DNA               containing DNA
scheme)         sequencing        sequence
                                  electropherogram
                                  files.

Orlando         DNA extraction    (1) Two
et al           and               preextracted
(EQUAL),        amplification     DNA samples, (2)
(65) 2007                         whole blood
(pilot)                           samples, and (3)
                                  primer sets.

Ramsden         Real-time         (1) ABL gene
et al           PCR               primers,
(EQUAL),                          (2) 5 -FAM/3 -
(66) 2006                         TAMRA-labeled
(pilot)                           probes, (3) 5 ABL
                                  standard plasmids
                                  (10E2-10E5), (4) 3
                                  test cDNA samples,
                                  and (5) 2 cell
                                  samples.

Ahmad-Nejad     Sequencing        (1) Primer sets,
et al                             (2) 2 plasmid
(EQUAL),                          DNA
(62) 2006                         samples (1, a
(pilot)                           vector; 1, a
                                  mixture
                                  of 2 different
                                  plasmids
                                  generated to
                                  provoke comments
                                  about template
                                  quality), (3)
                                  a purified
                                  PCR product,
                                  and (4) a finished
                                  sequencing
                                  reaction to be
                                  purified and
                                  analyzed.

Patton          Sequencing        (1) 450-bp, PCR-
et al                             amplified fragments
(EMQN),                           of CFTR covering all
(63) 2006                         main types of
(ongoing EQA                      sequence changes,
scheme)                           (2) a wild-type
                                  control, (3) primers
                                  (with location in
                                  sequence), (4)
                                  reference sequence,
                                  and (5) protein
                                  translation

Raggi et al     DNA               (1) Three primer
(ISS),(64)      extraction, PCR   pairs,
2003 (pilot)    performance,      (2) 1 reference
                interpretation    DNA,
                                  (3) 3 DNA
                                  calibrators,
                                  and (4) 1
                                  blood sample

(HGSA-ASOC),    Microarray        Extracted DNA
(43) (ongoing   analysis          from an
scheme)                           individual tested
                                   for a
                                  constitutional
                                  abnormality

(HGSA-MGSA),    DNA sequencing    (1) Amplified
(43) (ongoing   and               products
scheme)         interpretation    and (2) sequencing
                                  primers

Birch et al,    DNA extraction,   (1) Tubes
(76) 2004       PCR               containing high,
                amplification     medium, and
                                  low concentrations
                                  of bacteria
                                  in buffer; (2)
                                  control DNA in
                                  buffer; (3) PCR
                                  reagents (dNTPs,
                                  primers, Taq
                                  polymerase,
                                  reaction
                                  buffer);and
                                  (4) 100-bp
                                  DNA ladder, gel
                                  loading dye.

(DGKL),         DNA sequencing    Lyophilized DNA
(42) (ongoing   and
scheme)         interpretation

(DGKL),         DNA isolation,    Whole blood
(42) (ongoing   quantitation
scheme)         (and
                (and factor
                V genotyping)

Raggi et al     TaqMan            (1) Standard cDNA
(ISS), (77)     real-time         solution
2005 (pilot)    PCR               obtained by
                                  in vitro
                                  transcription of
                                  a fragment of
                                  hTERT cloned
                                  into a plasmid
                                  vector, (2)
                                  mix of primers,
                                  (3) 3 unknown
                                  Cdna samples,
                                  and (4) PCR
                                  conditions.

Source, y       PT protocol                 Laboratory Performance

Brothman        Laboratories must (1)       95.7% (472 of 493)
et al           analyze specimens and       of responses
(CAP),          identify and interpret      were concordant for an
(75) 2011       results for clinically      abnormality involving a
(ongoing PT     significant copy number     particular chromosomal
scheme)         abnormalities, and (2)      location. Errors
                identify and characterize   included designating an
                gains or losses and the     opposite change,
                cytogenetic location of     reporting a clinically
                any abnormalities           insignificant copy
                detected using standard     number change, and
                nomenclature.               failure to report
                                            a clinically significant
                                            abnormality.

(CAP), (22)     Laboratories must           No published data
(ongoing PT     interpret                   available.
scheme)         and report DNA
                sequence variants for
                inherited disease using
                standard nomenclature.

Orlando         Laboratories must (1)       25% (42 of 165) of
et al           perform DNA extraction      laboratories
(EQUAL),        by routine procedures,      performed poorly in
(65) 2007       (2) estimate DNA quality    quantification of >1
(pilot)         and quantity of both        of the 2 preextracted
                preextracted and            samples; 27% (46
                laboratory-extracted        of 166) of laboratories
                DNA, (3) perform PCR        had questionable
                with 100 ng of DNA          results for quality
                from all samples, (4)       and/or quantity of
                submit raw data from        blood sample extractions.
                DNA quantification and      High degree of
                post-PCR interpretation,    variability seen
                and (5) send aliquot of     with PCR
                DNA extracted from          performance of
                whole blood back to         all samples.
                EQUAL.

Ramsden         Laboratories must (1)       For the cDNA samples,
et al           construct a calibration     80% (74 of 93) of
(EQUAL),        curve; (2) estimate         laboratories provided
(66) 2006       cDNA copy numbers in        accurate values for
(pilot)         3 cDNA samples; (3)         all 3. For cell
                perform RNA extraction,     samples, approximately
                real-time PCR, and          35% (27 of 75, 25
                cDNA quantification of      of 74) of laboratories
                the 2 cell samples; and     provided 95% CI
                (4) provide Ct values for   limits that fell
                NTC, calibrators, and       outside the range
                unknowns, plus details      of standard
                of the testing platforms    dilutions. Wide
                used.                       variations in
                                            laboratory results
                                            were observed.

Ahmad-Nejad     Laboratories must (1)       For the cloning vector
et al           perform sequencing and      plasmid, the
(EQUAL),        provide longest possible    mean sequence
(62) 2006       sequence without any        stretch (of the
(pilot)         mistakes (primer-walking    2355-bp insert)
                not allowed) for DNA        was 537 bases
                plasmids and purified       and only 14 of 43
                PCR product and (2)         participants (33%)
                purify and analyze the      identified the insert
                finished sequencing         correctly. For the
                reaction.                   mixture of 2
                                            different plasmids,
                                            21% (9 of 43)
                                            of laboratories did
                                            not register
                                            any kind of
                                            difficulty; however,
                                            some excellent specific
                                            comments were received.
                                            For the finished PCR
                                            product, 57% (24
                                            of 43) of laboratories
                                            made no effort
                                            at identification.

Patton          Laboratories must           59% (36 of 61
et al           (1) return                  laboratories) scored
(EMQN),         form with genotyping        the maximum (12.0
(63) 2006       results and (2) return      marks); 19 genotyping
(ongoing EQA    color copies of             errors were made
scheme)         electropherograms and       from 346 genotypes
                electronic copies of        analyzed (5%); 10
                sequence data files.        of 19 (53%) were
                                            false-negative results,
                                            and 9 of 19 (47%)
                                            were false-positive
                                            results; 59%
                                            (27 of 46) of
                                            errors were made
                                            in naming mutations.
                                            Most laboratories
                                            produced data with
                                            acceptable
                                            diagnostic quality.

Raggi et al     Laboratories must (1)       Variability was
(ISS),(64)      extract DNA from blood      high for DNA
2003 (pilot)    sample and measure          quantification
                quality and quantity,       (range, 0.012-0.54
                (2) measure                 ig/|iL;CV, 82%). The
                quantity and                260:280 ratios varied
                quality of 3 standard       (range, 0.8-2.5;CV,
                DNA samples, (3)            21%). Large
                perform PCR, and (4)        variability seen in
                return PCR results,         yield and
                extracted DNA aliquot,      quality of PCR
                and PCR product             performance.
                aliquots.

(HGSA-ASOC),    Assessment of               No published data
(43) (ongoing   laboratory                  available.
scheme)         processing and/or
                analytic skills,
                as well as
                interpretive comments.

(HGSA-MGSA),    Laboratories must (1)       No published data
(43) (ongoing   perform Sanger              available.
scheme)         sequencing using PCR
                products and primers
                provided, and (2)
                analyze and report
                results using standard
                nomenclature and
                provide biologic
                interpretation.

Birch et al,    Laboratories must (1)       Identified analytic
(76) 2004       extract DNA from            issues such
                suspended bacteria, (2)     as inappropriate
                PCR-amplify extracted       extraction or
                DNA according to            amplification procedures,
                protocol provided, and      PCR inhibition or
                (3) analyze using           contamination, poor
                agarose gel                 labeling
                electrophoresis.            or poor-quality gel
                                            photography, and
                                            failure to
                                            record results
                                            correctly.

(DGKL),         DNA sequencing and          No published data
(42) (ongoing   corresponding               available.
scheme)         diagnostic
                interpretation

(DGKL),         Laboratories must (1)       No published data
(42) (ongoing   isolate DNA, (2)            available.
scheme)         determine the
                concentration of DNA,
                (3) report 260:280
                ratio,
                and (4) identify the
                factor V genotype of the
                sample.

Raggi et al     Laboratories must (1)       Only 12 of 42
(ISS), (77)     prepare dilutions           laboratories (29%)
2005 (pilot)    of the                      gave results that
                provided standard, (2)      were both concise
                perform PCR and             and accurate for all
                analyze results, (3)        samples tested;
                report                      17 of 42
                Ct values for each          laboratories
                well,                       (40%) reported
                and (4) report              inaccurate data
                concentrations of           for >1 result.
                unknown samples.            Inaccuracy showed
                                            an inverse dose-dependent
                                            trend; 12 of 42
                                            laboratories were
                                            unable to
                                            measure a sample with low
                                            concentration.

Abbreviations: aCGH, array-comparative
genomic hybridization; ASOC, Australasian Society
of Cytogeneticists; bp, base pair; CAP, College of
American Pathologists; cDNA, complementary
DNA; CI, confidence interval; Ct, cycle
threshold; CV, coefficient of variation; dNTP,
deoxynucleotide triphosphate;EMQN,
European Molecular Genetics Quality Network;EQA,
external quality assessment;HGSA, Human Genetics
Society of Australasia; hTERT, human
telomerase reverse transcriptase; ISS, Istituto
Superiore di Sanita; MGQAP, Molecular Genetics Quality
Assurance Program; NTC, no template control;
PCR, polymerase chain reaction.
COPYRIGHT 2013 College of American Pathologists
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2013 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Kalman, Lisa V.; Lubin, Ira M.; Barker, Shannon; du Sart, Desiree; Elles, Rob; Grody, Wayne W.; Pazz
Publication:Archives of Pathology & Laboratory Medicine
Geographic Code:1USA
Date:Jul 1, 2013
Words:13926
Previous Article:Cytology and human papillomavirus screening test results associated with 2827 histopathologic diagnoses of cervical intraepithelial neoplasia 2/3.
Next Article:Screening for Down syndrome in the United States: results of surveys in 2011 and 2012.
Topics:

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters