Commutability Assessment of External Quality Assessment Materials with the Difference in Bias Approach: Are Acceptance Criteria Based on Medical Requirements too Strict?
Commutability of External Quality Assessment (EQA)  materials is a key requirement for their use in accuracy-based EQA surveys (1-3). In a recent paper, Korzun et al. (4) evaluated commutability of 4 frozen pools for measurements of direct HDL cholesterol (HDLC) and LDL cholesterol (LDLC). These pools were used in the CDC's Lipid Standardization Program to assess accuracy of direct HDLC measurements only (4).
Among the results presented using the medical requirement acceptance criteria for bias (4% for LDLC and 5% for HDLC), the authors found that 1 of the 4 frozen pools was commutable for most of the HDLC methods, whereas none were commutable for LDLC methods. The authors concluded that frozen pools prepared according to the CLSI C37 protocol may not always be commutable and especially for direct LDLC assays.
In 2013, Laboratoire national de metrologie et d'essais (LNE) organized a similar study to assess commutability of 5 freshly prepared frozen serum pools prepared according to the CLSI C37-A protocol for HDLC and LDLC. The pools were shipped frozen and analyzed along with 20-25 fresh clinical specimens by 31 medical laboratories operating HDLC and LDLC routine methods on the most popular clinical chemistry analyzers: Roche Cobas, Siemens Vista, Abbott Architect, Ortho CD Vitros, Beckman DxC, Beckman AU, Roche Modular, and Thermo Konelab.
As described by Korzun et al. (4), the difference in bias observed between the reference materials and a set of clinical specimens was used as a measure of commutability. For each combination of 2 different methods, we established a difference plot of ln (Mx) - ln (My) vs [M.sub.m], where Mx and My are the averages of triplicate measurements performed on each individual patient specimen with methods x and y, respectively, and [M.sub.m] is the average mean concentration obtained with the 2 methods. The commutability acceptance criterion C (%) was calculated as k * [s.sub.b], with a coverage factor k of 1.9 (corresponding to a one-sided test at the 5% significance level) and [s.sub.b] as SD of the differences between the log-transformed mean concentrations measured with the 2 methods. The expanded uncertainty of the difference in bias between the clinical specimens and a given reference material was calculated as
k x [square root of ([s.sup.2.sub.b]/q + [s.sup.2.sub.x] + [s.sup.2.sub.y]/r)],
where [s.sub.x] and [s.sub.y] are the pooled SDs from triplicates measurements performed on all patient specimens with routine methods x and y; q, the number of clinical specimens; and r, the number of replicates. Using this approach, we found that the pools were commutable in 78%-100% of pairwise comparisons for LDLC and 47%-81% of pairwise comparisons for HDLC (see Table 1).
One difference between the 2 studies is that Korzun et al. used the beta-quantification reference method as a comparison method, whereas in our study, pairwise comparisons exclusively involved routine methods. This could affect the outcome of the statistical analysis because sample-specific effects can affect routine and reference methods differently and thus will not be estimated in a comparable manner. Since the measurement procedures included in a commutability study must have similar selectivity for the measurand, we speculate that most materials in the Korzun et al. study were found noncommutable because field methods and the beta-quantification reference method have different specificities. In addition to our use of fewer clinical specimens (20-25 instead of 175), another difference between the 2 studies was that Korzun et al. measured the frozen pools in duplicate at the beginning and end of each run, whereas we performed triplicate measurements only one time in a single run, which did not allow us to consider position and run effects. Our statistical analysis approach was the same as that in the Korzun paper.
Consistent with the results of Korzun et al., we found that the criteria for commutability based on random error components (approximately 10%-11%) were approximately 2 times higher than those based on medical requirements (approximately 4%-5%). Although preferable, the application of commutability criteria based on medical requirements appears quite stringent. When examined as if they were a pool, only 23%-27% of the fresh clinical specimens (commutable by definition) were found commutable using criteria based on medical requirements (against 83%-87% using criteria based on random error components), which suggests that medical-based criteria are probably too stringent. The homogeneous methods have been reported to be influenced by specimen specific effects owing to nonspecificity that likely contributes to this observation (5). At the same time, the acceptance criteria based on random error components varied for different pairwise comparisons, making it difficult to define generally applicable criteria. Since acceptance criteria sometimes exceeded 17% for some method pairs, these criteria may not always be stringent enough to validate commutability of materials used as trueness controls. We suggest developing fixed criteria that are appropriate for the intended medical use and that take the performance characteristics of procedures in use into account.
Author Contributions: Allauthors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.
Authors' Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/orpotential conflicts of interest:
Employment or Leadership: None declared.
Consultant or Advisory Role: H.W. Vesper, College of American Pathologists.
Stock Ownership: None declared.
Honoraria: None declared.
Research Funding: None declared.
Expert Testimony: None declared.
Patents: None declared.
(1.) Miller WG. Specimen materials, target values and commutability for external quality assessment (proficiency testing) schemes. Clin Chim Acta 2003; 327: 25-37.
(2.) Vesper HW, Miller WG, Myers GL. Reference materials and commutability. Clin Biochem Rev 2007; 28:139-47.
(3.) Miller WG, Myers GL. Commutability still matters. Clin Chem 2013; 59:1291-3.
(4.) Korzun WJ, Nilsson G, Bachmann LM, Myers GL, Sakurabayashi I, Nakajima K, et al. Difference in bias approach for commutability assessment: application to frozen pools of humanserum measured by 8 direct methods for HDL and LDL cholesterol. Clin Chem 2015; 61:1107-13.
(5.) Miller WG, Myers GL, Sakurabayashi I, Bachmann LM, Caudill SP, Dziekonski A, et al. Seven direct methods For measuring HDL and LDL cholesterol compared with ultracentrifugation reference measurement procedures. Clin Chem 2010; 56:977-86.
Vincent Delatour,  * Qinde Liu,  and Hubert W. Vesper,  on behalf of the LNE-LABAC Working Group on Commutability
 Laboratoire national de metrologie et d'essais (LNE) Paris, France
 Chemical Metrology Division Applied Sciences Group Health Science Authority Singapore
 Division of Laboratory Sciences CDC, Atlanta, GA
* Address correspondence to this author at:
1 rue Gaston Boissier
Previously published online at DOI: 10.1373/clinchem.2016.261008
A list of participating laboratories can be found in the online supplement at http://www.clinchem.org/content/ vol62/issue12.
 Nonstandard abbreviations: EQA, External Quality Assessment; HDLC, HDL cholesterol; LDLC, LDL cholesterol; LNE, Laboratoire national de metrologie et d'essais.
Disclaimer: The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official views or positions of the Centers for Disease Control and Prevention/Agency for Toxic Substances and Disease Registry. Use of trade names and commercial sources is for identification only and does not constitute endorsement by the US Department of Health and Human Services or the US Centers for Disease Control and Prevention.
Table 1. Commutability results of 5 frozen serum pools prepared according toCLSI C37-A for HDLC and LDLC. (a) LDL-C Acceptance Acceptance criteria based criteria based on random error on medical components, % requirements, % [LDLC] Mean C I NC C I NC mg/dL (b) [absolute value of [D.sub.RM]] Pool 1 77.4 2.4 89 11 0 22 78 0 Pool 2 136.0 2.7 100 0 0 56 22 22 Pool 3 95.0 2.6 78 22 0 11 89 0 Pool 4 156.4 4.1 89 11 0 44 22 33 Pool 5 66.0 2.5 89 0 11 11 78 11 CS NA 0.0 84 16 1 23 53 25 HDL-C Acceptance Acceptance criteria based criteria based on random error on medical components, % requirements, % [HDLC], Mean C I NC C I NC mg/dL (b) [absolute value of [D.sub.RM]] Pool 1 49.9 3.3 81 19 0 38 59 3 Pool 2 62.1 4.2 81 19 0 22 66 13 Pool 3 42.4 8.8 47 28 25 9 41 50 Pool 4 63.0 7.5 75 22 3 13 53 34 Pool 5 36.5 4.8 59 34 6 28 53 19 CS NA 0.0 87 13 0 27 65 9 (a) The same 20 fresh clinical specimens (CS) were shipped to 7 laboratories and 12 different sets of 25 clinical specimens were shipped to 12 pairs of other medical laboratories. The fresh clinical specimens and the 5 frozen pools were measured in the same analytical sequence. C stands for commutable, I for inconclusive, and NC for noncommutable, which were all set as described by Korzun et al. (4). Commutability results were obtained using 2 different acceptance criteria based on random error components and medical requirements. The mean of absolute values of the difference for the reference material ([D.sub.RM]) was calculated as defined by Korzun et al. (4) from 9 pairwise comparisons between field methods for LDLC and 32 pairwise comparisons for HDLC. NA, notapplicable. (b) To convert concentrations from mg/dL to mmol/L, multiply by 0.02586.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Letters to the Editor|
|Author:||Delatour, Vincent; Liu, Qinde; Vesper, Hubert W.|
|Article Type:||Letter to the editor|
|Date:||Dec 1, 2016|
|Previous Article:||Necessity Is the Mother of Invention: The Development of Digital Genomics.|
|Next Article:||Impact of Pneumatic Tube System Transport of Whole Blood Samples on Free DNA Concentration in Human Plasma.|