Between-lot variation in external quality assessment of glucose: clinical importance and effect on participant performance evaluation.
Glucometers used at physicians' offices or for self-monitoring of blood glucose use either capillary or venous whole blood. Whole-blood glucose is not stable, and different additives have different effects on the various meters (5, 6). It therefore is difficult to use a single control material in EQAS for all glucose meters (7). Method-specific target values are commonly used (8). The Norwegian Quality Improvement of Primary Care Laboratories (NOKLUS) uses 2 different control materials in EQAS, depending on the type of meter. For each survey, NOKLUS registers the instrument as well as the lot number of glucose strips used by participating office laboratories. Our surveys have indicated that the results of glucose measurements are affected by the lot of the blood glucose testing strips used as well as the type of glucometer. These results raise 2 important questions. (a) Would between-lot variation detected by use of control materials also be observed in clinical situations in which fresh capillary blood is used? (b) Should participant assessment results be compared against a lot-specific target value rather than a method-specific target value, i.e., what effects does the use of different target values have on participant assessment results?
We present results obtained with different lots of glucose strips for 4 widely used meters, comparing capillary blood and 3 EQA control materials.
Materials and Methods
In the NOKLUS EQAS for glucose, control materials in 2 concentrations are sent to ~1900 participating Norwegian office laboratories twice a year. All participants return the results together with information about the instrument and lot number of strips used. More than 20 different types of glucometers are represented in the EQA scheme. The 4 instruments included in the present study were used by approximately one-third of the participants.
On the basis of routine EQAS carried out with Sugar Chex (Streck Laboratories) or stabilized EDTA blood (Sero AS) as control materials, 3 different lots of glucose strips were selected for each of the 4 glucometers: Accu-Chek Sensor (Roche Diagnostics), Precision Xtra (Abbott Laboratories/ Medisense), Ascensia Elite (Bayer Diagnostics), and HemoCue 201 (HemoCue) instruments (see section below on instrument and lot selection). The 12 lots were investigated in the laboratory by use of capillary blood samples and stabilized EDTA blood. All but the lots from the HemoCue 201 were further examined by use of Sugar Chex and a control material used for Norwegian hospital laboratory EQAS, a liquid frozen serum pool (Sero AS). For each instrument, differences among the 3 lots were first calculated for capillary blood material and thereafter for each of the 3 control materials. In addition, results with capillary blood were compared with those from a conventional reference method. Finally, results from the NOKLUS October 2003 EQAS for glucose were used to simulate the effects of between-lot variation on participants assessments.
INSTRUMENT AND LOT SELECTION
This study included the instruments that were most widely used and that demonstrated significant between-lot variation in the NOKLUS October 2003 EQAS for glucose. Glucose strips from each of 3 lots were selected for each instrument according to the following criteria: (a) one lot number had significantly different results from the two others and (b) one lot number had results that were similar to the average of all lot numbers. As an example, results from the EQA survey for the Ascensia Elite are shown with all different lot numbers as well as the lots selected for our study (Fig. 1).
Capillary blood samples were obtained from 12 persons: 5 healthy volunteers and 7 patients with diabetes. The mean plasma glucose value was 7.53 mmol/L (range, 5.11-14.78 mmol/L) as measured by the Advia 1650 (with reagents from Bayer). The 12 capillary blood samples were measured once with each of the 3 different lots on each instrument (total 12 measurements on the 4 instruments). In addition, capillary blood samples for reference measurements (see below) were taken before and after meter measurements. The order of the lot numbers was altered between patients, and the procedure time did not exceed 10 min. The difference in glucose values between the first and the last sample as measured by the conventional reference method had to be <4% (9). Two series of measurements were excluded because of differences >4%.
[FIGURE 1 OMITTED]
The control materials used were Sugar Chex Proficiency [7.1 mmol/L; lot no. 32660723 (expiration date, January 19, 2004)]; stabilized EDTA blood (4.08 mmol/L EDTA, 1.8 mmol/L sodium iodoacetate) from 1 donor, adjusted with D-glucose to 7.38 mmol/L (measured on Advia 1650); and fresh-frozen human serum [7.04 mmol/L, as measured by use of an isotope-dilution gas chromatography-mass spectrometry reference method (4)]. For each control material, 10 measurements were carried out with each of the 3 different lots on each type of instrument. The order of the 3 lot numbers was altered between each measurement.
CONVENTIONAL REFERENCE METHOD
The conventional reference method for plasma glucose was performed on the Advia 1650 (Glucose Hexokinase method II; product no. B01-4597-01). The method was verified by the control solution [frozen human serum pool targeted with the isotope-dilution gas chromatography-mass spectrometry reference method for glucose (4)] used by NOKLUS in the EQAS for glucose for hospital laboratories. For instruments that reported whole-blood values, i.e., the Accu-Chek Sensor and HemoCue 201, results were multiplied by 1.11 (10) to be comparable to plasma values.
PARTICIPANT ASSESSMENT RESULTS
Participant assessment results in the EQAS for glucose were designated as good for results within the target interval (the method-specific target value [+ or -] 0.1 mmol/L) [+ or -] 5%; acceptable for results between [+ or -] 5% and [+ or -] 10% outside the target interval; and poor for results more than [+ or -] 10% outside the target interval. The method-specific target value was calculated as the median of all results from participants using a particular instrument, after outliers had been excluded (11).
We used an assumed CV of 5% and calculated that 10 replicate measurements (or 10 patients in case of capillary whole blood) would be required to detect a difference of ~8% between 2 lots (90% power using a 2-sided 0.05 level test). Between-lot variation and instrument bias were assessed by t-tests, and statistical significance was set to 5%. When selecting lots for inclusion in the study, we set statistical significance to 0.01 because of multiple comparisons.
A simulation study was performed because the number of participants using each lot number was rather small because of the large number of lots in use. The simulation was based on results from the NOKLUS October 2003 EQAS for glucose, including the 4 instruments and lots that were part of the present study. Using the mean (SD) for each of the 12 lots in 2 concentrations (~7 and ~20 mmol/L), we created 24 gaussian distributions, each comprising 100 values.
[FIGURE 2 OMITTED]
EFFECT OF USE OF CAPILLARY BLOOD AND DIFFERENT QUALITY-CONTROL MATERIALS ON BETWEEN-LOT VARIABILITY
Between-lot variation was affected by the instrument, the lot of glucose strips, and the test material (Fig. 2). For the Precision Xtra, the capillary blood results mirrored those of the EQA control materials. For the Accu-Chek Sensor and Ascensia Elite, between-lot variations of up to 1.3 mmol/L were obtained with certain control materials and differed by as much as 0.8 mmol/L from between-lot variations obtained with capillary blood. For the HemoCue 201, there was a difference of <0.3 mmol/L between all 3 lots for EDTA blood but no difference when capillary blood was used.
As shown in Fig. 3, results from the HemoCue 201 using capillary blood showed no deviation from the conventional reference values, whereas results from the Accu-Chek Sensor were higher and results from the Precision Xtra and Ascensia Elite were lower for all 3 lots compared with the conventional reference method.
EFFECT OF BETWEEN-LOT VARIATION ON PARTICIPANT ASSESSMENT RESULTS
To compare the consequences of different target values on participant assessment results, we performed a simulation study (see Materials and Methods). For the 3 instruments that used Sugar Chex as the quality-control material in our EQAS for glucose, the simulated results were assessed by use of a common target value (assessment of method, lot, and participant), a method-specific target value (assessment of lot and participant), and a lot-specific target value (assessment of participant). With the HemoCue 201, which used another control material, the simulated values could be assessed only with a method-specific target value and a lot-specific target value. The percentage of participants with a poor assessment result decreased from 38% for participants who used the common target value to 10% and 4% for those who used a method-specific or a lot-specific target value, respectively. For detailed consequences related to lots and instruments see Table 1 and Fig. 1 in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/ content/vo151/issue9/.
EQAS for glucose analysis is a problem because it is difficult to provide a single control material commutable to capillary blood for all different types of meters. Method-specific target values are therefore used.
Between-lot variations for glucose strips were first described in 1986 (12) and subsequently in several reports (13-16). In a study performed in 1998, Skeie et al. (16) showed that differences in glucose-strip quality among lots might be a major factor in diminishing analytical quality. Another recent study reported measurement variability between test-strip lots for 5 hand-held glucose meter systems (15). The investigators in all of these studies used either fresh capillary blood or plasma supplemented with glucose in their evaluations.
The present study showed between-lot variation in results for both capillary blood and control materials (Figs. 1 and 2). Between-lot differences were as much as 1.3 mmol/L with control materials. Such differences are clinically important at critical decision limits for glucose (9,17). It is therefore important to undertake measures to determine whether the between-lot differences observed in EQAS are likely to occur when native blood is used. On the other hand, between-lot differences present in capillary blood will probably not always be reflected when control materials are used, and the absence of between-lot differences in EQAS is no guarantee that such differences do not occur when native blood samples are tested. Manufacturers should therefore be urged to produce more consistent glucose strips, and commutability studies should be carried out to ensure that control materials give valid results.
Because there is much variability among lots, the lot used by the most participants will influence the target values to a greater extent than the other lots, and participants with an aberrant lot will have a greater probability of obtaining a poor assessment result. If a lot-specific target value is used, these participants will have a much higher probability of obtaining a good result.
The simulation study showed that there is a large effect on participant assessment results whether a common target value for all methods, a method-specific target value, or a lot-specific target value is used. Therefore, lot-specific target values should be used for conducting participant assessments because the performance of the participants thus would not be adversely affected even if the method and lot used have poor quality. This approach will be difficult to implement, however, because there are many different lots of strips on the market at the same time and the number of participants using each of the lots is rather small. In our October 2003 EQAS for glucose, there were 1834 participants using 24 different instruments and ~400 different lots of strips. An alternative procedure for EQAS for glucose is to register lot numbers so that participants can be given a fair feedback report. For example, the feedback report may inform a participant assessed as poor that the lot used deviated significantly from the method target median.
Although evaluation of the trueness of meters and strips was not the main objective of this study, it is interesting to note that only the HemoCue results did not show any deviation from the results of the conventional reference method, a finding that is in agreement with some studies (18-20) but in contrast to others (21-23). For one of the instruments, the Accu-Chek Sensor, one lot showed a significantly larger deviation from the reference method than the other two lots. Between-lot variation is important in instrument evaluations, and more than one lot should always be included (24, 25).
In conclusion, clinically important between-lot variations detected by use of control materials may not be detected with capillary blood and vice versa. In addition, EQA organizers should register lot numbers of glucose strips so that information on lot variation can be included in feedback reports. The use of lot-specific target values, however, is not feasible.
We thank Professor Per Hyltoft Petersen for review of the manuscript. The instruments used in this study were kindly supplied by Roche Diagnostics, Abbott Laboratories, Bayer Diagnostics, and HemoCue.
(1.) Libeer JC, Baadenhuijsen H, Fraser CG, Petersen PH, Ricos C, Stockl D, et al. Characterization and classification of external quality assessment schemes (EQA) according to objectives such as evaluation of method and participant bias and standard deviation. External Quality Assessment (EQA) Working Group A on Analytical Goals in Laboratory Medicine. Eur J Clin Chem Clin Biochem 1996;34:665-78.
(2.) Sciacovelli L, Zardo L, Secchiero S, Plebani M. Quality specifications in EQA schemes: from theory to practice. Clin Chim Acta 2004;346:87-97.
(3.) ILAC Technical Accreditation Issues Committee. ILAC-G13: 2000-guidelines for the requirements for the competence of providers for proficiency testing schemes. http://www.ilac.org (accessed January 2005).
(4.) Thienpont LM, Leenheer AP, Stockl D, Reinauer H. Candidate reference methods for determining target values for cholesterol, creatinine, uric acid, and glucose in external quality assessment and internal accuracy control. II. Method transfer. Clin Chem 1993;39:1001-6.
(5.) Burrin J, Alberti K. What is blood glucose: can it be measured? Diabetic Med 1990;7:199-206.
(6.) Petersen P. Is it possible to create a perfect external control system? Scand J Clin Lab Invest 1998;58:265-8.
(7.) Sandberg S, Christensen NG, Jevnaker M, Thue G, Klovning A. [Quality of hemoglobin and glucose analysis in general practice. Results from the NOKLUS (Norwegian Health Authority) surveys 1994]. Tidsskr Nor Laegeforen 1996;116:25-9.
(8.) Wood WG, Hanke R, Meissner D, Reinauer H. Experience with an external quality assessment programme for point-of-care-testing (POCT) devices for the determination of blood glucose. Clin Lab 2003;49:151-9.
(9.) International Organization for Standardization. In vitro diagnostic test systems-requirements for blood glucose monitoring systems for self-testing in managing diabetes mellitus. ISO/TC 212/SC. Final draft International Standard ISO/FDIS 15197. Geneva, Switzerland: ISO, 2003.
(10.) Burnett RW, D'Orazio P, Fogh-Andersen N, Kuwa K, Kulpmann WR, Larsson L, et al. IFCC recommendation on reporting results for blood glucose. Clin Chim Acta 2001;307:205-9.
(11.) Burnett RW. Accurate estimation of standard deviations for quantitative methods used in clinical chemistry. Clin Chem 1975;21: 1935-8.
(12.) Bradley C, Moses JL. Evaluation of blood glucose measurement techniques: locating sources of error. Diabetes Res 1986;3: 53-8.
(13.) Albertson C, Davis C, Ellison J, Chu C. Clinical evaluation of a new, miniaturized biosensor for self-monitoring of blood glucose. Clin Chem 1998;44:2056-7.
(14.) Harrison B, Markes R, Bradley P, Ismail IA. A comparison of statistical techniques to evaluate the performance of the Glucometer Elite blood glucose meter. Clin Biochem 1996;29:521-7.
(15.) Louie RF, Tang Z, Chan KJ, Kost GJ. Measurement variability between test strip lots on five handheld glucose meter systems [Abstract]. Clin Chem 2000;46(Suppl o):A15.
(16.) Skeie S, Thue G, Nerhus K, Sandberg S. Instruments for self-monitoring of blood glucose: comparisons of testing quality achieved by patients and a technician. Clin Chem 2002;48:994-1003.
(17.) Skeie S, Thue G, Sandberg S. Patient-derived quality specifications for instruments used in self-monitoring of blood glucose. Clin Chem 2001;47:67-73.
(18.) Voss EM, Cembrowski GS. Performance characteristics of the HemoCue B-Glucose analyzer using whole-blood samples. Arch Pathol Lab Med 1993;117:711-3.
(19.) Ashworth L, Gibb I, Alberti KG. HemoCue: evaluation of a portable photometeec system for determining glucose in whole blood. Clin Chem 1992;38:1479-82.
(20.) Rassam AG, McLeod J, Burge MR, Schade DS. Use of the HemoCue blood glucose analyzer in research studies. Diabetes Care 1998;21:1369-70.
(21.) Torjman MC, Jahn L, Joseph JI, Crothall K. Accuracy of the HemoCue portable glucose analyzer in a large nonhomogeneous population. Diabetes Technol Ther 2001;3:591-600.
(22.) Wiener K. An assessment of the effect of haematocrit on the HemoCue blood glucose analyser. Ann Clin Biochem 1993;30: 90-3.
(23.) Karcher RE, Ingram RL, Kiechle FL, Sykes E. Comparison of the HemoCue R-glucose photometer and reflotron for open heart surgery. Am J Clin Pathol 1993;100:130-4.
(24.) Kristensen GB, Nerhus K, Thue G, Sandberg S. Standardized evaluation of instruments for self-monitoring of blood glucose by patients and a technologist. Clin Chem 2004;50:1068-71.
(25.) Chen HS, Kuo BI, Hwu CM, Shih KC, Kwok CF, Ho LT. Technical and clinical evaluation of an electrochemistry glucose meter: experience in a diabetes center. Diabetes Res Clin Pract 1998; 42:9-15.
GUNN B.B. KRISTENSEN, * NINA GADE CHRISTENSEN, GEIR THUE, and SVERRE SANDBERG
NOKLUS, Norwegian Quality Improvement of Primary Care Laboratories, Section for General Practice, Department of Public Health and Primary Health Care, University of Bergen, Bergen, Norway.
* Address correspondence to this author at: NOKLUS, Norwegian Quality Improvement of Primary Care Laboratories, Section for General Practice, Department of Public Health and Primary Health Care, University of Bergen, UNIFOB AS, Boks 6165 Postterminalen, 5892 Bergen, Norway. Fax 47-55586710; e-mail email@example.com.
Received February 4, 2005; accepted June 14, 2005.
Previously published online at DOI: 10.1373/clinchem.2005.049080
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Evidence-Based Laboratory Medicine and Test Utilization|
|Author:||Kristensen, Gunn B.B.; Christensen, Nina Gade; Thue, Geir; Sandberg, Sverre|
|Article Type:||Clinical report|
|Date:||Sep 1, 2005|
|Previous Article:||Role of N-terminal pro-B-type natriuretic peptide in risk stratification in patients presenting in the emergency room.|
|Next Article:||Influences of blood sample processing on low-molecular-weight proteome identified by surface-enhanced laser desorption/ionization mass spectrometry.|