Printer Friendly

Results and feasibility of an external quality assessment scheme for self-monitoring of blood glucose.

Studies show conflicting results for the effect of self-monitoring of blood glucose (SMBG) [3] in improving glycemic control in patients with type 1 or type 2 diabetes mellitus. The evidence for an effect is somewhat stronger for type 1 diabetes mellitus, whereas SMBG has not been shown to be useful for patients with type 2 diabetes (1, 2). Nevertheless, SMBG is recommended for most patients with diabetes (3-5), but the benefit of self-monitoring depends on the analytical quality achieved as well as on the actions taken based on the results. A recent study demonstrated that the analytical quality of blood glucose measurements attained by patients did not fulfill criteria stated by the American Diabetes Association (ADA) or the International Organization for Standardization (6-9). Successful SMBG requires the establishment of standardized control routines (3,10).

Patients performing SMBG may theoretically control their instruments in several ways: by comparing results from self-monitoring with results obtained in a hospital laboratory, a doctor's office, or a pharmacy; by using control materials recommended by the manufacturer; or by participating in an external quality assessment (EQA) program. There is little information about control routines for patients performing SMBG. However, Skeie et al. (9) reported that only approximately one third of patients performed instrument controls of some kind. These controls were performed mainly by duplicate measurements, comparing results from self-monitoring with results obtained at the doctor's office, or use of several SMBG instruments. Only 6% of the patients performed quality control with commercial control materials.

External quality assessment schemes (EQAS) are important for improving analytical quality (11) and are integrated into the workflow of the Norwegian Quality Improvement of Primary Care Laboratories (NOKLUS). The objective of this study was to investigate whether an EQA program designed for office laboratories could also improve the quality of SMBG measurements among diabetes patients.

Materials and Methods

We performed 6 glucose surveys (surveys 1-6) of diabetes patients during the period from October 2001 through March 2004. The surveys were coordinated with the NOKLUS biannual EQAS for glucose, in which 2 concentrations of control materials are sent to ~1900 laboratories associated with Norwegian general-practice offices. We sent control samples identical to those used in NOKLUS EQAS to the diabetes patients participating in the surveys.

PARTICIPANTS

Diabetes patients performing SMBG from 4 counties in Norway were recruited by an advertisement in the Journal of the Norwegian Diabetes Association, by distribution of an enrollment folder at pharmacies, and by an internet link on the Norwegian Diabetes Association home page. The study was designed in accordance with guidelines from the Regional Committee for Medical Research Ethics, and the participants gave informed consent to take part in the study.

Diabetes patients who participated in the first and second surveys (n = 171) were included in the study. Ten patients quit the program during the study period. Approximately 10% of the patients in each survey group used instruments classified to an instrument group with <12 participants (see the section on glucose instruments below), and we did not send control samples to these patients. Thus, 120 to 152 patients in each survey group received control samples, with 34 patients receiving control samples in all 6 surveys. The response rate was 71%-86%.

The characteristics of the participants, including background variables and use of SMBG, are shown in Table 1. This information was obtained with a questionnaire attached to the last survey (survey 6 in March 2004). Of the 161 patients included in this survey, 126 (78%) responded to the questionnaire. Almost one half of the diabetes patients who were included in the program were self-educated, more than one half performed SMBG more than 10 times per week, and 50% had more than one instrument (we did not ask patients who had more than one instrument to specify whether these were identical or different types).

STUDY DESIGN

Diabetes patients participating in the EQAS received 2 control samples by post twice a year. Participants measured each control sample in duplicate in accordance with written instructions and returned the results to NOKLUS together with information about the instrument and the lot number and expiration date of the strips used. In response, participants received an assessment of the analytical performance within 1 month. Participants who got a "poor" evaluation were given written instructions on how to use their specific instrument. They were also contacted by phone (by G.B.B.K. or K.N.) and offered guidance and advice concerning the poor result. We focused on a thorough review on measurement performance, including sample volume, calibration code, and storage and expiration date of strips, as well as mixing and applying the control samples properly before use.

CONTROL MATERIALS

NOKLUS uses 2 different control materials in 2 concentrations (~6 and ~20 mmol/L) in EQAS for glucose meters, depending on the type of instrument (12,13): Sugar Chex (Streck Laboratories) and EDTA blood, stabilized with NaI (Sero AS).

The control materials used in our EQAS differ in composition from native capillary blood, and method-specific target values were therefore used. The method-specific target value was calculated as the median of all results from participants who used a particular type of instrument, after outliers had been excluded (14). Stability tests were performed for 5 days in each survey. Mean values did not differ significantly for 3 days, and all samples were analyzed by patients within this period.

GLUCOSE INSTRUMENTS

Because we used method-specific target values, only diabetes patients with instruments that could be classified to an instrument group with more than 12 participants received control samples and participated in the EQAS. Thus the following instrument brands were included; OneTouch Basic/II/Profile and GlucoTouch (LifeScan); Accu-Chek Sensor (Roche Diagnostics); Glucometer Dex/ Dex2 and Ascensia Elite (Bayer Diagnostics); Precision QID, MediSense Pen, MediSense Card, and Precision Xtra (Abbott Laboratories /MediSense).

The distribution of instruments among the 120 diabetes patients who received control samples in the first survey in October 2001 and the corresponding distribution of these instruments in laboratories affiliated with Norwegian general practice offices at the same time are shown in Table 2.

PARTICIPANT ASSESSMENT

In our EQAS for the patients, we used the same quality specifications that NOKLUS has used in EQAS for glucose in office laboratories, taking biological variation and clinical needs into account. Participant performance was assessed as "good" if the result (mean of duplicate measurements) deviated from the target interval (the method-specific target value [+ or -] 0.1 mmol/L) by <5%; "acceptable" if their result deviated from the target interval by 5% to 10%, and "poor" if the result deviated more than 10% from the target interval. Thus, if the target value is 7.00 mmol/L, the participant will get the assessment "good" if the result is 6.55-7.45 mmol/L, "acceptable" if the result is 6.20-6.54 mmol/L or 7.46-7.80 mmol/L, and "poor" if the result is <6.20 mmol/L or >7.80 mmol/L.

Participant precision was not assessed, but if the difference between 2 measurements of the same control sample was larger than the deviation between the method-specific target value and the acceptable lower limit for the assessment, their results in the EQAS were described as "uncertain".

To compare analytical quality throughout the surveys, we divided participants into 3 groups, and office laboratories constituted an additional group:

Study group 1. Results from all patients, i.e., patients who were assessed in at least 1 of the 6 surveys (n = 158-218 results in each survey).

Study group 2. Results from the subgroup of patients from study group 1 who were assessed in all 6 surveys (n = 68 results in each survey).

Study group 3. Results from the subgroup of patients from study group 1 who were assessed in at least 1 of the 6 surveys and who used instruments that were also used by office laboratories (n = 135-181 results in each survey).

Study group 4. Results from office laboratories using the same instruments as diabetes patients who were assessed (n = 955-1290 results in each survey).

The percentage distribution of instruments differed between diabetes patients and office laboratories, as exemplified by the results of the first survey (Table 2). Therefore, to enable an instrument-independent comparison of performance between diabetes patients and office laboratories, we weighted results from the office laboratories according to the percentage distribution of instruments among diabetes patients in each survey. Only instruments used by both patients and office laboratories were included in this procedure; thus, results from patients using the Glucometer Dex were excluded in study group 3 (Table 2).

STATISTICS

We calculated imprecision, [SD.sub.within], and [CV.sub.within] by use of duplicate measurements performed with the same control sample by all participants with a specific type of instrument, based on the formula:

[SD.sub.within] = [square root of [summation][d.sup.2]/2n]

where d is the difference between the 2 measurements, and n is the number of duplicate measurements. [CV.sub.total] was calculated with the mean of duplicate measurements performed with the same control sample by all participants with a specific type of instrument. The CV between all participants, [CV.sub.between], was calculated by the formula:

[CV.sub.between] = [square root of [CV.sup.2.sub.total] - [CV.sup.2.sub.within]/2]

the criterion promoted by Burnett (14) was used to detect outliers.

To assess changes in analytical quality, we used the binominal test for proportions, the F-test, and linear regression and compared variances. For each survey, we calculated an estimate of between-participant CV by pooling the [CV.sub.between] values for both control samples for all instruments. The between-participant CV and the percentage of poor performances for each survey were used to demonstrate analytical quality. The percentage outliers ranged from 0% to 5%. The level of statistical significance was set to 5%.

Results

CHANGES IN ANALYTICAL QUALITY AMONG DIABETES PATIENTS

The percentage of participants with poor performance for study groups 1 and 2 decreased during the study (Fig. 1A). The between-participant CVs (see Materials and Methods) also decreased throughout the study period, although the decrease was significant only for study group 2 (Fig. 1B). However, both study groups demonstrated a significantly lower percentage of poor performers and a significantly lower between-participant CV in the last survey compared with the first survey (1.9% compared with 11% and 3.7% compared with 5.5% for study group 1, respectively; and 0% compared with 16% and 3.0% compared with 6.0% for study group 2, respectively).

The participants demonstrated good precision in all 6 surveys, with pooled [CV.sub.within] values of 2.4% to 3.2%.

COMPARISON OF ANALYTICAL QUALITY BETWEEN DIABETES PATIENTS AND OFFICE LABORATORIES

In the first and second surveys, which evaluated the results obtained with the same kind of instruments, the percentage of poor results was significantly higher for diabetes patients than for office laboratories [14% compared with 8.8% (P = 0.04) and 22% compared with 13% (P = 0.003) for the first and second survey, respectively; Fig. 2A]. However, for diabetes patients, the percentage of poor assessments decreased throughout the 6 surveys, as demonstrated by linear regression. This decrease was not seen for office laboratories. The same decreasing trend was observed for pooled between-participant CVs (Fig. 2B). However, from having a significantly higher between-participant CV in the 2 first surveys, i.e., 5.9% compared with 5.4% and 7.7% compared with 6.4% in surveys 1 and 2, respectively, the diabetes patients demonstrated significantly lower between-participant CVs than did office laboratories in surveys 3, 5, and 6 (6.1% compared with 7.2%,4.8% compared with 6.6%, and 3.9% compared with 4.6%; Fig. 2B).

[FIGURE 1 OMITTED]

REASONS FOR POOR PERFORMANCE

For ~20% of diabetes patients with poor assessments, explanations such as the use of expired test strips or a deviant lot of test strips, not using the right calibration code on the instrument, or not mixing the control samples properly were detected by follow-up of participants. Variability among lots might also contribute to poor analytical quality (12); however, because the number of participants using each lot number was rather small (1-2 participants per lot number), it was impossible to estimate the extent of between-lot variability among the patients. For office laboratories, on the other hand, some lot numbers were detected that gave glucose values that differed significantly from the method-specific target interval. Some of these deviant lot numbers were also used by patients and might be a reason for poor performance.

Among patients who got a poor assessment, those who changed instruments obtained as high a percentage of poor results in the following survey as those who did not change instruments. Similar findings were seen with participants who received good/acceptable assessments in the first survey and then changed instruments. We did not give any recommendations concerning the selection of a new instrument. Approximately 10% of the participants changed instruments between the different surveys.

[FIGURE 2 OMITTED]

EVALUATION OF THE EQAS

To evaluate the EQAS, we attached a questionnaire to the last survey (March 2004). According to the survey results, 81% found it useful to participate in the program, only 5% did not, and 84% indicated that participation in the EQA surveys increased their confidence in SMBG results to some or a high degree. Of the participants who needed follow-up, 90% were very satisfied or satisfied with the follow-up.

Among the survey responders, 90% expressed interest in participating in a similar EQAS on a regular basis, and 65%, 17%, and 16% of these wanted to receive control samples 2, 4, or 1 time a year, respectively. One half were willing to pay for each survey, but not more than US $15.00 (80%) or US $30.00 (15%). The cost of participating in one survey was estimated to be US $60.00, of which approximately US $30.00 is allocated to distributing the control material.

Discussion

To our knowledge, this study is the first to evaluate the effect of an EQAS among patients using SMBG. In essence, we found that participating in an EQA program for a period of 3 years improved the quality of the glucose results and that the patients themselves thought it useful to participate in the program and were willing to pay for it. The ADA recommends quality control of all glucose meters used by patients, both at the start of use and at regular intervals thereafter (3). However, no standard quality-control procedure is available, and most patients have poor control routines (9).

Survey 2 revealed a deterioration of the analytical quality. This finding can be explained by the fact that before this survey Roche Diagnostics introduced an upgraded test strip for the Accu-Chek Sensor. Measurements performed with this test strip gave significantly different values than did measurements performed with the previous test strip. If separate target values were used to assess results for the 2 different strips (12), the percentage of poor assessments and the between-participant CVs would be reduced for both diabetes patients and office laboratories, e.g., the percentage of poor results in Fig. 2A would be reduced from 21.8% to 11.2% for diabetes patients and from 13.1% to 8.7% for office laboratories. Linear regression analysis of these results indicated an even more significant decrease (P = 0.002) in poor results than that shown in Fig. 2A for diabetes patients throughout the study. For office laboratories, the results from linear regression did not change.

The within-participant imprecision values obtained in this study were within the quality specifications of <5% set by the ADA and by the patients themselves (7,15). However, the within-participant CVs should be interpreted with caution because some of the participants might have analyzed the sample more than twice and reported the 2 best results (16). Regarding total error, because we used method-specific target values, it is difficult to compare the results from this study with the quality goals set by the ADA or ISO 15197 (6-8). The diabetes patients, however, demonstrated improved analytical quality during the study, and at the end of the study the quality was similar to or better than the quality of the office laboratories (Fig. 2). The results did not indicate that changing to newer instruments improved the analytical quality. Therefore, the manufacturers' redesign of SMBG instruments to improve their accuracy and precision throughout the study period did not seem to explain the improvement.

Because the participants in this study were self-selected and likely to be particularly capable and motivated, there might have been an overestimation of the quality of the SMGB measurements, especially at the start of the study. In addition, the "Hawthorne effect" (17)--participation in a program highlighting the importance of good analytical quality may be sufficient to cause people to change their behavior--might have played a role. Although it is difficult to sort out these different effects, we still think that an EQAS with a systematic feedback program will improve the quality of the measurements, and the Hawthorne effect would probably decrease over time. Most office laboratories have participated in the EQAS since 1993 with a similar decrease in the percentage of poor results from 17% to ~6% throughout the first 6 years (18). The percentage of poor results for office instruments used in this study (with the notable exception of HemoCue users) seems to have plateaued at ~10%, which may indicate that the analytical quality achieved is representative of current meter technology in an outpatient setting. Our results are in line with the findings of Rumley (19), who reported that after implementation of an EQAS for glucose in 1993 and improved meter technology, the frequency of unacceptable results fell to 5%-7% in 1997.

A limitation to the EQA program for diabetes patients, as in almost any EQA program, is that artificial control materials may not be commutable to native blood samples. Therefore, aberrant results for the artificial control materials can be difficult to interpret because the results might not reflect the situation when native blood is used and vice versa (12). Another limitation in EQA surveys is that there is no control of blood sampling and no immediate feedback. An alternative approach uses a split-sample design in which glucose measurements performed by patients are compared with a result from a standardized comparative method. This design implies direct observation of user performance; thus, user errors could be more easily detected. Some studies recommend this method for routine use (20-23). NOKLUS is currently evaluating a model in which patients use the split-sample design to control their SMBG performance at pharmacies or in doctors' offices. This decentralized model would require more resources than does a traditional EQA program.

In Norway, health authorities have stipulated that all SMBG instruments with test strips should be examined by a standardized procedure that implies testing simultaneously by an experienced technologist and a group of patients before test strips are reimbursed by the government (16). An additional effect of running an EQAS for glucose for diabetes patients is that it will give an opportunity to continuously monitor the quality of SMBG technology on the market. The cost of approximately US $120.00 per year for 2 control samples should be compared with the costs of SMBG, e.g., using on average 4 strips a day, which amounts to approximately US $1600.00 per year. However, the clinical usefulness of implementing such a program should be evaluated further, and costs as well as limitations of current EQAS for glucose in general should be taken into account.

We thank Professor Per Hyltoft Petersen for fruitful discussions of the manuscript.

References

(1.) Aarsand AK, Alter D, Frost SJ, Kaplanis B, Klovning A, Price CP, et al. Diagnosis and management of diabetes mellitus. In: Nichols JH, ed. The National Academy of Clinical Biochemistry. Laboratory Medicine Practice Guidelines: Evidence Based Practice for Point of Care Testing. pp. 167-237. http://www.nacb.org/impg/poct_lmpg-draft.stm (accessed March 2006).

(2.) Coster S, Gulliford MC, Seed PT, Powrie JK, Swaminathan R. Monitoring blood glucose control in diabetes mellitus: a systematic review. Health Technol Assess 2000;4:1-93.

(3.) American Diabetes Association. Standards of medical care in diabetes [erratum published in Diabetes Care 2005;28:990]. Diabetes Care 2005;28:S4-36.

(4.) Goldstein DE, Little RR, Lorenz RA, Malone JI, Nathan DM, Peterson CM. Tests of glycemia in diabetes. Diabetes Care 2003;26(Suppl 1):S106-8.

(5.) Sacks DB, Bruns DE, Goldstein DE, Maclaren NK, McDonald JM, Parrott M. Guidelines and recommendations for laboratory analysis in the diagnosis and management of diabetes mellitus. Clin Chem 2002;48:436-72.

(6.) American Diabetes Association. Consensus statement on self-monitoring of blood glucose. Diabetes Care 1987;10:95-9.

(7.) American Diabetes Association. Consensus statement on self-monitoring of blood glucose. Diabetes Care 1996;19:62-6.

(8.) International Organization for Standardization. In vitro diagnostic test systems. Requirements for blood glucose monitoring systems for self-testing in managing diabetes mellitus. ISO/TC 212/SC. International Standard ISO 15197. Geneva, Switzerland: ISO, 2003.

(9.) Skeie S, Thue G, Nerhus K, Sandberg S. Instruments for self-monitoring of blood glucose: comparisons of testing quality achieved by patients and a technician. Clin Chem 2002;48:9941003.

(10.) Barlow I, Beer S, Summerton N. Meta-analysis of diabetes care in general practice: all glucose meters must be subject to formal quality control measures. BMJ 1999;318:460; author reply, 461.

(11.) Libeer JC, Baadenhuijsen H, Fraser CG, Petersen PH, Ricos C, Stockl D, et al. Characterization and classification of external quality assessment schemes (EQA) according to objectives such as evaluation of method- and participant bias and standard deviation: external quality assessment (EQA) working group A on analytical goals in laboratory medicine. Eur J Clin Chem Clin Biochem 1996;34:665-78.

(12.) Kristensen GB, Christensen NG, Thue G, Sandberg S. Between-lot variation in external quality assessment of glucose: clinical importance and effect on participant performance evaluation. Clin Chem 2005;51:1632-6.

(13.) Wood WG, Hanke R, Meissner D, Reinauer H. Experience with an external quality assessment programme for point-of-care-testing (POCT) devices for the determination of blood glucose. Clin Lab 2003;49:151-9.

(14.) Burnett RW. Accurate estimation of standard deviations for quantitative methods used in clinical chemistry. Clin Chem 1975;21: 1935-8.

(15.) Skeie S, Thue G, Sandberg S. Patient-derived quality specifications for instruments used in self-monitoring of blood glucose. Clin Chem 2001;47:67-73.

(16.) Kristensen GB, Nerhus K, Thue G, Sandberg S. Standardized evaluation of instruments for self-monitoring of blood glucose by patients and a technologist. Clin Chem 2004;50:1068-71.

(17.) Polit DF, Hungler BP. Nursing Research Principles and Methods, 6th ed. Baltimore: Lippincott Williams & Wilkins, 1999:184-5.

(18.) Malde K, Nilsen ST, Svabo HM. Kvalitetssikringsfondene Arsmelding 1999. Oslo, Norway: The Norwegian Medical Association: Quality Assurance Foundation, 2000.

(19.) Rumley AG. Improving the quality of near-patient blood glucose measurement. Ann Clin Biochem 1997;34:281-6.

(20.) Solnica B, Naskalski JW. Quality control of SMBG in clinical practice. Scand J Clin Lab Invest Suppl 2005;240:80-5.

(21.) Schrot RJ, Foulis PR, Morrison AD, Farese RV. A computerized model for home glucose monitoring proficiency testing: efficacy of an innovative testing program. Diabetes Educ 1999;25:48-55.

(22.) Alto WA, Meyer D, Schneid J, Bryson P, Kindig J. Assuring the accuracy of home glucose monitoring. J Am Board Fam Pract 2002;15:1-6.

(23.) Bergenstal R, Pearson J, Cembrowski GS, Bina D, Davidson J, List S. Identifying variables associated with inaccurate self-monitoring of blood glucose: proposed guidelines to improve accuracy. Diabetes Educ 2000;26:981-9.

GUNN B.B. KRISTENSEN, [1] * KARI NERHUS, [2] GEIR THUE [2] and SVERRE SANDBERG [1,2]

[1] NOKLUS, Norwegian Quality Improvement of Primary Care Laboratories, Laboratory of Clinical Biochemistry, Haukeland University Hospital, Bergen, Norway.

[2] NOKLUS, Section for General Practice, Department of Public Health and Primary Health Care, University of Bergen, Bergen, Norway.

[3] Nonstandard abbreviations: SMBG, self-monitoring of blood glucose; ADA, American Diabetes Association; EQA(S), external quality assessment (scheme); and NOKLUS, Norwegian Quality Improvement of Primary Care Laboratories.

* Address correspondence to this author at: NOKLUS, Norwegian Quality Improvement of Primary Care Laboratories, University of Bergen, Boks 6165, 5892 Bergen, Norway. Fax 47-5558-6710; e-mail gunn.kristensen@isf.uib.no.

Received February 2, 2006; accepted April 14, 2006.

Previously published online at DOI: 10.1373/clinchem.2006.068114
Table 1. Characteristics of participants in the last survey
(n = 126).

Median (range) age, years 55 (6-84)
Median (range) duration of diabetes, years 9 (2-61)
Sex, % male 50
Type of diabetes, %
 Type 1 48
 Type 2 50
 Unknown 2
Treatment, %
 Insulin 66
 Tablets only 28
 Diet only 6
Median (range) duration of SMBG, years 9 (2-25)
Responses to questions on survey, %
 How did you learn to use your instrument?
 Self-educated 47
 Nurse 28
 Salesperson 20
 Physician 4
 Other 1
How frequently do you usually perform SMBG?
 1-3 times per month 8
 1-3 times per week 13
 4-6 times per week 8
 7-10 times per week 15
 >10 times per week 56
Do you have more than one instrument?
 Yes 50
If yes, how many instruments?
 2 50
 3 34
 4 or more 16
If yes, how many of these instruments do
 you use at least once per month?
 2 or more 39

Table 2. Distribution of instruments among diabetes
patients and office laboratories.

 Diabetes Office
 patients, % laboratories, %
Brand (n = 120) (n = 658)

OneTouch Basic/II/Profile 3 11
GlucoTouch 5 11
Accu-Chek Sensor 22 18
Glucometer Dex/Dex2 18 0
Ascensia Elite 17 51
MediSense Pen/Card/Precision 35 9
 QID/Precision Xtra
COPYRIGHT 2006 American Association for Clinical Chemistry, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2006 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Evidence-Based Laboratory Medicine and Test Utilization
Author:Kristensen, Gunn B.B.; Nerhus, Kari; Thue, Geir; Sandberg, Sverre
Publication:Clinical Chemistry
Date:Jul 1, 2006
Words:4242
Previous Article:Matrix-assisted laser desorption/ionization mass spectrometric analysis of DNA on microarrays.
Next Article:Predictors of vitamin [B.sub.6] and folate concentrations in older persons: the InCHIANTI study.
Topics:


Related Articles
Assessment of a reference procedure to collect and analyze glucose in capillary whole blood.
Approved IFCC recommendation on reporting results for blood glucose (abbreviated).
Analytical quality of near-patient blood cholesterol and glucose determinations.
Hardware and software in diabetes mellitus: performance characteristics of hand-held glucose testing devices and the application of glycemic testing...
Instruments for self-monitoring of blood glucose: comparisons of testing quality achieved by patients and a technician.
Postanalytical external quality assessment of blood glucose and hemoglobin [A.sub.1c] an international survey.
Quality assessment of blood glucose testing in general practitioners' offices improves quality.

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters