Effects of measurement frequency on analytical quality required for glucose measurements in intensive care units: assessments by simulation models.
A factor in the success of TGC maybe the performance of the devices used to measure glucose (9). The successful TGC outcomes (3-5) were achieved with a highly accurate glucose measurement system and blood samples collected from central lines. By contrast, most subsequent studies that failed to achieve reductions in mortality used less accurate measurement systems (9). Thus glucose measurement systems may play a role in the reported failures of TGC, but it is difficult to test the relationship of measurement error and glycemic control in clinical trials.
To explore the effects of measurement error on control of glycemia, we previously used in silico simulations of patients on TGC (10). In these in silico simulations (10), each modeled patient's true glucose was measured hourly by simulated devices with defined bias and imprecision. The measured glucose concentrations were used to select insulin infusion rates according to the Yale (11) and University of Washington (UW) (12) treatment regimens. Regardless of the treatment regimen used, increases in bias and imprecision of the measurements were found to be associated with higher rates of hypo- or hyperglycemia or both in the modeled patients (10). Moreover, increased imprecision was associated with higher variability in the patients' true glucose. We concluded that to avoid degradation of glycemic control, total error in the hourly measurements needed to be <10%-15%. Similarly, a current guideline on glucose meters for use in hospitals concludes that the quality specification for total error of glucose meters used in acute care facilities should be <12.5% at most concentrations of glucose and may need to be <10% for TGC in ICUs (13).
Recent efforts in TGC have focused on the possible use of continuous glucose monitoring (CGM) to replace glucose meters (14). More frequent measurements of glucose provided by CGM devices are expected to improve the control of glycemia and decrease occurrences of hypoglycemia (15). However, it is not currently known to what extent the relationship between measurement error and glycemic control maybe affected by increasing the frequency of glucose measurement. Here we report studies aimed at using our earlier modeling approach to address this question by comparing the control of glycemia when glucose measurements were made hourly (as in our earlier studies) vs 12 times per hour, as maybe seen with CGM.
The modeling scheme presented previously (10) was modified and extended to simulate CGM with glucose measurements made more frequently than once per hour. In brief, our approach, as before, used simulated glucose measurements by assays with specified amounts of inaccuracy and imprecision. Simulated assay bias was varied from -20% to 20% in increments of 5% to assess the effects of inaccuracy; simulated imprecision (expressed as CV) was varied from 0% to 20% in increments of 5%. We used measured glucose concentrations to select insulin infusion rates on the basis of 2 published insulin administration protocols (11, 12), referred to here as the Yale and UW regimens, respectively.
As in the previous study (10) modeled patients had various insulin sensitivities, starting glucose concentrations, and rates of gluconeogenesis and intravenous glucose administration, all set with use of random number generators. Starting glucose concentrations, for example, were randomly set with equal probabilities of occurrence in the range of 40-600 mg/dL (2.2-33.3 mmol/L). For each modeled patient, we measured glucose concentrations by a simulated assay having a preselected combination of bias and imprecision. We used the measured glucose concentrations to make hourly adjustments to insulin infusion rates as specified by either the Yale or UW regimen. One hundred patients were modeled at each combination of bias and imprecision for 100 h with each treatment regimen. We used a model of glucose homeostasis, patterned after the minimal model of glucose regulation (16), to determine glucose concentrations after each hour of insulin infusion at the chosen rate. Within-patient variability in rates of gluconeogenesis or glucose infusion was modeled by adding normally distributed random hourlymean (SD) increments of 50 (10) mg/dL to glucose concentrations predicted from the minimal model of glucose regulation.
Compared with our earlier study, the simulations using the UW regimen were enhanced to better reflect the specific steps suggested in that protocol for treatment of hypoglycemia. For this, the increment in actual plasma glucose 1 h after giving 12.5 g glucose intravenously was estimated to be 44 mg/dL, the concentration that would remain after the expected 141 mg/dL initial increment in a person with a typical 7.25-L volume of distribution (17, 18). To simplify calculations, we treated these hypoglycemic patients according to the UW protocol for conscious patients only.
MODELING OF CGM
Each modeled patient's "true" glucose was calculated hourly based on the change from the current glucose concentration predicted by the physiologic model for each hour's rate of insulin infusion specified by the TGC treatment regimen. For the simulations in which glucose was measured every 5 min, we calculated the hourly true glucose concentrations every 5 min from an assumed linear relationship between the initial glucose concentrations and the concentration predicted by the physiologic model (and the then-current insulin infusion rate) to occur at 60 min. Each of the 12 true glucose concentrations per hour was measured by a simulated glucose assay with a defined imprecision and bias. The results of the 12 measurements over each hour were fitted by linear least-squares regression, and the glucose concentration predicted from the regression line at 60 min (rather than the single measured glucose at 60 min) was used to determine the insulin infusion rate according to the chosen treatment regimen (UW or Yale) for that patient.
Thus, 100 patients were modeled for 100 h with each regimen with both the hourly and every-5-min measurement schemas under 45 combinations of simulated bias and imprecision. This modeling generated 11 700 000 simulated glucose measurements for the 2 regimens combined.
To investigate the effect of a CGM failing to provide a reading at a given time point, we conducted additional simulations, changing the model to randomly drop a mean of 10% of simulated CGM results. Because each result was assigned a 10% probability of being suppressed, and unavailable for use, in a given sequence of 12 between-hour results, it was possible to observe 0, 1, 2, or more missing results; by probability calculations, only 28% of sequences in this experiment would have all 12 results.
In a separate set of simulation runs, the variability of the glucose measurements at concentrations <100 mg/dL was expressed in terms of constant SDs (in mg/ dL) rather than as a CV. For each selected CV, the corresponding constant SD was calculated by multiplying 100 mg/dL times the modeled CV expressed as a fraction.
We conducted another set of simulation runs to assess the effect of immediate intervention on the basis of any finding of a single measured glucose in the hypoglycemic range between adjacent 1-hour time points. Such a hypoglycemic result caused an "early dropout" from the process of collecting 12 glucose measurements each hour, and the insulin treatment regimen was invoked immediately to perform treatment that was appropriate for the measured (low) glucose concentration.
In all experiments, the observed rates of hypoglycemia and hyperglycemia, the percentage of time that true glucose was in the target range, and the variability (SD) of hourly true glucose measurements were monitored as outcomes.
RATES OF HYPOGLYCEMIA
Fig. 1 shows the relationship between the frequency of true hypoglycemia and the imprecision (expressed as CV) of the glucose measurements. In this figure, the bias was held at zero. Compared with hourly measurements, measurements made every 5 min were associated with lower frequencies of (true) hypoglycemia in the modeled patients (Fig. 1). This reduced frequency of hypoglycemia with more frequent glucose measurements was seen at analytical imprecision (CVs) of 5% through 20% when using either the Yale or UW regimen for selecting insulin infusion rates (Fig. 1). With the Yale regimen, the rate of hypoglycemic episodes with measurements every 5 min at 10% CV was similar to that seen with hourly measurements at 5% CV.
When biases (constant errors) of -20% to 20% were introduced in the glucose measurement along with imprecision, the more frequent measurements again produced similar decreases of hypoglycemia at any level of bias (contour plots, Supplemental Fig. 1, which accompanies the online version of this article at http://www.clinchem.org/content/vol60/issue4). Increased frequency of measurement had no perceptible ability to decrease the effects of bias (see online Supplemental Fig. 1).
RATES OF HYPERGLYCEMIA AND TIME IN TARGET RANGE
Fig. 2 shows the effects of glucose measurement imprecision on the rates of hyperglycemia when the measurement system has zero bias. As with hypoglycemia, the frequency of hyperglycemia was lower, at any analytical imprecision, when measurements were made every 5 min rather than every hour. Similar effects of imprecision and frequency of measurement were seen when bias was added (see online Supplemental Fig. 2).
The time that modeled patients' true glucose was in the target range decreased with increasing imprecision (Fig. 3). Again, this adverse effect of imprecision was markedly blunted when glucose was measured every 5 min rather than every hour (Fig. 3).
The variability of the modeled patients' true glucose increased with increasing imprecision of the glucose measurements (Fig. 4). As before, this adverse effect of imprecision was blunted with more frequent measurements of glucose. The effect of increased frequency of measurements was seen with both the Yale and UW regimens. Even at a measurement imprecision (CV) as high as 10% (UW) or 15% (Yale), the glycemic variability in the modeled patients with measurements every 5 min was comparable to the glycemic variability seen with no imprecision when measurements were made hourly (Fig. 4). Introduction of bias with the imprecision had minimal or no detectable additional effect on variability of the true glucose (not shown).
The individual within-patient SDs for the 100 modeled patients at each glucose measurement imprecision (and zero bias) value are presented in online Supplemental Fig. 3. The detailed view illustrates that some modeled patients had extremely high glycemic variability (SDs). On investigation, these patients uniformly had starting glucose concentrations that were in the upper portion (>450 mg/dL) of the range of allowed values. When the studies were repeated with the range of starting glucose concentrations limited to 70-160 mg/dL, the within-patient variability of glucose was reduced at each frequency of measurement, but the findings of the study remained unchanged: Increasing the frequency of measurements decreased the glycemic variability and decreased the rates of hypoglycemia and hyperglycemia in the modeled patients.
EFFECTS OF OTHER VARIABLES
When the model was changed to randomly drop 10% of results (i.e., each result had a 10% chance of being suppressed and unavailable for use), no perceptible effect was observed on the measured outcomes (not shown). The regression fitting of the remaining results when approximately 10% of overall results were missing appeared sufficient to compensate for these missing values. Changing the imprecisions at low concentrations (<100 mg/dL) from a proportion of the measured value (CV) to a fixed concentration (fixed at the SD equivalent to the CV% at a glucose concentration of 100 mg/dL) produced no evident effect on glycemic control in the modeled patients (not shown). When the model was changed to allow immediate intervention on the basis of a single hypoglycemic result between the 1-hour time points, the frequencies of hypoglycemia were slightly decreased in some trials (not shown).
This simulation study demonstrated that increasing the frequency of measurement of glucose reduced the adverse impact of glucose measurement imprecision on the control of glycemia in modeled patients. This effect was seen at measurement imprecisions (CVs) of 5% to 20% and was seen with each of the 2 treatment regimens tested. More frequent measurements increased the time that modeled patients' true plasma glucose concentrations were in the target zone. Improvements were also seen in each of the 3 aspects of glycemic control that are most predictive of patient outcomes, namely glycemic variability and the rates of hypoglycemia and hyperglycemia. The effects of bias were not altered by increasing the frequency of glucose measurements.
The approach taken in these studies was to use our previously published modeling scheme (10) to examine the effect of changing the frequency of glucose measurement. We compared hourly measurements, which we had used previously to model TGC with glucose meters, with measurements made once every 5 min, as may be achieved with CGM. The previous simulation suggested that a total error of up to 10%-15% in hourly glucose measurements was compatible with control of glycemia when the measurements were used with the Yale or UW regimens (10). With both regimens, total errors of 10% or less produced little change of the control of glycemia from that seen with a "perfect" meter (10). Modeling studies by others have suggested similar or identical allowable total errors both for TGC (19) and for glucose meters used in self-monitoring of glucose (20). The latter study concluded that a threshold effect appeared to exist between 10% and 15% total error. A recent guideline from the Clinical Laboratory Standards Institute identified a goal of 12.5% total error for glucose meters used in acute and chronic healthcare facilities (13). The findings of the present study can be used to evaluate whether these conclusions and the Clinical Laboratory Standards Institute goal are appropriate when measurements are made more frequently than hourly. The results suggest that the above criteria for glucose meters in acute care hospitals may not apply to CGM devices that make glucose measurements more frequently than hourly.
The improvement in glycemic control that was seen with increased frequency of measurements was a result of limiting the adverse impact of measurement imprecision. Theoretically, as measurement frequency increases, the confidence in the combined information from the measurements increases in a fashion similar to that expected if the measurements were expressed as a moving average. The effect of random errors would be expected to decrease with the square root of the number of determinations used for the moving average. By use of regression analysis for prediction of the measured glucose at 60 min based on up to 12 glucose measurements over a 1-h interval, we have performed an analogous calculation, reducing the information from the multiple measurements into a single extrapolated measurement with improved precision. Such effects of increased frequency of measurement are not limited to CGM devices but would also be expected when more frequent measurements are made with glucose meters or other measuring systems.
Glycemic control in our studies was not noticeably improved by the use of the frequent measurements to detect hypoglycemia that occurred between the hourly measurements. The lack of a major positive effect may reflect offsetting, adverse effects that occur when the imprecision of single measurements generates false signals of hypoglycemia and thus infusion of glucose that is not warranted. An advanced algorithm, perhaps evaluating trends in glucose concentrations over short time periods, maybe better able than the regimens used here to exploit the potential advantage of earlier detection of hypoglycemia while largely avoiding the penalty from imprecision of single measurements.
This study has several limitations. The model of glucose homeostasis is a simplified model and may not reflect true physiologic behaviors accurately in all circumstances, nor does it allow for lag in the sensing of changes of glucose concentrations. The aim of this study, however, was to explore how our previously published results, using this model, would be affected when the frequency of measurements was increased. Moreover, for glucose meters, our conclusions using this model (10) were similar to those of Breton and Kovatchev (20), who used a highly sophisticated model of glucose control, albeit in a different setting (self-monitoring of blood glucose). A limitation of the model is that it assumes a linear relationship of true plasma glucose concentrations and time between hourly time points. Thus a linear least-squares fit was adequate for the regression analysis of measured glucose concentrations vs time. In reality, the time courses will, in some time segments, be nonlinear, which will require fitting with nonlinear equations. We could not evaluate that approach with our model. A related limitation of our study is that the modeling scheme may not have adequately accounted for factors such as oral feeding, steroid injections, and infections that occur in clinical practice and alter insulin sensitivity and other variables related to glycemia, nor did it address effects of changes in analytical bias (or imprecision) that may occur during the 100 h of monitoring of each modeled patient. The simulation did, however, introduce considerable variability in the hour-to-hour increments in glucose attributed to gluconeogenesis/glucose infusion as described in Methods. Moreover, this same limitation applied to the 2 frequency-of-glucose-measurement schemas that were being compared. Finally, these studies addressed only the imprecision and bias components of analytical error; the studies did not address extrinsic random and nonrandom variation generated by the clinical context. The datareported therefore represent a best-case scenario for glycemic control at the indicated levels of bias and imprecision. This is a clear limitation, but the model nonetheless allows exploration of the effects of changes in the selected inputs for the model, as was the intent of the present experiments.
This study has several strengths. The simulation used a model of analytical error that distinguished effects of imprecision and bias rather than relying solely on the concepts of total error or mean absolute relative difference. Such a modeling scheme appears to be important for at least 2 reasons: a) Imprecision and bias produced different effects on the modeled patients' glycemic control, as can be seen in the contour plots in online Supplemental Figs. 1 and 2; and b) each of these components of analytical error may be more or less problematic for a given CGM device. A second strength is that the same modeling conditions were applied to both frequencies of measurement, allowing direct comparison of the two approaches. Other variables were unchanged, including the frequency of changing insulin infusion rates. The use of the Yale and UW regimens to determine the magnitude, direction, and frequency of changes of insulin and glucose infusions seems reasonable because such regimens are representative of those likely to be used in the initial implementations of CGM devices. Studies like ours may provide insights that inform decisions about analytical quality requirements and design goals for CGM devices. Finally, Finfer et al. (21) recently called for randomized controlled trials to compare the abilities of CGM and intermittent monitoring to achieve and maintain glycemic control. The relationships found in modeling studies like ours may provide insights that are useful in the design and interpretation of such clinical trials.
In the future, closed-loop algorithms likely will be used to control insulin infusion rates on the basis of measured glucose concentrations; the effect of glucose measurement error on each such closed-loop algorithm will need to be assessed individually. Instinctively, one may expect that the power of advanced algorithms for these devices, combined with high measurement frequencies, will mitigate the effect of measurement error. If these algorithms make changes in insulin infusion rates on the basis of small changes in glucose concentrations, however, the effect of small measurement errors could be magnified.
In summary, the studies described here show that the predicted quality requirements for glucose measurements depend on the frequency of measurement. We conclude that analytical quality requirements proposed for glucose meters with hourly measurements for use in ICUs are inappropriate for CGMs with more frequent measurements.
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, oranalysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.
Authors' Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest:
Employment or Leadership: J.C. Boyd, Clinical Chemistry, AACC. Consultant or Advisory Role: None declared.
Stock Ownership: None declared.
Honoraria: None declared.
Research Funding: This study was supported by funding to the University of Virginia from Edwards Life Sciences.
Expert Testimony: None declared.
Patents: None declared.
Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, or preparation or approval of manuscript.
(1.) American Diabetes Association. Standards of medical care in diabetes-2012. Diabetes Care 2012;35 Suppl 1:S11-63.
(2.) Jacobi J, Bircher N, Krinsley J, Agus M, Braithwaite SS, Deutschman C, et al. Guidelines for the use of an insulin infusion for the management of hyperglycemia in critically ill patients. Crit Care Med 2012;40:3251-76.
(3.) Van den Berghe G, Wouters P, Weekers F, Verwaest C, Bruyninckx F, Schetz M, et al. Intensive insulin therapy in critically ill patients. N Engl J Med 2001;345:1359-67.
(4.) Van den Berghe G, Wilmer A, Hermans G, Meersseman W, Wouters PJ, Milants I et al. Intensive insulin therapy in the medical ICU. N Engl J Med 2006;354:449-61.
(5.) Vlasselaers D, Milants I, Desmet L, Wouters PJ, Vanhorebeek I, van den Heuvel I et al. (2009) Intensive insulin therapy for patients in paediatric intensive care: a prospective, randomised controlled study. Lancet 2009;373:547-56.
(6.) NICE-SUGAR Study Investigators, Finfer S, Chittock DR, Su SY, Blair D, Foster D, Dhingra V et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med 2009;360: 1283-97.
(7.) Wiener RS, Wiener DC, Larson RJ. Benefits and risks of tight glucose control in critically ill adults: a meta-analysis. JAMA 2008;300:933-44.
(8.) Griesdale DE, de Souza RJ, van Dam RM, Heyland DK, Cook DJ, Malhotra A et al. Intensive insulin therapy and mortality among critically ill patients: a meta-analysis including NICE-SUGAR study data. CMAJ 2009;180:821-7.
(9.) Scott MG, Bruns DE, Boyd JC, Sacks DB. Tight glucose control in the intensive care unit: are glucose meters up to the task? Clin Chem 2009; 55:18-20.
(10.) Boyd JC, Bruns DE. Monte Carlo simulation in establishing analytical quality requirements for clinical laboratory tests: meeting clinical needs. Methods Enzymol 2009;467:411-33.
(11.) Goldberg PA, Siegel MD, Sherwin RS, Halickman JI, Lee M, Bailey VA et al. Implementation of a safe and effective insulin infusion protocol in a medical intensive care unit. Diabetes Care 2004; 27:461-7.
(12.) Trence DL, Kelly JL, Hirsch IB. The rationale and management of hyperglycemia for in-patients with cardiovascular disease: time for change. J Clin Endocrinol Metab 2003;88:2430-7.
(13.) Sacks DB, Bruns DE, Horton J, Lindberg S, Mahoney JJ, Manzella S et al. Point-of-care blood glucose testing in acute and chronic care facilities: approved guideline. Clinical Laboratory Standards Institute 2013;33:1-48.
(14.) Damiano ER, El-Khatib FH, Zheng H, Nathan DM, Russell SJ. A comparative effectiveness analysis of three continuous glucose monitors. Diabetes Care 2013;36:251-9.
(15.) Pickup JC, Freeman SC, Sutton AJ. Glycaemic control in type 1 diabetes during real time continuous glucose monitoring compared with self monitoring of blood glucose: meta-analysis of randomised controlled trials using individual patient data. BMJ 2011;343:d3805.
(16.) Toffolo G, Bergman RN, Finegood DT, Bowden CR, Cobelli C. Quantitative estimation of beta cell sensitivity to glucose in the intact organism: a minimal model of insulin kinetics in the dog. Diabetes 1980;29:979-90.
(17.) Cunningham VJ, Heath DF. An interpretation of the intravenous glucose tolerance test in the light of recent findings on the kinetics of glucose and insulin in man. Clin Sci Mol Med 1978;54:161-73.
(18.) Duncan LJ. The intravenous glucose tolerance test. Exp Physiol 1956;41:85-96.
(19.) Karon BS, Boyd JC, Klee GG. Glucose meter performance criteria for tight glycemic control estimated by simulation modeling. Clin Chem 2010; 56:1091-7.
(20.) Breton MD, Kovatchev BP. Impact of blood glucose self-monitoring errors on glucose variability, risk for hypoglycemia, and average glucose control in type 1 diabetes: an in silico study. J Diabetes Sci Technol 2010;4:562-70.
(21.) Finfer S, Wernerman J, Preiser JC, Cass T, Desaive T, Hovorka R et al. Clinical review: consensus recommendations on measurement of blood glucose and reporting glycemic control in critically ill adults. Crit Care 2013;17:229.
James C. Boyd  and David E. Bruns  *
 Department of Pathology, University of Virginia, Charlottesville, VA.
* Address correspondence to this author at: Department of Pathology, Box 800168, University of Virginia School of Medicine and Health System, Charlottesville, VA 22908. E-mail email@example.com.
Received September 17, 2013; accepted December 18, 2013.
Previously published online at DOI: 10.1373/clinchem.2013.216366
 Nonstandard abbreviations: ICU, intensive care unit; TGC, tight glucose control; UW, University of Washington; CGM, continuous glucose monitoring.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Endocrinology and Metabolism|
|Author:||Boyd, James C.; Bruns, David E.|
|Date:||Apr 1, 2014|
|Previous Article:||Phase I and II cannabinoid disposition in blood and plasma of occasional and frequent smokers following controlled smoked cannabis.|
|Next Article:||Publication and reporting of test accuracy studies registered in ClinicalTrials.gov.|