The effect of a graphical interpretation of a statistic trend indicator (Trigg's Tracking Variable) on the detection of simulated changes.
We were involved in a recently reported study of micro-simulations, which were used to explore various facets of data presentations (9). One finding of this study was that a graphical display led to faster detection of changes than a numeric display, primarily because more information per unit of time could be presented. We have also been interested for some time in the use of simple statistical trend detection tools, in particular Trigg's Tracking Variable (TTV) (10), which was described in detail by Lewis (11) and first implemented in an anaesthetic context by Hope in 1973 (12).
The calculation of TTV is outlined in the Appendix. In essence, TTV is a measure of the deviation of the actual value at each iteration with a prediction based on a time-weighted moving average of previous values, thereby incorporating the effect of recent variability. Values for TTV range between -1 and +1 with the magnitude representing the likelihood that a change is occurring while the sign indicates the direction of the change. TTV makes no assumption about the suitability of any given value; it only indicates whether a change is likely to be taking place.
We have implemented TTV within a locally developed trend display system that incorporates a number of other features such as forward prediction of inhaled anaesthetic effect site and end-tidal concentrations (13). Rather than display the actual value of TTV, we have chosen to display the value of TTV as a number of "+" or "-" symbols next to the current (numeric) value depending on the magnitude or direction of changes. These steps, which are shown in Table 1, were originally determined empirically. We have related these values of TTV to the likelihood of detecting a true change in this model (10). We have now undertaken a study to evaluate the utility of TTV in facilitating trend detection in the presence of a graphical trend display.
This study was approved by the Canterbury Regional Ethics Committee and written informed consent was obtained from all participants. Our micro-simulation system has recently been described in detail (9). Four separate variables are displayed as the (numeric) current value and a graphic trend. After running at 'baseline' for a random period of time each variable changes to a new randomly determined value between 10 and 40 units away from the baseline and following an exponential 'wash-in' with a time constant randomly selected between three and 10 seconds. In 10% of the simulations there is no change from baseline. The basic signal is modified with a sinusoidal variation and random noise. Each simulation lasts 300 seconds, with one simulation cycle per second. The mean change after five seconds is 12 units (range 4 to 39); after 10 seconds, 17 units (6 to 40) and after 20 seconds, 20 units (9 to 40).
[FIGURE 1 OMITTED]
Participants observed 20 sets of simulations of the four variables. Since in 1 in 10 instances no change is generated, each participant would be expected to see at least 70 changes. Each set was randomly allocated to include an indication of TTV or not. Randomisation was performed using random numbers generated within the simulation adjusted after each simulation to ensure equal numbers in each group.
Figure 1 depicts a partial screen shot during a simulation showing the graphical display with TTV indications incorporated. The value of TTV was displayed in two ways. First with a series of "+" or "-" signs, as shown in the figure, adjacent to the current numerical value. Second, as a series of horizontal bars across the bottom of the display showing the time periods when a change was occurring. The width of the bar was between one and three pixels representing TTV values greater than 0.7 and corresponding to two, three or four "+" or "-" symbols. For all variables the TTV time weighting factor, alpha, was set to 0.2. In those simulations randomised to no TTV, the screen showed the graphical trends and the numerical current values without the "+" or "-" symbols or the horizontal bars.
Participants were instructed to identify changes as quickly as possible. Participants indicated changes by pressing a specific, clearly labelled key for each variable and direction of change. The program then recorded the response as correct (a change was occurring) or incorrect (no change was occurring). For correct responses, the time from the onset of each change (according to the algorithm) until it was detected was recorded defined as the latency of the response.
[FIGURE 2 OMITTED]
This study was undertaken at Christchurch Hospital during August and September 2006 once recruitment of subjects to our previous study9 had been completed at this site and prior to completion of data collection at the second centre. Since this study builds on our previous investigation of display formats and uses a very similar protocol, presentation of the results of the current study needed to wait for publication of the display format study. Ten anaesthetists were recruited and studied. Prior to the study, participants were shown at least three examples of each display format and practised the method of indicating changes in a training version of the system. The studies were conducted in a quiet room, free from distraction. For each subject, data from all simulations with the TTV indicator were pooled and those from all simulations without the TTV indicator were pooled. The two groups were compared using a paired t-test or Wilcoxon test as appropriate and P values and 95% confidence intervals calculated. P[less than or equal to]0.05 was designated as statistically significant. Statistical analysis was performed using GraphPad Prism for Macintosh v 4.0c.
In our previous study of the effect of the presence of a graphical trend display, the mean difference in time to detect a change was 3.0 seconds and the standard deviation of the difference was 2.9 seconds. We used GraphPad StatMate 2.0 to determine the sample size for a paired t-test with this standard deviation of the difference and this showed that using 10 pairs will detect a difference of 2.72 at a power of 80%.
All ten participants competed the simulation session.
Table 2 shows the results for the individual participants, while Table 3 summarises the results. Changes were detected 10.9% faster with the trend indicator present (mean 13.1 [SD 3.1] cycles vs 14.6 [SD 3.4] cycles, 95% confidence interval 0.4 to 2.5 cycles), P=0.013. There was no difference in accuracy of detection (median with trend detection 97% [interquartile range 95 to 100%], without trend detection 100% [98 to 100%]), P=0.8, although our study was not powered to detect small changes. Figure 2 shows the pattern of results for individual participants. All subjects detected changes more rapidly with the TTV display, with individual changes between 1% and 42% (mean 12%, 95% confidence intervals 2 to 20%).
These results suggest adding a graphical indicator of trends facilitates earlier detection of changes in simulated displays of data over and above the improvement seen in the presence of a graphic trend display alone. All participants' performances were improved by the presence of the display. The decrease in time to detect changes was above the 10% level for clinical significance determined by Lampotang et al (14). Three participants improved by more than 15%.
We expected that in this study environment, which involved a single task and no other distractions, participants would detect changes readily. In particular we expected attention would be more focussed on the monitor than in clinical practice. This should mean that changes are detected more rapidly than in a clinical setting, although we have not tested this assertion. We have already demonstrated that the presence of a graphical trend display results in more rapid detection of changes than current numeric values alone (9). Given these factors, any intervention leading to further decreases in latency in our micro-simulation should be of value in a clinical setting, and TTV appears to be such an intervention. We felt that exploring the effect of TTV in the presence of a graphic trend would be a more demanding test than the effect if only current numeric values were displayed.
The main limitation of this study is that the changes in trend were arbitrary and have no physiological significance. The types of parameters routinely monitored during anaesthesia have a wide range of baseline values and types and rates of change. The use of micro-simulation provides a convenient and highly controllable means of examining individual factors influencing performance We might expect changes to be more readily detected in the study setting than in the clinical environment (with its various distractions). Recent results suggest that anaesthetists glance at monitor screens, rather than study them in detail (2). Therefore, a clear indication that a change is occurring should be of value, which supports the applicability of our findings in clinical situations. However, confirmatory research would be necessary, perhaps using high fidelity simulation (17). A confounding variable is that the position and order of the numeric values do not always match the order of the trend graphs in this simulation, since all data are plotted on the same y-scale, while the numeric values retain fixed positions. We observe that this disconnection between numeric values, waveforms or trends is reasonably common in clinical monitoring systems, and our previous results suggest that using a higher resolution y-axis assists in the detection of changes despite the consequent overlap of traces9. The sampling interval of one update every second is also artificial. The overall simulation, including the generation of changes was designed to represent 10 times real time. This approach combined with a statistical approach to the generation of changes was chosen to allow users to be exposed to many changes within a reasonable time period.
Trigg's Tracking Variable is one of many trend detection algorithms (15). More complex and possibly more accurate systems such as wavelet analysis (16) have been described and show promise. Trigg's Tracking Variable has the advantage of being simple to implement and therefore study, but has well-recognised deficiencies (14). We would expect the more sophisticated techniques to be at least as useful and possibly better than TTV.
In anaesthesia, just as when driving, there are several steps between recognising an impending problem and responding to it effectively. Much of the emphasis in anaesthesia crisis management training is on processes after the problem is recognised. Enhanced monitoring could reduce the time between a perturbation occurring and its recognition by the practitioner. It could, perhaps, also facilitate effective decision making. We have previously shown, in association with Warman and Webster, that trend graphs speed detection of changes and now have demonstrated further enhancement in detection with trend signalling. As shown in Figure 2, the response time of all our subjects was improved, with three showing a greater than 15% improvement. Even a modest improvement in performance by a minority of participants may have a beneficial effect on patient care.
In summary, this study has demonstrated that, in a micro-simulation, the presence of a simple trend detection algorithm leads to faster detection of changes in time series of data. Tools such as TTV may be of value in clinical monitoring systems.
Trigg's Tracking Variable
Adapted from Hope (12).
Initialisation requires an approximate current value (V)
From this are calculated initial values for
The time weighted average of the previous sampling period
[u.sub.t-1] = V
the previous error in prediction
[s.sub.t-1] = V/100
and the previous mean absolute deviation
[MAD.sub.t-1] = V/10
The main routine consists of the following steps:
The prediction for the next period:
[u.sub.t] = [alpha][d.sub.t] + (1-[alpha])[u.sub.t-1]
where [d.sub.t] is the current value [alpha] is a constant between 0 and 1 which determines the weighting
typical values are 0.1 or 0.2 (11)
The error in the prediction
[e.sub.t] = [d.sub.t]-[u.sub.t-1]
The smoothed error
[s.sub.t] = [alpha][e.sub.t]+(1-[alpha])[s.sub.t-1]
and the mean absolute deviation for the error
[MAD.sub.t] = [alpha][absolute value of [e.sub.t]] + (1-[alpha]) [MAD.sub.t-1]
Trigg's Tracking Variable is then calculated
[T.sub.t] = [s.sub.t]/[MAD.sub.t]
Finally variables relating to the previous time period are updated:
[u.sub.t-1] = [u.sub.t]
[s.sub.t-1] = [s.sub.t]
[MAD.sub.t-1] = [MAD.sub.t]
(1.) Gorges M, Staggers N. Evaluations of physiological monitoring displays: a systematic review. J Clin Monit Comput2008; 22:45-66.
(2.) Ford S, Birmingham E, King A, Lim J, Ansermino JM. At-a-glance monitoring: covert observations of anesthesiologists in the operating room. Anesth Analg 2010; 111:653-658.
(3.) Slagle JM, Weinger MB. Effects of intraoperative reading on vigilance and workload during anesthesia care in an academic medical center. Anesthesiology 2009; 110:275-283.
(4.) Westhorpe RN. Ergonomics and monitoring. Anaesth Intensive Care 1988; 16:71-75.
(5.) Drews FA, Westenskow DR. The right picture is worth a thousand numbers: data displays in anesthesia. Hum Factors 2006; 48:59-71.
(6.) Michels P, Gravenstein D, Westenskow DR. An integrated graphic data display improves detection and identification of critical events during anesthesia. J Clin Monit 1997; 13:249-259.
(7.) Gurushanthaiah K, Weinger MB, Englund CE. Visual display format affects the ability of anesthesiologists to detect acute physiologic changes. A laboratory study employing a clinical display simulator. Anesthesiology 1995; 83:1184-1193.
(8.) Jungk A, Thull B, Hoeft A, Rau G. Evaluation of two new ecological interface approaches for the anesthesia workplace. J Clin Monit Comput 2000; 16:243-258.
(9.) Kennedy RR, Merry AF, Warman GR, Webster CS. The influence of various graphical and numeric trend display formats on the detection of simulated changes. Anaesthesia 2009; 64:1186-1191.
(10.) Kennedy RR. A modified Trigg's Tracking Variable as an 'advisory' alarm during anaesthesia. Int J Clin Monit Comput 1995; 12:197-204.
(11.) Lewis CD. Statistical monitoring techniques. Med Biol Eng 1971; 9:315-323.
(12.) Hope CE, Lewis CD, Perry IR, Gamble A. Computed trend analysis in automated patient monitoring systems. Br J Anaesth 1973; 45:440-449.
(13.) Kennedy RR. The effect of using different values for the effect-site equilibrium half-time on the prediction of effect-site sevoflurane concentration: a simulation study. Anesth Analg 2005; 101:1023-1028.
(14.) Lampotang S, Gravenstein JS, Euliano TY, van Meurs WL, Good ML, Kubilis P et al. Influence of pulse oximetry and capnography on time to diagnosis of critical incidents in anesthesia: a pilot study using a full-scale patient simulator. J Clin Monit Comput 1998; 14:313-321.
(15.) Avent RK, Charlton JD. A critical review of trend-detection methodologies for biomedical monitoring systems. Crit Rev Biomed Eng 1990; 17:621-659.
(16.) Melek WW, Lu Z, Kapps A, Fraser WD. Comparison of trend detection algorithms in the analysis of physiological time-series data. IEEE Trans Biomed Eng 2005; 52:639-651.
(17.) Merry AF, Weller JM, Robinson BJ, Warman GR, Davies E, Shaw J et al. A simulation design for research evaluating safety innovations in anaesthesia. Anaesthesia 2008; 63:1349-1357.
R. R. KENNEDY *, A. F. MERRY ([dagger])
Department of Anaesthesia, Christchurch Hospital and University of Otago-Christchurch, Christchurch, New Zealand
* M.B., Ch.B., Ph.D., F.A.N.Z.C.A., Specialist Anaesthetist and Clinical Associate Professor.
([dagger]) M.B., Ch.B., A.N.Z.C.A., F.F.P.M.A.N.Z.C.A., F.R.C.A., Professor, Deaprtment of Anaesthesiology, University of Auckland, Auckland.
Address for correspondence: Dr R. Kennedy, email: email@example.com
Accepted for publication on April 10, 2011.
TABLE 1 Ranges of values for Trigg's Tracking Variable and corresponding indicators Absolute value of TTV Proportion Indication of changes by current detected value TTV [greater than or equal No indicator to] 0.55 0.55 < TTV [greater than or 33% "+" or "-" equal to] 0.70 0.70 < TTV [greater than or 60% "++" or "--" equal to] 0.80 0.80 < TTV [greater than or 92% "+++" or "---" equal to] 0.97 TTV >0.97 94% "++++" or "----" Absolute value of TTV Width (pixels) of bar TTV [greater than or equal No indicator to] 0.55 0.55 < TTV [greater than or No indicator equal to] 0.70 0.70 < TTV [greater than or 1 equal to] 0.80 0.80 < TTV [greater than or 2 equal to] 0.97 TTV >0.97 3 The "+" or "-" are shown beside the current value while the bars are shown under the trend graph. The "proportion of changes" column shows the percentage of actual changes detected by the upper level of Triggs Tracking Variable in this type of simulation (10). TTV=Trigg's Tracking Variable. TABLE 2 Results for individual participants No TTV TTV present Number of Time to Changes Time to changes detect correctly detect correctly change identified/ change identified/ actual actual A 38/38 10.9 37/39 10.7 B 34/35 14.8 39/40 10.4 C 36/38 11.4 33/38 10.0 D 39/39 22.8 37/37 19.5 E 39/39 16.2 33/37 16.0 F 36/36 13.2 39/39 13.1 G 39/39 15.4 37/38 10.2 H 40/40 12.0 36/36 10.2 I 37/38 13.5 39/39 11.9 J 34/34 15.3 34/36 15.1 Changes identified are shown with the number of actual changes in that simulation. TTV=Trigg's Tracking Variable. TABLE 3 Summary of results No TTV TTV present Accuracy, median [IQR], % 97.4 [95-100] 100 [98-100] P=0.8 Latency (delay) in 14.6 (3.4) 13.1 (3.1) P=0.013 detection, mean (SD), s Accuracy is the proportion of correctly identified changes. The delay, or latency, is the average time to detect correctly identified changes from the algorithmic onset of the change. Results are shown with IQR, or SD as appropriate. TTV=Trigg's Tracking Variable, IQR=interquartile range.
|Printer friendly Cite/link Email Feedback|
|Author:||Kennedy, R.R.; Merry, A.F.|
|Publication:||Anaesthesia and Intensive Care|
|Date:||Sep 1, 2011|
|Previous Article:||The association between intraoperative electroencephalogram-based measures and pain severity in the post-anaesthesia care unit.|
|Next Article:||Safety of exposure of malignant hyperthermia non-susceptible patients and their relatives to anaesthetic triggering agents.|