Case study: using control charts to track physician productivity.

Control charts provide a multitude of statistical information in a single set of graphs that can be highly useful in various health care applications.

K. Lee and C. McGreevey provided insight into the use of control charts to evaluate performance measures to satisfy Joint Commission on Accreditation of Healthcare Organizations (JCAHO) standards (1), and Raymond G. Carey has provided examples of application of the use of control charts for assessing health care data. (2,3)

An article in the Italian Heart Journal demonstrated the use of control charts to examine trends in medical error rates, as well as identify special cause targets for intervention and common cause variation that indicated the need for process redesign. (4,5)

The configuration of a control chart involves a centerline that represents the mean of the data set, and "control limits" that are calculated to represent [+ or -]3 standard deviations (or standard errors of the mean).

A line consisting of sample data is then plotted on the graph, and the configuration of the line relative to the centerline and control limits is examined for specific attributes that provide insight into the process underlying the data. (6)

Although the basic configuration of control charts appears consistent, calculating the values for the centerline and control limits vary depending on the type of data and configuration of the data set. The effective use of control charts requires attention to the type of chart selected.

Case study

Control charts are also being used to track physician productivity at Shriners Hospitals for Children (SHC), a 22-hospital system caring for children with orthopedic conditions, burns and spinal cord injuries.

Care at the hospitals is provided at no cost to patients or families, and SHC is renowned for patient centered care that considers all of the needs of children with special health care needs. The Lexington, Ky., Shriners Hospital specializes in pediatric orthopedics and has the characteristics noted in Table 1.

Chester Tylkowski, MD, the chief of staff of the Lexington Shriners Hospital, practices orthopedic surgery at the hospital and also has responsibility for managing the medical staff, credentialing, clinical research, resident education and clinical quality improvement.

Tylkowski encouraged physicians on staff to become more productive during his tenure, and over the past nine years developed a strategy of performance improvement that is evidence based, involving the use of control charts.

Data analysis

The use of clinical data to monitor and improve performance is an old concept, but the traditional approach involved observation of a clinical measure from month to month and reacting to the individual practitioner's volume statistics at the end of each time period. This approach is characterized by Figure 1.

Managing Dr. X's productivity from month to month using the data in Figure 1 would create a number of problems, since the variation in the measure is so great. For example, a conference with Dr. X in April 2002 would probably include a "pep talk" to encourage her to increase her clinical activity.

On the other hand, in June of 2002, Dr. X's mentor would have a much more difficult time determining how to counsel her (other than lauding her ambition), since her clinical activity almost tripled between April and June of that year.

Then, of course, her clinical activity returned to more representative levels, but the quandary would remain regarding her performance--why is her hospital activity varying so much? Are there special circumstances underlying the variation or is it within a range that would be considered normal variance?

One method of trying to make sense of her activity is to add a trend line using traditional linear regression, as in Figure 2. The trend line shows an increasing trend during the 12-month period that would have been difficult to detect through simple analysis of the data.

[FIGURE 1 OMITTED]

Using the trend line, Dr. X's mentor probably would not consider an aggressive approach to changing her behavior, since her utilization is trending in the right direction. Analyzing performance solely with trend analysis misses an important issue, however.

As can be seen in Figures 1 and 2, the inpatient days measure demonstrates substantial variation over the analysis period.

Control charts afford another way of looking at the metric. This approach provides a wealth of information in addition to evaluation of the raw data.

Figure 3 presents a control chart analysis of the inpatient days' performance measure.

Two charts are presented in Figure 3, since the individual X-moving range (IX-mR) chart is based on an assumption that the data is approximately normally distributed. The moving range (mR) chart tests that assumption by determining if the differences between each observation (the moving range) fall between [+ or -]3 standard deviations from the mean.

If the moving range values fall outside the control limits, then the underlying assumptions that validate the use of the control chart approach may not be legitimate, and other analytic approaches must be applied. In the case presented in Figure 3, the mR chart is in control, and so the IX chart can be considered valid.

Control limits for the IX chart represent [+ or -]3 standard deviations from the mean, which is plotted as the center line for the chart. The individual values of inpatient days are plotted on the graph, and the values are compared with the control values to determine if they fall within the control limits (common cause variation) or outside the control limits (special cause variation).

Special cause variation connotes a value in the data set that has less than three chances in 1000 of occurring by chance. As a result, a special cause value indicates that some reason for the value can be found, and that the value is most likely not due to process variation. On the other hand, common cause variation is considered to be due to variation due to process design.

Another useful approach combines the trend line in Figure 2 with the control chart in Figure 3 to create the graph in Figure 4 that provides information about both the trend and the variation in the metric.

With the additional analysis, both parameters can be evaluated to give a more complete picture of the underlying issues surrounding the measure. For example, Figure 4 indicates that the inpatient days measure for Dr. X is demonstrating an upward trend, and variation is within control limits. So, Dr. X's performance would be considered favorable for the hospital.

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

Using control charts for evaluating physician activity has a number of advantages.

First, the significance of the variation in a clinical activity measurement is readily apparent. Control limits establish the bounds for determining whether the variation should be treated as a special cause or if it is related to expected variability in the physician's practice patterns.

Secondly, addition of the least squares line as in Figure 4 provides information about the trend of the physician's activity.

Finally, combining the trend and control chart analyses affords a more complete representation of the doctor's clinical activity. The inpatient days measure chosen for this paper is only one example used in producing a physician profile, which at the Lexington Shriners' Hospital include:

* Average length of stay

* Number of admissions

* Outpatient new visits

* Outpatient return visits

* Outpatient total visits

* Outreach clinic visits

* Number of procedures

* Number of surgical cases

* Outpatient relative value units (RVUs)

* Inpatient relative value units (RVUs)

* Surgical relative value units (RVUs)

* Total relative value units (RVUs)

[FIGURE 4 OMITTED]

The physician's evaluation utilizes all of these measures to identify opportunities for improvement, which comprise the discussion topics for the semiannual physician reviews.

Physician reviews

Physicians respect data. Although most doctors are not well versed in the use of statistical techniques like control charts (which have only recently been applied in health care), they are quite aware of the implication of descriptive statistics (mean, median, standard deviation), and data analyses that provide measures of statistical significance allow a greater level of reassurance than simple attempts to make inferences from raw data. The approaches afforded by statistical process control (SPC) can add value to discussions of trends and performance.

Physicians dislike annoyances. In particular, physicians dislike being required to explain normal variation as though it were a special circumstance. In SPC terms, physicians do not appreciate tampering or treating common cause variation as a special cause.

For example, the data in Figure 1, when tracked month by month, most likely would trigger an administrative reaction each month to the up-and-down behavior of the data, with concern during the months when performance apparently declined and delight during the months when performance increased.

This lack of consistency in an administrator's responses can be particularly frustrating since all of the variation is expected due to the physician's inherent practice patterns and the hospital's system for providing care.

Although the doctor can alter practice patterns, such changes for the physician in this analysis would not be achieved by looking for a specific cause of the variation, but rather examining the processes by which the doctor performs her work.

Using the control charts, physician activity can be evaluated for statistically significant variation that can then be targeted with root cause analysis and design of specific interventions for improving either practice processes or the methods by which the hospital provides care.

The approach taken in the Lexington Hospital involves tracking several measures as part of a physician profile that is used by the chief of staff in working with the medical staff to detect opportunities for improvement.

Use of a profile makes the analysis more germane to the physician, since the profile provides a more complete depiction of the physician's clinical activity for a particular time period. For example, if the doctor has spent a substantial amount of time in outreach clinics during the month, then surgical activity would be expected to drop.

Comparison of the metrics in the profile helps the chief of staff balance the physician's activity for the time period and could help identify the explanation for apparent special cause variation in some cases. Each physician also has the opportunity to compare an individual profile with overall averages for all physicians, allowing the physician to better understand his/her productivity in the context of all the hospital's providers.

Data is blinded so that physicians are afforded anonymity with peers, but still have the advantage of discerning where they stand in relation to their peers. This comparison arouses the competitive nature of most physicians and encourages them to become more productive if they are on the low end and rewards them if they are above most in the group or the leader of the group. Ranking can also serve to reward high producers, depending on the financial structure of the organization.

Additionally as activity is reviewed periodically, achieving balance between related measures also can prove of substantial value. For example, if reducing the average length of stay is an important goal of the hospital, then increasing the number of procedures performed in the ambulatory surgery center may be a concurrent objective. The combined control chart and trending analysis can provide a more complete evaluation and analysis.

Control chart and trend analysis are useful adjuncts for analysis of physician practice patterns, particularly when utilized to find opportunities for improvement. Differentiating between special cause and common cause variation directs interventions more appropriately, either to find unusual issues that can be remedied with a direct intervention (special causes) or to identify the need for a complete process review and root cause analysis to determine methods of improving the system of care.

Computer hardware and software for performing these analyses are inexpensive and readily available, making the approach easily deployed in virtually any organization.

Table 1 Shriners Hospital in Lexington, KY

Pediatric conditions: Orthopedics, Arthritis

Full Time Medical Staff:
Pediatric orthopedics                  5
Pediatrics                             1
Nursing staff                         40
Total employees                      212
Surgical procedures per year       1,000
Outpatient visits per year        15,000

References

1. Lee K, McGreevey C. "Using control charts to assess performance measurement data." Joint Commission Journal of Quality Improvement. Feb. 2002, 28(2): 90-101.

2. Carey, R. "Improving patient satisfaction: a control chart case study." J Ambulatory Care Manage. July 2002, 25(3): 78-83.

3. Carey R. "How do you know that your care is improving? Part II: Using control charts to learn from your data." J Ambulatory Care Manage. April 2002, 25(2): 78-88.

4. Carroll CA, Cox KS, Santos SR, Simon SD. "Using standard desk-top tools to monitor medical error rates." Seminars in Nursing Management. June 2002, 10(2): 95-9.

5. Cisbani L, Berti E, Tassinari C, Giovannelli N, Vizioli M, Grilli R. "Learning from experience. Applications of control charts to the monitoring of performance in cardiology." Italian Heart Journal. April 2002, 3(4 Suppl):426-32.

6. Lighter D, Fair D. Principles and Methods of Quality Improvement in Health Care, Chapter 4. Denver: Aspen, 2000.

By Donald E. Lighter, MD, MBA, FAAP and Chester M. Tylkowski, MD

Donald E. Lighter, MD, MBA, FAAP, is AMD for clinical outcomes programs with Shriners Hospitals for Children in Tampa, Fla. He can be reached at 813-281-7158 or [e-mail]

[ILLUSTRATION OMITTED]

Chester Tylkowski, MD, is chief of staff of the Lexington Shriners Hospital in Lexington, Ky. He can be reached at 859-266-2101 or [e-mail].

[ILLUSTRATION OMITTED]