Using the Balanced Scorecard as a performance management tool.

A frequent complaint from department chiefs in academic medical centers is that their performance is increasingly being judged, both by hospitals and schools of medicine, based on financial indicators, such as billings, collections and research grant funding.

While they concede that financial performance is a major indicator of success, it fails to provide a "big picture" assessment of their department's overall performance. They recognize that other factors may be equally important in determining their success and often are linked to their ultimate financial performance.

How can you develop a way to measure and link performance from these various factors?

The Balanced Scorecard is a performance management tool that is used to provide an integrated perspective of an organization. Developed by Kaplan and Norton (1) as a performance measurement tool, it is widely used in both the manufacturing and service industries.

The scorecard is based on the concept that organizations must manage and evaluate their business from at least four perspectives:

1. External

Analyze how customers view the organization. For an academic practice, this could be measured by patient satisfaction, number of outside referrals, invited lectures given or articles published in peer-review journals.

2. Internal

Look at where an organization must succeed and improve. This could include measures of operating room cases, consultations performed, clinic cancellations or length of stay.

3. Growth and learning

Examine how the organization grows and learns. This could include measures of internal and external clinical program development or research development.

4. Financial

Study financial performance of the organization. Measures could include management of expenses, research grant funding or measures of billings/collections and days in receivables.

The important point of the scorecard is that each of these perspectives influences the other. To concentrate on only one, such as financial performance, fails to recognize the contribution and balance of the other three perspectives.

Analyzing performance from each of these perspectives allows the organization to develop an integrated strategy around them.

Developing a scorecard

How do yon develop a balanced scorecard?

First the organization must decide what it is going to measure in each of the four perspectives. Each measure must be based on data that can be easily collected and analyzed. The measure must have meaning to the overall performance of the organization. Just because something is easy to measure does not mean that it should be used.

In addition, each of the perspectives may not hold any equal weight to the others. The same is tree for the criteria to be measured within each perspective,

For example, having a "days in receivables" less than 90 days may carry a greater weight than total research grant funding when examining the financial perspective.

Figure 1 is an example of a simplified balanced scorecard for an academic surgical department. As you can see, easily measurable criteria with target measures are determined for each of the four balanced scorecard perspectives.

After figuring out the criteria and target measures, how do you determine the weights of each of these criteria and ultimately evaluate overall performance?

Fletcher and Smith (2) showed that by using the technique of Analytic Hierarchy Processing (AHP) to weight the criteria within the perspectives of the scorecard, you may develop an integrated performance index to monitor and assess overall performance.

AHP developed by Saaty (3) is a method for breaking down a complex problem into a hierarchy, assigning numeric values to subjective judgments through pair-wise comparisons, then synthesizing those judgments to give priorities to the variables. (For a full discussion of AHP and its calculations, please see The Nuts and Bolts of Business, "Determining Physician Compensation: Part 2" in the May-June 2002 issue of The Physician Executive).

Figure 2 is an example of a hierarchy and the results of pair-wise comparisons for the criteria in our model scorecard. The model was developed using the software, Expert Choice.

[FIGURE 2 OMITTED]

AHP Structured Hierarchy

For simplicity, each of the perspectives was weighted equally. Pair-wise comparisons were then made for each of the criteria within each perspective.

For example, in the external perspective of this model, patient satisfaction was judged to be three times more important than invited lectures and published articles, and twice as important as referrals.

The results of the comparisons are shown in the figure. The criteria are then weighted by dividing the weight of each individual criterion by the total value of all the weights in each perspective. The results are shown here as AHP weight.

You can see that certain criteria, such as clinical program development carry an unusually high weight. This is because we weighted each of the four perspectives equally.

Had we weighted the financial or external perspectives higher than the growth perspective (which likely would be the case in most real scorecards), the criteria within those categories would be weighted higher than those in the growth perspective.

Performance counts

Alter developing the balanced scorecard criteria and target measures, and weighted each of the perspectives and criteria, it's time to develop an overall performance index. (2)

Measurement of the performance index is accomplished by taking the actual performance measure for each criterion, determining the percent of the target measure achieved, and multiplying this percentage by the AHP target weight to determine the performance index for that criterion.

The total of each of these indices gives the overall performance index. A performance index table allows you to examine those areas where performance was less than target measure so that improvements in those criteria may be undertaken.

Long-term success for any organization requires performance assessment that monitors the value of efforts across all functions of the organization and not just financial results.

By using a combination of the balanced scorecard with AHP, you may develop a useful performance management tool that allows you to develop and assess strategy implementation.

Figure 1
Academic Surgical
Practice Balanced Scorecard

External Perspectives

Goals Target           Measures

Patient Satisfaction   95%
Invited Lectures       20/year
Referrals              600/year
Publications           12/year

Financial Perspective

Goals              Measures

Mgt. of Expenses   100% to budget
Research Grants    $1 million/year
Days in AR         90 days
Collection Rate    40%

Growth Perspective

Goals                          Measures

Clinical Program Development   1/year
Faculty Promotion              2/year
New Research Projects          10/year

Internal Perspective

Goals                 Measures

OR Cases              1200/year
Avg. Length of Stay   3.5 days
Case Cancellations    25/year

Figure 3
Determining the Performance Index

Performance          AHP     Target     True      Percent   Performance
Measure             Weight   Measure   Measure   of Target     Index

Patient
Satisfaction        10.2%      95%       93%        98%        10.0%

Referrals            8.0%      600       450        75%         6.0%

Invited
Lectures             2.6%       20        17        85%         2.2%

Published
Articles             4.1%       10        11       110%         4.5%

OR Cases            14.6%     1200      1150        96%        14.0%

Avg. Length
Of Stay              7.0%      3.5       3.5       100%         7.0%

Case
Cancellations        3.4%       25        50        50%         1.7%

Expense Mgt          5.6%     100%       93%        93%         5.2%

Grant Funding        2.4%      $1m     $1.2m       120%         2.9%

Days in AR           7.2%       90       100        90%         6.5%

Collection
Rate                 9.9%       40%      42%       105%         0.4%

Clinical Program
Development         12.3%        1         1       100%        12.3%

Faculty Promotion    4.9%        2         2       100%         4.9%

New Research         7.8%        8         8        80%         6.2%
Projects

Total                100%                                      93.8%

References:

(1) Kaplan. RS. and Norton, DP. "The Balanced Scorecard: Measures that Drive Performance.' Harvard Business Review. 70: (Jan-Feb) 1992, pp. 71-79.

(2.) Fletcher, HD. and Smith, DB. "Managing for Value: Developing a Performance Measurement System Integrating Economic Value Added and The Balanced Scorecard in Strategic Planning." J Business Strategy 2003 (In print)

(3.) Saaty, TL. The Analytic Hierarchy Process Pittsburgh, Pa.: RWS Publications; 1996.

David P. Tarantino, MD, MBA is the executive medical director of Shock Trauma Associates, P.A., a 50+ physician, multispecialty practice associated with the University of Maryland School of Medicine. In addition he is the chief executive officer of The MD Consulting Group, LLC, a health care management consulting firm in Baltimore. He can be reached by phone at 410-328-2036 or by e-mail at [e-mail]