Keeping score: scorecards, profiles and report cards rapidly expanding to track physician performance.

About five years ago, physicians and executives at three central New Jersey hospitals that encompass Meridian Health were concerned that scorecards developed by health plans and others might portray its doctors unfairly.

"We wanted to know the data better than an outside agency so we could challenge it if it was wrong," says Jeffrey Borell, Meridian's manager of outcomes measurement.

[ILLUSTRATION OMITTED]

"Data has been available to insurance companies for years. Doctors needed to know what the companies had on them," adds Douglas Bechard, MD, Meridian's senior vice president for clinical effectiveness. So Borell and members of Meridian's teaching hospital staff at Jersey Shore University Medical Center, created their own scorecard from scratch. "We looked at what data we already had and started to formulate something," Borell says.

Benchmarks to compare physicians came from 3M's APR-DRG software, the New Jersey Department of Health, JCAHO, and the CMS Web site. Once executives created the scorecard, they stacked it up against physician profiles at other facilities, and then tweaked it more. "It's a work in progress, a good foundation. There will be changes in the future," Borell says.

Meridian's scorecard has endured several revisions following distribution and physician complaints. In the current incarnation, doctors are rated in three areas:

1. Quality, such as giving prophylactic antibiotics prior to surgery

2. Clinical effectiveness, like mortality and length of stay

3. Financial issues, such as percentage of managed care days denied.

Cardiologists, for example, receive grades on 10 quality measures, four clinical effective measures and three financial measures.

The scorecards compare physician performance to members of his or her own group, members of the same specialty who work at their hospital, and to other doctors who treat similar conditions at their hospital.

For readability and ease of use, Borell and his team limited color-coded scorecards to one page. (See Figure 1)

"We felt if it was a booklet, doctors wouldn't read it," Borell says. To assure doctors didn't discard it, Meridian sent it to physician homes with a sticker emblazoned with bold red type: "This is some of the most important information you will receive this year."

Meridian judges 600 of its 1,750 physicians twice annually. Only the largest admitters, those with more than 10 inpatient discharges per six months, qualify for the program since the data are not deemed statistically significant otherwise. Internists as a group receive the most scorecards, followed by cardiologists, pediatricians and surgical subspecialists.

It costs the system about $40,000 annually in manpower to produce the scorecards, excluding technology and mailing. Though Meridian has installed CPOE, it is not yet helpful in data collection. Information manually accumulated from nearly every chart is fed online to QuadraMed, a Reston, Virginia, information management company. QuadraMed then spits it back with results.

For its investment, Borell estimates at least 10 percent of doctors improve their performance each time scorecards are issued. One male cardiologist, for example, received quality scores of 75-96 percent in 2004.

"In the first six months of this year, the most recent period for which we have data, he has all 100 percent," Borell says.

[ILLUSTRATION OMITTED]

"After presenting him with 2004 data, he immediately changed his practice," Bechard adds. "Physicians are under pressure. They don't want to be left behind." Bechard and Borell do not attribute improvement solely to Meridian's scorecards. "Physicians today are bombarded with quality information," Borell says. Less frequently, doctors score lower than their previous profiles.

Meridian has had an overall reduction in length of stay and noticeable quality gains, a result that pleases the system's board of directors. "The board of Meridian realizes improved quality from the profiles has an economic benefit even though there is a cost to doing it," Borell says. "We believe we are saving money."

Hospital accountants call the savings "funny money" because when doctors avoid a $15,000 expense by discharging a patient sooner, it's not cash in your pocket," he adds. Bechard is reticent to mention cost benefits. "We don't want physicians to believe this is a financial tool masquerading as a quality report," he says. "Sometimes quality saves money by preventing infections and lowering readmission, but not always. Quality can also cost more."

Meridian physicians are neither rewarded for high scores nor penalized for low ones. "We don't play 'gotcha,'" says the manager. And profiles are not tied to recredentialing. However, when scores are persistently low, doctors are summoned to meetings with their department chairs. Bechard sits in if necessary.

About 10 percent of the 600 who receive the cards, or 60 doctors, are called in twice a year. "We have conversations with individuals who do not do well. The department chair says, 'I've sent you your profile. Are there areas you are concerned with? We don't say 'your mortality rate is 20 percent higher than the group,'" Bechard says.

"Sometimes physicians will say they did something. We tell them it's not documented. Then their documentation improves," Borell says.

Next year, Meridian plans to include "patient satisfaction." In the future, "complications," a more controversial measure, will appear, Borell notes. Readmission rates aren't considered because it's difficult to benchmark. "It counts if someone comes in for heart failure and a month later, with a broken leg," he says.

Bechard prefers the term "profile" to "scorecard" or "report card." "Scorecard and report card have negative connotations that you are being scored for reimbursement or for staying on a medical staff. That's a limited use of these. You get much more out of this when you think of them as collaborations," he says.

To reinforce a culture of education rather than blame, confidentiality is paramount. Only four copies are made of each scorecard. One is sent to physician homes, one resides with the physician's department chair, one goes to the medical staff office and one remains in hospital files under lock and key.

Meridian uses scorecards now because of technological developments. "We didn't have the sophistication of data to produce them before," Borell says. There was also a cost consideration. It takes three full-time analysts two months out of the year to compile the data.

One goal of the scorecards is to standardize medical practices more quickly than what has taken place in the past. "It takes 15 to 17 years for a new concept to become routine, such as ace inhibitors for congestive heart failure and beta blockers for myocardial infarction," Bechard says. "If you use profiles as quality measures, it could take a lot shorter than 15 years."

Bechard says he wants Meridian doctors to become comfortable with the inevitability of public reporting on performance. "This is the first step--trying to get our physicians used to it."

Experts believe that the nation is moving toward a system where anyone searching for a doctor will be able to read an online review of his or her performance like a box score in a newspaper. Patients may comparison-shop doctors and hospitals as they do cameras and cars. Scorecards then become a Consumer Reports-type analysis of doctors in place of dishwashers.

The push for scorecards comes from multiple sources--employer groups, CMS, Institute of Medicine and the National Quality Forum, says Geoffrey Baker, CEO, Med-Vantage, a San Francisco-based health informatics company that develops profiles.

"I think the future is that physician services are too important to patients' physical well being not to be subjected to accountability," says national profiling expert Arnold Milstein, of Mercer Human Resource Consulting, San Francisco.

"Gradually, increasing details on individual physician performance on resource use, price, quality and customer service will become available. Insurers and the federal government will take the lead in sharing this information with the public. It's already happening."

More than 250 health plans profile physicians, but only 20 share performance and cost information with consumers, Med-Vantage's Baker says. Humana, Pacificare, HealthPartners, Aetna as well as many state and regional Blue Cross and Blue Shields are among them. The information comes from health plan administrative data--claims, pharmacy, eligibility--as well as CMS QIO hospital information.

Profiling has been around for at least 15 years. Nearly every urban practicing physician already receives report cards' from either the hospitals where they work, the HMOs with whom they contract, or both, estimates Milstein.

"Profiling has only begun to get mainstream over the last several years," Milstein says. "There are two reasons: health insurance premiums began to rise after the 'decline' of managed care. Consumers started to get desperate over cost increases eating into rent and food."

For example, the annual cost of health insurance for a California family of four is equivalent to the annual earnings of a fully employed minimum wage worker, according the 2005 Employer Health Benefits Survey by the Kaiser Family Foundation and Health Research and Educational Trust.

Moreover, he adds, there were numerous Institute of Medicine reports with the core message that inconsistent quality, varied outcomes and unacceptable error rates are rampant in American health care. Research published in 2003 in the New England Journal of Medicine indicated that an average American has only a 55 percent chance of getting care that current science says they should receive.

While some Meridian doctors agree with the need to standardize medicine, they take issue with the methods used to grade them. "The most difficult facet of scorecards is achieving physician buy-in. It takes time, and not all of our doctors accept them to this day," Borell says. "The more we put these out, the more acceptance we get because they see us not going away."

Physicians disagree with the benchmarks, they claim the data is inaccurate, and they are skeptical about intent and relevance. Many doctors feel it will be used punitively rather than to identify opportunities for improvement. And they are scared.

"At medical staff meetings, physicians have raised concerns about trial attorneys using scorecards against them," Bechard says. It's never happened, though he acknowledges it is a valid concern.

Despite Meridian's enormous effort to produce the profiles, random calls to Meridian admitters reveal that doctors don't remember receiving them, easily confuse them with those sent by insurers and sometimes don't give them a lot of consideration.

"I've gotten low scores. I don't care. I can't do any better," says Sharon Angelo, DO, an internist who has been told she is one of the largest admitters to Meridian's Riverview Medical Center, Red Bank. "I am an excellent diagnostician. Length of stay is not a good parameter of my abilities."

Angelo argues she is not always the reason her patients remain in the hospital. Orders get lost, orders don't get done in a timely manner, and there are equipment failures, staffing and scheduling problems. "If you admit a patient and a test can't be done until the next day, it's a black mark against the physician. The responsibility is not shared by the institution," Angelo says.

Meridian interventional cardiologist, Christopher Pierson, MD, has found errors. "They had me attached to a practice I have not been with for years and at the wrong address. It makes you wonder about the accuracy of the other data," he says. "This should not be the only yardstick."

In addition, Pierson suggests scorecards are inadequate measurement tools. "Patients don't always fit within established protocols," Pierson says.

Still, Bechard defends their use. "There are always a small percentage of patients that cannot be standardized. But it would be unusual for a doctor to have all patients that don't fit the models," he says. "We know there are best practices and if doctors don't use them, there are poor outcomes."

Pierson adds that the results are suspect. "As a doctor, you have very few ways to confirm the numbers they throw at you because you'd have go back and examine all of your charts," he says.

Borell admits scorecards are imperfect but better than nothing. "If the profile shows a lot of things out of compliance, it gives a hint of other issues. If doctors meet 100 percent of quality issues, it hints the doctor may be of good quality."

One complaint of Meridian doctors, echoed by physicians elsewhere, is that scorecards can be gamed.

"You can lower your mortality by walking away from the sickest patients, by referring them to other doctors," Pierson says. "There are ways to get out of a case. You can tell a patient, 'I'm not the right doctor for you.' You see surgeons who take on low risk patients so their numbers look good. I know doctors who refer patients to other hospitals," Pierson says. "These scorecards could hurt patient care."

Recent press reports indicate New York state heart surgeons refrain from angioplasty on patients most at risk because it skews their state scorecards and gives them high mortality rates.

"The process by which New York surgeons are mortality adjusted is insufficient," contends Bechard. "Overall, it could produce adverse selection of patients. It doesn't mean we should not give out the data."

Milstein says gaming is a valid charge. "It's going to drive improvement in the measures so doctors are not penalized unintentionally for taking on the sickest patients." Some Meridian doctors support scorecards. "It's helpful as a reminder to maintain the standard of care in the community," says cardiologist Jeffrey Daniels, MD, who practices at Jersey Shore.

Still, Daniels admits there are problems, "The person reading the record may not understand the implication of what they are reading. If a patient is sent home without aspirin at discharge, it doesn't mean the doctor did something wrong. It could mean the patient has bleeding ulcers. But the reviewer only sees the patient is not on aspirin which violates the quality issue."

Daniels notes the information may be inaccurate if a doctor is a member of a group. "There's so much mixing of doctors with individual patients. If I do the admission, all the information about the patient is attributed to me. But my partners could see the patient, administer treatment, and do the discharge. To be fair, doctors in groups should not be judged individually but as a group."

Pierson argues scorecards take the art out of medicine. "They are trying to micromanage us as doctors," Pierson says. "If they want to use scorecards, they might as well have a computer generate a protocol and have non-doctors carry out it out. They're removing the physician experience, trying to rob us of our experience and turn us into technicians. The scorecard is not a standalone assessment of how good a doctor I am."

Bechard remains firm. "If you can't quantify it; if you can't measure it, you can't improve it. Can you grade a doctor on this alone? No. There is a lot that goes into being a good doctor," he says.

Milstein, the San Francisco expert, supports Bechard. "There are some facets of interpersonal care and diagnostic nuance that remain beyond our ability to measure. But the bulk of the value physicians provide to patients can be evaluated by quantified metrics," he says. "As measures get better, that subset of services that defy measurement will continue to decline."

Borell and others developed Meridian's scorecard to refute information about its doctors by others that may be incorrect. To date, they have found none.

Maureen Glabman is a Miami-based health care reporter and recipient of the 2000 Reuters Fellowship in Medical Journalism at Columbia University's Graduate School of Journalism. She can be reached at [e-mail]

RELATED ARTICLE: History of Scorecards

The earliest scorecards were created in the early 1980s by East Coast health plans, (like US Healthcare, now part of Aetna), says Arnold Milstein, MD, a nationally known scorecard expert with Mercer Human Resource Consulting, San Francisco. "The focus was on measuring quality and resource use."

US Healthcare pioneered the physician bonus system, the precursor to pay for performance. In the late 1980s, as health plans began paying medical groups a fixed payment for members, i.e., capitation, large California practices, like Sharp Rees-Stealy, started doing the same thing--comparing physicians, providing feedback and offering economic incentives for improvement.

In technological developments, a group of Rochester, NY-based ex-Blue Cross employees created comparison software used by health plans and IPAs, called Peer-A-Med. DRG's precipitated hospital use of software to track simple quality measures and length of stay. Adjustments for severity of illness and comorbidities started to develop just after DRGs in the 1990s.

In the mid 1990s, says Milstein, New York state broke the ice with public reporting of physician performance through its release of risk-adjusted death rates for cardiac surgeons.

About that time, US Healthcare began making its profiles available to patients through a program called, "Pick A Doc." "Software became more sophisticated, comparing doctor performance over a longitudinal period of time, such as a year's worth of chronic care," Milstein says.

RELATED ARTICLE: Five Common Complaints About Scorecards

1. Doctors can "game" the profiling system by refusing to take on complicated patients with comorbidities that might extend their length of stay or skew mortality ratings.

[ILLUSTRATION OMITTED]

2. Patients of group doctors see multiple doctors from the same group during a single hospital stay, though all records of the admission are attributed on scorecards to the admitting physician, rendering the information inaccurate.

3. Physicians fear plaintiff attorneys could subpoena the scorecards to use against them in a medical malpractice action.

4. Ratings for length of stay are attributed to doctors but do not take into account hospital errors, such as lost orders, orders not completed in a timely manner, equipment failures, staffing and scheduling problems.

5. Some patients do not fit neatly into evidence-based medicine protocols. Doctors who depart from the protocols for good reasons are rated poorly.

RELATED ARTICLE: Meridian Health

Meridian Health's three hospitals could not be more different. Located in Monmouth and Ocean counties in New Jersey, the combined institutions have annual revenues of $725 million, 7,500 employees, and 1,750 physicians.

[ILLUSTRATION OMITTED]

Riverview Medical Center in Red Bank is in a bedroom community of New York City. It has the smallest number of Medicare and Medicaid patients--about 40 percent. Patients are mostly middle and upper-middle class. Many commute to Manhattan and are covered by employer health plans.

Jersey Shore University Medical Center, in Neptune, is a teaching hospital for Robert Wood Johnson Medical School. Patients come from all over the state for the cardiac facility, which performs a large volume of open-heart surgeries, 800-900 annually. The hospital also has a Level 2 trauma center that has the third or fourth largest volume in the state. There are many indigents. About 55 percent of all patients are either Medicare or Medicaid patients.

Ocean Medical Center, in Brick, is located in a middle-class retirement community and a deeply religious Jewish community where families are known to have many children. There are a high number of Medicare recipients, more than 60 percent.

Meridian Health is a 2005 winner of the John M. Eisenberg Patient Safety and Quality Award given by the National Quality Forum and the JCAHO. The system won for innovation in patient safety and quality at a local or organizational level. Just months after embedding evidence-based best practice guidelines into its computer-based physician order entry system, Meridian physicians embraced the use of the interactive practice guidelines for online ordering, resulting in the implementation of a dozen best practices into the system.

Figure 1 Scorecards like this one used by Meridian Health are gaining
popularity. But critics say scorecards can be gamed. "You can lower your
mortality by walking away from the sickest patients, by referring them
to other doctors," says one physician.

Dr.Jane Doe -- Cardiologist

Quality Measures                           Physician
                             Jan-Dec                  Jan-Dec
Indicator Description        2004        Num/Denom    2003

AMI -- Aspirin at Arrival    (d) 100%    12/12         88%
AMI -- Aspirin at Discharge  (d) 100%    2/2           50%
AMI -- ACE Inhibitors for    n/a         n/a          100%
  LVSD
AMI -- Beta Blockers at      (d) 100%    2/2          100%
  Discharge
AMI -- Beta Blockers at      (b) 91%     10/11         86%
  Arrival
HF -- LVF Assessment         (d) 100%    13/13        100%
HF -- ACE Inhibitors for     (d) 100%    4/4           67%
  LVSD
SIP -- Prophylactic          n/a         n/a          n/a
  antibiotic received
  w/i 1hr prior to
  surgical incision
SIP -- Prophylactic          n/a         n/a          n/a
  antibiotic
  discontinued w/i 24hrs
  after surgery end time
CPOE                         (b) 26-50%  1,541/4,231  26-50%

Quality Measures             Physician    Group      Specialty  Hospital
                                          Jan-Dec    Jan-Dec    Jan-Dec
Indicator Description        Num/Denom    2004       2004       2004

AMI -- Aspirin at Arrival    7/8          97%        93%        88%
AMI -- Aspirin at Discharge  1/2          100%       94%        91%
AMI -- ACE Inhibitors for    1/1          n/a        100%       89%
  LVSD
AMI -- Beta Blockers at      2/2          86%        94%        93%
  Discharge
AMI -- Beta Blockers at      6/7          93%        85%        84%
  Arrival
HF -- LVF Assessment         17/17        98%        95%        96%
HF -- ACE Inhibitors for     2/3          100%       88%        88%
  LVSD
SIP -- Prophylactic          n/a          n/a        n/a        83%
  antibiotic received
  w/i 1hr prior to
  surgical incision
SIP -- Prophylactic          n/a          n/a        n/a        59%
  antibiotic
  discontinued w/i 24hrs
  after surgery end time
CPOE                         2,702/7,866  0-25%      0-25%      0-25%

Quality Measures                     Benchmarks
Indicator Description        Threshold  Target       Maximum

AMI -- Aspirin at Arrival    95%        98%          99%
AMI -- Aspirin at Discharge  96%        98%          99%
AMI -- ACE Inhibitors for    81%        95%          98%
  LVSD
AMI -- Beta Blockers at      91%        97%          99%
  Discharge
AMI -- Beta Blockers at      91%        95%          98%
  Arrival
HF -- LVF Assessment         94%        98%          99%
HF -- ACE Inhibitors for     85%        93%          97%
  LVSD
SIP -- Prophylactic          69%        85%          92%
  antibiotic received
  w/i 1hr prior to
  surgical incision
SIP -- Prophylactic          46%        75%          88%
  antibiotic
  discontinued w/i 24hrs
  after surgery end time
CPOE                         26-50%     51-75%       76-100%

Clinical Effectiveness               Physician
                          Jan-Dec     *Eligible Cases  Jan-Dec
Indicator Description      2004       Num/Denom        2003

Overall Mortality         (b) 3.3%      4/121          5.5%
All Payer Overall         (b) 4.53    548/121          4.65
  Average Length of
  Stay
ALOS for Highest Volume   (c) 5.44     87/16           5.00
  APR-DRG: 194 -- Heart
  failure
Medicare Overall Average  (b) 5.06    349/69           5.01
  Length of Stay

Clinical Effectiveness    Physician    Group      Specialty  Hospital
                                       Jan-Dec    Jan-Dec    Jan-Dec
Indicator Description     Num/Denom    2004       2004       2004

Overall Mortality             7/128    3.2%       3.1%       3.6%
All Payer Overall           595/128    4.97       4.83       4.87
  Average Length of
  Stay
ALOS for Highest Volume     100/20     6.00       5.88       6.16
  APR-DRG: 194-Heart
  failure
Medicare Overall Average    411/82     5.52       5.38       5.97
  Length of Stay

Clinical Effectiveness               Benchmarks
                            Sev Adj      Sev Adj      Sev Adj
Indicator Description       NJ Avg     NJ Top 25th  NJ Top 10th

Overall Mortality             4.2%       2.0%          0.8%
All Payer Overall             5.01       4.09          3.44
  Average Length of
  Stay
ALOS for Highest Volume       7.22       6.04          5.43
  APR-DRG: 194-Heart
  failure
Medicare Overall Average      5.27       4.28          3.61
  Length of Stay

Finance                                Physician
                           Jan-Dec                   Jan-Dec
Indicator Description      2004       Num/Denom      2003

Clinical Denials-% of      (d) 3.4%    6/174         19.4%
  Managed Care Denied
  Days
Lab-CT Ratio               (d) 1.09    12/11         1.70
Overall Direct Cost per    $2,720      125 cases     $2,519
  Admission

Finance                   Physician    Group      Specialty  Hospital
                                       Jan-Dec    Jan-Dec    Jan-Dec
Indicator Description     Num/Denom    2004       2004       2004

Clinical Denials-% of     25/129       5.8%       10.6%      7.3%
  Managed Care Denied
  Days
Lab-CT Ratio              17/10        1.15       1.41       2.00
Overall Direct Cost per   128 cases    $3,228     $3,225     $3,526
  Admission

Finance                           Benchmarks
Indicator Description     Threshold  Target       Maximum

Clinical Denials-% of     6.3%       6.1%         5.9%
  Managed Care Denied
  Days
Lab-CT Ratio              1.60       1.50         1.40
Overall Direct Cost per
  Admission

*Eligible cases are only those cases that can be benchmarked; may vary
from actual cases.
(a) Below Threshold
(b) Between Threshold and Target
(c) Between Target and Maximum
(d) Meets/Exceeds Maximum