Using a benchmarking system to improve patients care and assist in technology assessment.

The earliest examples of benchmarking date back to military strategies for reconnaissance hundreds of years ago. When the armies of Alexander the Great surveyed the strength and position of the enemy and revised tactical and strategic decisions, they were benchmarking best practices.

The Xerox corporation is credited with beginning formal benchmarking in this country in 1979. At the time, its market share had fallen from 80 percent to 35 percent because of intense competition. The company began a process of "product quality and features comparisons." It identified differences in the manufacturing process with its Japanese subsidiary, Fuji-Xerox. In addition, it compared its distribution processes to that of L.L. Bean in this country. Since beginning this program, the company's annual productivity has increased from less than 3 percent to more than 10 percent.

Over the past two decades, a number of other major corporations, such as, Southwest Airlines and Eli Lilly, have conducted ongoing analysis and comparisons of performance of practices, products, and services with those of major competitors. Examples of areas that have been the focus of strategic benchmarking include customer service, distribution, manufacturing, operations, finance, human resources, and marketing.

Types of Benchmarking

As shown in table 1, page 11, there are five types of benchmarking that an organization can undertake. Depending on the industry, the environment, and available resources, comparisons can be either internal or external. With the formation of new networks and alliances in the health care industry, there will be increasing opportunities for sharing and comparing information among organizations in these systems. For example, the 65 academic medical centers that make up the University Health System Consortium are currently involved in a number of clinical and operational benchmarking projects that compare information for high-volume or high-cost areas.

Table 1. Types of Benchmaking

Internal      Comparisons of similar processes within the same
              organizations.
Competitive   Comparisons with the toughest competitors in the
              same business.
Functional   Comparisons to world-class companies in the same
             industries.
Group        Sharing and comparison of information among
             organizations in an alliance.
Generic      Evaluation of organizations with outstanding,
             noteworthy performance in similar processes that
             are not necessarily in the same service industry
             or business.

Clinical benchmarking as a tool of CQI seeks to improve outcomes and reduce unnecessary variation by identifying and establishing "best practices" instead of focusing on care that falls below an established threshold, as is done in traditional quality assurance activities. Traditional quality assurance seeks to identify hospitals and providers that demonstrate behavior that is not normative and eliminate it. This approach is accomplished through establishing what is often an arbitrary threshold to separate acceptable quality that does not require corrective action from unacceptable quality that does. The intended goal of the thresholds is to identify poor quality events or providers. In doing so, the thresholds simply separate "bad apples" and seek to improve them without improving the entire process. In order to be successful, clinical benchmarking should focus on comparing an organization's current performance with what could be achieved if improvements in the process of care that have been identified through benchmarking were implemented.

Identifying Areas for Clinical Benchmarking

The area selected for benchmarking should fit at least one of the following criteria:

* Be of strategic importance to the organization.

* Represent a competitive area of service.

* Be a critical success factor for the organization.

* Represent a problem area for the group.

* Be significant in terms of quality, cost, time.

Our strategy has been to identify fundamental measures, such as average length of stay, that can be improved. Given the increasing emphasis on cost reduction, this particular benchmarking satisfies a number of the selection criteria identified above. Furthermore, length of stay allows for benchmarking among providers in the same institution as well as with tough competitors or members of a health care alliance.

In undertaking a clinical benchmarking project, it is important to specifically identify a benchmarking target and demonstrate its feasibility. Before beginning the process, it is crucial to gain physician participation and administration support while identifying benchmarking partners. After the benchmarking project is selected by the organization, the data elements that characterize the process of care must be well defined so that they can be collected and compared. The institution must utilize these data to identify processes to be improved and then compare the processes to similar ones in other institutions in order to determine which practices make a difference. In order to be successful, the focus must be on what could be achieved if the improvements are implemented instead of on the gap between the institution and other organizations.

New practices should be developed into clinical pathways--written guidelines that describe the process of care in terms of sequence and time in order to map its progression. Clinical pathways should have at least two indicators of outcome in areas such as quality, cost, and patient satisfaction. As data are collected, the pathways should be reviewed at least quarterly and continuously improved. This process of plan, collect, analyze, and adapt is summarized in tables 2 and 3, page 12.

[TABULAR DATA 2 and 3 OMITTED]

Benchmarking Pitfalls

While conceptually clinical benchmarking is often straightforward and simple, there are several pitfalls that should be avoided in order to improve the likelihood of a successful outcome.

* Benchmarking is conducted as a one-time event. The process cannot end with the first successful review of outcome indicators. It must become part of a day-to-day routine.

* Support and attention from medical staff and administrative leadership are lost After the fanfare, there is little interest, training stops, funding is lost, teams cannot visit other sites, and recommendations are met with indifference. Any process that facilitates continuous improvement in quality must have continuous support.

* The best practices of another organization are directly adapted. This never works and is not benchmarking. Information collected from a benchmark partner must be integrated with other information and considered in the environment in which it will be used. Benchmarking is not imitation or copying; it is an opportunity to learn and assimilate new data.

* It is seen as easy and quick. Without adequate investment of planning, funding, instruction, staff time, and support, the return from a project will be little if any.

* Benchmarking is viewed as just a fad. Benchmarking must be integrated into an organization. It cannot just be endorsed by leadership, it must be practiced by it. To be successful, it must be one of the quality tools used by an organization. Employees must be supported in its use, and the organization must integrate it into its decision-making process.

* Things are measured differently for the organization being benchmarked and other sources of data. A constant set of "metrics" must be used. Data must be comparable. This extends from case mix indexing to defining a FTE.

Harold L. Paz, MD, MS. is Dean, UMDNJ Robert Wood Johnson Medical School, and CEO, University Medical Group, New Brunswick, N.J. Jean Livingston, RN, MSN, is Associate Director, Clinical Evaluative Science Program, University Health System Consurtium, Oak Brook, Ill. The authors may be reached through Dr. Paz at 125 Paterson St., Suite 1400, Clinical Academic Bldg., New Brunswick, N.J. 08901, 908/235-6300, FAX [phone].