Awards recognizing quality in health care.

In the postwar recovery period, the Deming Award[1] was established in Japan to recognize organizations that excelled in meeting the needs and expectations of customers with high-quality products and services. Eventually, companies in this country were awarded Deming prizes. The Malcolm Baldrige National Quality Award was created here in 1987 to promote awareness of quality excellence, to recognize quality achievements of U.S. companies, and to publicize successful strategies. Individual states have established their own quality awards. For example, New York has had the Governor's Excelsior Award since 1991, recognizing the public sector, the private sector, and the education sector. In 1995, 10 states, including New York, had awards for the health care sector. The Malcolm Baldrige Award has created pilot teams for health care and education sectors as a prelude for formal awards. Those who have worked in health care have always been comfortable with the thought that we are providing high-quality health care. We tend to look at the outcome of patient care with an eye toward a litigious society that has caused the costs of professional liability insurance to go into orbit. Thus, we took great comfort in the results of the Harvard Study[2] in New York State. This proved that the results of care were satisfactory in more than 96 percent of cases and actual negligence was on the order of one to two percent. We were quite comfortable, therefore, that we were providing high-quality care.

Berwick[3] has sensitized us to the need for process analysis as a change from just looking for the "bad apples." Total quality is management of all the processes involved in health care. A large adjustment has to be made in the thinking and attitude of those who have been comfortable with the traditional concept of quality in health care. In the examination of an application for a health care award such as the Baldrige or the Excelsior, the total system that provides care is evaluated. No one will receive a 98 percent rating based on a review limited only to patients, freedom from negligent care.

This more global view of quality has to be understood in light of the many changes coming with health care reform. We have to make the difficult shift from income to knowing the actual cost of providing care for each procedure. This will be possible when there's an understanding of all the processes involved in providing care and not just lump sum guesses about the cost of categories of care. Cost will be managed as a result of improving the several processes involved in all care. Savings will result from improvement of these processes, and ratings will be based on a review of the total care effort. As we seek to improve the quality of care, we must concentrate on improving processes and not just react to problems. Quality has to move beyond problem solving to process improvement.

The Evaluation Model used in these award programs is divided into three areas. First, the applicant is evaluated to determine the role of Senior Leadership in enabling the creation and continuation of a total quality program. Second, a system has to be established that consists of information and analysis, strategic quality planning, human resource excellence, and management of process quality. Third, the goal of customer or constituent satisfaction is achieved as verified by the measures of progress found in quality and operational results. Within each of these categories there are two to seven more specific criteria. Comparison of the Baldrige4 and Excelsior5 programs reveals some differences in emphasis in these criteria. Similar differences are found in the total possible score for each criteria. For example, organized labor has been supportive of the Excelsior program in New York State and the role of labor in the functioning of Excelsior applicants is considered specifically. While such differences are to be expected and are appropriate, the main thrust of all the programs is consistent with the objective of evaluating the level of quality achievement by each applicant.

To expand on this concept of total involvement by an applicant organization, consider some generalizations about the different categories.

* Leadership. A quality program will never succeed without strong support and commitment of senior leadership. This means excellent communication, good relations with employees, providing an example, and giving direction consistent with strategy and plans. There must be a demonstration that this involves all levels in commitment to quality methods, with cooperation and continuous evaluation and improvement of all efforts. In the community, the organization must integrate its public responsibilities with its quality values and be a leader as a corporate citizen.

* Information and Analysis. Meaningful data have to be collected and analyzed to allow understanding of what is being done, how this activity relates to goals and planned strategy, and where modifications will result in improvement. Competitive comparisons and benchmarking become increasingly essential as progress is made. The relation of quality efforts to customer satisfaction, development of priorities, and related operations needs to be indicated. This has value in determining possible changes in methods of analysis.

* Strategic Quality Planning. This is all about how the organization develops strategies and business plans to satisfy customers, quality concerns. There has to be a description of how plans are developed and then deployed, followed by evaluation after application and further improvement. The scope of planning has to be both short- and long-term. There has to be an idea of what resources will have to be committed to achieving the goals contained in the plans.

* Human Resource Development and Management. This examines how the work force is enabled to develop its full potential to achieve the goals of the organization. Efforts to build and maintain an environment for quality excellence must be noted. The criteria include resource planning and management, employee involvement and empowerment, employee education and training, employee recognition and performance, and employee well-being and satisfaction.

* Management of Process Quality. This looks at how new products and services are selected and developed, along with how all services and products are delivered, evaluated, improved, and analyzed. This requires an evaluation of how the several parts of the overall operation relate and contribute to each other and to results. There has to be recognition of support services and the quality of suppliers. This area needs to indicate the overall assessment of quality of all the parts of the operation and how the assessment is carried out.

* Quality and Operational Results. This is where achievement levels and improvement trends are described and calls for comparisons with competitors. There has to be consideration not only of overall results, but also of business and support results and supplier quality results. This is the area where traditional health care outcomes will receive primary attention, but the entire process in reaching these outcomes has to be included in the evaluation.

* Customer Focus and Satisfaction. The patient is the primary customer in health care, and there needs to be documentation of more than simply the objective attainment of a treatment result. Patients have both current and future expectations. There has to be a good relationship between the patient and the health care organization and its personnel. This includes preventive care, counseling, and education. The organization must be able to demonstrate its commitment to its patients and to the community. The satisfaction of all customer groups must be noted and related to results obtained by competitors.

An application is provided by the award program after a determination of an organization's eligibility for an award. The application format is a written document with a limit on the number of pages that can be used to provide answers and details. Because there is overlap and linkage among the application categories, answers need to be crafted to be complete and balanced.

Application is a completely voluntary step. Confidentiality is stressed. Each examiner is required to avoid any indication to anyone about who an applicant is and what information is contained in the application. All applications are returned on completion of the evaluation process. Release of information is totally controlled by the applicant both before and after the process. If there is a site visit, all notes made are collected and destroyed to further ensure confidentiality. Examiners are prohibited from discussing details of their activity after their work is finished. Extreme efforts are made to avoid conflicts of interest on the part of examiners. There is a sufficient number of examiners to make this possible.

With the application received, the first stage consists of a review by five to eight examiners who independently go over the information in the written application and score the answers. This scoring activity includes preparation of an overview of all the business factors that are important in the activity of the applicant. The answer for each category and each item within a category are studied. Strengths described in the answer and areas for improvement are noted. A standard scoring system is used to assign a value to indicate the degree of quality for each item studied. This same approach is used at all stages that follow. Objectivity is a paramount goal of the examiners. In particular, when areas for improvement are noted, they are described in terms of what can be done, not how, why, or when. The area is merely noted, and no comment is intended to be a prescription for action on the part of the applicant.

The initial review is followed by an evaluation by judges to determine which applications merit further study and which applications are unlikely to have high enough scores to be given an award. If an applicant is not passed to the next stage, a feedback report to the applicant indicates areas of strength and areas for improvement. This can be very important in pointing out a lack of balance in an applicant's quality effort, a need for additional experience that will come with sufficient data, for example, to indicate trends, and other efforts in process improvement that go beyond just problem solving.

The next stage consists of a meeting of examiners to review applications that merit additional consideration. Examiners share their evaluations and concepts and arrive at a consensus about each and every item. Each examiner initially will have put in 2040 hours of study and evaluation. The exchange of information that comes out of a consensus meeting allows the development of a score for the application. Ideally, there will be unanimity, but a scorecard that indicates what the final decision of each examiner was will reveal whether the consensus score also represents averaging or acceptance of the median of a range of values.

These consensus scores go to the judges, who decide which applicants are to be selected for site visits. Most important at this consensus stage, again, is the preparation of feedback reports that go to applicants who are not selected for site visits. This continues the process of objective assessment to be used by the applicants for their own efforts at continued improvement.

At each stage preceding the site visit, each examiner indicates those issues he or she feels can be resolved only by a site visit. A site visit team of five to eight examiners is developed and holds a preliminary meeting to define the issues that will be investigated at the site visit. There are two purposes for the site visit. As the application has been considered, issues have been raised for which more information would be desirable. The first purpose is to verify that information in the application is valid. Files can be reviewed, personnel interviewed, and inspections made to confirm that reported items are accurate and complete. This step is not designed just to detect exaggeration of claims, but also to reveal additional supporting information that was missing from the application. The second purpose is to clarify information in the application. Conflicting statements or incomplete statements of fact can be investigated. The degrees of difference between what has been written in an application and what exists may not otherwise become known. This is not to say that a site visit is a search for misrepresentation. It is an acknowledgment that skills vary in organizing and writing applications when there are so many items to be presented and there is a limit on the number of pages to cover everything.

Again, a feedback report is sent to applicants who receive site visits. There is no additional scoring because of the site visit, but the team can recommend that scores be increased or decreased as a result of the visit. The team prepares a summary report that brings together all of the results of study to date. These reports are considered by judges who recommend which applicants receive awards. This becomes necessary when the number of awards in each category is limited, or if there is a judgment that no one should be declared a winner in a given year. There is a move in award programs toward recognizing more applicants who evidence high-quality operations. For example, the New York Excelsior award may now recommend that certain applicants be recognized for "exemplary practices" in one or more categories.

A lot of work has to be done to gather the information needed to prepare an application. This necessitates an examination of what is going on at all levels and in all areas of an organization. These observations are linked to conclusions that everything is not exactly what might have been assumed. It is this self-evaluation that moves an applicant to a winning position. When the self-evaluation is reinforced by a feedback report prepared by independent examiners, the value of the process is multiplied. If completing the process is a learning experience for an applicant, improvement should result. It is reasonable to expect that repeat applications will be made as applicants return to the process for confirmation of desired progress.

As health care organizations begin to apply and are evaluated, the clinical results of many will be good to excellent, but deployment of total quality management to business and support areas will be limited and the approach to total quality will be anecdotal or represent beginning efforts at a systematic program of improvement. Some organizations will have excellent results because of their approach and deployment of quality methods. We need to identify these organizations, which will then be able to serve as benchmarks for others that seek to improve.

Health care organizations are encouraged to learn more about the Malcolm Baldrige Award and other awards such as the New York State Governor's Excelsior Award. We have always taken pride in the quality of health care in this country. These award processes are a method to measure, understand, and appreciate health care quality.

References

[1.] Walton, M. The Deming Management Method. New York, N.Y.: Perigree Books, Putnam Publishing Group, 1986. [2.] A Report by the Harvard Medical Practice Study to the State of New York. "Patients, Doctors and Lawyers: Medical Injury, Malpractice Litigation, and Patient Compensation in New York." Cambridge, Mass.: President and Fellows of Harvard College, 1990. [3.] Berwick, D. "Continuous Improvement as an Ideal in Healthcare." New England Journal of Medicine 320(1):53-6, Jan. 5, 1989. [4.] Malcolm Baldrige National Quality Award, 1994 Award Criteria, U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technology, Gaithersburg, Md., 1994. [5.] The Governor's Excelsior Award, 1994 Application Guidelines, New York State Department of Labor, Albany, N.Y., 1994