Toward a definition of quality.

(Tables and illustrations not shown)

Providers of health care are concerned about their ability to continue providing high-quality care in the face of diminishing resources. Purchasers of health care are concerned that the cost has exceeded their capacity and willingness to finance it. Both need more quantifiable measures of quality; both seek ways of assessing value.

While measures of direct cost may be identifiable, measures of quality have been more elusive. In part, their elusiveness has been fostered by nonagreement on what the definition of quality of health care truly is. In part, defining quality has been complicated by the subjective nature of some of the components of that definition. Descriptions of quality of care vary with the point of view and role of the observer, who may be a patient, a clinician, a purchaser, or a manager. Descriptions of quality also depend on the clinical setting, on patient expectations, and on severity of illness.

The purpose of the study reported in this article is to determine if there can be agreement on the relative importance of nine elements of a quality of care definition as evaluated by physician executives who manage health care institutions. A second purpose of the study is to determine if the relative importance of these elements is different in different clinical scenarios.

Previous Studies

Benson and others established 10 parameters for evaluating quality in the ambulatory care setting. Their evaluation matrix included practitioner performance, appropriateness, accessibility, satisfaction, continuity, minimization of risk, staff performance, and others.

Forty-eight reviewers for the Joint Commission on Accreditation of Healthcare Organizations were asked to rank and weight the 10 parameters according to their impact on patients' health care status. Benson's work reveals two important concepts: reviewers could reach statistically significant agreement on weighting the importance of the parameters, and some parameters were found to be twice as important as others.

Panniers found that there is a correlation between severity of illness and quality of care; a decrease in quality was associated with an increase in severity of illness.

Elbeik's review of perceptions of hospital care found that technical skill, interpersonal relationships, access, continuity, and coordination of care are important components of quality. His study measured the relative importance of diagnosis, treatment, care, and education from the perspective of clients, administrators, nursing staff, and medical staff. Significant differences in the weighting of these parameters were found among the four groups.

Relative values for 13 components of quality of care index for family planning have been suggested by Hirschhorn and others.

Study Design

Nine components of a quality of care definition were selected for study. Availability of health care services and access to them are commonly assumed to be part of that definition but were not assessed in this study. Six clinical scenarios were defined. The scenarios differ in the severity of the inness and the occurrence (absence) of death during the hospitalization.

Members of the American Academy of Medical Directors (AAMD), now the American College of Physician Executives, were randomly selected as evaluators. Eighteen hundred members were divided into three cohorts. Each cohort was asked to rank the relative importance of the nine components in assessing the quality of care for a control scenario and two of the six clinical scenarios. Total weights for each scenario total 100, but respondents could assign any weight to any element, including a weight of zero.

Statistical Design

Five statistical examinations were performed. Kendall's coefficient of concordance (W) was used to determine if the respondents agreed on rankings and if there was agreement between ranks and weights. The Mann-Whitney test was used to determine if there was a difference in ranking of elements between scenarios. Differences in weights between scenarios were tested with a "T" test. Multiple regression analysis sought correlations between the demographics of respondents and their choice of ranks and weights.

Results

There were 568 responses (31.5 percent response rate) from physician executives in 639 management sites, representing hospitals (51.5 percent), HMOs (21.8 percent), group practices (21.88 percent), and other (9.5 percent). Most respondents are engaged in at least half-time management (74 percent); many are fulltime (32 percent). All responded to the control scenario. The numbers of responses to the clinical scenario are unequal:I(87), II(102),III(37),IV(52), V(49), and VI(51).

Control Group

The attributes, activities, and capabilities encompassed by "physician-technical" were found to be the most important element in terms of both rank and weight (6.2 to 18.3). Severity of illness was the second most important element in assessing quality.

Clinical Scenarios

There is agreement among respondents on the elements. Kendell's coefficient of concordance (W) ranged from 0.18 to 0.41 (P < 0.001). The most agreement occurred in ranking elements in scenario six (W = 0.4141), where a severely ill patient dies. There was also a modest amount of agreement in assigning weights. Kendall's coefficient ranged from 0.23 to 0.68. Again, the most agreement was observed for scenario six. No element received a zero weight, although "hospital-subjective" showed the greatest variation of weightsand the lowest coefficient of concordance (W = 0.25).

Scenario Comparisons

Comparing scenarios with different severities of illness upon admission and different outcomes as measured by mortality had little effect on the relative rank order of the elements. Physician technical skills, hospital technical capabilities, and the presence or absence of complicadons remain most important,in that order. Hospital subjective attributes and client satisfaction remain the lowest ranked indicators.

Changing scenarios does have a significant effect on the relative importance of the elements, as manifested by weights. Comparisons of scenarios three and four and scenarios five and six hold severity of presenting illness constant while outcomes worsen from discharged alive to in hospital death. Three significant changes were found (T test, P < 0.001). For patients who were moderately HI, the weight of physician's technical expertise increased by 25.4 percent, the importance of the hospital's technical capabilities increased by 22.4 percent, and continuity measures fell 36.1 percent. For patients who were critically ill upon admission, there were nonsignificant changes among the parameters measured, even though the outcome was far worse.

Comparisons of scenarios three and five and scenarios four and six hold outcome constant while the severity of presenting illness increases from moderately ill to critically ill. Four significant changes were identified (T test, P < 0.001). For patients discharged alive, the importance of physician technical expertise increased 34 percent, the weight of hospital's technical capabilities increased 24 percent, and both hospital subjective and client satisfaction decreased. For patients who died, there was no significant difference in the relative weights as the severity of presenting illness increased.

There was no correlation (multiple regression, P < 0.001) between the demographic data collected on respondents and their choices of ranks and weights.

Conclusions

A complete description of quality of health care requires a multifactorial definition that encompasses numerous elements. Reliance on mortality rates that do not consider severity of illness is misleading; overemphasis on client satisfaction without factoring in complications and continuity may be myopic. While our survey did not offer to its participants the option of adding additional definitional elements to those studied, neither did it preclude a participant from assigning no value to any of those reviewed.

Determination of quality of health care seems to require a multivariable eqation solved simultaneously by several publics. Physician executives, the professional now responsible for making management decisions on quality for health care institutions, concur that assessing quality requires the inclusion of many elements.

The study demonstrates that all important elements in defining and measuring quality do not have the same rank order of importance. The clinicians "technical" preparedness and ability remained most critical to the evaluation process regardless of outcome or severity of illness. Physicians' capabilities and performance in these matters is assessed by training, board certification, peer review, experience, utilization review, liability history, conformity with quality assurance standards, and outcome indicators. They account for up to 29 percent of the total composite of this study's description of quality measures. The importance of assessing the clinician's technical ability increases significantly as the severity of illness increases and remains the signal element for evaluating quality if there is a patient death. Because weightings change between scenarios three and four, the study suggests that evaluations of quality that do not assess severity of illness may be incomplete and misleading. Indeed, mortality ranked fifth and weighted sixth among the nine measured elements in the control group. The necessity of adjusting for severity of illness prior to meaningfully comparing mortality rates has also been observed by Dubois and others.

Client satisfaction and hospital subjective attributes such as ambiance, "caring attitude," and reputation consistently ranked lowest. The ranking does not change with the outcome of care or with the severity of illness. These results are similar to those in several previous studies of the importance of patient satisfaction in assessing quality. Benson and others found that patient satisfaction received a weight only one-third of physician performance in the outpatient setting. Jensen reported that environmental attributes such as private rooms, good surroundings, and courteous employees were not ranked high by consumers selecting from a list of subjective factors used in selecting a hospital. Satisfaction surveys were found to be deceptive by Cassileth, who found that customers can assess hospitals with considerable sophistication and objectivity prior to admission but revert to using affective qualitative terms once admitted. Patients also tend to be biased toward the "completely satisfied" end of the evaluation scale.

Although the order of importance of the nine elements did not vary greatly with clinical circumstances, their importance relative to one another did change. The importance of physician and hospital technical capabilities and performance increases with the severity of illness and with the worsening of outcome. Their importance remains high in assessing quality among severely ill patients who die. These conclusions seem supported by the observation that severity of illness is a heavily weighted variable within the control group for whom neither outcome nor severity were specified. Physician executives in this study conclude that, to fully assess quality, the patient's input to the process as manifested by severity of illness is an important variable. The study demonstrates that a working agreement on the relative importance of nine elements to the quality definition can be achieved within at least one group of participants. Other publics have demonstrated wide differences of opinion on the quality of health care and is assessment. These parties in the health system will need to have input to the process of defining quality and to assigning importance to its components. Further studies that seek to test the hypothesis that different publics can agree on a definition should be pursued.

Further Read

The following additional sources of information on quality measurement were obtained through a computerized search of databases. Copies of the articles cited are available from the College for a nominal charge, For further information on citations, contact Gwen Zins, Director of Information Services, at College headquarters, 813/287-2000.

Ammentorp, W., and others. "Assuring Quality Care: Measuring and Monitoring the Caring Process." Journal of Quality Assurance 10(2):10-2, Spring 1988.

Berwick, D. "Toward an Applied Technology for Quality Measurement in Health Care." Medical Decision Making 8(4):2538, Oct.-Dec. 1988.

Fanizzo, B., and Kehl-Rose, K. "Management Scheme, Profile Can Produce Measurable Quality." Michigan Hospitals 23(10) :7-12, Oct. 1987.

Nathanson, P. "Measuring and Marketing Quality: How To Stan." Health Care Strategic Management 7(2) :12-4, Feb. 1989.

O'Leary, D. "Develop Relevant Internal Performance Measures To Evaluate Ouality of Care." Trustee 40(8) :14-5, Aug . 1987.

Payne, B., and others. "Relationships of Physician Characteristics to Performance Quality and Improvement."Health Services Research 19(3):307-32, Aug. 1984.

Sommers, L., and others. "Physician Involvement in Quality Assurance." Medical Care 22 (12):1115-38, Dec. 1984.

Vuori, H. "Patient Satisfaction-An Attribute or Indicator of the Quality of Care?" ORB 13(3):106-8, March 1987.

Wyszewianski, L. "Ouality of Care: Past Achievements and Future Challenges." Inquiry 25(1) :13-22, Spring 1988

Wyszewianski, L. "The Emphasis on Measurement in Quality Assurance: Reasons and Implications." Inquiry 25(4):424-36, Wnter 1988.

Input

1. Clinical severity

"Moderately ill" encompasses the clinical range from elective plastic surgery to minimally justifying admission by PRO, HMO, or other third-party criteria.

"Severely ill" encompasses those clinical situations when admission to an acute care hospital would easily be justified, the need for care is urgent, and some risk of unfavorable outcome can be anticipated.

2. Physician-technical

Capabilities and characteristics of physicians as reflected by training, board certification, continuing education, liability history, medical records, and level of activity (numbers of procedures performed or diagnoses made).

3. Hospital-technical

Licenses, accreditation, and other"official evaluation of institutionalcompetence. " Examples include JCAHO findings, university affiliation, current technology, nurse staffing ratios, level of activity (numbers of procedures performed), and liability history.

Process

4. Physician-art

Art of care characteristics of physicians as reflected by peer review findings, rapport with patients, availability, listening, and instructing.

5. Hospital-subjective

Hospital characteristics include staff attitudes, service orientation, appearance, ambience, guest amenities, reputation, and empathy.

6. Continuity of care

The degree to which a comprehensive spectrum of health care services exists. This includes services such as education, preadmission planning, discharge planning, and follow-up care, as appropriate.

Outcome

7. Mortality

Patient outcomes resulting in death.

8. Morbidity

Complications or other outcomes that are less than ideal occur. Examples would include hospital-acquired infections; returns to surgery within 24 hours; repeated procedures; lengths of stay more than 2 standard deviations above the mean; and, if discharged, delayed returns to work or readmissions for the same diagnosis.

9. Customer satisfaction

Evaluation (both subjective and objective) of patient's care from the perspective of lay parties such as patients, families, employers, and/or other purchasers.