THE IMPACT OF CHANGE IN PHYSICIAN REMUNERATION ON CLINICAL PRACTICE: A FEASIBILITY STUD Y FOCUSSING ON UPPER GI ENDOSCOPY.

ABSTRACT

In July 1994 clinicians in the Southeastern Ontario Health Sciences Center (SEOHSC) in Canada replaced fee-for-service billing with an Alternative Funding Plan (AFP) that provided block funding to SEOHSC. Both SEOHSC and Ontario's Ministry of Health anticipated benefits from the change despite limited evidence that funding changes have any impact on clinical practice. The main objective of this study was to establish a method for assessing the impact of a change in physician remuneration on clinical practice. Specifically, the relationship of changes in the volume and clinical practice of upper gastrointestinal (GI) endoscopy to the different funding environments was examined.

A longitudinal, pre-post design using discharge data and et control site were employed. The data provided procedure-specific information for four years surrounding the AFP implementation (n = 21,037). Primary data from local endoscopists helped defined the setting, confounding factors, and diagnosis measures. The main measures were volume and patient types. Patient types were categorized by relevant outcome diagnoses: Important/Group A (unquestionably beneficial) or Group B (debatable benefit).

The results obtained demonstrated that SEOHSC's 34 percent decrease in volume paralleled an increased probability of Important/Group A and decreased probability of et Group B outcome diagnoses. The increase in volume seen in the control center differed significantly from the decrease in SEOHSC. Concurrently the control center saw et smaller increased probability of an outcome Group A diagnosis and larger decreased probability of a Group B diagnosis outcome. Changes in SEOHSC's volume of upper GI endoscopies were more comparable though to provincial trends then were changes in the control site. Despite significant changes in SEOHSC's post-AFP clinical practice of upper GI endoscopy, causality can not yet be inferred. This study provides a foundation for future research into the impact of funding changes on clinical practice.

Fiscal problems facing the Southeastern Ontario Health Sciences Center (SEOHSC) in Canada and Ontario Ministry of Health (Moll) prior to July 1, 1994 were not unique. Academic Health Science Centers (AHSCs) across the province were experiencing budgetary cuts and restrictions. However, through lengthy deliberations, SEOHSC employed a unique response to these issues. The solution was an alternative funding plan that was accepted by a large majority (81%) of the geographical full-time clinicians practicing in the SEOHSC and implemented July 1, 1994. The block funding, the amount of which was derived from previous fee-for-service billings and funds from several other sources, was designed to provide the SEOHSC with financial stability not found in the traditional fee-for-service system. It also enabled the Moll to accurately forecast expenditures in the region.

The AFP was expected, along with other benefits, to provide greater opportunity for SEOHSC clinicians to increase their involvement in research and educational activities. It was hypothesized that the ability to reorganize clinical, educational, and research commitments might lead to a decline in the volume of clinical services (Birch & Makela 1994; Labelle, Hurley & Rice 1989; Wright 1996). Further, it was suggested that a reduction in volume of procedures performed could be obtained, while maintaining or improving the health of the SEOHSC's population. This would be achieved by selectively reducing those procedures that may have been performed for questionable benefits (JAMA 1993; Westbrook et al. 1993).

The practice of high volume, high cost (Gentleman et al. 1996; Cromwell et al. 1989), discretionary procedures (Labelle, Hurley & Rice 1989; Gentleman et al. 1996; Gibson 1997) was considered the most susceptible to change induced by remuneration changes (Labelle, Hurley & Rice 1989; Gentleman et al. 1996). A methodology to evaluate the impact of the AFP on clinical practice in the SEOHSC was established. The methodology was developed as a template for evaluating changes in the clinical practice of a variety of procedures on the basis of administrative data (Kosecoff et al. 1987; Knowles 1996; Roos et al. 1982; Hawker et al. 1997; Wray et al. 1995; Huston & Naylor 1996). The methodology can best be detailed by focussing on the practice of a single procedure--upper gastrointestinal (GI) endoscopy. Upper GI endoscopy had been identified through a decision matrix as satisfying all three criteria (high volume, high cost and discretionary) (Gibson 1997). This current study was designed to address two main questions. Was there a change in the volume of upper GI endoscopies performed in the SEOHSC that was associated with the implementation of the AFP? Second, were changes in volumes associated with changes in the clinical practice of SEOHSC clinicians?

METHODS

Focussing on procedures performed in non-restrictive (i.e., non-pediatric), active treatment hospitals affiliated with an AHSC, the study used a retrospective pre-post design, included a comparison AHSC and used hospital discharge data for a four year period--July 1, 1992 through June 30, 1996 (two years before and two years after implementation of the AFP). To identify procedure codes (Statistics Canada 1993) for inclusion in the study, specialists from the hospital's medical records department and area clinicians were consulted. The final list of codes included upper GI endoscopies performed for diagnostic purposes(1).

All of the administratively recorded procedures performed under the Department of Medicine or Surgery during the study period in each AHSC were included in the study(2). This provided information about 21,037 procedures for analysis (SEOHSC = 6,414; comparison AHSC = 14,623). The data included record level information about each procedure. The early decision not to restrict inclusion of procedures to "principle procedures" dictated the need to include primary diagnoses and all comorbidities (up to a total of 16 diagnoses per procedure) recorded in the database during analysis. While it would have been ideal to obtain secondary data from a single source, Canadian Institute of Health Information (CIHI), it was too costly. Thus, administrative data were collected from Strategic Information Development System (SIDS) for the SEOHSC and from the Medical Records Departments (MRDs) of the hospitals in the comparison center. The data provided by SIDS had also originated in the MRDs of hospitals in the SEOHSC and was validated using the different sources. A simple comparison of SIDS vs. MRD data in the SEOHSC found less than an annual 1 percent (0.006) margin of error and further confirmed the validity of this approach.

Primary data were also collected to augment the interpretation of the secondary data. Clinicians, who performed GI endoscopies in the SEOHSC, were the main source of the primary data. These data were used to identify the procedure codes and categorize diagnoses related to the codes. Information from the clinicians was also used to define confounding factors and the context in which changes in clinical practice were being examined (Giraud & Jolly 1992).

There was a two stage process for collecting and consolidating information from the clinicians. The first stage involved surveying individual clinicians who agreed to participate (n = 11). All interviews were conducted by a single researcher who provided clarification and direction as needed. Each clinician was provided with a preliminary list of diagnoses that covered a range of relevant health/digestive system problems and an invitation to add other pertinent diagnoses. They were asked to rate the importance of an endoscopy based on the diagnoses. Clinicians were also asked to identify and explain diagnoses for which patient management had changed during the study period.

Initial analysis of clinician input was carried out to facilitate categorization of the outcome diagnoses. The procedure related diagnoses were categorized as "Important" if the clinicians ratings, based on a scale of 1 to 10, scored a mean of 6.5 to 10. The resulting diagnoses group was presented during the second stage of the primary data collection. This stage involved a focus group meeting to which all area endoscopists were invited. During the session the lack of the diagnoses group's sensitivity to change was discussed. Two diagnoses groups were developed and accepted by the clinical participants (Table 1). Group A was a subset of the selected Important outcome diagnoses and Group B included outcome diagnoses for which the benefit of having performed a GI endoscopy could be questioned.

TABLE 1

CONTENTS OF DIAGNOSIS GROUPS

Selected Important Diagnoses          Group A Diagnoses
Varices                               Stricture
Iron Deficiency                       Upper GI Bleeding
Dysphagia                             Upper GI Cancers
Esophagitis
Dyspepsia
Barrett's Follow-Up
Gastric, Peptic & Esophageal Ulcer
Celiac
Foreign Body Obstruction
Stricture
Upper Gl Bleeding
Cancer
Screening (with positive history of
   Digestive Disease or Cancer)

Selected Important Diagnoses                 Group B Diagnoses
Varices                               Duodenal Ulcer
Iron Deficiency                       Heartburn
Dysphagia                             Reflux (without bleeding)
Esophagitis                           Screening (with negative
Dyspepsia                             history of Digestive Disease
Barrett's Follow-Up                   or Cancer)
Gastric, Peptic & Esophageal Ulcer
Celiac
Foreign Body Obstruction
Stricture
Upper Gl Bleeding
Cancer
Screening (with positive history of
   Digestive Disease or Cancer)

The study used outcome diagnoses available in the administrative database to characterize the health problems ("type") of patients who had undergone the procedure. This was based on Donabedian's theory that purported inferences about "process" could be made by examining "outcome." Thus, if there is a valid relationship between the outcome and the process of health care provision, either approach may be used for evaluation, depending on the ease and accuracy of measuring (Donabedian 1978, 1981, 1988). Figure 1 illustrates the hypothesized relationship between the outcome diagnoses, the content of clinical practice (patient "type") and clinical practice. The symptomology continuum represents the type of patients seen by the clinician, the pathology continuum represents the outcome diagnoses and the area between the two displays how both relate to the threshold for performing a procedure and thereby the clinical practice. According to Donabedian theory, the distribution of diagnostic outcomes could be directly affected by a change in the decision-making process of service provision. This is depicted by the suggested change in the threshold for performing an endoscopy. It is also recognized that changes in the proportion of outcome diagnoses may not always be directly related to changes in the processes of providing care. However, aside from changes in the prevalence of disease in the general population, a change in outcomes would be related to a change in the process of service provision in some way or another (i.e., the posited change in the referral population was influenced, in part, by a change in the clinical practice of endoscopy).

This relationship between outcome diagnoses and the process (clinical practice) facilitates the use of administrative data to evaluate the practice of upper GI endoscopy.

Had the outcome diagnoses had been used as indicators for performing a procedure, characterizing patients by diagnoses/health problems and using the diagnoses as a proxy for clinical content would have allowed for misclassification. That is, any procedure performed on a patient who had presented with symptoms associated with a Group B diagnosis would only be classified as a Group B if the procedure did not result in a selected Important or Group A diagnosis. Patients who had presented with a Group A diagnosis, on the other hand, would remain in the Group A category. Using the administrative diagnoses as post-procedure diagnoses avoided this misclassification and allowed for accurate classification of those who had been scoped. However a possibility that bias, stemming from the involvement of area clinicians in defining which diagnosis indicated an important, or not so important, procedure was performed, need be acknowledged. This could result in "overinclusion" of diagnoses in the important diagnosis group. The results of the comparison between the pre and post periods would not be influenced by this bias though since the diagnosis groups were consistent throughout the study.

ANALYSIS

Hierarchical analysis was performed using a repeated measure analysis of variance (Proc Mixed in SAS 6.12) to establish differences in the pre-post mean measures of the continuous measures (patient age, number of diagnoses recorded and time between repeated procedures) within SEOHSC. Categorical repeated measure analysis of variance (Proc CatMod in SAS) was used to determine differences in probability for the dichotomous variables (clinical service, institution type, repeated procedure, diagnosis of cancer, selected Important diagnosis, Group A diagnosis and Group B diagnosis) because of its ability to partition the variation from the mean of these dichotomous variables.

Preliminary analysis was performed to determine the significance of differences between the pre-post changes within SEOHSC. Analysis then was conducted to define differences between the AHSCs for the full study period. This provided a basis for comparing the magnitude and direction of changes during the study.

RESULTS: WITHIN SEOHSC

There was a marked decrease in the number of upper GI endoscopies performed in SEOHSC after the AFP was implemented (33.6%, p [is less than or equal to] 0.01). No significant changes were found in the age or gender ratio of patients or in the recording practice of diagnoses (Table 2).

TABLE 2

CHANGES IN UPPER GI ENDOSCOPY ACTIVITY WITHIN SEOHSC

                                  Pre-AFP   Post-AFP     Change

Volume (Totals)                   3854       2560       -33.60%(***)
Gender Ratio (F:M)                   0.96       0.92     -0.04
Age (yrs.)(a)                       56.30      56.70      0.40
No. of Diagnoses Recorded(a)         3.12       3.38      0.26
Selected Important Diagnosis(b)      0.50       0.56      0.06(***)
Group A Diagnosis(b)                 0.25       0.31      0.06(***)
Group B Diagnosis(b)                 0.24       0.20     -0.04(***)

(a) mean value

(b) probability

(***) p [is less than or equal to] 0.01

The increase in the probability of an upper GI endoscopy patient having a selected Important or a Group A diagnosis (6%, p [is less than or equal to] 0.01) that occurred along with the decrease in volume provided evidence that clinicians continued to perform procedures on patients whose diagnoses clearly warranted endoscopic investigation. The decrease in the probability of an upper GI endoscopy resulting in a Group B diagnosis (-4%, p [is less than or equal to] 0.01) further demonstrated a change in the content of clinical practice toward a better match between patient need and service provision. No significant differences in change were found between the clinical services under which the procedures where performed during the study period.

DISCUSSION

Evaluation of changes in volume of the selected endoscopy procedures performed in the SEOHSC was carried out in response to one of the main research questions: Had there been a change in the volume of procedures performed following the implementation of the AFP? Analysis revealed that there had been a significant (34%) decrease in the volume of these procedures.

The second major research question asked whether changes in volumes over the study period were associated with changes in the clinical practice of SEOHSC clinicians. General consistency in the average age and gender ratio of the GI endoscopy patients over the four year period suggested that the demographics of the population scoped remained stable. This finding accompanied marked changes in the outcome diagnoses associated with an upper GI endoscopy. Patients who had been scoped were more likely to have had a Group A outcome diagnosis and less likely to have had a Group B outcome diagnosis in an AFP environment than prior to the implementation of the AFP. Combined, these changes indicated a significant change in the type of patients who underwent endoscopies, and therefore a change in the content of the clinical practice of upper GI endoscopy after the AFP was implemented.

A number of factors that could have influenced the provision of the GI endoscopic procedures during the study period were identified (Figure 2). These included newly recommended approaches based on evidence introduced prior to July 1, 1992 regarding the effectiveness of the H-Pylori breath test for diagnosing and the drug Omeprazole for the medical treatment of peptic ulcer disease. The GI laboratory in SEOHSC then began accepting referrals from the GI division for H-Pylori breath tests in April, 1993 and from the broader medical community in July, 1994. Concurrently, there was a deceleration of the rapidly increasing Ontario Drug Benefit costs for Omeprazole that had began in 1994/95. This suggested that the use of Omeprazole, to manage physical problems associated with peptic ulcer disease, had become common place (Anderson et al. 1996).

[Figure 2 ILLUSTRATION OMITTED]

Along with having reduced the need to perform diagnostic upper GI endoscopies for hyper-acidity problems, scientific advances would have had an impact on the population of patients being referred to the endoscopists. Specifically, open-access HPylori breath tests enabled family physicians to diagnose and manage (with Omeprazole) without intervention from endoscopists. In short, medical evidence had promoted volume-reducing changes in the clinical practice of GI endoscopy.

Other potential factors that may have influenced the changes found in the study relate to facility and organizational changes within the SEOHSC. Early in 1993 gastroenterology services in the two SEOHSC institutions were amalgamated. The new Endoscopy Unit, completed in the summer of 1993, circumvented the need to book an operating suite for procedures. This change enhanced the SEOHSC's capacity to perform endoscopies and initially that is what happened (Figure 2). However the volume of upper GI endoscopies began to drop markedly coinciding with the implementation of the AFP and open access H-Pylori breath tests. Levels of upper GI endoscopies continued to fall until 1996. It must be recognized that potential increases in volume stemming from the facility changes may have been tempered by a combination of the new funding environment, changes in the practice guidelines for endoscopy, a reduction in the number of hospital beds (from 18 to 12) available to the GI Division, and increased efforts in continuing medical education activities (CME) by the GI Division clinicians. These clinicians were active participants in CME, informing referring physicians of the new diagnostic and patient management approaches for hyper-acidity problems.

In addition to facility changes, there were personnel changes: a new endoscopist began practicing in the SEOHSC mid-way into the pre-AFP period, one endoscopist left coinciding with the onset of the AFP, and the clinical services of another clinician were reduced for three to five months during the post-AFP period. Despite these staff changes, all within the Department of Surgery, the number of clinicians performing the majority of the endoscopies (6 from the Department of Medicine, 8 from the Department of Surgery) remained relatively consistent throughout the study period. The lack of impact that the Department of Surgery personnel changes appeared to have (the proportion of procedures performed under the Department of Surgery did not change significantly) may have been related to an interdepartmental shift in responsibilities.

In July, 1994 a shift in departmental responsibility for other GI endoscopic procedures was also instituted. This shift involved the GI division assuming responsibility for performing endoscopic retrograde cholangiopancreatography (ERCP--a more involved, time consuming procedure) that had previously been performed under the Department of Surgery. In effect this inter-departmental shift in the division of work could increase the amount of available time for clinicians in the Department of Surgery and reduce the time available for the clinicians in the Department of Medicine to perform the upper GI endoscopies considered in the study and, thereby, possibly offsetting the impact of personnel changes.

The change in volume was found to correlate, in part, with local changes in the facilities and personnel as well as widely disseminated changes in practice recommendations and new technology (Hayward et al. 1997). However, the importance of exploring the possibility that the direction and/or magnitude of the change may have been tempered, or augmented, by the implementation of the AFP can not be dismissed. This required comparing the change in the SEOHSC to those within another Ontario AHSC.

While the comparison site was selected because of it's geographical proximity and similar political environment, it was important to recognize basic differences between the two centers. The comparison center provided services to a much larger catchment population and performed over twice as many procedures during the four year study period than did the SEOHSC (Table 3). It must be stressed however that the study compared only the magnitude and direction of change in the volume and content of the clinical practice of endoscopy. If changes in a comparison AHSC were broadly similar to those found in SEOHSC, the changes could be attributed to systemic factors. On the other hand, if the changes were dissimilar it would be less likely that systemic factors accounted for the changes seen in either center.

TABLE 3

DIFFERENCES BETWEEN AHSCS

                                              Comp.
                                   SEOHSC     AHSC        Diff.

Volume (Totals)                    6,414      14,623      ***
Gender Ratio (F:M)                     0.92        1.00   **
Age (yrs.)(a)                         56.5        57.0
No. of Diagnoses Recorded(a)           3.25        3.85   **
Selected Important Diagnosis(b)        0.52        0.43   ***
Group A Diagnosis(b)                   0.28        0.20   ***
Group B Diagnosis(b)                   0.22        0.22

(a) mean value

(b) probability

(***) p [is less than or equal to] 0.01

(**) p [is less than or equal to] 0.05

It was ascertained that there had been no substantial change in or to the comparison facility, organization or personnel within the study period. This was supported by analysis that found the bulk of the comparison AHSC's procedures were performed by three clinicians under the Department of Medicine and four clinicians under the Department of Surgery during both study periods. No attempt was made to standardize "procedures per clinician" since the number of clinicians in each AHSC was relatively consistent over the study period and the study procedures accounted for only a portion of the practice of GI endoscopy.

The 34 percent decrease in volume of upper GI endoscopy (Table 4) activity within the SEOHSC contrasted directly with the 8 percent increase seen in the comparison center (p [is less than or equal to] 0.01). No practical significance was found in the difference between SEOHSC and the comparison center's change in the age and gender ratio of upper GI endoscopy patients. There was a much greater increased probability that upper GI endoscopy patients had a selected Important outcome diagnosis in the SEOHSC than in the comparison center (p [is less than or equal to] 0.01).

The study found similar directional change in the probability of a Group B diagnoses following an endoscopy that implied an unverified agreement about Group B diagnosis group components among clinicians in both AHSC's. A significant difference was found though in the magnitude of the change. The comparison's decrease was greater (p [is less than or equal to] 0.01).

However, the SEOHSC's 4 percent decrease in Group B diagnoses accompanied a 34 percent decrease in volume and suggested the volume change was related to the decrease in the proportion of procedures performed on patients with a Group B diagnosis. In turn, the 10 percent decrease in the proportion of Group B diagnosis that accompanied the 7 percent increase in the volume at the comparison center suggested the increased volume was not a result of more procedures having been performed on Group B type patients. This implied that changes in the clinical practice of GI endoscopy within the SEOHSC varied from those within the comparison AHSC (Figure 3).

[Figure 3 ILLUSTRATION OMITTED]

The difference in changes found between the AHSCs appeared to suggest that systemic influences did not account for the changes documented in SEOHSC. Yet a preliminary comparison of SEOHSC's downward trend in volume was found to be much more in line with the provincial trend than was the trend in the comparison center (Figure 4).

[Figure 4 ILLUSTRATION OMITTED]

This pattern of parallel change suggests the difference between the AHSCs could have been related more to idiosyncratic changes in the comparison center than to changes in the funding mechanism within SEOHSC. In turn, comparable trends in the SEOHSC and provincial volume indicates a possibility that a systemic effect--such as changing standards-of/guidelines-for clinical practice--may have exerted a greater influence on the clinical practice of GI endoscopy in the SEOHSC than had the implementation of the AFP.

The results of the study showed significant change in clinical volume and practice content within the SEOHSC that corresponded with the implementation of the AFP. Along with the marked decrease in volume, a change in the "type" of patients that were scoped was identified. The consistency of changes in clinical practice with evidence-based suggestions for change indicated that the changes found were, in part, related to newly articulated clinical guidelines. It may be that the changes were more acceptable in the AFP environment. However, the extent of the influence of an AFP environment on these changes could not be firmly established.

Preliminary inferences based on the differing volume and content of clinical practice changes between the two AHSCs supported the hypothesis that the AFP had played a role. The possibility that the AFP provided an environment which more readily facilitated change in accordance with clinical guideline changes was seriously considered. Still, the similarity between the SEOHSC and provincial volume trends generated resistance to prematurely crediting the AFP environment with having influenced the changes in SEOHSC. Further comparison of changes in the type of patients who underwent endoscopies within the SEOHSC to changes in all of Ontario's AHSCs would be necessary before ascribing more concretely that the AFP was a factor which influenced clinical practice change in SEOHSC.

CONCLUSIONS

The limitations stemming from the use of a single comparison site were highlighted by the comparison of the study findings for volumes to provincial trends. The study also faced limits imposed by a retrospective design, two data sources, and a dearth of information about the general population "needs" and the "under-serviced"(3) segment of the population. However, the direction of the changes in the outcome diagnosis groups would not suggest a growth of the under-serviced population. Finally, the study examined only one component of the practice of gastroenterology and the findings can not be generalized to all aspects of that sub-specialty's activities.

This research was conducted to determine whether changes in clinical practice could be determined using administrative data. The results support the approach, while at the same time emphasize the need to expand the comparison base by including all the AHSCs in Ontario. It would also be imperative to collect/utilize detailed information about clinical practice and the environment within each comparison AHSC. The template could then be used as a foundation for examining the nature and extent of change in the clinical practice.

(1) CCP codes: 01.11-01.16, 12.11-12.13, 54.214, 54.814, 54.894, 55.41,56.814, 56.824, 56.894, 57.114.

(2) Medicine: Medicine Emergency, and Gastroenterology; Surgery: General and Thoracic Surgery.

(3) The under-serviced are those who should but do not either seek or undergo medical diagnosis or treatment.

REFERENCES

Anderson, G., T. Axceli, B. Chan, & J. Carter (1996) The Use of Acute Care Hospitals, Physician and Diagnostic Services, and Prescription Drugs in Ontario's Health Planning Regions," In V. Goel, J. Williams, G. Anderson, P. Blackstien-Hirsch, C. Fooks & D. Naylor (eds.). Patterns of Health Care in Ontario, The ICES Practice Atlas. 2nd Edition.

Birch, S., L. Goldsmith & M. Makela (1994) Paying The Piper and Calling the Tune: Principles and Prospects for Reforming Physician Payment Methods in Canada, McMaster University, Hamilton Ontario.

Cromwell, J., J. Mitchell, M. Rosenbach, W. Stason & S. Hurdle (1989) "Using Physician Time and Complexity to Identify Mispriced Procedures," Inquiry, 26:7-23.

Donabedian, A. (1978) "The Quality of Medical Care: Methods for Assessing and Monitoring the Quality of Care for Research and for Quality Assurance Programs," Science, 200: 856-864.

Donabedian, A. (1981) "Criteria, Norms and Standards of Quality: What Do They Mean?" American Journal of Public Health, 71 (4):409-412.

Donabedian, A. (1988) "The Quality of Care. How Can it be Assessed?" Journal of the American Medical Association, 260(12): 1743-48.

Gentleman, J., M. Vayda, G. Parsons & M. Walsh (1996) Surgical Rates in Subprovincial Areas Across Canada: Rankings of 39 Procedures in Order of Variation, CJS, 39(5):361-367.

Gibson, L. (1997) Decision Matrix: Identifying High Volume, High Cost and Discretionary Procedures, Unpublished.

Giraud, A. & D. Jolly (1992) "How to Include Physicians to Engage in Quality Assurance Activities in a University Hospital: A Policy," Quality Assurance in Health Care, 4(1): 19-24.

Hawker, G.A., P.C. Coyte, J.G. Wright, J.E. Paul & C. Bombardier (1997) "Accuracy of Administrative Data for Assessing Outcomes After Knee Replacement Surgery," Journal of Clinical Epidemiology, 50(3):265-273.

Hayward, R., G. Guyatt, K. Moore, K. McKibbon & A. Carter (1997) "Canadian Physicians' Attitudes About and Preferences Regarding Clinical Practice Guidelines," Canadian Medical Association Journal, 156(12):1715-23.

Huston, P. & D. Naylor (1996) "Health Services Research: Reporting on Studies Using Secondary Data Sources," Canadian Medical Association Journal, 155(12): 1697-17

JAMA (1993) "Toward Fewer Procedures and Better Outcomes," Journal of the American Medical Association, 269 (6) :794-796.

Knowles, K. (1996) Briefing Note: Accuracy of Canadian Health Administrative Databases, The Institute for Clinical Evaluative Sciences, Toronto, ON.

Kosecoff, J., A. Fink, R. Brook & M. Chassin (1987) "The Appropriateness of Using a Medical Procedure--Is Information in the Medical Record Valid?" Medical Care, 25(3):196-201.

Labelle, R., J. Hurley & T. Rice (1989) Financial Incentives and Medical Practice: Evidence from Ontario on the Effect of Changes in Physician Fees on Medical Care Utilization, Physician Payment Review Commission, Washington, DC.

Roos, L.L. Jr., N.P. Roos, S.M. Cageorge & J.P. Nicol (1982) "How Good are the Data? Reliability of One Health Care Data Bank," Medical Care, 20(13):266-276.

Statistics Canada (1993) Canadian Classification of Diagnostic, Therapeutic, and Surgical Procedures, Second printing, Ministry of Industry, Science, and Technology, Ottawa ON.

Westbrook, J., R. Rushworth, M. Rob & G. Rubin (1993) "Diagnostic Procedures and Health Outcomes," The Medical Journal of Australia, 159:242-45.

Wray, N.P., C.M. Ashton, D.H. Kuykendall & J.C. Hollingsworth (1995) "Using Administrative Databases to Evaluate Quality of Medical Care: A Conceptual Framework," Social Science and Medicine, 40(12): 1707-1715.

Wright, C (1996) "Physician Remuneration Methods: The Need for Change and Flexibility," Canadian Medical Association Journal, 154(5):678-680.

Sally E. Stanton Queen's University (Canada)

The authors would like to thank all of the clinicians whose thoughtful input helped to strengthen the project's methodology. Special thanks going to Dr. D. Naylor, Dr. S. Lewis, and Dr. J. Turnbull for their review and insightful comments on an earlier draft. This study was funded by the Southeastern Ontario Academic Medical Association. Results and conclusions are those of the authors and no official endorsement by the Southeastern Ontario Academic Medical Association is intended nor should be inferred.

Address for correspondence: Sally E Stanton, Queen's Health Policy Research Unit, 3rd Floor Abramsky Hall, Queen's University, Kingston, ON K7L 3N6 CANADA, [e-mail].