The Development of an Instrument to Measure Satisfaction With Physical Therapy.

Quality of care continues to be a major concern for health care providers and a major focus for health services research.[1] Although many operational definitions of "quality of care" focus on the personal knowledge, skills, and expertise of the clinician rather than on other aspects of the treatment experience, patient satisfaction, in our opinion, constitutes a dimension of care outside of the physical therapist's immediate control. Yet, technical quality and patient satisfaction are synergistically linked to influence the outcomes of care. Ellwood and Paul[2] imply, for example, that dissatisfied patients are less likely to use professional advice, thus undermining both primary and secondary prevention. Therefore, in light of the hypothesized relationships among the technical expertise of the care provider, the experience of the person receiving the care and how that person values care, and measures of outcomes of the care provided, any comprehensive formulation of an operational definition of "quality" in health care should state that patient satisfaction is a necessary construct.[3-6]

Satisfaction and Quality of Care

Even though it is recognized as an extremely important concept, satisfaction with the provision of physical therapy, or any health care services, is often difficult to define.[7,8] Satisfaction can refer to a health care recipient's reaction to aspects of the service delivered and satisfaction over time which result in overall perceptions of quality of service.

Job satisfaction literature from the field of organizational psychology can be an important source of hypothesis generation for research on patient satisfaction. For example, equity theory[9] posits that a person's degree of satisfaction with his or her salary is contingent on the expectation that the outcome, as measured by salary, is either greater or less than the person's perceived work input. Linder-Pelz[10] has applied these theories of satisfaction in the workplace to satisfaction with the provision of health care. When patients' expectations of care are exceeded, their level of satisfaction is high. Likewise, if expectations of care exceed actual delivery, dissatisfaction will result. Satisfaction, therefore, is always relative to the patient's expectations. Satisfaction changes when the patient's expectations or standards of comparison change, even though the object of comparison (the actual health care received) may stay constant. Thus, satisfaction measures, although they may be objective (ie, have reliability), are actually reflecting subjective phenomena and are quite distinct from other types of evaluation of the provision of care.

In addition to the importance to the clinician of a patient's level of satisfaction with care as part of the patient-therapist relationship, maintaining a high level of patient satisfaction may also have an economic impact on the clinician. Patients who are satisfied with the services they have received are more likely to remain loyal to the provider (ie, the therapist). Even if a "successful" physical therapist may never again see a patient who has been rehabilitated to an optimal functional level or changes in insurance or personal considerations preclude returning to a particular physical therapist, levels of satisfaction, in our view, may contribute to more informal word-of-mouth communication with other people who may become patients. Dissatisfaction with a particular provider broadly voiced in the community in which the therapist practices may cause a potential patient to seek another physical therapist for intervention as the need arises.

Benefits of Patient Satisfaction Surveys

One method of collecting data on patient satisfaction is to use standardized survey instruments.[11,12] Patient satisfaction surveys provide several benefits for physical therapists. First, the information, in our opinion, may be used to evaluate provider services and facilities by characterizing the structure, process, and outcome of care. Second, we contend that patient satisfaction data may be used to predict patient behavior on the assumption that differences in levels of satisfaction can influence clinical outcomes to at least a small degree. Finally, data from patient satisfaction surveys, in our view, can help health care providers develop strategies for provision of care that will facilitate the retention of current patients or the recruitment of new patients.

Domains of Satisfaction

The benefits of patient satisfaction questionnaires are clear and have been documented.[2,7] The determination of which variables to include in an instrument that measures patient satisfaction is a more difficult task. Several researchers.[8,10,13,14] have posited that patient satisfaction is a multidimensional concept. Indeed, recent research[15] has indicated that little is known about which information is most important to consumers when making decisions regarding the selection of health care providers. Different types of consumers likely have different needs. Although there is no "gold standard" for the measurement of patient satisfaction, recent research by Nelson[13] is helpful in determining the areas that comprise patient satisfaction. Nelson performed a content analysis on surveys from 18 selected health care institutions and attempted to match questions to indicators of quality as described by Donabedian.[3] Based on this framework, he concluded that access, administrative technical management, clinical technical management, interpersonal management, and continuity of care are the domains that define patient satisfaction (Tab. 1).

Table 1. Dimensions of Patient Satisfaction

1. Access: Physical location of facility, hours of operation,
telephone access, appointment waiting time, waiting time in
waiting room.

2. Administrative Technical Management: Ambience of
facility, parking, payments/claims processing, quality-assurance
assurance programs.

3. Clinical Technical Management: Qualifications of staff,
including clinical skills of physical therapists, technical
skills of physical therapist assistants, technical skills of
any others on staff providing care, explanation of care given.

4. Interpersonal Management: Responses to complaints or
suggestions, warmth/friendliness of physical therapist(s),
warmth/friendliness of other staff members, appropriate
amount of time spent with each patient, respect for patient
privacy.

5. Continuity of Care: Intent to continue to have condition
managed by provider, knowledge of patient's history by the
therapist, patient's recommendation of the therapist to others,
general satisfaction with intervention received. These last 2
items infer that the patient will continue care with the same
therapist if he or she is satisfied with the intervention received.

These domains were also well represented in several patient satisfaction survey instruments currently used by physical therapists across diverse practice settings.[16] The patient satisfaction survey instruments that we have seen contain, on average, 30 questions. Questions typically are answered by use of a scale that allows respondents to communicate different levels of agreement with a particular item. Less typical are questions of a "yes/no" variety and open-ended questions.[13]

Administration of Surveys

A number of different survey methodologies have been used to administer patient satisfaction surveys.[13] Differences can be found among frequency of conducting surveys, method for selecting patients, method of asking questions, and strategies for distributing survey instruments. Nelson[13] contends that most health care organizations collect patient satisfaction data on an ongoing basis throughout the course of a year according to a set schedule (eg, monthly or quarterly).

The number of patients surveyed and the manner in which patients are surveyed differ among providers as well.[13] Within health care overall, we believe it is as likely that all discharged patients will be surveyed as it is that a random sample of discharged patients will be surveyed. However, among physical therapists, it appears that it is much less likely that all patients seen by a physical therapist will be surveyed.

Finally, numerous methods are used to distribute patient satisfaction survey instruments. According to Nelson,[13] the standard method is to provide the survey instruments to patients at a facility on conclusion of services and request them to return the instruments by mail. A slightly less common method is to mail survey instruments to patients. Administering surveys by telephone is a technique not as widely used, and face-to-face interviews are used least often. Regardless of sampling frame, Nelson[13] has indicated that approximately 60% of surveys do not require respondents to provide their names. This feature of the survey maintains confidentiality and increases the likelihood that respondents will be objective. In addition, it has the benefit of increasing response rate.

Measurement Issues

Patient satisfaction surveys are usually designed to measure and define patients' expectations of, and satisfaction with, the care they receive. Some critics believe that patient satisfaction should not be included in the definition of quality of care. Among other arguments, they cite factors that may compromise the reliability and validity of measurements obtained from such surveys. Among these factors are:

* Patients cannot accurately recall all aspects of the process of the delivery of care.

* Patients lack the knowledge to assess accurately the technical competence of health care personnel.

* Patients are reluctant to disclose negative attitudes toward a health care provider because of a sense of dependency on patient-provider communication.

* Good "bedside manner" may inflate patient ratings of care beyond its merits.

To address some of these problems, in particular threats to reliability and validity, patient satisfaction instruments should undergo evaluation to establish their psychometric properties. Such evaluation, however, cannot eliminate the conceptual argument against the use of patient satisfaction measures to reflect quality of care. Often, however, these studies have not been undertaken. A notable exception to the dearth of such studies is the work of Ware and colleagues.[11,12,17-19] Over an approximately 10-year period, these researchers developed and refined the Patient Satisfaction Questionnaire and the Group Health Association of America's Consumer Satisfaction Survey.[12,15] These 2 research efforts, the former funded through a grant from the Agency for Health Care Policy and Research (now called the Agency for Healthcare Research and Quality) and the latter developed at the request of the Group Health Association of America, represent 2 of the most rigorous efforts to develop instruments for measuring patient satisfaction. These survey instruments, however, deal with consumer reactions to the provision of health (ie, medical) care. They do not deal specifically with the provision of physical therapy services. Adapting them to assess satisfaction with the provision of physical therapy would necessitate the measurement of reliability and validity of a new instrument.

Physical therapists have been confronted with the dilemma of either adopting instruments developed for other purposes or using instruments for physical therapists that have had a less than rigorous psychometric evaluation. As far as we can determine, survey instruments currently in use have been developed within a single location and generally have not been subjected to psychometric evaluations. Conversely, instruments that have undergone rigorous evaluation have not been developed to answer questions about patient satisfaction with intervention by physical therapists.

The purpose of this study was to develop an instrument to assess patient satisfaction with the provision of physical therapy services. The instrument was field tested across a number of practice settings to determine its psychometric properties. The availability of an instrument specific to physical therapy would allow therapists to be less reliant on instruments developed primarily for the assessment of other types of health care delivery, and the instrument could be used to better assess satisfaction with the provision of physical therapy services. The development process that we used included an evaluation of the psychometric properties of the instrument. One goal of this endeavor was to advance the assessment of patient satisfaction by physical therapists beyond the use of instruments that have passed only the test of face validity.

Methods

Instrument Content

We used the 5 hypothesized domains of patient satisfaction cited by Nelson[13] as a guide in the generation of the items in the instrument. In the process of devising the items, we gave consideration to the inclusion of items from each of the hypothesized domains. Items were generated by adapting items from the survey instruments contained in Patient Satisfaction Instruments: A Compendium[16] (the Compendium) as well as other instruments found in the literature.

The Compendium was compiled by the American Physical Therapy Association (APTA) in 1995 from responses to a call for patient satisfaction instruments that was published in PT Bulletin and included in a letter to all members of APTA's Private Practice Section. Approximately 75 instruments were received in response to this call. The Compendium includes 36 of these instruments. The instruments were selected for inclusion in the Compendium on the basis of their overall utility and coverage of the domains of patient satisfaction. These instruments were in use by physical therapists in a variety of settings, including acute care hospitals, subacute rehabilitation hospitals, and private outpatient offices, at the time they were submitted to APTA.

From the instruments in the Compendium and other sources, 20 items were generated for the instrument that we used in our study. The items were selected to represent the domains postulated by Nelson[13] as well as additional domains that we deemed relevant to patient satisfaction at the time the study was undertaken. The goal was to maintain a reasonable number of questions that would represent aspects of the delivery of care that would affect patient satisfaction. Table 2 contains a listing of domains and a categorization of items by domain as determined by the authors. As illustrated in the table, a total of 11 domains are listed. Some of these domains represent Nelson's domains, and some represent new or additional domains of patient satisfaction. Three such domains not covered in Nelson's list are (1) cost (domain 4), (2) convenience of appointment time (domain 6), and (3) ease of scheduling an appointment (domain 7).

Table 2. Domains of Patient Satisfaction(a)

                        Questions

Domain                  No.   Content

Treatment                11   Satisfied with treatment by PT
                         17   Satisfied with services by PTAs
                         19   PT understood problem
                         20   Instructions by PT were helpful
                         21   Satisfied with overall quality of
                                PT care
Privacy                   7   Privacy was respected
Convenience of           15   Convenient location
  appointment time       18   Parking available
Cost                     24   Cost was reasonable
                         25   If had to, I would pay for these
                                physical therapy services myself
Billing                  16   Bills were accurate
Ease of scheduling an    10   Scheduling appointments at
  appointment                   convenient times
Scheduling               12   First visit scheduled quickly
                         13   Subsequent visits scheduled
                                easily
Wait time                14   Seen promptly
Courteous staff           9   Staff courteous
PT courteous              8   PT was courteous
Overall satisfaction     22   Would recommend to family
                                and friends
                         23   Would return to this facility for
                                physical therapy in the future
                         26   Overall satisfaction with the
                                physical therapy experience

(a) PT = physical therapist, PTA = physical therapist assistant.

Patients' opinions of service in each domain were measured using 5-point Likert-type scales that ranged from "strongly disagree" to "strongly agree." The Likert-type scales were selected based on advantages cited by Ware and colleagues,[17] who believed that this type of scale facilitates the task of survey completion for the respondent and allows the developer to more easily revise the survey instrument. In addition to the items designed to assess patient satisfaction, 6 additional items were included to gather the following information: (1) age, (2) sex, (3) the condition requiring physical therapy intervention, (4) the manner in which the patient learned about the practice, (5) identification of the visit as the patient's first experience with that particular facility, and (6) identification of the visit as the patient's first experience with physical therapy (Appendix). These descriptive variables were included to allow for the potential to conduct additional analyses to determine whether they exerted differential effects on the ratings of patient satisfaction.

Survey Administration

Data were collected at 12 practice settings that were part of a large physical therapy practice. These settings represented diverse locations, including a hospital-based outpatient clinic and a private physical therapy office. Thus, patients and clients were not limited to those from a private outpatient office. A total of 35 physical therapists were employed across all practice locations. The average daily number of patients seen was 366, with a range of 4 patients at the smallest location to 80 patients at the largest location. Orthopedic conditions were those typically seen most often. More specifically, patients were treated for back, knee, shoulder, elbow, and hand problems. However, patients were not restricted to those with orthopedic conditions. One of the clinics provided services to patients with stroke. Another clinic was devoted solely to providing services for women's health issues, including fibromyalgia and pelvic dysfunction.

Patients were asked to complete the instrument upon leaving a facility. Patients were assured that their responses were voluntary and would remain confidential. To ensure confidentiality, patients completed the instrument in the waiting room outside the view of the clinic therapists and other staff, and they were instructed to immediately place it in a locked box. This data collection effort was typical of this practice. Patient satisfaction data had been collected routinely, using a different instrument than the one being developed, for 2 years prior to this data collection effort. Data were collected over a period of 1 month. Instruments were then hatched and returned to the researchers to begin processing and analysis. Responses were entered into a data file, and data analysis was accomplished using the SPSS/PC+ statistical package.[20],(*)

Analysis Plan

Descriptive statistics and estimates of reliability and validity were computed for the instrument. Reliability was assessed using the Cronbach alpha coefficient, which measures the internal consistency of measurements obtained with an instrument. The internal consistency, or homogeneity, is a measure of the extent to which items assess the same characteristic.[21] This type of reliability analysis was chosen because it has been used to assess the reliability of measurements obtained with other patient satisfaction instruments.[22,23] It does not, however, address whether a measure is stable over time.

For example, we strongly believe that test-retest reliability would likely be influenced by factors most related to satisfaction. Responses to the second administration could be influenced by the manner in which respondents completed the instrument the first time. Furthermore, as time passes, respondents may experience "selective forgetting," causing them to perceive aspects of an intervention differently as more time elapses between the intervention and completing the instrument. Another method not selected was split-halves reliability. This technique correlates scores from one half of an instrument with scores from the other half. The Cronbach alpha coefficient is very similar to the split-halves reliability method, except that the split-halves reliability method considers only one split, whereas the Cronbach alpha coefficient accounts for all possible splits.[24]

Validity was assessed in 3 ways. First, an assessment of validity was made in terms of content (ie, content validity). This type of validity deals with the extent to which an instrument reflects the meaningful elements of the content without extraneous elements.[25] There is no quantitative index available for this type of validity. Content validity is often judged simply by comparing the content of an instrument with the domains that are intended as the areas to be measured, and sometimes it is judged by seeking expert opinion.

The second assessment of validity was formulated as an assessment of concurrent validity. Concurrent validity is a measure of the degree to which an instrument correlates with a criterion variable that is available at the time the instrument is administered.[24] This criterion variable is known to be a valid measure of the construct under investigation. Thus, if the instrument has a high correlation with the criterion variable, then the instrument is known also to be a valid measure of the construct. The criterion variable used for this assessment was overall satisfaction (domain 11 in Tab. 2). The items in this domain represent the most broad aspect of satisfaction, and, therefore, all other domains of satisfaction should correlate with domain 11.

The third type of validity that was assessed was construct validity, which refers to the degree to which an instrument accurately measures the construct under investigation. Several approaches are utilized to assess construct validity. In an approach to assessing construct validity, multitrait-multimethod matrixes are evaluated for the consistency of inter-item correlations. If items measuring different constructs or traits correlate highly within methods of measurement and to a lesser extent across methods of measurement, then the instrument is consistent with the hypothesized constructs; thus, it has construct validity. The evaluation of construct validity used in our study reflects the use of the multitrait-multimethod matrix. An inter-item correlation matrix was evaluated using the domains listed in Table 2 as a model for the patterns of correlation.

In addition, a factor analysis was performed to evaluate the factor structure relative to the domains presented in Table 2 and relative to the domains presented by Nelson.[13] One aspect of factor analysis is the estimation of communalities for each item of an instrument. The communalities of an item reflect the extent to which it shares a common variance with other items. In the case of this instrument, communalities can be interpreted as measures of the same underlying factor, patient satisfaction.

Results

Respondent Characteristics

A total of 289 questionnaires were completed, returned, and entered into the data file. The average age of the patients was 45.7 years (SD=17.3, range=10-92); the largest percentage (63.7%) were women. The patients typically were being managed for lower back, shoulder, knee, or neck conditions (31.1%, 26.6%, 19.4%, and 18.7%, respectively). They typically were referred by a physician (82.4%) and had no prior experience with the facility (75.1%) about which they completed a survey instrument. However, nearly half the respondents (47.6%) indicated that this episode was not their first experience with physical therapy.

Psychometric Characteristics of the Instrument

The Cronbach alpha coefficient computed for the instrument was .99. Table 3 contains item analysis results and the Cronbach alpha coefficients that would be generated if each item were to be deleted from the instrument. These results illustrate the fact that question 25 (ie, "If I had to, I would pay for these physical therapy services myself") does not measure the underlying construct of the survey to the same extent as the other questions. The correlation between question 25 and the total for the questionnaire was only .59, whereas the next lowest correlation of a single question with the total was .80.

Table 3. Reliability Analysis With Each Question Deleted

No. of              Corrected
Deleted    Scale    Item Total    Cronbach
Question   Mean     Correlation   Alpha

 7         88.640     .9047       .9891
 8         88.512     .9517       .9888
 9         88.528     .9663       .9887
10         88.552     .9038       .9891
11         88.496     .9667       .9887
12         88.552     .9440       .9888
13         88.512     .9698       .9887
14         88.552     .9636       .9887
15         88.600     .9395       .9889
16         88.800     .7978       .9901
17         88.528     .9673       .9887
18         88.608     .8988       .9892
19         88.528     .9665       .9887
20         88.536     .9710       .9887
21         88.520     .9623       .9887
22         88.504     .9599       .9888
23         88.512     .9651       .9887
24         88.744     .8404       .9896
25         89.120     .5852       .9929
26         88.520     .9658       .9887

We inferred that the content validity for this survey instrument was good because items were included from each of the domains of patient satisfaction cited in the patient satisfaction literature.[13] We based this inference on (1) an initial assumption that patient satisfaction is a multidimensional construct and (2) the fact that items of the instrument had previously been included in instruments used by physical therapists. The items of our instrument also appeared in the instruments included in the Compendium. Thus, the instruments were influenced by a number of physical therapists who had an awareness of, and interest in, the topic of patient satisfaction. Only minimal changes were made to the items selected from the instruments included in the Compendium. Specifically, wording was changed to make the items consistent with each other. The changes were necessary as the wording differed among those surveys included in the Compendium. Making these minor changes allowed each question to be consistent with the others, and it was anticipated that completion of the survey would be facilitated. However, the intent of the items was maintained from those included in the Compendium. The domains that were initially hypothesized to be indicators of patient satisfaction (Tabs. 1 and 2), in our view, were adequately covered by the instrument.

To assess concurrent validity, 3 of the 20 items were removed so that they could be used as criterion measures. The 3 items that were removed were: (1) "Would recommend to family and friends" (question 22), (2) "Would return to this facility for physical therapy in the future" (question 23), and (3) "Overall satisfaction with the physical therapy experience" (question 26). A case can be made that these items have face validity as the best overall indicators of satisfaction. The first and second items are the best behavioral indicators of satisfaction. The third item asks directly about overall satisfaction. The remaining 17 items were then used to form a summary score for the remainder of the survey instrument. This summary score was correlated with each of the criterion variables. The obtained correlations were r=.95 (P [is less than] .01) for question 22 and the summary score, r=.96 (P [is less than] .01) for question 23 and the summary score, and r=.96 (P [is less than] .01) for question 26 and the summary score. These correlations indicate a high level of agreement between the summary score and each of the criterion variables.

The multitrait-multimethod matrix for the assessment of construct validity is presented in Table 4. The questions in the matrix are ordered according to the domains listed in Table 2. That is, all of the questions from domain 1 are grouped together and appear first. Likewise, all of the items from domain 2 are grouped together and appear second, and so on. The correlations were consistent with the notion that patient satisfaction has an underlying dimension to which all of the domains are highly related. This is evident, in our view, because of the high correlations in all positions of the matrix. However, there is also evidence of distinct domains. As noted, an instrument that yields measurements that have construct validity has higher correlations for items measuring the same domain than for items measuring different domains. The upper left portion of Table 4 contains the correlations among items from domain 1. These correlations averaged around .9. The correlations in the upper middle and right-hand portions of the table represent the relationships among items from domain 1 and the other domains. With a few exceptions, these correlations, although still positive, were lower than the correlations within items from domain 1. This result is consistent with construct validity. The same pattern of correlations is evident for domains 7 and 11 as well. It should be noted that this type of evaluation is not possible for domains with only one item.

Table 4. Multitrait-Multimethod Matrix(a)

                                     Domain

                                       1

Domain   Question
 No.       No.       11        17       19      20       21

   1        11      1.000    .8854    .9194    .8782    .9270
            17              1.000     .9117    .8687    .9157
            19                       1.000     .9236    .9476
            20                                1.000     .9214
            21                                         1.000
   2         7
   3        15
            18
   4        24
            25
   5        16
   6        10
   7        12
            13
   8        14
   9         9
  10         8
            22
  11        23
            26

                                  Domain

                        2            3                4

Domain   Question
 No.       No.          7       15       18       24      25

   1        11       .8432    .7535    .7486    .7887    .5501
            17       .8313    .7795    .7642    .7896    .5078
            19       .8511    .7903    .8021    .7837    .5359
            20       .8292    .7365    .7683    .7604    .5189
            21       .8501    .7721    .7963    .7821    .5296
   2         7      1.000     .7323    .7449    .7448    .5391
   3        15               1.000     .7430    .8021    .4749
            18                        1.000     .7270    .5643
   4        24                                 1.000     .6352
            25                                          1.000
   5        16
   6        10
   7        12
            13
   8        14
   9         9
  10         8
            22
  11        23
            26

                                       Domain

                        5       6            7            8

Domain   Question
 No.       No.         16       10       12       13       14

   1        11       .7634    .8686    .8589    .8936    .9972
            17       .7882    .8712    .8480    .8683    .8786
            19       .7739    .8950    .9017    .9145    .8956
            20       .7723    .8271    .8813    .8660    .8650
            21       .7634    .8731    .8958    .8985    .8940
   2         7       .7369    .8015    .8206    .8498    .8387
   3        15       .7482    .7508    .7913    .7908    .7769
            18       .7349    .7404    .7813    .7999    .7589
   4        24       .9057    .7613    .7921    .7876    .8029
            25       .5765    .5380    .5165    .5598    .5150
   5        16      1.000     .7441    .7851    .7807    .8011
   6        10               1.000     .8643    .9173    .8781
   7        12                        1.000     .9283    .9027
            13                                 1.000     .9263
   8        14                                          1.000
   9         9
  10         8
            22
  11        23
            26

                                   Domain

                       9        10               11

Domain   Question
 No.       No.         9         8       22      23        26

   1        11       .8763    .9014    .8912    .8967    .9149
            17       .8858    .8986    .8799    .8866    .8778
            19       .8830    .9184    .9152    .9246    .9309
            20       .8693    .8746    .8937    .8917    .9147
            21       .8812    .9014    .9359    .9345    .9379
   2         7       .8529    .8710    .8787    .8821    .8398
   3        15       .7802    .7926    .7632    .7701    .7501
            18       .7824    .7719    .7885    .7804    .7468
   4        24       .7507    .7509    .7834    .7862    .7984
            25       .5010    .4957    .5613    .5520    .5557
   5        16       .7355    .7378    .7562    .7590    .7774
   6        10       .8578    .8916    .8435    .8624    .8648
   7        12       .8828    .9034    .8803    .8733    .8551
            13       .9171    .9296    .8891    .9080    .8843
   8        14       .9299    .9102    .8849    .8980    .8657
   9         9      1.000     .9277    .8716    .8832    .8329
  10         8               1.000     .8988    .9175    .8810
            22                        1.000     .9792    .9111
  11        23                                 1.000     .9281
            26                                          1.000

(a) Domains:

1 = treatment,

2= policy,

3 = convenience of appointment time,

4 = cost,

5 = billing,

6 = ease of scheduling an appointment,

7 = scheduling,

8 = wait time,

9 = courteous staff,

10 = physical therapist courteous,

11 = overall satisfaction.

All correlation coefficients are significant (P<.01).

Construct validity, as estimated through the application of factor analysis, yielded results consistent with those from the multitrait-multimethod matrix. A principal-components analysis yielded one factor that accounted for nearly 83% of the variance (Tab. 5). This was the only factor to meet the minimum eigenvalue criterion of an eigenvalue greater than or equal to 1. The factor loadings and communalities for each variable are presented in Table 6. As shown in the table, all of the variables, with the exception of question 25 (ie, "If I had to, I would pay for these physical therapy services myself"), loaded highly on factor 1. Typically, an initial principal-components analysis is followed by a rotation that aids in the interpretation of the factors.[26] Rotation in this case was unnecessary because there was only one factor.

Table 5. Results of Principal-Components Factor Analysis

                                      Cumulative
                      Percentage      Percentage
Factor                of Variance     of Variance
No.      Eigenvalue   Accounted For   Accounted For

 1       16.5540         82.8             82.8
 2        0.8007          4.0             86.8
 3        0.4924          2.5             89.2
 4        0.3413          1.7             90.9
 5        0.2904          1.5             92.4
 6        0.2402          1.2             93.6
 7        0.2274          1.1             94.7
 8        0.1715          0.9             95.6
 9        0.1510          0.8             96.3
10        0.1343          0.7             97.0
11        0.1150          0.6             97.6
12        0.0894          0.4             98.0
13        0.0777          0.4             98.4
14        0.0714          0.4             98.8
15        0.0596          0.3             99.1
16        0.0526          0.3             99.3
17        0.0470          0.2             99.6
18        0.0367          0.2             99.8
19        0.0323          0.2             99.9
20        0.0152          0.1            100.0

Table 6. Factor Loadings and Communalities

Question   Factor
No.        Loadings   Communalities

 7         .90162       .81293
 8         .94992       .90235
 9         .93426       .87284
10         .91957       .84560
11         .94167       .88675
12         .93703       .87802
13         .95580       .91355
14         .94546       .89390
15         .84227       .70942
16         .84784       .71884
17         .93583       .87578
18         .84353       .71154
19         .96257       .92653
20         .93179       .86823
21         .95948       .92061
22         .95035       .90317
23         .95609       .91412
24         .86314       .74501
25         .60590       .36711
26         .94214       .88763

Discussion

Test developers typically strive for an instrument with a coefficient for reliability in the range of .80 to .90.[27] The coefficient for reliability computed from this questionnaire exceeded that criterion. In addition, there are other indications of the internal consistency for this instrument. For example, all of the correlations among items, except for those involving question 25, exceeded .7. As noted, question 25 appears to measure a slightly different dimension than the other questions. The correlations between question 25 and the other questions were in the range of .47 to .56, whereas the correlations among the other items ranged from .73 to .93. The findings of the factor analysis also are indicative of a high degree of internal consistency for the instrument.

The discovery of only one factor and a high coefficient for reliability may be a cause for concern. Several researchers[8,10,13,14] have hypothesized that patient satisfaction is a multidimensional phenomenon. However, there is no consensus on this point. Other researchers[15] have argued that there is little data regarding the type of information that contributes most to patients' satisfaction with the provision of health care. Perhaps, patient satisfaction is best described as a unidimensional construct, or as a construct that has one predominant dimension. Ratings of overall satisfaction may not be influenced to a great extent by experiences with ancillary aspects of care such as courtesy of support staff or the availability of parking.

The notion of a predominant dimension of satisfaction is consistent with our data on concurrent validity. In our analyses, the 3 measures of overall satisfaction were shown to be correlated highly with a composite score that was based on questions from all domains of satisfaction. Likewise, the analysis of construct validity via the multitrait-multimethod matrix and the factor analysis indicate that a single dimension accounts for most of the variance in the data.

Limitations

The generalizability of our results is limited by the fact that a sample of convenience (289 subjects selected from a network of 12 clinics) was used. Convenience samples are characterized by the inclusion of individuals who are conveniently available to participate in a study. These individuals may or may not be representative of the larger population. In our investigation, all of the study participants were patients from a single clinical network. Although the operation is large, and has several sites, each with multiple physical therapists, the fact that only one organization was used makes the sample homogeneous with respect to geographical region. More positively, however, it should be noted that the case mix with respect to presenting condition and treatment appears to be representative of the types of conditions for which physical therapists provide intervention.[28,29]

The fact that the organization we studied is large and growing is also supportive of the inference that it is a successful practice. The growth is evidence from the inclusion of 3 new settings as part of the practice within the past year. A successful practice in a competitive market must do a good job with respect to patient satisfaction, with all other factors held constant. Therefore, a broad range of ratings for satisfaction is not available on which to establish and evaluate the psychometric properties of the instrument. That is, the variability of the responses is limited because all of the ratings were highly positive. Although this narrow dispersion still permits the identification of questions that are not consistent with patient satisfaction, the results are limited because the properties of the instrument under less favorable conditions are unknown. Unfortunately, a recurring problem in this type of research is the fact that practices that anticipate lower patient satisfaction are less likely to participate in such studies.

Another possible limitation of the study concerns the evaluation of validity. Although the instrument has been shown to exhibit some content validity, concurrent validity, and construct validity, the degree of predictive validity was not assessed. Predictive validity is the degree to which an instrument correlates with an actual outcome. For example, the predictive validity of measurements obtained with a test for selecting lighter pilots might be evaluated by comparing test predictions with ratings from flight instructors after completion of flight school. In the case of patient satisfaction, outcome measures might be whether a patient returns to a facility or whether the patient recommends the facility to other people. No outcomes were measured in our study, and the predictive validity of measurements obtained with the instrument remains unknown.

Summary and Conclusions

Patient satisfaction remains an important concept for health care providers. We are not alone in believing that health care providers can no longer rely on sources of information that omit the patient's perspective in the definition of quality of care.[13] Based on the large number of instruments developed for practices,[16] physical therapists appear to have recognized the value of feedback from patients.

The instrument that we studied was shown to yield reliable measurements, and the measurements appear to have some content, concurrent, and construct validity. The results from both the reliability analysis and the. factor analysis suggest that a single dimension underlies patient satisfaction. This dimension appears to be manifested in all of the questions, but it is perhaps represented best in the questions pertaining to overall satisfaction. The analysis also revealed that only one question (ie, "If I had to, I would pay for these physical therapy services myself") was less correlated with the other questions on the instrument. Although this question addresses an important issue, that is, the cost equity of the provision of physical therapy services, it appears to be less related to a measure of patient satisfaction. Therefore, we do not recommend its inclusion in a composite measure of patient satisfaction.

Most instruments of patient satisfaction for physical therapists appear to have been evaluated only informally and with respect to content validity. Very few of the instruments contained in the Compendium were assessed for reliability or validity. Those that did undergo testing were analyzed within settings even more limited than the one in our study. Our instrument was subjected to some psychometric evaluation, and, although we believe that this evaluation is a step in the right direction, additional research assessing the instrument across a larger number of offices, clinics, or departments is warranted. This research will be essential to establish the utility of the instrument across the spectrum of quality in the provision of physical therapy services.

(*) SPSS Inc, 444 N Michigan Ave, Chicago, IL 60611.

References

[1] Brook RH, McGlynn EA. Maintaining quality of care. In: Ginzberg E, ed. Health Services Research. Cambridge, Mass: Harvard University Press; 1991:284-314.

[2] Ellwood PM Jr, Paul BA. But what about quality? Health Aff(Millwood). 1986;5:135-140.

[3] Donabedian A. The quality of care: how can it be assessed? JAMA. 1988;260:1743-1748.

[4] Hulka BS, Zyzanski SJ, Cassel JC, Thompson SJ. Scale for the measurement of attitudes towards the physician and primary medical care. Med Care. 1970;8:429-435.

[5] Vuori H. Patient satisfaction: an attribute or indicator of quality of care? Qual Rev Bull. 1987;13:106-108.

[6] Elliot KM, Hall MC, Stile GW. Service quality in the health care industry: how are hospitals evaluated by the general public? Journal of Hospital Marketing. 1992;7:113-124.

[7] Linder-Pelz SU, Struening EL. The multidimensionality of patient satisfaction with a clinical visit. J Community Health. 1985;10:42-54.

[8] Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality of care. Inquiry. 1988;25:25-36.

[9] Adams JS. Toward an understanding of inequity. In: Yukl GA, Wexley KN, eds. Readings in Industrial and Organizational Psychology. New York, NY: Oxford University Press; 1971:194-210.

[10] Linder-Pelz SU. Toward a theory of patient satisfaction. Soc Sci Med. 1982;16:577-582.

[11] Davies AR, Ware JE Jr, Kosinski M. Standardizing health care evaluations. Medical Outcomes Trust Bulletin. 1995;3(4):2-3.

[12] Davies AR, Ware JE Jr. GHAA's Consumer Satisfaction Survey and User's Manual. Washington, DC: Group Health Association of America; 1991.

[13] Nelson CW. Patient satisfaction surveys: an opportunity for total quality improvement. Hospital and Health Services Administration. 1990;35:409-425.

[14] Keith RA. Patient satisfaction and rehabilitation services. Arch Phys Med Rehabil. 1998;79:1122-1128.

[15] Edgman-Levitan S, Cleary PD. What information do consumers want and need? Health Aff (Millwood). 1996;15:42-56.

[16] Patient Satisfaction Instruments: A Compendium. Alexandria, Va: American Physical Therapy Association; 1995.

[17] Ware JE Jr, Davies-Avery A, Stewart AL. The measurement and meaning of patient satisfaction. Health Med Care Serv Rev. 1978;1:3-15.

[18] Ware JE Jr, Snyder MK, Wright WR, Davies AR. Defining and measuring patient satisfaction with medical care. Eval Prog Plann. 1983;6:247-263.

[19] Ware JE Jr, Hays RD. Methods for measuring patient satisfaction with specific medical encounters. Med Care. 1988;26:393-40.

[20] SPSS: Statistics Guide. Chicago, Ill: SPSS Inc; 1990.

[21] Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. East Norwalk, Conn: Appleton & Lange; 1993.

[22] Carey RG, Seibert JH. A patient survey system to measure quality improvement: questionnaire reliability and validity. Med Care. 1993;31:834-845.

[23] Loeken K, Steine S, Sandvik L, Laerum E. A new instrument to measure patient satisfaction with mammography: validity, reliability, and discriminatory power. Med Care. 1997;35:731-741.

[24] Magnusson D. Test Theory. Reading, Mass: Addison-Wesley Publishing Co; 1967.

[25] Rothstein JM, Echternach JL. Primer on Measurement: An Introductory Guide to Measurement Issues. Alexandria, Va: American Physical Therapy Association, 1993.

[26] Harman HH. Modern Factor Analysis. 3rd rev ed. Chicago, Ill: University of Chicago Press; 1976.

[27] Anastasi A. Psychological Testing. New York, NY: Macmillan; 1968.

[28] Practice Profile [unpublished data]. Alexandria, Va: American Physical Therapy Association; 1998.

[29] Roush SE, Sonstroem RJ. Development of the Physical Therapy Outpatient Satisfaction Survey (PTOPS). Phys Ther. 1999;79:159-170.

Appendix.

[TABULAR DATA NOT REPRODUCIBLE IN ASCII]

MS Goldstein, EdD, is Director, Research Services, Division of Practice and Research, American Physical Therapy Association, 1111 N Fairfax St, Alexandria, VA 22314 (USA) ([e-mail]). Address all correspondence to Dr Goldstein.

SD Elliott, PhD, is Director of Analytic Support, Division of Practice and Research, American Physical Therapy Association.

AA Guccione, PT, PhD, FAPTA, is Senior Vice-President, Division of Practice and Research, American Physical Therapy Association.

All authors provided concept/research design, writing, and project management. Dr Goldstein and Dr Guccione provided data collection, and Dr Goldstein and Dr Elliott provided data analysis. The authors acknowledge the assistance of participating physical therapists at clinics throughout Louisville, Ky, without whom this research could not have been completed. Special thanks go to Laurence Benz, PT, MA, OCS, who played a vital role in the design of the study and coordinated the data collection phase of the research.

This article was submitted December 1, 1999, and was accepted May 10, 2000.