Development of the Physical Therapy Outpatient Satisfaction Survey (PTOPS).

Background and Purpose. The purposes of this S-phase study were (1) to identify the underlying components of outpatient satisfaction in physical therapy and (2) to develop a test that would yield reliable and valid measurements of these components. Subjects. Three samples, consisting of 177, 257, and 173 outpatients from 21 facilities, were used in phases 1, 2, and 3, respectively. Methods and Results. In phase 1, principal component analyses (PCAs), reliability checks, and correlations with social desirability scales were used to reduce a pool of 98 items to 32 items. These analyses identified a 5-component model of outpatient satisfaction in physical therapy. The phase 2 PCA, with a revised pool of 48 items, indicated that 4 components rather than 5 components represented the best model and resulted in the 34-item Physical Therapy Outpatient Satisfaction Survey (PTOPS). Factor analyses conducted with phase 2 and phase 3 data supported this conclusion and provided evidence for the internal validity of the PTOPS scores. The 4-component scales were labeled "Enhancers," "Detractors," "Location," and "Cost." Responses from subsamples of phase 3 subjects provided evidence for validity of scores in that the PTOPS components of "Enhancers," "Detractors," and "Cost" appeared to differentiate overtly satisfied patients from overfly dissatisfied patients. "Location" and "Enhancer" scores discriminated subjects with excellent attendance at scheduled physical therapy sessions from those with poor attendance. Conclusion and Discussion. In this study, we identified components of outpatient satisfaction in physical therapy and used them to develop a test that would yield valid and reliable measurements of these components. [Roush SE, Sonstroem RJ. Development of the Physical Therapy Outpatient Satisfaction Survey (PTOPS), Phys Ther. 1999;79:159-170.]

Key Words: Patient satisfaction, Physical therapy, Survey development.

Patient satisfaction has become an increasingly important issue in health care.[1,2] Many contemporary trends, including managed care and continuous quality improvement, have highlighted the importance of the consumer's perspective in the delivery of health care.[3,4]

We believe that patient satisfaction has not been closely monitored in physical therapy. Reports in the literature for determining the dimensions of patient satisfaction in physical therapy or for developing tests that will yield reliable measurements are lacking. This deficiency contrasts sharply with the information for other health care professions (eg, medicine,[5-7] nursing[8,9]). One approach that has been taken to measure satisfaction in physical therapy has utilized instruments designed for other disciplines.[10] Although it could be advantageous to show that patient satisfaction can be uniformly measured across varied health care disciplines and that a physical therapy-specific instrument is not necessarily needed, there may be discipline-specific differences in the delivery of care that make generic satisfaction measures impractical. Another approach taken to measure patient satisfaction in physical therapy has used informally developed physical therapy-specific instruments.[11] Typically, these instruments rely solely on face validity or content validity, which limits the utility of the results. Psychometric data on reliability and validity are typically not available.

The lack of an appropriate instrument means that satisfaction cannot be used as a variable in clinical research in physical therapy. The purposes of our study, therefore, were (1) to identify the underlying components of satisfaction in physical therapy and (2) to develop a test that would yield reliable and valid measurements of these components. A multistage process consisting of 3 phases, each with its own sample, was used. This study focused on outpatients and resulted in the creation of the Physical Therapy Outpatient Satisfaction Survey (PTOPS).

Literature Review

Interest in patients' evaluations of the care they receive has arisen primarily out of the consumer movement.[12] Advocates contend that health care should exist within a negotiated model of shared power and responsibility between the consumer and the service provider. This approach makes assessment of patient satisfaction a key component of comprehensive program evaluation in health care.[13,14]

Patients report high levels of satisfaction with their health care,[15] although race and sex differences have been reported.[3] Hsieh and Kagle[3] summarized the literature on correlates of patient satisfaction with physicians and found that men were less satisfied than women, African Americans were less satisfied than Caucasians, and elderly people were more satisfied than members of other age groups. Additionally, people in poor health were less satisfied than those in good health, and people receiving care through prepaid group practices were less satisfied than those receiving care through fee-for-service practices.

Patient satisfaction has been conceptualized in recent years as a multidimensional construct.[16] The multi-dimensional nature means that a person may be highly satisfied with one or more aspects of a health care encounter and simultaneously dissatisfied with other aspects. Numerous satisfaction dimensions have been identified in the literature, although there is no agreed-on list of standard dimensions. Similarities that appear to be mentioned most often include (1) provider conduct (eg, technical competence, personality attributes),[3,5,6,17] (2) accessibility and convenience,[3,6,7,14] (3) finances (ie, the ability to pay for treatment or make arrangements for payment),[17,18] (4) the physical environment in which the care is offered, including seating, lighting, and noise level,[3,16,14] and (5) expectations.[3,6,14]

Patient satisfaction is not well understood in physical therapy. Although the use of informal patient satisfaction surveys is increasing in the profession.[11] we believe that the lack of a standardized data collection instrument has hindered formal research. Several points about patient satisfaction in physical therapy, however, have emerged. The available data show that patients are often highly satisfied with their physical therapy,[10] as they are with other health care professionals.[15] Also consistent with the literature from other health care professions, provider conduct has been identified as a factor in the satisfaction of patients with physical therapy.[10] Data show that provider characteristics of friendliness and caring are most highly regarded by patients receiving physical therapy.[10] Satisfaction differences between male and female patients have not been reported, and data from Roush[20] showed no correlation between satisfaction and degree of disability in a group of patients with multiple sclerosis. This latter result contrasts with the positive relationship reported between patient's health and satisfaction with physicians.[3]

The development of a test that yields valid measurements of patient satisfaction requires removing response biases. A response bias is "a systematic tendency to respond to a range of questionnaire items on some basis other than the specific item content."[21](p17) Responding to items in a manner believed to be socially desirable is a response bias that can be controlled by measuring it along with the survey content and deleting survey items that correlate highly with social desirability scores.

Paulhus[22] has determined that social desirability consists of 2 distinct components: impression management (IM) and self-deceptive enhancement (SDE). Both of these components are assessed by his Balanced Inventory of Desirable Responding (BIDR).[22] Impression management represents the conscious distortion of responses so as to convey the best impression possible. This construct has developed large correlations with conventional lie scales.[22] Self-deceptive enhancement is a tendency to present self-reports that are thought to be honest but positively biased (exaggerated). In our study, we used IM as the criterion indicative of deception.

Phase 1 -- Initial Development

Method

Data collection. Seven hospital-based and private outpatient physical therapy practices in southeastern New England participated in the data collection for phase 1 of this study. These facilities varied in size and setting (urban/rural) and employed between 2 and 9 outpatient therapists, although not all therapists were seeing outpatients full-time. Sampling included all outpatients scheduled for physical therapy services on a day when peak patient volume was expected. The following patients were excluded: (1) patients under 18 years of age, (2) patients being seen for initial evaluation on the day of data collection, and (3) patients who had a cognitive inability to respond to the survey (even when the survey was read to them).

On the selected day, subjects were approached by one of the investigators during nontreatment time and were asked to complete the satisfaction survey and the BIDR. Informed consent was obtained from each subject (in this and all subsequent phases) in accordance with the policies of the University of Rhode Island's Institutional Review Board. Data collection was anonymous; subjects were instructed not to identify themselves or their therapists on the survey instrument. Subjects were encouraged to ask questions if clarification was needed, and the investigators were available to read the survey instructions to subjects if they requested. Each participating subject received a token gift valued at 50 cents (eg, playing cards, notepaper, scented soap, refrigerator magnets) after completing the survey.

There were 258 eligible subjects scheduled for physical therapy on the days of data collection. Complete data were obtained from 177 subjects, for a 69% return rate. The major reasons for nonparticipation were incomplete survey instruments and survey instruments not returned. Overt subject refusal was minimal across all phases of this study. Descriptive statistics for these subjects are provided in Table 1.

Table 1.

Descriptive Subject Data for Phases 1, 2, and 3

                                      Phase 1      Phase 2

n                                     177          257

Age (y)
 Range                                19-76        18-87
 [bar] X                              45.2         46.7
 SD                                   15.6         16.31

Gender (M/F%)                         40/60        43/57

Self-reported diagnostic categories
 (percentages per phase)

 Orthopedic
   Upper extremity                     17.5        19.1
   Lower extremity                     25.4        20.6
   Trunk/spine                         27.7        34.2
   Other                                8.4         7.4
   Total                               79.1        81.3

 Neurological                           6.2         7.4

 Other                                  2.0         5.8

 Unspecified                           11.9         5.4

 Totals (%)                           100          99.9(b)

                                        Phase 3
                                        Subsample(a)

                                       A           B

n                                      65          49

Age (y)
 Range                                 18-79       25-74
 [bar] X                               40.9        46.9
 SD                                    16.3        10.3

Gender (M/F%)                          39/61       30/70

Self-reported diagnostic categories
 (percentages per phase)

 Orthopedic
   Upper extremity                     27.7        0.0
   Lower extremity                     33.8        0.0
   Trunk/spine                         24.6        0.0
   Other                                3.1        0.0
   Total                               89.2        0.0

 Neurological                           6.1      100.0

 Other                                  1.5        0.0

 Unspecified                            3.1        0.0

 Totals (%)                            99.9(b)   100.0

                                   Phase 3
                                   Subsample(a)

                                       C           Total

n                                      59          173

Age (y)
 Range                                 23-74       18-79
 [bar] X                               61.1        49.4
 SD                                    15.6        16.9

Gender (M/F%)                          37/63       36/64

Self-reported diagnostic categories
 (percentages per phase)

 Orthopedic
   Upper extremity                      8.5        13.3
   Lower extremity                     24.4        21.4
   Trunk/spine                         13.5        13.9
   Other                               30.5        11.6
   Total                               77.9        60.2

 Neurological                          16.9        36.4

 Other                                  5.1         2.3

 Unspecified                            0.0         1.2

 Totals (%)                            99.9(b)    100.0

(a) Subsample A=physical therapy outpatient, subsample B=self-report subjects, subsample C=attendance subjects.

(b) Rounding error.

Item pool development. Because there is a degree of inconsistency in the health care literature concerning satisfaction dimensions, item pool development in this study focused on the dimensions reported most frequently in health care research. Following the general guidelines of Fitzpatrick[23] and Marks,[24] 98 survey items were written to assess the following identified health care satisfaction dimensions: (1) provider conduct, (2) accessibility/convenience, (3) cost, (4) physical environment, and (5) expectations. These 5 dimensions were considered as hypothetical components of outpatient satisfaction in physical therapy. Both positively and negatively worded items were used, and the survey used a 5-point Likert scale with responses ranging from "strongly disagree" to "strongly agree."

The developed item pool was reviewed for content validity by a physical therapist who has studied patient satisfaction.[25] Suggestions for improvement were incorporated into the survey.

In addition to the satisfaction items, subjects also completed the BIDR. The BIDR uses a 7-point Likert scale and contains 40 items (20 items for each subscale, as previously described). Cronbach reliability alphas ranging from .80 to .86 have been reported for IM and from .70 to .82 for SDE. Five-week test-retest Pearson coefficients (r) have been reported at .65 and .69 for IM and SDE, respectively.[21]

The phase 1 results are based on principal component analysis (PCA). Principal component analysis is a statistical technique that is used to reduce the number of variables into a smaller number based on similarities (or correlations) among these variables. Components are believed to represent underlying concepts that account for the relationships among the original variables.

With PCA, a correlation matrix of variables (in this case, the satisfaction inventory items) is used to identify, or extract, combinations of these variables to form components. A new correlation matrix, between the original variables and these newly identified components, is formed. The correlations between a component and its variables are called component loadings. Items with large component loadings on only one component are retained. Large component loadings indicate a more reliable component.

Interpreting the results of a PCA can be problematic due to the complexity and number of relationships among components and variables. Rotation of a component solution is a technique that can be used to increase the interpretability of a PCA solution, but at the same time does not change its fundamental qualities. Rotation is a process that improves the component loadings between components and their respective variables. Different rotational options are available, including rotations that keep the components uncorrelated, or orthogonal, and other rotations that allow the components to be intercorrelated or nonorthogonal.

Determining the number of factors to rotate is an extremely important but, in our view, somewhat imprecise part of the PCA process. We used the Scree Test to determine the number of components to rotate. The Scree Test evaluates a plot of eigenvalues and attempts to identify a point where the slope of the decreasing eigenvalues begins to level off. An eigenvalue is a mathematical expression associated with each component and represents the amount of variance explained by that component. Further information on PCA is available in other sources.[26,27]

Results

Principal component and reliability analyses. Inventory items failed to form distinct components in initial PCAs. This finding was attributed to the high positive responses given to many items, which is frequently seen with patient satisfaction data.[15,28] This similarity in item response served to reduce differences among items and prevented the separation of items into components based on differences across people. In an attempt to increase item response differences, 15 items with means greater than 4.5 (or less than 1.5 if negatively worded) were deleted from subsequent analyses. To gain a general look at the component structure of the data, the remaining 83 items were submitted to a PCA with subsequent varimax rotation.

As previously stated, the determination of the number of components to be rotated and included in an inventory is a somewhat imprecise procedure. Therefore, 4-, 5-,and 6-component models were all tested. Items were deleted when their largest component loading was less than .40 or when the difference between this largest component loading and any other component loading was less than .20. On the basis of item meaning, and as supported by the Scree Test,[29] the 5-component solution was found to provide the best fit between components and items. Items were next deleted to improve component reliability or because of significant associations with BIDR scales. This left 32 items as best representing the 5-component model. When those 32 items were used in a second PCA, the identified 5-component structure was reproduced with all 32 items loading on their respective component. Component loading was high (.49-.83) and indicated component independence (ie, items loaded on a single component without sizable loadings on other components). Cronbach alphas ranged from .71 to .85, representing acceptable to very good values.[30] The components were tentatively labeled as follows: component 1="Satisfaction Enhancers," component 2="Satisfaction Detractors," component 3="Location," component 4="Cost," and component 5="Expectation." This 5-component solution from phase 1 is summarized in Table 2.

Table 2.

Satisfaction Components Identified in Phases 1 and 2

Phase   Solution      Component

1       5-component   Enhancers
                      Detractors
                      Location
                      Cost
                      Expectations

2       5-component   Enhancers
                      Detractors
                      Location
                      Cost
                      Expectations

2       4-component   Enhancers
                      Detractors
                      Location
                      Cost

Phase   Accounted   No. of   Cronbach
        Variance    Items    Alpha

1       47.9%        7       .75
                     9       .80
                     5       .84
                     6       .77
                     5       .71

2       47.5%       14       .84
                     7       .74
                     7       .88
                     7       .71
                     3       .46

2       46.7%       10       .86
                    10       .82
                     7       .87
                     7       .71

Phase 2 -- Instrument Refinement

Method

Item pool enhancement. In order to better define dimensions and to maximize reliability of PTOPS measurements, a pool of 48 items was developed. The 32 items identified in phase 1 were the basis for this pool. New items were added, however, to increase the reliability of scores for the projected components, and several of the phase I items were rewritten in an effort to better reflect content. The resultant 48-item pool consisted of 8 "Location" items, 8 "Cost" items, 9 "Expectation" items, 11 "Detractor," and 12 "Enhancer" items. Items in the "Expectation" component were rewritten to avoid repeated use of the phrase "I expect." All of the "Expectation" items identified in phase 1 contained this phrase We hypothesized that subjects might be responding to this consistent phrase and not the full content of these items, thus producing a component based on an artifact, not content. We also hypothesized that the phase 2 analyses would identify the same 5 dimensions as identified in phase 1.

Eleven hospital and private outpatient physical therapy practices in southeastern New England participated in the phase 2 data collection. Again, the facilities varied in size and setting, employing between 1 and 8 outpatient therapists. Procedures were identical to those used in phase 1. There were 323 eligible subjects scheduled for treatment on the days of data collection. Usable data were obtained from 257 subjects, giving an 80% return rate. This 80% return rate was an improvement over the 69% return rate in phase 1 and was perhaps attributable to the shortened questionnaire. The shortened questionnaire contained only 48 items, which did not include BIDR items. Descriptive statistics for these 257 subjects are provided in Table 1.

Results

Principal component analysis. The 48 inventory items were submitted to a PCA.[29] The Scree Test[28] again suggested retaining a 5-component solution. Further analyses considered the 4-, 5-, and 6-component solutions, with both varimax and oblimin rotations. Varimax requires orthogonal relationships (ie, it attempts to develop independent components that are essentially unrelated).[29] We used it in phase 1 in an effort to better define components underlying the satisfaction items. Oblimin rotation permits associations among components and is believed to better represent actual relationships among attitude or belief components.[29] Therefore, oblimin rotation was used as the method of record in this final PCA. Across all analyses in both phases, almost identical results were obtained for the 2 methods. The criteria for retaining items were a minimum loading of .40 with a loading difference of at least .10 between the largest loading and any other component loading.

These PCAs identified 39 items in the model A 5-component solution and 36 items in the model B 4-component solution. The 6-component model produced a component consisting of 2 items with no apparent similarity in meaning. Thus, we discounted it as an effective solution. One item was subsequently deleted from both models A and B in order to improve reliability. The model A and B pools of 38 and 35 were resubmitted to PCAs and reliability analyses, which produced the results shown in Table 2. Model A retained all 38 items, and model B was reduced by 1 to 34 items. Without exception, components identified in the first PCA analysis of phase 2 reappeared in the second series. The variance explained was 47.4% for model A and 46.7% for models B. Again, component loadings were high, which indicates excellent component saturation, identified by Guadagnoli and Velicer[31] as the best predictor of model validity, as obtained by factor analysis.

We concluded, subject to confirmatory factor analysis (CFA), that the 4-component solution represented the best model for the following reasons. First, a scale composed of only 3 items (ie, "Expectation" in model A) tends to be unreliable.[32] Second, we concluded that an expectancy component does not actually exist in outpatient satisfaction with physical therapy as developed by the present data. Ten items were written for the "Expectancy" dimension. Three items loaded on component 1, and only 3 items loaded on a unique component. The 3-item cluster that we obtained was susceptible to a variety of interpretations. Finally, the "Detractor" component of the 4-component solution contained 3 additional items and was more reliable than the "Detractor" component of the 5-component solution.

Confirmatory factor analysis. Although PCA represents an exploratory and descriptive approach to data analysis, CFA requires that hypotheses be developed in advance and that relationships between measures (in this case, items) and latent variables (in this case, components) be identified.[33] The CFA then tests these associations after correcting for measurement error. The presence of 3 to 14 items per component permits an estimation of measurement error. A successful test of the model requires that all of the associations between items and their hypothesized components be significant and that the model fit the data well. It is also possible to statistically examine the goodness of fit between data and a hypothesized model. A single satisfactory goodness-of-fit statistic, however, does not exist. The following 4 statistics (with corresponding evaluative standards) are generally used to evaluate CFA results (ie, measure the degree of fit between models and the data): (1) a smaller chi-square value and a ratio of chi-square to degrees of freedom less than 5,[34] (2) an average absolute standardized residual (AASR) equal to or less than 06,[34] (3) a nonnormed fit index (NNFI) greater then .75,[35] and (4) a comparative fit index (CFI) greater than .90.[36] Although a more detailed explanation of these statistics is beyond the scope of this article, the reader is encouraged to refer to the references.

The EQS computer program[33] was used for these analyses. Because of earlier reported skewness and kurtosis in the data, a generalized least squares solution (elliptical distribution theory) was used. Coefficients were fixed at 1.00 for one item of each component to provide for measurement equivalence across components.

Phase 2 CFAs are summarized in the first 4 rows of Table 3. Fit indices for model A (5-component solution) represent an excellent data fit. This data fit, however, was improved upon for all fit indices by model B (4-component solution). Model B's ratio of chi square to degrees of freedom was only 1.24. The NNFI and CFI values shown in Table 3 represent excellent fit values. All items loaded on their hypothesized components and on no others in both models. The data fit of model A and the data fit of model B were compared by subtracting chi-square values and degrees of freedom for model B from those same values for model A. The fourth row of Table 3 shows that the difference in chi-square values relative to degrees of freedom was significant ([chi-square] = 228.23, df = 521, P [is less than] .01). This finding indicates that model B provides a superior fit to phase 2 data as compared with model A. These results assure us that the 4-component solution (model B) provides an excellent data fit that is also superior to that of the 5-component solution (model A).

Table 3.

Fit Indices for Confirmatory Factor Analyses in Phases 2 and 3

Model         [chi-square]   p         df

Phase 2
 Model A      875.40         <.001     655
 Model B      647.17         <.001     521
 Difference   228.23         <.01      134
   (A-B)
Phase 3
  Model B     671.73         <.001     521

Model         AASR(a)        NNFI(b)   CFI(c)

Phase 2
 Model A      .049           .964      .967
 Model B      .046           .977      .978
 Difference
   (A-B)
Phase 3
 Model B      .057           .972      .974

(a) AASR=average absolute standarized residual.

(b) NNRI=nonnormed fit index.

(c) CFI=comparative fit index.

Phase 3 -- Structural and External Validity

Method

Three subsamples of data were used to examine the structural and external validity of the PTOPS. Subsample A data were collected at 3 outpatient facilities with procedures identical to those utilized in phases 1 and 2. These facilities employed between 2 and 4 outpatient therapists. There were 86 eligible subjects scheduled on the days of data collection, and 65 subjects provided usable data, for a 76% return rate. The BIDR data were also collected from these subjects.

Data collection procedures for subsamples B and C were different from the previously utilized protocol. An elaboration of the process of construct validation will explain the necessity of this change. Construct validation entails comparing survey results with hypothesized related behavior and was indicated in the development of the PTOPS because there is no patient satisfaction "gold standard." Two types of related, collaborative evidence were considered in this study: self-reported satisfaction and scheduled treatment attendance. The self-report subjects (subsample B) were recruited through the Multiple Sclerosis societies of Connecticut, Massachusetts, New Jersey, and Rhode Island. Subjects were asked to complete the PTOPS if they had had a particularly positive or a particularly negative physical therapy experience. We hypothesized that subjects reporting high satisfaction with their physical therapy experience would obtain higher scale scores on the PTOPS than those reporting low satisfaction. Sixty-nine subjects responded to recruitment efforts, and usable data were obtained from 49 self-report subjects, for a 71% return rate. Thirty-seven of these subjects reported high satisfaction, and 12 subjects reported low satisfaction.

Subsample C (those subjects selected in relationship to their scheduled treatment attendance) were recruited through a large outpatient physical therapy department associated with an 82-bed rehabilitation facility. The attendance records were reviewed over a 7-month period, and 2 categories of subjects were identified: those who attended 85% or more of their scheduled appointments and those who attended 50% or fewer of their scheduled appointments. Data from the subsample C were collected through the mail, after an introductory telephone call. Exclusion criteria utilized in phases 1 and 2 were used for both subsamples B and C. Subjects receiving workers' compensation were excluded from the subsample C. We hypothesized that subjects who had high attendance would score higher than those who had low attendance. Survey instruments were sent to 84 subjects in subsample C, and 59 subjects provided usable data, for a 70% return rate. Fifty-one of these subjects had high attendance, and 8 subjects had low attendance.

In summary, subsample A data (PTOPS and BIDR) came from 65 physical therapy outpatients, subsample B data (PTOPS only) came from 37 subjects reporting high satisfaction and 12 subjects reporting low satisfaction with their physical therapy, and subsample C data (PTOPS only) came from 51 subjects with high attendance and 8 subjects with low attendance. Descriptive statistics for each phase 3 subsample and for the total sample are provided in Table 1.

Results

Model validity (as obtained through factor analysis). The CFA of the total phase 3 sample of 173 subjects was used to examine the model validity and reliability of the PTOPS scores. The results of this analysis are presented in Table 3. The last row of Table 3 indicates that model B's structure (as identified in phase 2) provided an excellent fit to data from a new sample of subjects. Its fit indices, especially the NNFI and CFI values, were excellent and confirm that satisfaction responses of patients receiving physical therapy are explained very well by the PTOPS. All associations between individual items and hypothesized components were significant, with an average regression coefficient (as identified in the CFA) of .61.

Intercomponent relationships. As an additional test of component independence, Pearson correlation coefficients were calculated among components for both phase 2 data (upper values in Tab. 4, n=257) and phase 3 data (lower values in Tab. 4, n= 173). Without exception, coefficients were very similar across the 2 samples, indicating the presence of stable relationships among components. From these 2 analyses, we concluded that the PTOPS possesses a reliable structure that is reproducible across groups of physical therapy outpatients. In terms of between-component associations (Tab. 4), none of the coefficients of determination ([r.sup.2]) were greater than .37. We may conclude, therefore, that PTOPS components are independent. Coefficients of determination greater than .5 indicate that a majority of variance is shared or mutual, rather than unique to independent components. Table 4 data clearly show that the "Enhancer" and "Detractor" components were most closely related, although negatively. The "Cost" component contained the largest amount of unique variance; that is, it was least related to the other 3 components.

Table 4.

Physical Therapy Outpatient Satisfaction Survey Subscale
Intercorrelations Between Phase 2 and Phase 3 Samples and
Phase 3, Subsample C Associations With Social Desirability

                                                     BIDR(a)
             Phase 2(n=257)                          (n=67)
Phase 3
(n=175)     Enhancer   Detractor   Locator   Cost     IM      SDE

Enhancer                -.50(b)     .46(b)   -.11     .17     .13
Detractor   -.61(b)     -.30        .03      -.07            -.25(c)
Locator      .43(b)     -.47(b)              -.00     .23     .12
Cost        -.13         .11        .05              -.15    -.01

(a) BIDR=Balanced Inventory of Desirable Responding,[22] IM=impression

(b) P [is less than] .01.

(c) P [is less than] .05.

Social desirability relationships. To determine whether efforts had been successful in developing a satisfaction inventory free of social desirability influences, the PTOPS subscales were correlated with the IM and SDE subscales of the BIDR for the 65 subjects in subsample A. The sixth column of Table 4 shows that none of the 4 components correlated significantly with IM (recall that IM measures conscious distortion). The seventh column of Table 4 indicates that "Detractors" was the only PTOPS component to relate with SDE (r=-.25, P[is less than] .05). This finding tells us that low scores on the "Detractor" component were associated with high SDE scores. It would appear that some respondents who scored low on the "Detractor" component (consistent with high levels of overall satisfaction) were motivated by socially desirable reasons to exaggerate a lack of dissatisfaction.

The BIDR sample means were slightly higher than means reported in the BIDR manual that were elicited under "play up your good points" instructions.[22] We were somewhat surprised, therefore, that associations between PTOPS and BIDR scales were not larger. The BIDR manual reports BIDR correlations with 9 major personality and life-adjustment standardized scales. The median correlation with SDE was .38, with values as large as .52. The median correlation with IM was .17, with values as large as .35. Similar-sized BIDR relationships with self-esteem and other adjustment variables have been obtained.[37] The data presented in Table 4 indicate that PTOPS components are less related to social desirability measures than many better known standardized tests.

External validity. Stepwise discriminant function analysis was used to test the ability of the PTOPS to statistically separate the 2 satisfaction groups in subsample B. Component means of the high and low satisfaction groups are displayed in Table 5. A significant discriminant function was formed by the selection of "Enhancers" first, "Detractors" second, and "Cost" third. A canonical R correlation of .79 (P [is less than] .001) was obtained. When function scores were used to classify subjects as having either high or low satisfaction, the function correctly classified 10 of 12 reporting low satisfaction and 35 of 36 subjects reporting high satisfaction. Overall, correct classification was 93.8%. The pooled within-group correlation coefficients indicate that the "Enhancer" and "Detractor" scales were most important in separating the groups, whereas financial considerations improved the differentiation slightly (Tab. 5). The nature of the criterion variable (satisfaction versus dissatisfaction), the selection of the "Enhancer" and "Detractor" scales, and the strength of the association provides the PTOPS with strong content validity and construct validity.

Table 5.

Validity of the Physical Therapy Outpatient\Survey (PTOPS): Mean
PTOPS \Subscale Scores Across Four Criteria

                                 Component Scale
Group              n    Enhancer   Detractor   Locator    Cost

Self-report
satisfaction
 High              37   4.38       1.81        3.90       2.89
 Low               12   3.04       3.38        3.36       3.06
 Functional              .803(d)    .837(d)    -.256       .131(d)
 coefficient(c)

Attendance
 High              51   4.09       1.92        4.12       3.07
 Low                8   3.65       2.16        3.39       3.11
 Functional             .803(d)    -.483        .903(d)   -.060
 Coefficient

Patient Sex
 Male              37   3.93       2.13        3.80       3.02
 Female            70   4.10       2.03        3.96       3.01
 F                      1.74       0.37        0.80       0.01

Therapist sex
 Male              17   4.13       1.79        4.12       3.12
 Female            81   4.10       2.07        3.93       2.99
 F                      0.03       1.83        0.69       1.62

                   Canonical   P      Classification
                   R(a)               Accuracy(b)

Self-report
satisfaction
 High
 Low
 Functional        .79         .001   93.8%
 coefficient(c)

Attendance
 High
 Low
 Functional        .39         .01    88.1%
 coefficient

Patient Sex
 Male
 Female
 F

Therapist sex
 Male
 Female
 F

Stepwise discriminant function was again used to test the ability of the PTOPS to separate the previously identified subjects with high and low attendance in subsample C. Means for these 2 groups are presented in Table 5. A significant discriminate function was obtained that selected "Location" and "Enhancers" in that order. The canonical R correlation was .39 (P [is less than] .01). The functional coefficients demonstrate the powerful effect of facility location, first, and overall enhancing features, second, at influencing attendance at scheduled physical therapy sessions. The presence of "Location" as the premier predictor of patient attendance replicates exercise adherence research, where facility location has been identified as the most reliable predictor of adherence.[38] Although our data lack the temporal sequence necessary in testing true predictive validity, they support the development of hypotheses for future research in patient attendance.

We also examined scores to determine whether patient gender and therapist gender would need to be considered in subsequent PTOPS research. The PTOPS scale means for male and female subjects and F values for analysis of variance (ANOVA) comparison tests are presented in Table 5. Discriminant function analyses failed to obtain a significant function for either of these 2 independent variables. In addition, a single-classification ANOVA demonstrated no PTOPS scale differences between male and female patients, or between patients of male and female therapists. These results indicated that the PTOPS can be administered to all patients receiving physical therapy and that the values shown in Table 5 apply to patients of both genders and to patients of both male and female therapists.

PTOPS summary. The PTOPS item statements, along with administration, scoring, and interpretation instructions, are shown in the Appendix. Definitions for the 4 outpatient satisfaction dimensions follow:

   Enhancers -- contentment with the physical environment and the personal
   interactions associated with a clinic visit. The issues relate to
   enhancements that enrich a patient's experience beyond a minimally
   acceptable level.

   Detractors -- acknowledgment of a patient's basic physical and
   interpersonal needs that, if not present, create negative feelings, but
   that, if present, are not necessarily associated with positive feelings.
   Perceptions of the provider's behavior are particularly salient.

   Location -- ease of locating and traveling to a clinic.

   Cost -- compatibility between the perceived value of the provided service
   and the cost, which relates to both money and convenience.

The development of norms is beyond the scope of this article. Means and standard deviations for data obtained in this study are listed in Table 6.

Table 6.
Physical Therapy Outpatient Satisfaction Survey Descriptive
Statistics Across 2 and 3 Samples

            Phase 2       Phase 3
            (n=257)       (n=173)

            X      SD     X      SD

Enhancer    4.28   0.56   4.19   0.59
Detractor   1.79   0.60   1.87   0.72
Locator     3.99   0.82   4.06   0.81
Cost        2.95   0.31   2.97   0.39

Discussion

In this multiphase project, we identified 4 underlying dimensions of outpatient satisfaction in physical therapy and developed a measure that we contend has acceptable reliability and validity to assess that satisfaction. These dimensions are both similar and dissimilar to the satisfaction literature for other health care professionals.

Location and cost have been identified as satisfaction or adherence factors in other disciplines, and they were manifested in our analyses. Their strong presence in the PTOPS reinforces the premise that outpatient satisfaction in physical therapy is greatly influenced by nonclinical issues. Location may influence outpatient satisfaction in physical therapy because of the high frequency of visits typical of physical therapy in this setting (as opposed to physician visits) and because patients may perceive that all physical therapy facilities offer the same generic level of care, with no individual facility worthy of long or complicated travel. We believe that the stability of the "Cost" component reinforces the idea that financial matters, particularly perceived financial value and convenience, are part of outpatient satisfaction in physical therapy. We anticipate that, as managed care continues to influence the delivery of health care, financial value will continue to be a consideration for patients.

The "Enhancer" and "Detractor" components seem to conceptually parallel the industrial management concepts of "Satisfiers" and "Dissatisfiers," as conceptualized by Herzberg.[39] A brief review of Herzberg's work may be helpful in illustrating these parallels. Herzberg studied worker satisfaction in the employment sector in the 1960s, defining his Motivation/Hygiene Theory of worker satisfaction.[39] He conceptualized satisfiers and dissatisfiers (also known as motivators and hygiene factors, respectively) as separate constructs, not opposite ends of the same continuum. Satisfiers are those aspects of a job that motivate an employee. Conversely, dissatisfiers are those aspects of a job that can contribute to lessened motivation when they are not adequately present, but that do not heighten motivation when present. Classic dissatisfiers include pay, supplemental benefits, and working conditions. Satisfiers include recognition, achievement, and advancement. Employment satisfiers and dissatisfiers parallel the PTOPS "Enhancer" and "Detractor" components, respectively.

In physical therapy, the "Detractor" component relates to patients' basic physical and interpersonal desiderata when receiving therapy. Following Herzberg's theory, fulfillment of these desires will lessen dissatisfaction but will not necessarily ensure satisfaction. Conversely, items within the "Enhancer" component address issues related to personal care enhancement that enrich a patient's physical therapy experience beyond the minimally acceptable level. These issues include patient privacy, respect, and affirmation by the facility staff. Satisfaction with these enhancing issues promotes overall satisfaction with the physical therapy experience. Consistent with Herzberg's conceptualization, the present analysis shows that "Enhancers" and "Detractors" in physical therapy are separate constructs, not opposite ends of a single construct. Just as employers have used Herzberg's theory to enhance worker productivity by addressing both hygiene and motivational factors, physical therapy professionals can develop techniques to improve patient satisfaction by addressing both the enhancing and detracting elements of the experience.

A striking feature of the PTOPS is the lack of a distinct "Provider Conduct" component, as is typically seen in the physician satisfaction literature.[3,6,5,17] In our research, therapist conduct was embedded in many "Enhancer" and "Detractor" items, indicating that general enhancement and detraction provided stronger bases for explaining satisfaction then did a component of provider conduct.

A consistent finding in the patient satisfaction literature for physicians is that expectations account for the most variance.[3,6,14] Although our data initially suggested an "Expectation" component, further analyses discounted its existence. There could be many reasons why expectations may not be paramount in outpatient satisfaction with physical therapy. One consideration is the varying characteristics of typical patient/professional encounters in physical therapy and medicine. Physician visits are often limited in number, but are often clouded by fear and uncertainty as the patient seeks a diagnosis. In contrast, patients often see a physical therapist for a number of ongoing sessions with an already diagnosed condition. Additionally, the failure to identify "Expectations" as a stable component in this research suggests that perhaps patient expectations are not well defined in physical therapy. Patients may be unfamiliar with the nature of physical therapy services and, therefore, do not know what to expect when they attend therapy sessions. It appears that physical therapy outpatients have few, if any, clearly defined expectations. This situation affords therapists the opportunity to influence developing expectations, potentially resulting in more realistic expectations.

Other potential interpretations should to be considered, including the possibility that the high degree of satisfaction may have limited the identification of other meaningful factors in outpatient satisfaction with physical therapy. Although it may be reassuring to recognize that physical therapy is met with such a high degree of satisfaction, future endeavors should attempt to contact sizable numbers of people who are dissatisfied with their therapy, are poor attendees, or have discontinued therapy. It is possible that a larger pool of dissatisfied patients could influence the structure of satisfaction in physical therapy as well as the validity data reported here. The data, however, indicate that the PTOPS possesses excellent sensitivity in discriminating between satisfied and dissatisfied patients, and between patients with low and high attendance in small samples.

In our opinion, the PTOPS appears to be an excellent inventory to assess outpatient satisfaction in physical therapy. The fit indices of Table 3 must be considered excellent by any standard. This evaluation becomes enhanced when it is realized that the number of CFA measures (items) negatively affects goodness-of-fit indicators and, particularly, creates an increased and biased chi-square value. [40] Therefore, a large majority of CFAs involved with test development use item parcels (ie, the practice of combining 2 or more items to form a single entry).[40] The use of item parcels has the effect of reducing the number of measures and subsequently provides more favorable fit statistics. The fact that we did not use item parcels and that the excellent PTOPS fit statistics were obtained with 34 measures, in our view, speaks strongly to the validity of this satisfaction model. Additionally, the overall size of regression coefficients obtained from the CFA indicates excellent correspondence between items and their respective components. Twenty-eight of the 34 standardized regression coefficients were above .50.

In order to better understand outpatient satisfaction in physical therapy, a limitation of our research should be considered. Although the participation rates of 69% to 80% represent good to very good response rates when viewed by customary survey literature,[41,42] the data were characterized by consistently high satisfaction scores. It is logical to believe that some percentage of the non-responders represented dissatisfied patients. Until it becomes possible to obtain returns from this potentially dissatisfied population (even though relatively small), a complete understanding of the satisfaction variable in physical therapy will not be realized. It is possible that satisfaction components for a distinct population of dissatisfied patients may be different from the components identified in this research.

Summary

This multiphase research resulted in the development of the PTOPS, which provides the physical therapy profession with what we believe is a reliable and valid measure of outpatient satisfaction. By applying psychometric analyses to the measurement of satisfaction, this research fills a previous void.

Patient satisfaction research in physical therapy is in its infancy, and many avenues of inquiry need to be pursued. Further evidence of the validity of PTOPS scores could be obtained by testing the stability of components over time and by ascertaining that component scores were not highly related with reading level or intelligence. Most importantly, efforts should be made to administer the PTOPS to sizable numbers of dissatisfied patients. Additionally, sufficient sample sizes by gender (both patients and therapists) should be obtained in order to ascertain that similar satisfaction components actually exist across genders.

Correlates of satisfaction should be examined. For example, what is the relationship between outpatient satisfaction and patient age, educational level, reimbursement source, previous experience with physical therapy, socio-economic status, diagnosis, and treatment outcomes? Additionally, the effect of manipulating service and setting factors on PTOPS component scores can be examined, with a goal of developing greater satisfaction in clinical settings.

References

[1] Lewis J. Patient views on quality care in general practice: literature review. Soc Sci Med. 1994;39:655-670.

[2] Greene MG, Adelman RD, Friedmann E, Charon R. Older patient satisfaction with communication during an initial medical encounter. Soc Sci Med. 1994;38:1279-1288.

[3] Hsieh M, Kagle JD. Understanding patient satisfaction and dissatisfaction with health care. Health Soc Work. 1991;16:281-290.

[4] Schlenoff D. Growing demands on time versus the physician/patient relationship. Maryland Family Doctor. Summer 1994:6-10.

[5] Comstock LM, Hooper EM, Goodwin JM, Goodwin JS. Physician behaviors that correlate with patient satisfaction. J Med Educ. 1982;57: 105-112.

[6] Linder-Pelz S. Social psychological determinants of patient satisfaction: a test of five hypotheses. Soc Sci Med. 1982;16:583-589.

[7] Baldwin LM, Inui TS, Stenkamp S. The effect of coordinated, multidisciplinary ambulatory care on service use, charges, quality of care, and patient satisfaction in the elderly. J Community Health. 1993;18:95-108.

[8] Forbes ML, Brown HN. Developing an instrument for measuring patient satisfaction. AORNJ. 1995;61:737-739, 741-743.

[9] Bond S, Thomas LH. Measuring patients' satisfaction with nursing care. J Adv Nurs. 1992;20:52-63.

[10] Roush SE. The satisfaction of patients with multiple sclerosis regarding services received from physical and occupational therapists. International Journal of Rehabilitation and Health. 1995; 1:155-166.

[11] Patient Satisfaction Instruments: A Compendium. Alexandria, Va: American Physical Therapy Association; 1995.

[12] Reeder LG. The patient-client as a consumer: some observations on the changing professional-client relationship. J Health Soc Behav. 1972; 13:406-412.

[13] Friedman ML, Churchill GA. Using consumer perceptions and a contingency approach to improve health care delivery. Journal of Consumer Research. 1987;13:492-510.

[14] Zimney L, McClain MP, Batalden PB, O'Connor JP. Patient telephone interviews: a valuable technique for finding problems and assessing quality in ambulatory medical care. J Community Health. 1980;6:35-42.

[15] Health Services Research Group. A guide to direct measures of patient satisfaction in clinical practice. CMAJ. 1992;146:1727-1731.

[16] Linder-Pelz S, Struening EL. The multidimensionality of patient satisfaction with a clinic visit. J Community Health. 1985;10:42-54.

[17] DiMatteo MR, Hays R. The significance of patients' perceptions of physician conduct: a study of patient satisfaction in a family practice center. J Community Health. 1980;6:18-34.

[18] Hulka BS, Kupper LL, Daly MB, et al. Correlates of satisfaction and dissatisfaction with medical care: a community perspective. Med Care. 1975;13:648-658.

[19] Guzman PM, Sliepcevich EM, Lacey EP, et al. Tapping patient satisfaction: a strategy for quality assessment. Patient Education and Counseling. 1988;12:225-233.

[20] Roush SE. Examining the Relationship Between Health Care Professionals and Persons With Disabilities [Dissertation]. Seattle, Wash: University of Washington; 1990.

[21] Paulhus DL. Measurement and control of response bias. In: Robinson JP, Shaver PR, Wrightsman LS, eds. Measures of Personality and Social Psychological Attitudes. Orlando, Fla: Academic Press Inc; 1991: chap 2.

[22] Paulhus DL. BIDR Reference Manual for Version 6. Vancouver, British Columbia, Canada: University of British Columbia Press; 1991.

[23] Fitzpatrick R. Surveys of patient satisfaction, II: designing a questionnaire and conducting a survey. BMJ. 1991;302:1129-1132.

[24] Marks R. Evaluating patient satisfaction: general considerations, I: designing a questionnaire. New Zealand Journal of Physiotherapy. December 1993:35-38.

[25] Grisetti GC. The Effect of Oral Communication Between the Physical Therapist and Outpatients on Patient Satisfaction [Dissertation]. New York, NY: Teachers' College of Columbia University; 1989.

[26] Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. East Norwalk, Conn: Appleton & Lange; 1994:540-548.

[27] Tabachnick BG, Fidell LS. Using Multivariate Statistics. 2nd ed. New York, NY: Harper & Row; 1989: chap 12.

[28] Williams B. Patient satisfaction: a valid concept? Soc Sci Med. 1994;38:509-516.

[29] Norusis MJ/SPSS Inc. SPSS Base System User's Guide. Chicago, Ill: SPSS Inc; 1990.

[30] Aiken LR. Psychological Testing and Assessment. 7th ed. Boston, Mass: Allyn and Bacon; 1976.

[31] Guadagnoli E, Velicer WF. Relation of sample size to the stability of component patterns. Psychol Bull. 1988;103:265-275.

[32] Comrey AL. Factor-analytic methods of scale development in personality and clinical psychology. J Consult Clin Psychol. 1988;56: 754-761.

[33] Bentler PM. EQS: Structural Equations Manual. Los Angeles, Calif: BMDP Statistical Software; 1989.

[34] Hayduk LA. Structural Equation Modeling With LISREL. Baltimore, Md: The Johns Hopkins University Press; 1987.

[35] Bentler. PM, Bonnett DG. Significance tests and goodness of fit in the analysis of covariance structures. Psychol Bull. 1980;88:588-606.

[36] Bentler PM. On the fit of models to covariances and methodology to the Bulletin. Psychol Bull. 1992;112:400-404.

[37] Sonstroem RJ, Potts SA. Life adjustment correlates of physical self-concepts. Med Sci Sports Exerc. 1996;28:619-625.

[38] Dishman RK. Exercise Adherence: Its Impact on Public Health. Champaign, Ill: Human Kinetics Inc; 1988.

[39] Herzberg F. One more time: How do you motivate employees? Harvard Business Review. 1968;46:53- 62.

[40] Marsh HW, Hau KT, Balla JR Grayson D. Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multi Behav Research. In press.

[41] Fowler FJ. Survey Research Methods. 2nd ed. Newbury Park, Calif: Sage Publications Inc; 1993.

[42] Babbie ER. Survey Research Methods. Belmont, Calif: Wadsworth Inc; 1973.

Appendix.

Physical Therapy Outpatient Satisfaction Survey (PTOPS): Items, Administration, Scoring, and Interpretation(a)

1. The cost of treatment is more than I expected.

2. I enjoy listening to my therapist.

3. I expect the facility to be quieter than it is.

4. The facility is flexible about payment options.

5. The distance required to get to the facility is acceptable to me.

6. I expect my therapist to spend more time with me than he/she does.

7. I am given privacy when I need it.

8. It is difficult for me to get into the facility from the parking lot.

9. I am charged a reasonable amount for my therapy.

10. This facility could be more conveniently located for me.

11. I feel my therapist overcharges me.

12. The office staff is attentive to my needs.

13. My therapist acts like he/she is doing me a big favor by treating me.

14. The facility is in a desirable location.

15. My therapist could communicate with me more.

16. I have to wait too long between appointments.

17. The quality of the care I receive is not compatible with the cost.

18. This facility is a nice place to get my therapy.

19. It is somewhat difficult for me to reach this PT facility.

20. The facility is too crowded.

21. I have to travel too far to receive my treatment.

22. I can get around easily inside of the facility.

23. I don't really enjoy talking with my therapist.

24. My therapist seems to have a genuine interest in me as a person.

25. My therapist does not expect me to pay significantly more than what my insurance covers.

26. I anticipate my questions will be answered clearly.

27. My therapist doesn't give me a chance to say what is on my mind.

28. I should not have to travel this far for therapy.

29. This facility appreciates my business.

30. It could be easier to make the arrangements to pay for my therapy.

31. My therapist should be more thorough in my treatment.

32. The physical therapy facility is conveniently located for me.

33. My therapist should listen more carefully to what I tell him/her.

34. I get along well with everyone in this PT facility.

Administration: 5-point Likert scale: 1=Strongly Disagree; 2=Disagree; 3=Uncertain; 4=Agree; 5=Strongly Agree; Administration Time=approximately 6 to 10 minutes

Items to be recoded: 9, 10, 16, 17, 19, 21, 25, 28, 30 (1=5, 2=4,4=2,5=1)

Mean subscale scores (four separate scores)

Enhancers: 2, 7, 12, 16, 18, 22, 24, 26, 29, 34

Detractors: 3, 6, 8, 13, 15, 20, 23, 27, 31, 33

Location: 5, 10, 14, 19, 21, 28, 32

Cost: 1,4, 9, 11, 17, 25, 30

Interpretation: Enhancers and Location are positive scales, ie higher scores indicate greater satisfaction; Detractors and Cost are negative scales, ie higher scores indicate less satisfaction.

(a) The PTOPS items are presented exactly as they appear in the survey instrument.

SE Roush, PhD, PT, is Associate Professor, Physical Therapy Program, University of Rhode Island, 25 W Independence Way, Kingston, RI 02881 (USA) ([e-mail]). Address all correspondence to Dr Roush.

RJ Sonstroem, PhD, is Professor Emeritus, Physical Education Department, University of Rhode Island.

Dr Roush was project manager. Dr Roush and Dr Sonstroem provided the concept, writing, data analysis, facilities, and equipment. Data collection and fired procurement were conducted by Dr Roush and Laura Carson, Betsy Coakley, Jim Cowher, Melissa Foley, Elizabeth France, Karen Gdowski, Mike Haley, Mark Libby, John Kier, Tracy Miner, Heather Murphy, Laura Perrault, Dawn Rykiel, Kristin Riley, Francine Tanyag, Susan Vanston, and Trudy Williams, graduates of the University of Rhode Island Physical Therapy Program, who participated in this research in partial fulfillment of the requirements for the Master of Science degree.

This study was approved by the Institutional Review Board at the University of Rhode Island.

This study was supported in part by the Graduate Programs Fund of the College of Human Sciences and Services, University of Rhode Island.

This article was submitted February 23, 1998, and was accepted October 8, 1998.