Printer Friendly

Can additional patient experience items improve the reliability of and add new domains to the CAHPS[R] Hospital Survey?

Patient perceptions about care received during the hospital stay have been a major focus of hospital management (Bell, Krivich, and Boyd 1997), and interest in patients' perceptions has burgeoned along with implementation of performance-reporting systems and payment mechanisms based on performance (Hibbard, Stockard, and Tusler 2005; Scanlon et al. 2005). Patient experience data provide an important insight into care received from the patient's perspective and have been used by hospitals internally for many years for quality improvement. These data have not been available universally for statewide or national cross-hospital comparisons or public reporting (PR) initiatives because of the lack of a standardized survey instrument.

As part of the Consumer Assessment of Health Care Providers and Systems (CAHPS[R]) project, a standardized patient experience survey was developed to assess care provided to adult hospitalized patients (Crofton et al. 2005; Darby, Hays, and Kletke 2005). The CAHPS Hospital Survey has been endorsed by the National Quality Forum and will be included in the reporting requirements and pay-for-performance programs for the Centers for Medicare and Medicaid Services and the Hospital Quality Alliance in 2008 (AHRQ 2006). The CAHPS Hospital Survey includes 16 questions assessing specific aspects of care and two hospital rating questions (plus demographic and screener questions) (Darby, Hays, and Kletke 2005). This survey is a major step in producing a standardized core set of items, and it addresses key aspects of care such as nurse communication, doctor communication, nursing services, physical environment, pain control, communication about medicines, and discharge information. However, the survey also has some identified weaknesses. The two-item discharge information composite has relatively low internal consistency reliability ([alpha] = 0.45) (Keller et al. 2005). In addition, the survey does not assess some important aspects of the patient experience. For example, coordination of care is a domain identified by Gerteis, Edgman-Levitan, and Daley (1993) and highlighted by the Institute of Medicine (IOM) (Institute of Medicine 2001) in Crossing the Quality Chasm as a key aspect of patient care, but it is not included in the CAHPS Hospital Survey. In this paper, we describe a statewide project conducted in California with the goal of improving the reliability and expanding the scope of the CAHPS Hospital Survey.

METHODS

Supplementing the CAHPS Hospital Survey

In 2004, the California Hospitals Assessment and Reporting Taskforce (CHART) began meeting with the goal of establishing a voluntary PR initiative in California (California Health Care Foundation 2005). The consensus measure set that evolved included a variety of clinical measures along with a patient experience survey. Although standardization with national measures was an important goal in CHART, the CHART group wanted to expand on the CAHPS Hospital Survey to improve its reliability and address an additional IOM (Institute of Medicine) domain.

CHART hospitals agreed to include additional questions in their patient surveys based on an analysis of psychometric data from the CAHPS Hospital Survey three-state pilot study. These data showed a benefit of increased reliability by adding additional questions used in California through the Patient's Evaluation of Performance in California (PEP-C) project in the areas of discharge information and coordination of care. We defined coordination of care (based on the definition developed by the IOM [2004]) as:
 To establish and support a continuous healing relationship, enabled
 by an integrated clinical environment and characterized by the
 proactive delivery of evidence-based care and follow-up. Clinical
 integration is further defined as the extent to which patient care
 services are coordinated across people, functions activities and
 sites over time so as to maximize the value of services delivered
 to patients.


The CHART project then asked all the major vendors of patient experience surveyance to submit questions and supporting psychometric data in these areas. Based on preliminary estimates of the psychometric properties as provided by the vendors (these data have not been published), the group decided to test the performance of discharge and coordination of care questions submitted by NRC+Picker Inc. Two items were added to improve the reliability of the discharge information composite ([28]. Did someone on the hospital staff explain the purpose of the medicines you were to take at home in a way you could understand? and [29]. Did they tell you what danger signals about your illness or operation to watch for after you went home?). Five questions were added to allow consideration of alternative approaches to the new coordination of care composite ([23]. How organized was the admission process?, [24]. If yon had to wait to go to your room, did someone from the hospital explain the reason for the delay?, [25]. Were your scheduled tests and procedures performed on time?, [26]. Staff checked ID band before giving reeds/treatment/tests, and [27]. Sometimes in the hospital, one doctor or nurse will say one thing and another will say something quite different. Did this happen to you?). In addition, the group decided that interpreter availability was an important aspect of care not addressed in the CAHPS Hospital Survey. Two questions (one, a screener question) that had been used in California through the PEP-C project were added to assess the need for and provision of interpreter services, an important issue in California. ([30]. [Screener question] An interpreter is someone who repeats or signs what one person says in a language used by another person. Did you need an interpreter to help you speak with doctors or other health providers? and [31]. When you needed an interpreter to help you speak with doctors or other health providers, how often did you get one?).

Study Subjects

A total of 41,701 surveys were obtained from patients discharged from 186 hospitals between December 2005 and February 2006. After deleting 91 observations that completed < 50 percent of the items, 19 observations that were under 18 years of age and 1,419 observations from five hospitals that administered only CAHPS Hospital Survey items without the additional CHART questions, there were 40,172 observations from 181 hospitals available for the analyses (average of 222 observations per hospital). The survey was available in three languages (English, Spanish, and Chinese) and administered using a two-wave mail-only method. Ninety-five percent of respondents completed the survey in English and 5 percent in Spanish. The lag time between discharge and survey completion ranged from 10 to 210 days.

Fifty-one percent of the participating hospitals had bed sizes ranging from 100 to 299, 24 percent from 300 to 499, 19 percent < 100, and 6 percent over 500. The number of patients per hospital ranged from 30 to 523. Forty-nine percent of hospitals had sample sizes of between 201 and 300 patients and 28 percent between 101 and 200 patients.

Just under half (49 percent) of the 40,172 respondents were 65 or older; 17 percent were 18-34 years old (Table 1). The majority of the sample was female (63 percent). More than 12 years of education was reported by 61 percent of the sample. Sixty-five percent of respondents were white, 19 percent Hispanic, 10 percent Asian/Pacific Islander, and 5 percent black. Sixty-one percent of respondents were admitted to the hospital through physician/ clinic/health maintenance organization referral, 34 percent through the emergency room, 3 percent were transfers from another hospital, and 1 percent were skilled nursing facilities or other transfers. Ninety percent were discharged to home with self-care or home health services, 8 percent to other intermediate or nursing facilities, 1 percent to another short-term general hospital, and 1 percent left against medical advice. The percentage of the sample from medical was 49 percent, surgery was 33 percent, and maternity care was 17 percent (Table 1).

Data Analysis

We estimated item descriptive statistics for the 18 CAHPS Hospital Survey report and rating items plus the nine additional questions added for the CHART project. Next, we estimated product--moment correlations of the CAHPS Hospital survey items and additional CHART items with the seven CAHPS Hospital Survey composites (scales). Item-scale correlations were corrected for overlap when necessary. Then, we estimated internal consistency reliability for multi-item scales, hospital-level reliability for items and scales, and the correlations of these items and scales with two "bottom-line" indicators of hospital care: the global rating of the hospital and willingness to recommend the hospital to friends and family items. We also evaluated the CAHPS Hospital Survey discharge information composite, augmented by the two CHART discharge items, and the new hypothesized coordination of care composite created in three versions, using three, four, or five new questions, respectively.

We regressed the global rating of the hospital and willingness to recommend items on the CAHPS Hospital Survey and CHART hospital items to identify the total and unique variance (the increase in [R.sup.2] obtained by adding CHART items) accounted for by the report items, adjusting for age, gender, education, ethnicity, admission source, reason for admission, self-rated health, and discharge status. We corrected standard errors in the regression models for clustering within hospital (White 1980). Finally, we computed rank--order correlations of hospital-level scores between the CAHPS Hospital Survey discharge information and the augmented discharge information composite as well as between an overall summary score based on the CAHPS Hospital Survey items and an overall summary score based on the CAHPS[R] items plus the CHART items.

RESULTS

Psychometric Properties of Composites and Items

The percentage of completes for applicable items ranged from 85 percent for availability of interpreter to 99 percent for nurses' response. Missing values were primarily due to structured item nonresponse (not applicable).

Table 2 shows correlations of items with the seven CAHPS Hospital Survey composites, an augmented discharge information composite that included two of the CHART items, and coordination of care composites based on three, four, or all five CHART items (see online appendix). The item-scale correlations for hypothesized scales were generally supportive of item convergence and discrimination of the CAHPS survey items, but the physical environment items (Q8, Q9) and one nursing service item (Q4) correlated more highly with nurse communication than with their hypothesized composites.

The two CHART discharge items ([Q28]. Did someone on the hospital staff explain the purpose of the medicines you were to take at home in a way you could understand? and [Q29]. Did they tell you what danger signals about your illness or operation to watch for after you went home?) had noteworthy correlations with the CAHPS Hospital Survey discharge information composite. Another CHART item ([Q27]. Received different information from doctors and nurses?) correlated with the nurse communication (0.33) and doctor communication (0.30) as well as with the CHART coordination of care composites (0.29, 0.26, and 0.28). The interpreter availability item (Q31) had a high rate of nonapplicable responses (87 percent) and was not correlated with any of the scales; hence, we did not consider the item or the screener question any further.

Table 3 provides internal consistency and hospital-level reliability estimates for the CAHPS Hospital Survey composites and composites formed from the CHART items as well as individual CAHPS and CHART items. Internal consistency reliability for all CAHPS Hospital Survey composites except for physical environment and discharge information exceeded the 0.70 threshold for group comparisons (Nunnally and Bernstein 1994). The reliability of the discharge information composite was enhanced by adding the two CHART items: internal consistency reliability increased from 0.45 to 0.72 and hospital-level reliability from 0.75 to 0.81. While the CAHPS Hospital Survey physical environment composite had a low level of internal consistency reliability, it had the highest hospital-level reliability. Communication about medicine was the only CAHPS Hospital survey composite that did not have a hospital-level reliability >0.70. The three-, four-, and five-question coordination of care composites had adequate internal consistency (0.58, 0.66, and 0.70, respectively) and strong hospital-level reliability (0.84, 0.84, and 0.87, respectively). Hospital-level reliability was very high for two of the CHART questions (Q23--Organized admission process = 0.84 and Q26--Check ID band = 0.84) and only surpassed by two CAHPS items (Q4--Got help when pressed call button = 0.85 and Q9--Room quiet at night = 0.88). Two of the CHART items (Q24--Explained reason for room delay and Q27--Received different information from doctors and nurses) and three CAHPS items (Q13--Pain well controlled, Q16--Staff told what new medicine for, and Q17--Staff described side effects of new medicine) had hospital-level reliabilities lower than 0.70.

Correlations among the multi-item composites and with the global rating of the hospital and willingness to recommend items are provided in Tables 4 and 5, respectively. The strongest correlations with the global rating of the hospital and the willingness to recommend items were found for nurse communication (r's = 0.62 and 0.60, respectively) and the CHART coordination of care composites (three-item correlations = 0.54 and 0.53, four-item correlations = 0.58 and 0.56, and five-item correlations = 0.60 and 0.59, respectively). The nursing service and pain control composites also showed high correlations with the global rating of the hospital and the willingness to recommend the hospital to friends and family. The CAHPS Hospital Survey discharge information composite did not correlate very highly with these bottom-line indicators (r's of 0.27 and 0.26), the lowest correlations of any composites. This was significantly improved by the additional CHART questions (r's of 0.44 and 0.41); however, discharge information has the lowest correlations of any of the composites with overall hospital rating and willingness to recommend.

Information Uniqueness of the CHART Survey Items

Addition of the CHART items increased the variance in global ratings explained by responses to the survey questions. The seven CHART items accounted for only about 2 percent of unique variance in the global rating and the willingness to recommend.

Table 6 shows regression coefficients of the composites and patients' characteristics on the global rating and the willingness to recommend, which were estimated in the regression models with CAHPS composites and the models with CAHPS and CHART composites. The CAHPS composites and patient characteristics accounted for 47 percent of variance in the global rating of the hospital, and the variance explained increased to 49 percent when the discharge information composite was replaced by the CHART-augmented discharge information composite and any of the CHART coordination of care composites were added to the model. In the regression model for the willingness to recommend the hospital, the percent of variance explained by the model increased from 42 to 44 percent by including the augmented CAHPS discharge composites and any of the CHART coordination of care composites.

Rank-order correlation at the hospital level (n=181) between the CAHPS discharge information composite and the augmented CHART discharge information composite was 0.91; correlation of the sum of the CAHPS items with the sum of CAHPS and CHART items was 0.95.

DISCUSSION

The CAHPS Hospital Survey is an assessment tool soon to be in use across the country that was designed to be supplemented with additional questions to reflect user-specific needs. This study was undertaken to assess the psychometric properties of the survey and to evaluate benefits of the additional survey items used in the CHART project.

The study findings are generally consistent with previous evaluations of the psychometric properties of the instrument (Keller et al. 2005; Sofaer et al. 2005; Arah et al. 2006). However, the original two-item CAHPS discharge information and physical environment composites had relatively low internal consistency reliabilities. The augmentation of the discharge information composite by two CHART items improved the psychometric properties of the composite as well as the associations with the global rating and the willingness to recommend the hospital. The results of this study also indicated a need for improvement in the two-item CAHPS communication about medicine composite. The hospital-level reliability of both items and the composite were relatively low.

Consistent with findings in other research, the nurse communication composite within the CAHPS Hospital Survey had the strongest association with the ratings and willingness to recommend (Jenkinson et al. 2002; Sofaer et al. 2005; Arah et al. 2006). For example, the regression coefficients for explaining variation in the willingness to recommend the hospital in the CAHPS Hospital Survey was 0.013 for nurse communication compared with 0.005 for the physical environment--the composite that had the second strongest association. Hence, nurse communication is the major driver of bottom-line perceptions of hospital care. When the CHART coordination of care composites are considered along with the CAHPS Hospital Survey, all three had stronger associations with the hospital ratings and willingness to recommend than the physical environment composite, making these the composites with the second strongest associations with overall hospital rating and willingness to recommend.

The strong performance of the CHART coordination of care composites supports the reconsideration of including a coordination of care domain to the CAHPS Hospital Survey. Although there has been significant discussion regarding the inclusion of a coordination of care domain within the CAHPS Hospital Survey, the discussion has been based on a broad concept of coordination of care, much of which may not be visible or detectable by the patient, about which, therefore, the respondent would not be expected to be a knowledgeable informant. It has also been suggested that coordination of care may only be recognized in its absence (Levine, Fowler, and Brown 2005). Questions that were considered and deleted during the original CAHPS Hospital Survey development asked, for example, "did staff members who cared for you know about your condition without having to ask you?"

The questions used in the CHART coordination of care domain are more specifically focused on elements of coordination that may be directly experienced and understood by the patient (e.g., "scheduled tests were performed on time"). In this study, we chose to examine three different composites in coordination of care: a three-item composite that included three questions that had been clearly mapped to coordination of care in pilot studies, a four-item composite that included the "delay going to room explained" question that mapped well in this analysis, and a five-item composite that included an additional question--"ID band checked"--that also mapped to this domain. Although all of these composites performed well in this analysis, the five-item composite is slightly superior psychometrically.

There is always tension surrounding the inclusion of additional items within any patient survey--the benefit of increased reliability of the survey as questions are added must be balanced against the additional response burden on patients. Thus, the decision to add questions to improve the discharge information domain or add a coordination of care domain must be made in the context of overall patient survey strategies. For instance, some hospitals might already be asking questions beyond the CAHPS Hospital Survey that address areas of their own choosing, and using an expanded CAHPS Hospital Survey may reduce their capacity to address these local issues. However, it will increase the amount of standardized information available if many hospitals agree to use a specific expanded version of the CAHPS Hospital Survey, as has occurred in CHART.

There is also an issue of using proprietary versus public domain questions in a patient survey. The CAHPS Hospital Survey questions are now in the public domain, but the process for moving questions from proprietary to public domain status has not yet been worked out. Furthermore, we cannot state whether the questions we used in this study are superior to similar questions from other sources. For instance, the three-item Care Transitions Measures developed by Coleman, Mahoney, and Parry (2005) might function as well as the four-item discharge information domain presented here. Further investigation of alternatives to improving the discharge information domain (and the physical environment domain) of the CAHPS Hospital Survey is needed, as is research on how best to measure coordination of care. However, our results suggest that these improvements in patient surveys are feasible and can generate important information.

In summary, the findings of this study provide further support for the reliability and validity of the CAHPS Hospital Survey. Because of its brevity, it is feasible to add items to the CAHPS Hospital Survey to provide a more psychometrically sound assessment of the current domains and to assess domains not represented. The findings of this study illustrate how such additional items can be assessed and that additional items can improve our measurement of patients' hospital experiences.

ACKNOWLEDGMENTS

Joint Acknowledgment/Disclosure Statement: We would like to acknowledge the support of the California Health Care Foundation that was responsible for the funding of the CHART project within which the original data were generated. Dr. Dudley's work on this project was partially funded by a Robert Wood Johnson Foundation Investigator Award in Health Policy. Data analyses were supported by cooperative agreements from the Agency for Healthcare Research and Quality with Dr. Hays. Dr.Hays was supported by grants from the National Institutes on Aging and the UCLA Center for Health Improvement in Minority Elderly/Resource Centers for Minority Aging Research. We would also like to acknowledge the work of NRC+Picker in providing proprietary questions and data support to develop the CHART survey. We also would like to thank The California Hospital Assessment and Reporting Taskforce and all participating California CHART hospitals that submitted data through their vendors. In addition, we would like to thank the staff at the National CAHPS Benchmarking Database (NCBD) who developed a portal for data submission and carried out initial cleaning and analysis of the CAHPS data from hospitals. Beth Thew and Edie Wade at UCSF have also contributed their editing skills to read and reread many versions. And, most importantly, we acknowledge the thousands of patients who took the time to complete the surveys and share their experiences with the public. Many thanks to all.

Disclosures:. None.

REFERENCES

AHRQ: 2006. "CMS Initiates National Implementation of the Hospital Survey." CAHPS Connection 3: 5.

Arah, O. A., A. H. A. ten Asbrook, D. M.J. Delnoij, J. S. de Koning, P.J.A. Stam, A. H. Poll, B. Vriens, P. F. Schmidt, and N. S. Klazinga. 2006. "Psychometric Properties of the Dutch Version of the Hospital-Level Consumer Assessment of Health Plan Survey Instrument." Health Services Research 41: 284-301.

Bell, R., M.J. Krivich, and M. S. Boyd. 1997. Charting Patient Satisfaction. Marketing Health Services 17: 22.

California Health Care Foundation. 2005. "Hospitals, Public Reporting: California Hospital Assessment and Reporting Task Force" [accessed on January 22, 2007]. Available at http://www.chcf.org/topics/hospitals/index.cfm?itemID=l11065

Coleman, E. A., E. Mahoney, and C. Parry. 2005. "Assessing the Quality of Preparation for Posthospital Care from the Patient's Perspective: The Care Transitions Measure." Medical Care 43 (3): 246-55.

Crofton, C., C. Darby, M. Farquhar, and C. Clancy. 2005. "The CAHPS Hospital Survey: Development, Testing, and Use." Joint Commission Journal on Quality and Patient Safety 31: 655-9.

Darby, C., R. D. Hays, and P. Kletke. 2005. "Development and Evaluation of the CAHPS Hospital Survey." Health Services Research 40: 1973-6.

Gerteis, M., S. Edgman-Levitan, and J. Daley. 1993. Through the Patient's Eyes: Understanding and Promoting Patient-Centered Care. San Francisco: Jossey-Bass.

Hibbard, J. H., J. Stockard, and M. Tusler. 2005. "Hospital Performance Reports: Impact on Quality, Market Share, and Reputation." Health Affairs 24: 1150-60.

Institute of Medicine (IOM). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century, pp. 51-3. Washington, DC: National Academy Press.

--. 2004. First Annual Crossing the Quality Charm Summit: A Focus on Communities. Chapter 4, Board on Health Care Services, p. 47.

Jenkinson, C., A. Couker, S. Bruster, N. Richards, and T. Chandola. 2002. "Patients' Experiences and Satisfaction with Health Care: Results of a Questionnaire Study of Specific Aspects of Care." Quality and Safety in Health Care 11: 335-9.

Keller, S., A.J. O'Malley, R. D. Hays, R. A. Matthew, A. M. Zaslavsky, K. A. Hepner, and P. D. Cleary. 2005. "Method Used to Streamline the CAHPS Hospital Survey." Health Services Research 40: 2057-77.

Levine, R. E., F.J. Fowler, and J. A. Brown. 2005. "Role of Cognitive Testing in the Development of the CAHPS Hospital Survey." Health Services Research 40: 2037-56.

Nunnally, J. C., and I. H. Bernstein. 1994. Psychometric Theory, 3rd Edition. New York: McGraw Hill.

Scanlon, D. P., S. Swaminathan, M. Chernew, J. E. Bost, and J. Shevock. 2005. "Competition and Health Plan Performance: Evidence from Health Maintenance Organization Insurance Markets." Medical Care 43: 338-46.

Sofaer, S., C. Crofton, E. Goldstein, E. Hoy, and J. Crabb. 2005. "What Do Consumers Want to Know about the Quality of Care in Hospitals?" Health Services Research 40: 2018-36.

White, H. 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity." Econometrica 48: 817-30.

SUPPLEMENTARY MATERIAL

The following supplementary material for this article is available online:

Appendix SA1: Author Matrix.

Appendix SA2: CAHPS[R] Hospital Survey Items.

This material is available as part of the online article from http:// www.blackwell-synergy.com/doi/abs/10.1111/j.1475-6773.2008.00867.x (this link will take you to the article abstract).

Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.

Address correspondence to Arlyss Anderson Rothman, Ph.D., M.H.S., R.N.-C., F.N.P., UCSF, Phillip R. Lee Institute for Health Policy Studies, 401 Miner Road, Orinda, CA 94563; e-mail: arlyssar@aol.com. Hayoung Park, Ph.D., is with the Seoul National University, Seoul, South Korea. Ron D. Hays, Ph.D., is with the UCLA Department of Medicine/Division of General Internal Medicine and Health Services Research, Los Angeles, CA. Carol Edwards, B.A., is with the Rand Corporation, Santa Monica, CA. R. Adams Dudley, M.D., M.B.A., is with the UCSF, Phillip R. Lee Institute for Health Policy Studies, San Francisco, CA.
Table 1: Characteristics of Study Subjects

Characteristics (Total N = 40,172) Percent

Age
 18-34 17
 35-64 34
 65+ 49
Gender
 Male 37
 Female 63
Education
 No or [less than or equal to] high school graduate 39
 Some college+ 61
Ethnicity
 White 65
 Hispanic 19
 Asian/Pacific Islander 10
 Black 5
 Other 1
Language spoken at home
 English 88
 Spanish 8
 Some other 4
Self-reported overall health status
 Excellent 16
 Very good 28
 Good 30
 Fair 19
 Poor 7
Reason for admission
 Maternity care 17
 Medical 49
 Surgical 33
Admission source
 Physician/clinic/HMO referral 61
 Transfer from other hospital 3
 Transfer from SNF or other 1
 Emergency room 34
 Other 0
Discharge status
 Home with or without home health services 90
 Another short-term general hospital 1
 Other secondary/nursing/long-term care facility 8
 Left against medical advice 1

HMO, health maintenance organization; SNF, skilled nursing facility.

Table 2: Item to Scale Correlations

 Scale

 Nurse Doctor Nursing
Item Communication Communication Services

Nurse communication
 Q01 Nurses treat with 0.71 * 0.41 0.52
 courtesy and
 respect
 Q02 Nurses listen 0.75 * 0.46 0.55
 carefully
 Q03 Nurses explain 0.68 * 0.49 0.50
 things
 understandably
Doctor communication
 Q05 Doctors treat with 11.44 0.73 * 0.31
 courtesy and
 respect
 Q06 Doctors listen 0.47 0.79 * 0.94
 carefully
 Q07 Doctors explain 0.48 0.73 * 0.34
 things
 understandably
Nursing services
 Q04 Received help when 0.60 0.35 0.44 *
 pressed call
 button
 Q11 Received help with 0.43 0.29 0.44 *
 bathroom
Physical environment
 Q08 Hospital room and 0.44 0.26 0.39
 bathroom kept
 clean
 Q09 Hospital room area 0.36 0.25 0.35
 quiet at night
Pain control
 Q13 Pain well 0.43 0.36 0.41
 controlled
 Q14 Staff helped with 0.59 0.43 0.50
 pain management
Communication about medicines
 Q16 Staff tell what new 0.47 0.39 0.38
 medicine is for
 Q17 Staff describe side 0.44 0.33 0.38
 effects of new
 medicine
Discharge information (19, 20)
CHART-augmented discharge information (19, 20, 28, 29)
 Q19 Reversed: talk 0.22 0.20 0.19
 re help after
 discharge
 Q20 Reversed: written 0.22 0.23 0.21
 information
 about re symptoms
 Q28 Explain medicines 0.44 0.40 0.34
 to be used
 at home
 understandably
 Q29 Danger signals to 0.40 0.37 0.34
 watch at home
Composite for three CHART items (23, 25, 27)
Composite for four CHART items (23, 24, 25, 27)
Composite for five CHART items (23, 24, 25, 26, 27)
 Q23 Organized 0.40 0.30 0.34
 admission process
 Q24 Explained reason 0.22 0.19 0.18
 for room delay
 Q25 Scheduled test 0.42 0.39 0.36
 performed on time
 Q26 Check ID before 0.39 0.29 0.31
 giving medicines/
 treatment/test
 Q27 Reversed: received 0.33 0.30 0.23
 different
 information from
 doctors and
 nurses
 Q31 Interpreter 0.05 0.06 0.05
 available

 Scale

 Communication
 Physical Pain about
Item Environment Control Medicines

Nurse communication
 Q01 Nurses treat with 0.42 11.47 0.40
 courtesy and
 respect
 Q02 Nurses listen 0.40 0.50 0.46
 carefully
 Q03 Nurses explain 0.41 0.47 0.49
 things
 understandably
Doctor communication
 Q05 Doctors treat with 0.27 0.37 11.31
 courtesy and
 respect
 Q06 Doctors listen 0.29 0.39 0.37
 carefully
 Q07 Doctors explain 0.29 0.38 0.40
 things
 understandably
Nursing services
 Q04 Received help when 0.42 0.44 0.39
 pressed call
 button
 Q11 Received help with 0.35 0.41 0.94
 bathroom
Physical environment
 Q08 Hospital room and 0.32 * 0.33 0.32
 bathroom kept
 clean
 Q09 Hospital room area 0.32 * 0.30 0.30
 quiet at night
Pain control
 Q13 Pain well 0.32 0.66 * 0.37
 controlled
 Q14 Staff helped with 0.40 0.66 * 0.44
 pain management
Communication about medicines
 Q16 Staff tell what new 0.30 0.41 0.55 *
 medicine is for
 Q17 Staff describe side 0.36 0.37 0.55 *
 effects of new
 medicine
Discharge information (19, 20)
CHART-augmented discharge information (19, 20, 28, 29)
 Q19 Reversed: talk 0.17 0.20 0.28
 re help after
 discharge
 Q20 Reversed: written 0.15 0.19 0.32
 information
 about re symptoms
 Q28 Explain medicines 0.28 0.36 0.50
 to be used
 at home
 understandably
 Q29 Danger signals to 0.27 0.34 0.49
 watch at home
Composite for three CHART items (23, 25, 27)
Composite for four CHART items (23, 24, 25, 27)
Composite for five CHART items (23, 24, 25, 26, 27)
 Q23 Organized 0.34 0.35 0.32
 admission process
 Q24 Explained reason 0.16 0.19 0.20
 for room delay
 Q25 Scheduled test 0.35 0.37 0.33
 performed on time
 Q26 Check ID before 0.28 0.31 0.37
 giving medicines/
 treatment/test
 Q27 Reversed: received 11.21 0.26 0.21
 different
 information from
 doctors and
 nurses
 Q31 Interpreter 0.05 0.05 0.05
 available

 Scale

 CHART- Composite
 Augmented for Three
 Discharge Discharge CHART
Item Information Information Items

Nurse communication
 Q01 Nurses treat with 0.21 0.32 0.44
 courtesy and
 respect
 Q02 Nurses listen 0.24 0.36 0.48
 carefully
 Q03 Nurses explain 0.26 0.40 0.45
 things
 understandably
Doctor communication
 Q05 Doctors treat with 11.21 0.31 11.41
 courtesy and
 respect
 Q06 Doctors listen 0.24 0.35 0.43
 carefully
 Q07 Doctors explain 0.26 0.38 0.42
 things
 understandably
Nursing services
 Q04 Received help when 0.20 0.31 0.41
 pressed call
 button
 Q11 Received help with 0.22 0.30 0.33
 bathroom
Physical environment
 Q08 Hospital room and 0.18 0.26 0.35
 bathroom kept
 clean
 Q09 Hospital room area 0.14 0.21 0.32
 quiet at night
Pain control
 Q13 Pain well 0.22 0.31 0.37
 controlled
 Q14 Staff helped with 0.23 0.35 0.45
 pain management
Communication about medicines
 Q16 Staff tell what new 0.30 0.44 0.35
 medicine is for
 Q17 Staff describe side 0.35 0.48 0.35
 effects of new
 medicine
Discharge information (19, 20)
CHART-augmented discharge information (19, 20, 28, 29)
 Q19 Reversed: talk 0.29 * 0.37 * 0.19
 re help after
 discharge
 Q20 Reversed: written 0.29 * 0.49 * 0.19
 information
 about re symptoms
 Q28 Explain medicines 0.38 0.50 * 0.39
 to be used
 at home
 understandably
 Q29 Danger signals to 0.51 0.61 * 0.35
 watch at home
Composite for three CHART items (23, 25, 27)
Composite for four CHART items (23, 24, 25, 27)
Composite for five CHART items (23, 24, 25, 26, 27)
 Q23 Organized 0.21 0.30 0.41 *
 admission process
 Q24 Explained reason 0.14 0.21 0.26
 for room delay
 Q25 Scheduled test 0.20 0.31 0.45 *
 performed on time
 Q26 Check ID before 0.24 0.35 0.36
 giving medicines/
 treatment/test
 Q27 Reversed: received 11.10 11.18 0.29 *
 different
 information from
 doctors and
 nurses
 Q31 Interpreter 0.03 0.04 0.05
 available

 Scale

 Composite Composite
 for Four for Five
 CHART CHART
Item Items Items

Nurse communication
 Q01 Nurses treat with 0.42 0.47
 courtesy and
 respect
 Q02 Nurses listen 0.46 0.49
 carefully
 Q03 Nurses explain 0.44 0.48
 things
 understandably
Doctor communication
 Q05 Doctors treat with 0.39 0.41
 courtesy and
 respect
 Q06 Doctors listen 0.41 0.42
 carefully
 Q07 Doctors explain 0.40 0.42
 things
 understandably
Nursing services
 Q04 Received help when 0.39 0.42
 pressed call
 button
 Q11 Received help with 0.32 0.34
 bathroom
Physical environment
 Q08 Hospital room and 0.33 0.37
 bathroom kept
 clean
 Q09 Hospital room area 0.30 0.31
 quiet at night
Pain control
 Q13 Pain well 0.35 0.38
 controlled
 Q14 Staff helped with 0.43 0.47
 pain management
Communication about medicines
 Q16 Staff tell what new 0.35 0.40
 medicine is for
 Q17 Staff describe side 0.35 0.10
 effects of new
 medicine
Discharge information (19, 20)
CHART-augmented discharge information (19, 20, 28, 29)
 Q19 Reversed: talk 0.20 0.23
 re help after
 discharge
 Q20 Reversed: written 0.19 0.22
 information
 about re symptoms
 Q28 Explain medicines 0.40 0.15
 to be used
 at home
 understandably
 Q29 Danger signals to 0.36 0.41
 watch at home
Composite for three CHART items (23, 25, 27)
Composite for four CHART items (23, 24, 25, 27)
Composite for five CHART items (23, 24, 25, 26, 27)
 Q23 Organized 0.42 * 0.45 *
 admission process
 Q24 Explained reason 0.26 * 0.27 *
 for room delay
 Q25 Scheduled test 0.43 * 0.47 *
 performed on time
 Q26 Check ID before 0.35 0.35 *
 giving medicines/
 treatment/test
 Q27 Reversed: received 11.26 * 0.28 *
 different
 information from
 doctors and
 nurses
 Q31 Interpreter 0.05 0.05
 available

* Item-scale correlation, corrected for item overlap with the scale
total score. CHART, California Hospitals Assessment and Reporting
Taskforce.

Table 3: Reliability Estimates for Multi-Item Scales and Items

 Reliability

 Internal Hospital
Item/scale Consistency Level

Nurse communication 0.85 0.85
 Q01 Nurses treat with courtesy and respect 0.81
 Q02 Nurses listen carefully 0.82
 Q03 Nurses explain things understandably 0.80
Doctor communication 0.87 0.76
 Q05 Doctors treat with courtesy and respect 0.69
 Q06 Doctors listen carefully 0.72
 Q07 Doctors explain things understandably 0.71
Nursing services 0.71 0.85
 Q04 Received help when pressed call button 0.85
 Q11 Received help with bathroom 0.71
Physical environment 0.49 0.87
 Q08 Hospital room and bathroom kept clean 0.81
 Q09 Hospital room area quiet at night 0.88
Pain control 0.81 0.71
 Q13 Pain well controlled 0.62
 Q14 Staff helped with pain management 0.70
Communication about medicines 0.73 0.64
 Q16 Staff tell what new medicine is for 0.60
 Q17 Staff describe side effects of new 0.59
 medicine
Discharge information (19, 20) 0.45 0.75
CHART-augmented discharge information 0.72 0.81
 (19, 20, 28, 29)
 Q19 Reversed: talk re help after discharge 0.71
 Q20 Reversed: written information about re 0.74
 symptoms
 Q28 Explain medicines to be used at home 0.69
 understandably
 Q29 Danger signals to watch at home 0.79
CHART three-item coordination of care composite 0.58 0.84
 (23, 25, 27)
CHART four-item coordination of care composite 0.66 0.84
 (23, 24, 25, 26, 27)
CHART five-item coordination of care composite 0.70 0.87
 (23, 24, 25, 26, 27)
 Q23 Organized admission process 0.84
 Q24 Explained reason for room delay 0.54
 Q25 Scheduled test performed on time 0.76
 Q26 Check ID before giving medicines/ 0.84
 treatment/test
 Q27 Reversed: received different 0.64
 information from doctors and nurses

 Reliability

 Hospital-Level
Item/scale ICC

Nurse communication 0.02
 Q01 Nurses treat with courtesy and respect 0.02
 Q02 Nurses listen carefully 0.02
 Q03 Nurses explain things understandably 0.02
Doctor communication 0.01
 Q05 Doctors treat with courtesy and respect 0.01
 Q06 Doctors listen carefully 0.01
 Q07 Doctors explain things understandably 0.01
Nursing services 0.03
 Q04 Received help when pressed call button 0.03
 Q11 Received help with bathroom 0.02
Physical environment 0.03
 Q08 Hospital room and bathroom kept clean 0.02
 Q09 Hospital room area quiet at night 0.03
Pain control 0.02
 Q13 Pain well controlled 0.01
 Q14 Staff helped with pain management 0.02
Communication about medicines 0.01
 Q16 Staff tell what new medicine is for 0.01
 Q17 Staff describe side effects of new 0.01
 medicine
Discharge information (19, 20) 0.02
CHART-augmented discharge information 0.02
 (19, 20, 28, 29)
 Q19 Reversed: talk re help after discharge 0.01
 Q20 Reversed: written information about re 0.02
 symptoms
 Q28 Explain medicines to be used at home 0.02
 understandably
 Q29 Danger signals to watch at home 0.02
CHART three-item coordination of care composite 0.02
 (23, 25, 27)
CHART four-item coordination of care composite 0.02
 (23, 24, 25, 26, 27)
CHART five-item coordination of care composite 0.03
 (23, 24, 25, 26, 27)
 Q23 Organized admission process 0.02
 Q24 Explained reason for room delay 0.01
 Q25 Scheduled test performed on time 0.02
 Q26 Check ID before giving medicines/ 0.03
 treatment/test
 Q27 Reversed: received different 0.01
 information from doctors and nurses

ICC, intra-class correlation coefficient; CHART, California Hospitals
Assessment and Reporting Taskforce.

Table 4: Intercorrelations among Scales

 Inter-Scale Correlation

 Communication Communication Nursing
Scale with Nurse with Doctor Services

Communication 1.00 0.52 0.61
 with nurse
Communication 1.00 0.37
 with doctor
Nursing services 1.00
Physical
 environment
Pain control
Communication
 about medicines
Discharge
 information
Discharge
 information,
 CHART
 augmented
CHART three-item
 coordination of
 care composite
CHART four-item
 coordination of
 care composite
CHART five-item
 coordination of
 care composite

 Inter-Scale Correlation

 Communications
 Physical Pain about
Scale Environment Control Medicines

Communication 0.48 0.57 0.52
 with nurse
Communication 0.31 0.43 0.41
 with doctor
Nursing services 0.44 0.50 0.43
Physical 1.00 0.40 0.39
 environment
Pain control 1.00 0.44
Communication 1.00
 about medicines
Discharge
 information
Discharge
 information,
 CHART
 augmented
CHART three-item
 coordination of
 care composite
CHART four-item
 coordination of
 care composite
CHART five-item
 coordination of
 care composite

 Inter-Scale Correlation

 Discharge CHART
 Information, Coordination of
 Discharge CHART Care Three-Item
Scale Information Augmented Composite

Communication 0.26 0.45 0.53
 with nurse
Communication 0.26 0.42 0.48
 with doctor
Nursing services 0.22 0.36 0.44
Physical 0.19 0.32 0.41
 environment
Pain control 0.23 0.38 0.47
Communication 0.37 0.54 0.41
 about medicines
Discharge 1.00 0.90 0.23
 information
Discharge 1.00 0.40
 information,
 CHART
 augmented
CHART three-item 1.00
 coordination of
 care composite
CHART four-item
 coordination of
 care composite
CHART five-item
 coordination of
 care composite

 Inter-Scale Correlation

 CHART CHART
 Coordination of Coordination of
 Care Four-Item Care Five-Item
Scale Composite Composite

Communication 0.56 0.60
 with nurse
Communication 0.49 0.50
 with doctor
Nursing services 0.45 0.48
Physical 0.43 0.45
 environment
Pain control 0.49 0.52
Communication 0.46 0.51
 about medicines
Discharge 0.28 0.30
 information
Discharge 0.46 0.50
 information,
 CHART
 augmented
CHART three-item 0.93 0.90
 coordination of
 care composite
CHART four-item 1.00 0.97
 coordination of
 care composite
CHART five-item 1.00
 coordination of
 care composite

CHART, California Hospitals Assessment and Reporting Taskforce.

Table 5: Correlations of Scales and Items with Global Rating and
Willingness to Recommend

 Willingness
 Global to
Item/Scale Rating Recommend

Nurse communication 0.62 0.60
 Q01 Nurses treat with 0.55 0.54
 courtesy and respect
 Q02 Nurses listen carefully 0.57 0.54
 Q03 Nurses explain things 0.51 0.49
 understandably
Doctor communication 0.43 0.42
 Q05 Doctors treat with 0.38 0.38
 courtesy and respect
 Q06 Doctors listen carefully 0.40 0.39
 Q07 Doctors explain things 0.38 0.37
 understandably
Nursing services 0.51 0.48
 Q04 Received help when 0.50 0.46
 pressed call button
 Q11 Received help with 0.48 0.44
 bathroom
Physical environment 0.48 0.45
 Q08 Hospital room and 0.43 0.40
 bathroom kept clean
 Q09 Hospital room area quiet 0.37 0.34
 at night
Pain control 0.52 0.48
 Q13 Pain well controlled 0.43 0.40
 Q14 Staff helped with pain 0.53 0.49
 management
Communication about 0.46 0.42
 medicines
 Q16 Staff tell what new 0.42 0.39
 medicine is for
 Q17 Staff describe side 0.41 0.37
 effects of new medicine
Discharge information 0.27 0.26
 (19, 20)
CHART-augmented 0.44 0.41
 discharge
 information (19,
 20, 28, 29)
 Q19 Reversed: talk re help 0.23 0.22
 after discharge
 Q20 Reversed: written 0.21 0.20
 information about re
 symptoms
 Q28 Explain medicines to be 0.42 0.41
 used at home
 understandably
 Q29 Danger signals to watch 0.37 0.36
 at home
Coordination of care 0.54 0.53
 composite for three
 CHART items (23,
 25, 27)
Coordination of care 0.58 0.56
 composite for four
 CHART items (23,
 24, 2.5, 27)
Coordination of care 0.60 0.59
 composite for five
 CHART items (23,
 24, 25, 26, 27)
 Q23 Organized admission 0.43 0.42
 process
 Q24 Explained reason for room 0.39 0.39
 delay
 Q25 Scheduled test performed 0.42 0.41
 on time
 Q6 Check ID before giving 0.35 0.35
 medicines/treatment/test
 Q27 Reversed: received 0.31 0.30
 different information
 from doctors and nurses

CHART, California Hospitals Assessment and Reporting Taskforce.

Table 6: Regression Coefficients of and Patients' Recommend Composites
Characteristics on the Global Rating and the Willingness to Recommend

 Global Rating

 CARPS
 CARPS + CHART
Composite ([dagger]) (Three Items)

Intercept 0.583 ** 0.076
Communication with nurse 0.038 ** 0.033 **
Communication with doctor 0.012 ** 0.006 **
Nursing services 0.010 ** 0.009 **
Physical environment 0.018 ** 0.015 **
Pain control 0.008 ** 0.006 **
Communication about 0.003 ** 0.002 **
 medicines
Discharge information 0.004 **
Discharge information, 0.006 **
 CHART augmented
CHART coordination of 0.023 **
 care composite
Age [greater than or equal 0.221 0.166 **
 to] 6.5
Gender = male 0.068 ** 0.056 **
Education = some college+ -0.056 ** 0.053 **
Ethnicity = Hispanic 0.170 ** 0.188 **
Ethnicity = Asian/Pacific 0.129 ** 0.163 **
 Island Islander
Ethnicity = black 0.036 0.055
Ethnicity = all other except -0.087 -0.050
 for white
Admission -0.008 -0.028
 source = physician/HMO
 referral
Reason for 0.005 0.007
 admission = medical
Reason for 0.037 -0.013
 admission = surgical
Health status = fair/poor -0.001 0.024
Discharge status = expired/ -0.397 ** -0.300 *
 LAMA
Survey language = Spanish 0.264 ** 0.224 **
Survey language = other -0.078 -0.042
Model [R.sup.2] 0.469 0.493

 Global Rating

 CARPS CARPS
 + CHART + CHART
 (Four Items) (Five Items)
Composite ([double dagger]) ([double dagger])

Intercept 0.086 0.009
Communication with nurse 0.033 ** 0.032 **
Communication with doctor 0.007 ** 0.007 **
Nursing services 0.009 ** 0.009 **
Physical environment 0.016 ** 0.016 **
Pain control 0.006 ** 0.006 **
Communication about 0.002 ** 0.001 **
 medicines
Discharge information
Discharge information, 0.006 ** 0.005 **
 CHART augmented
CHART coordination of 0.022 ** 0.025 **
 care composite
Age [greater than or equal 0.166 ** 0.160 **
 to] 6.5
Gender = male 0.058 ** 0.050 **
Education = some college+ -0.047 * -0.048 *
Ethnicity = Hispanic 0.178 ** 0.182 **
Ethnicity = Asian/Pacific 0.146 ** 0.158 **
 Island Islander
Ethnicity = black 0.050 0.054
Ethnicity = all other except -0.054 -0.057
 for white
Admission -0.017 -0.012
 source = physician/HMO
 referral
Reason for -0.001 -0.008
 admission = medical
Reason for -0.002 -0.010
 admission = surgical
Health status = fair/poor 0.026 0.015
Discharge status = expired/ -0.318 * -0.313 *
 LAMA
Survey language = Spanish 0.2.50 ** 0.261 **
Survey language = other -0.051 -0.044
Model [R.sup.2] 0.491 0.492

 Willingness to Recommend

 CARPS
 + CHART
 CARPS (Three Items)
Composite ([dagger]) ([double dagger])

Intercept 0.973 ** 0.792 **
Communication with nurse 0.013 ** 0.011 **
Communication with doctor 0.004 ** 0.002 **
Nursing services 0.003 ** 0.003 **
Physical environment 0.005 ** 0.004 **
Pain control 0.003 ** 0.002 **
Communication about 0.OO1 ** 0.000 **
 medicines
Discharge information 0.001 **
Discharge information, 0.002 **
 CHART augmented
CHART coordination of 0.008 **
 care composite
Age [greater than or equal 0.067 ** 0.048 **
 to] 6.5
Gender = male 0.032 ** 0.027 **
Education = some college+ 0.015 * 0.016 *
Ethnicity = Hispanic -0.002 0.004
Ethnicity = Asian/Pacific 0.024 * 0.036 **
 Island Islander
Ethnicity = black -0.040 ** -0.033 *
Ethnicity = all other except -0.048 * -0.035
 for white
Admission 0.012 0.004
 source = physician/HMO
 referral
Reason for -0.001 0.000
 admission = medical
Reason for 0.007 -0.011
 admission = surgical
Health status = fair/poor 0.008 0.017 *
Discharge status = expired/ -0.159 ** -0.125 **
 LAMA
Survey language = Spanish 0.080 ** 0.066 **
Survey language = other -0.012 0.000
Model [R.sup.2] 0.419 0.444 *

 Willingness to Recommend

 CARPS CARPS
 + CHART + CHART
 (Four Items) (Five Items)
Composite ([double dagger]) ([double dagger])

Intercept 0.796 ** 0.766 **
Communication with nurse 0.011 ** 0.011 **
Communication with doctor 0.003 ** 0.003 **
Nursing services 0.003 ** 0.003 **
Physical environment 0.004 ** 0.004 **
Pain control 0.002 ** 0.002 **
Communication about 0.000 ** 0.000 **
 medicines
Discharge information
Discharge information, 0.002 ** 0.002 **
 CHART augmented
CHART coordination of 0.008 ** 0.009 **
 care composite
Age [greater than or equal 0.048 ** 0.045 **
 to] 6.5
Gender = male 0.028 ** 0.025 **
Education = some college+ 0.018 ** 0.018 **
Ethnicity = Hispanic 0.001 0.002
Ethnicity = Asian/Pacific 0.029 ** 0.034 **
 Island Islander
Ethnicity = black -0.03.5 * -0.033 **
Ethnicity = all other except -0.036 -0.037 *
 for white
Admission 0.008 0.010
 source = physician/HMO
 referral
Reason for -0.003 -0.005
 admission = medical
Reason for -0.007 -0.010
 admission = surgical
Health status = fair/poor 0.018 * 0.014
Discharge status = expired/ -0.131 ** -0.129 **
 LAMA
Survey language = Spanish 0.075 ** 0.079 **
Survey language = other -0.003 0.000
Model [R.sup.2] 0.442 0.443 **

** <.01;
* p<.05.

([dagger]) Regression models with the CAHPS composites and patients'
characteristics.

([double dagger]) Regression models with the six CAHPS composites,
CHART item-augmented discharge information composite, and the three-,
four-, and five-item CHART coordination of care composites, and
patients' characteristics.

CAHPS, Consumer Assessment of Health Care Providers and Systems;
CHART, California Hospitals Assessment and Reporting Taskforce; HMO,
health maintenance organization; LAMA, left against medical advice.
COPYRIGHT 2008 Health Research and Educational Trust
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2008 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Methods and Research Briefs
Author:Rothman, Arlyss Anderson; Park, Hayoung; Hays, Ron D.; Edwards, Carol; Dudley, R. Adams
Publication:Health Services Research
Date:Dec 1, 2008
Words:7999
Previous Article:Is neighborhood access to health care provision associated with individual-level utilization and satisfaction?
Next Article:Accuracy of physician billing claims for identifying acute respiratory infections in primary care.
Topics:

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters