Composite health plan quality scales.
Although much attention has been given to development of quality measurement systems such as HEDIS[R] and more recently physician-centered quality measures being developed by the National Center for Quality Assurance (NCQA), CMS, and the American Medical Association, comparatively little research has focused on the problem of building composite measures of health care service quality from discrete and seemingly unrelated measures of clinical quality. This article tackles the problem of constructing a credible unidimensional composite HMO quality scale from cross-sectional HEDIS[R] scores reported by commercial HMOs in calendar year (CY) 1999. Evidence is uncovered that composite subscales by categories of care and a unidimensional composite HEDIS[R] quality measure are feasible.
The quality of medical care is likely to have many possible dimensions, but it is conceivable that the quality of medical care rendered by an HMO may also have an organizational dimension, reflecting a common approach across all types of medical care (Wholey et al., 2003). If a plausible summary measure of overall HMO output quality could be developed, it is likely to be useful to many actors: employers assembling sets of health plan choices to offer employees, consumers faced with health plan choices, HMO regulators needing a way to screen firms for regulatory attention, and researchers in need of quality variables to add a quality dimension to models of HMO behavior, e.g., in a model of HMO costs like the CMS risk- adjustment model.
Some research concerning composite health care quality measures has taken place ancillary to studies of consumer choice of health plans in which researchers have sought to determine whether consumers actually use quality information that is provided to them. The literature on consumer choice of health plans usually includes quality variables in binomial or multinomial logit models of utility maximization (Chernew and Scanlon, 1998; Scanlon and Chernew, 1999; Feldman, Christianson, and Schultz, 2000; Chernew, Gowrisankaran, and Scanlon, 2001; Scanlon et al., 2002). Recent work by Chernew et al. (2003) that investigates employer decisions about which health plans to offer employees contains a good example of how researchers have used limited amounts of HEDIS[R] data to construct a quality variable by rolling several HEDIS[R] variables together and thereby obtaining a quality construct usable as a regressor to represent quality in a more general regression model. This literature does not address measurement properties of the quality measures that are implemented, and the measures implemented appear to be simply the best available proxy for the construct the researchers wished to include in their models of consumer behavior.
The most sophisticated HMO-level output quality scales so far reported have been constructed by researchers whose primary objective was to construct composite scales that might eventually be disseminated to consumers (Lied et al., 2002; Zaslavsky et al., 2002). Both of these studies relied on a single year of quality data taken from Medicare+Choice health plans. Lied and colleagues constructed a composite HEDIS[R] score for each health plan in a cross-section of Medicare+Choice plans by averaging percentile scores for 17 indicators chosen primarily from available HEDIS[R] effectiveness of care measures. Zaslavsky and colleagues used factor analysis to reduce 20 quality measures, consisting of 12 HEDIS[R] and 8 consumer satisfaction CAHPS[R] measures, into 4 interpretable constructs that explained 65 percent of the variance across all measures. The four constructs (office care, access/customer service, vaccinations, and clinical quality) were then used to create averaged scales that exhibited good evidence of internal consistency. In each case Cronbach's (1951) alpha statistics computed to test the composite scales returned evidence of internal consistency, an alpha score of 0.88 for the scale constructed by Lied and colleagues, and alpha scores ranging from 0.86 to 0.96 for the scales constructed by Zaslavsky and colleagues.
These studies adopted different approaches to handle the problem of missing quality data, a significant obstacle to the construction of composite scales. Lied and others dropped 10 percent of the observations in their original sample when quality reporting was very limited or suspect. Only 64 of 160 remaining observations reported all 17 of the HEDIS[R] measures, and the mean number of reported measures in their final sample was 13.58 per observational unit. Each HMO's composite quality score was thus actually an average of its percentiles for the HEDIS[R] indicators that it happened to report.
To avoid losing HMO-level HEDIS[R] observations, Zaslavsky and others used procedures to impute missing HEDIS[R] scores, employing as predictor variables other reported HEDIS[R] scores for HMOs that did report the measure to be imputed and also special dummy variables keyed to patterns of missing variables in each included observation. This imputation method generated complete sets of HEDIS[R] measures that could then be used to compute factor analysis results. Although variances for observations with many imputed values were indeed larger than for observations with few missing HEDIS[R] scores, the effect of larger variances seemed to be muted when measures were standardized and averaged into composite measures.
In different ways both studies grappled with evidence that quality of care is multidimensional. When Lied's group subjected their data to factor analysis, the researchers found evidence that the variance in the HEDIS[R] measures decomposed into three components rather than the single component scale that they had constructed, a result conceivably related to having disproportionately many measures from particular categories of medical care. For example, 6 of the 17 measures pertained specifically to diabetes care. Zaslavsky's group encountered a similar problem as a result of factor analyzing data from HEDIS[R] and CAHPS[R] together; the four factor structure that they uncovered is diffuse in its range, and it generally appears that HEDIS[R] and CAHPS[R] measures contribute to different factor loadings.
Thus, recent quality scaling work using HEDIS[R] data encountered difficulties when researchers used conventional scaling techniques to convert multiple measures into a smaller number of quality-related variables. Gaps in reported quality scores present a fundamental problem because algorithms needed to estimate factor analytic models require complete data with no missing values in any of the observations used. Imputation of missing values is unavoidable if the sample is to include the broadest possible set of HMOs.
Indiscriminate inclusion of all raw HEDIS[R] performance measures in a single factor analytic model may complicate identification of a unidimensional underlying organizational-level quality scale. As previously noted, the studies that have implemented factor analysis of HEDIS[R] data have included all available variables in the same factor analysis with disproportionate numbers of variables for certain disease categories such as diabetes.
An alternative approach is to implement a second-order factor model by first using HEDIS[R] variables to create subscales by categories of care (e.g., diabetes, women's care, childhood immunizations, etc.) and then using category-of-care subscales in an exploratory factor analysis intended to uncover underlying quality constructs at the health plan level (McArdle, 1980; Hershberger, 1994; Kline, 2005). Availability of CAHPS[R] survey results also makes it feasible to consider evidence for the external validity of a composite HEDIS-based HMO quality scale.
This article reports implementation of a two-stage approach to composite scaling of HEDIS quality data together with evidence for the internal consistency and external validity of the resulting unidimensional quality scale.
Data and Data Preparation
Data available for this study included 31 HEDIS[R] process of care and outcome measures designed and collected by NCQA from 380 HMOs concerning medical care provided to under age 65 commercial subscribers in CY 1999. The insured population covered by the reporting HMOs represented almost the entire under age 65 non-Medicaid, non-Medicare population that received its health care through HMOs in CY 1999. A list of the HEDIS[R] measures with descriptive statistics appears in Table 1. All of the measures are interval measures suitable for factor and correlation-based analysis and normality plots indicate that the measures appear to be normally distributed.
Reporting of HEDIS[R] measures by HMOs was often incomplete, though less incomplete than was the case in the study by Zaslavsky and others that also employed multiple imputation methods for missing data. Almost all HEDIS[R] measures in the data set had missing values, but the problem is most acute for the the battery of measures for asthma first introduced in the measurement year. Multiple imputation techniques using algorithms packaged in SAS[R] Proc MI and Proc MIANALYZE were employed to maintain the range of measures needed for the construction of a broad-based composite quality measure. The imputation algorithms assume that data are missing at random (MAR), which means that the value of a missing score is assumed independent of the likelihood that the score will be missing (Rubin, 1976, 1987, 1991; Allison, 2002; SAS Institute, 1999). If the MAR assumption is satisfied, multiple imputation methods lead to estimates that are approximately unbiased and efficient, with standard errors that take into account variance added by imputation (Allison, 2002).
Implementation of multiple imputation methods occurs in two stages. First, the imputation algorithm is used to generate multiple versions of the data that is to be analyzed, with each version having its own randomly assigned values for missing scores. Second, another algorithm uses the multiple versions of the data to compute a single set of relevant statistics with adjusted standard errors. Where, as here, correlations between measures are needed for factor analysis, a SAS-supplied variant of the algorithm is employed in order to correct for bias that would otherwise be present due to skewness of the distribution of sample correlation coefficients (SAS Institute, 1999).
In accord with the research strategy of generating a set of quality scales by categories of care and then using those scales to build a composite scale, multiple imputation methods were implemented separately for each battery of care measures: diabetes, heart, asthma, mental health, women's care, childhood immunization, and adolescent immunization. When an HMO had not reported values for a given battery of care measures, no imputation of values for that HMO for the battery of care occurred and that observation was ignored in constructing a scale for the battery of care. Once category of care scales had been constructed, multiple imputation was used again to fill in missing scales before proceeding with construction of the quality composite. Each implementation of multiple imputation methods employed the Markov Chain Monte Carlo method for imputing values for arbitrary non-monotone missing value patterns. Where missing data patterns were monotone, alternative imputation algorithms capable of exploiting the presence of monotonicity were implemented, but use of alternative algorithms for multiple imputation did not affect subsequent analysis.
Data available for external validation of HEDIS-based composite scales consisted of CAHPS[R] survey data for CY 1999 collected by NCQA from 361 out of the 380 HMOs that reported HEDIS[R] data. Specifically, for each of six CAHPS[R] composite measures for each HMO, the total percentage of respondents who gave their HMO a satisfactory or good rating was available. Depending on the CAHPS[R] composite this was the the percentage of respondents who said they were not having a problem or the percentage who said they were usually or always encountering the care or behavior covered by the CAHPS[R] composite.
Methods of Analysis
The methods of analysis consisted of exploratory factor analysis, summated scale construction and testing, and linear regression. Factor analysis was used to identify grounds for data consolidation through scale construction. Summated scales by categories of care and finally as a composite of the category of care scales are the principal output of the study and their internal consistency was validated using the Cronbach's (1951) alpha statistic. Linear regression was used to evaluate the external validity of the final composite quality of care scale by examining its relationship with CAHPS[R] survey results.
As in prior studies that have sought to reduce the dimensionality of HEDIS[R] health plan quality data, exploratory factor analysis was used to identify the component or components of variance common to multiple HEDIS[R] measures. Factor analysis is a statistical technique that originated in the field of psychometrics, but is used widely in social science research. The method seeks to explain the variation of observed random variables in terms of a smaller set of unobserved random variables, known as factors (Nunally and Bernstein, 1994). Algorithms that implement factor analysis model observed variables as the linear combination of unobserved variables and a random error term.
A matrix of correlations between observed variables is sufficient to compute a factor analysis model. As previously noted, the skewness of the distribution of sample correlations was taken into account in assembling correlation matrices for factor analysis. Confidence intervals for each correlation in each matrix were also available, making it feasible in each case to compute models for low- and high-range correlations as well as for set of estimated correlations. In this way it was possible to verify that results did not vary substantially within the range of the confidence intervals for the correlation matrices.
The output of a factor model is a set of loadings or factor pattern which is the set of estimated coefficients that would be applied to unobserved variables to produce the observed variables. In each case the factor pattern was analyzed by standard procedures (Nunally and Bernstein, 1994). A HEDIS[R] measure was deemed to have a significant loading if it had a single loading on a single unobserved variable with an absolute value equal to or exceeding 40 percent and no significant loading on any other unobserved variable. Varimax (orthogonal) and promax (oblique) rotation methods were employed to enhance interpretability of factor patterns when the factor solution indicated that more than a single unobserved factor could be present. In multifactor situations, the meaning of each factor (each column in a factor pattern) was inferred from the pattern of meaning implied by the HEDIS[R] measures having significant loadings on that factor (Nunally and Bernstein 1994).
Summated scales were constructed based on the results of the exploratory factor analyses. As the term summated scale implies, such scales are constructed by summing the items that make the scale (Nunally and Bernstein, 1994; Spector, 1992). Measurement theory behind summated scale construction postulates that error terms associated with each variable included in a scale should cancel out due to their randomness, permitting the true value of the construct to be revealed. The theory behind summated scales also implies that the variables making up a scale should be highly correlated with each other, and this intuition lies behind Cronbach's (1951) alpha, the standard test statistic for evaluating the internal consistency of summated scales (Spector, 1992; Hatcher, 1994). Generally, the alpha statistic will increase as the correlation between variables increases and the number of variables included in the scale increases. By convention, an alpha statistic equal to 0.7 is deemed minimally adequate evidence of internal consistency for a summated scale, though obviously the closer that the alpha statistic is to one, the stronger the evidence of internal consistency.
External validity of composite quality scales was investigated with linear regressions each of which used a composite quality scale as the dependent variable and a set of CAHPS[R] survey results as regressors. The set of regressors consisted of the six CAHPS[R] composites that involve aspects of health plan performance: (1) getting needed care, (2) getting care quickly, (3) physician communication, (4) staff courtesy, (5) customer service, and (6) claims processing. The presence of significant coefficients in the regression and the presence of explanatory power in the regression would be interpreted as evidence supporting the hypothesis that both HEDIS[R] and CAHPS[R] are measuring the same unobservable factor: health plan quality.
To document continuity with prior research results, an all measures factor analysis similar to the factor models in previous measurement studies of HEDIS[R] data included all of the 31 HEDIS[R] process of care measures, and as in those studies, the all measures model pointed to a multidimensional factor solution whose pattern on promax rotation is shown in Table 2. The factor pattern was interpreted variable by variable in accord with the rule previously described, looking for variables that have a strong loading (absolute value of a loading > 40 percent) on a single unobserved factor. All immunization variables loaded strongly and exclusively on the second factor, as did the variable for controlling blood pressure. All diabetes and mental health variables along with some women's health and coronary care variables loaded strongly and exclusively on the first factor. Only asthma care variables loaded strongly and exclusively on the third factor.
The factor pattern for the all measures model shown in Table 2 is derived, as are all factor models in this article, from an analysis using standardized variables. The pattern demonstrated in another run with unstandardized variables was substantively the same, which would be expected with variables not having widely discrepant variances. The all measures factor solution accounts for 90 percent of the variance contained in the 31 underlying measures. A scree plot raised the issue of adding a fourth factor to the solution, but the additional variance associated explained by a fourth factor was negligible (< 3 percent) and on examination its loadings were uninterpretable without even one variable having a strong and exclusive loading. On this basis a solution including more than three factors was rejected. Overall, the factor pattern provides evidence to support the existence of asthma care, immunization, and and a third factor that represents four different categories of care (diabetes, mental health, women's health, and coronary care). The three factors are labeled in Table 2 as Four Categories, Immunizations, and Asthma. The fact that so many categories of care were loading on a single factor was taken as a promising indication that the intended research strategy employing subscales would work as hoped.
In accord with the plan of research, factor analysis was performed by categories of care to examine whether measures whose subject matter is identical hang together in ways indicative of disease-specific or treatment-specific scales. Table 3 summarizes the results of eight different factor model estimates, displaying the loadings for a single-factor solution on the variables included in a given category of care model. Table 3 displays the proportion of total variance explained by each category of care factor solution and also the Cronbach's (1951) alpha statistics for the summated category of care scales that were created based on the factor analyses results obtained.
Sets of variables defined by categories of care turned out with minor and easily handled exceptions to lead to factor analyses that pointed to persuasive single factor solutions with strong loadings of comparable magnitude on most variables included in each set of estimating variables. The only set of variables that did not lead to a clear single factor solution was the battery of measures for mental health. The results of an analysis on all mental health variables pointed clearly to a two-factor solution, one defined by depression care and the other defined by followup after hospitalization. The implication of this result seemed to be to allocate these variables to separate factor models; thus, the loadings for these variables in Table 3 reflect the single factor solutions for the two different mental health factor models that were then implemented.
The proportion of variance reported by the SAS[R] algorithms that estimated the factor models frequently reported that the models as estimated explained more than 100 percent of the variance contained in the included items. In part this reflects the high correlation among measures and in part this reflects the possibility inherent in factor analysis models of extracting factors that explain either negative or positive shares of the variance contained within the correlation matrix used by the estimating algorithm, subject to the requirement that the variance explained by all factors sums to 100 percent (Hatcher, 1994). When this occurred, the proportion of variance explained was reported (Table 3) as simply 100 percent.
The fact that loadings within each given category of care tended to have broadly similar magnitudes may be interpreted as evidence that construction of summated scales is warranted. A summated scale was constructed for each of the categories of care (Table 3). With two exceptions satisfactory alpha coefficients were obtained using for each category of care all of the variables associated with the category of care. For coronary care and mental health II (depression care), the internal consistency of the summated scale could be substantially improved by excluding the variable with the smallest loading. In each case the magnitude of the outlier loading was substantially smaller than the loadings for the rest of the variables, and exclusion was consistent with the theory of summated scale construction. The alpha coefficients for coronary care and mental health II (Table 3) are for the summated scales that exclude the variable that had the lowest loading in the estimate for the corresponding factor model.
Six out of the eight category of care scales have internal consistency scores equal to 0.86 or better and two of the scales yielded high alpha coefficients substantially exceeding 0.90. Only one of the scales for coronary care, has a minimally acceptable alpha coefficient (Table 3).
Four alternative factor models were then estimated using the summated category of care scales created in the previous phase of the research and alternative summated composite scales were created for the set of variables in each model. In every case there was a clear single factor solution. The single factor loadings for each of the four different composite factor models appear in Table 4. In every model the single factor solution explains virtually all variance in the included items. Loadings on the scales for asthma care and mental health II were pronouncedly smaller in magnitude than those for the other scales. Unsurprisingly, removal of those items from composite models improved the internal consistency of the summated composite constructed from the remaining category of care scales.
Finally, a summated HEDIS[R] composite scale created from six category of care scales (Table 4) was used as the dependent variable in a regression that used CAHPS[R] composites as regressors (Table 5). The regression was significant and the adjusted R-squared statistic was 0.36. Coefficients on all variables were statistically significant or close to statistically significant. The coefficients for the physician communication and staff courtesy variables had negative signs. This suggests an interesting question bearing on the validity of CAHPS[R] results as quality measures: whether social circumstances of care that are perceived positively by patients are necessarily indicators of good medical care. However, the many positive coefficients in the regression and the statistical significance of all coefficients are sufficient to conclude that there is strong external evidence that the six category HEDIS[R] scale is a measure of the quality of medical care in the reporting health plans. In particular, the regression results provide reassurance that performance on the aggregate quality scale is not simply an artifact of which HMOs have good recordkeeping systems.
The results reported in this article are consistent with measurement theory. Reliable scales were constructed around categories of care and then analyzed together to construct a reliable measure of output quality. In this way, a broadly based composite measure emerged, reflecting six types of care. Validating regression results establish a strong connection between the six-category composite quality scale and the dimensions of consumer satisfaction constructed from CAHPS[R] survey data. The six-category scale exhibits plausible and persuasive evidence of internal consistency and external validity. Although it would be desirable to have a unidimensional scale that incorporated still more categories of care, the breadth of the scale that has been developed may plausibly be seen as a measure of quality at an organizational level.
Nevertheless, it is important to acknowledge significant limitations in the results obtained and a need for caution in how these results are used. First, this was an exploratory study on a single year of cross-sectional quality data for a single type of insured population. It is unknown whether the results will remain robust when the approach is used with additional years of quality data and for Medicare and Medicaid HMOs as well as for the commercial HMOs represented in this study. With new data a formal confirmatory factor analysis model should be evaluated, and because second-order factor models are difficult to identify econometrically, applied studies that incorporate quality scales within larger models could also provide important insight into whether the scales appear to add a plausible quality dimension (McArdle, 1980; Hershberger, 1994; Kline, 2005). For example, a composite HEDIS[R] scale (or category of care scales) could be included in a risk-adjustment model or other model of HMO economic behavior.
A second limitation on the substantive results obtained is the matter of missing data. No study of the measurement properties of HEDIS[R] measures has implemented a fully satisfactory solution to the missing data problem. The MAR assumption behind the multiple imputation methods implemented here is questionable: It is difficult to believe that in all cases the failure of an HMO to report a particular score had nothing to do with the level of the unreported score. Fortunately, the external validity regression provides tangible evidence that the composite HEDIS[R] scale is measuring something real despite the problematic nature of the MAR assumption. An actual program affecting payments to plans could not implement any kind of composite measure that relied on imputed data. It is to be hoped that the problem of missing data will disappear as HMOs become more and more accustomed to annual collection of HEDIS[R] data and purchasers of HMO services become more insistent that the data be thoroughly reported.
In light of the research limitations that have been discussed the immediate usefulness of the scales developed here is limited. Certainly, the scales as they have been developed could be helpful as guides to HMOs on needed quality improvement, particularly the category of care scales. Beyond that possible application, the most immediate use of the scaling results is as a basis for further research to test and hopefully validate the approach. If the modeling approach implemented here in an exploratory way can be confirmed on a more substantial body of quality data, then the results would provide essential information needed for practical development of a pay-for-performance payment system for HMOs. Additional composite quality scaling research might also lead to refinements in the underlying components of aggregate scales, such as their relative weighting. More generally, if these methods can be broadly validated, they might be applied to the problem of physician performance measurement using the battery of physician-level process measures that NCQA is currently promulgating.
Although the results achieved here have limitations, they should be encouraging to policymakers and to institutions like CMS that are endeavoring to build the technical infrastructure needed to make pay-for-performance in health care a reality. The overarching message is that aggregate quality measures that would greatly facilitate the development of pay-for-performance payment systems may be practically feasible.
Much effort and expense have been expended developing the HEDIS[R] performance measurement system, but up until now evidence has been meager that multiple discrete HEDIS[R] measures can be assembled into a plausible organization-level composite measure of health care quality. The methods used in this study demonstrate that HEDIS[R] measures may carry information relevant both to the specific care contexts in which they arise and more generally to output quality at the organizational level. That the HEDIS[R]based composite quality scale is highly correlated with composites in the CAHPS[R] consumer satisfaction survey provides important evidence of external validity, but also suggests the possibility of regularly using such methods as a check on the ongoing validity of both measurement systems. As policymakers become more determined to pay providers based on their performance, solutions to the problem of health care quality measurement are urgently needed. The results obtained here should encourage researchers and institutions that sponsor research to focus more attention on development and testing of aggregate health care quality scales.
Reprint Requests: Todd Caldis, Ph.D., J.D., Centers for Medicare & Medicaid Services, Office of the Actuary, Mail Stop N3-02-02, 7500 Security Blvd. Baltimore, MD 21244-1850. E-mail: todd. email@example.com
Allison, P.D.: Missing Data. Sage Publications USA. Thousand Oaks, CA. 2002.
Chernew, M. and Scanlon, D.P.:Health Plan Report Cards and Insurance Choice. Inquiry 35(1):9-22, Spring 1998.
Chernew, M., Gowrisankaran, G., and Scanlon, D.P.: Learning and the Value of Information: The Case of Health Plan Report Cards. National Bureau of Economic Research Working Paper No. 8589. November 2001.
Chernew, M., Gowrisankaran, G., McLaughlin, C., et al.: Quality and the Employers' Choice of Health Plan. National Bureau of Economic Research Working Paper No. 9847. July 2003.
Cronbach, L.J.: Coefficient Alpha and the Internal Structure of Tests. Psychometrika 16(3):297-334, 1951.
Engberg, J., Wholey, D., Feldman, R., et al.: The Effect of Mergers on Firm's Costs: Evidence from the HMO Industry. The Quarterly Review of Economics and Finance 44 (4):574-600, September 2004.
Feldman, R., Christianson, J., and Schultz, J.: Do Consumers Use Information to Choose a HealthCare System? The Milbank Quarterly 78(1):47-77, March 2000.
Hatcher, L.: Step-by-Step Approach to Using SAS[R] System for Factor Analysis and Structural Equation Modeling. SAS Institute. Cary, NC. 1994.
Hershberger, S.L.: The Specification of Equivalent Models Before the Collection of Data. In Von Eye, A. and Clogg, C. (eds.): Latent Variable Analysis. SAGE Publications, Inc. Thousand Oaks, CA. 1994.
Kim, J.O. and Mueller, C.W.: Factor Analysis: Statistical Methods and Practical Issues. Sage Publications, Inc. Beverly Hills, CA. 1978.
Kline, R.B.: Principles and Practice of Structural Equation Modeling, 2nd Edition. The Guildford Press. New York, NY. 2005
Lied, T., Malsbary, R., Eisenberg, C., et al.: Combining HEDIS[R] Indicators: A New Approach to Measuring Plan Performance. Health Care Financing Review 23(4):117-129, Summer 2002.
McArdle, J.: Causal Modeling Applied to Psychonomic Systems Simulation. Behavior Research and Instrumentation 12(2):193-209, 1980.
Nunally, J.C. and Bernstein, I.H.: Psychometric Theory, Third Edition. McGraw-Hill, Inc. New York, NY. 1994.
Rubin, D.B.: Inference and Missing Data. Biometrika 63:581-592, 1976.
Rubin, D.B.:Multiple Imputation for Nonresponse in Surveys. John Wiley and Sons. New York, NY. 1987.
Rubin, D.B. and Schenker, N.: Multiple Imputations in Health-Care Databases: An Overview and Some Applications. Statistics in Medicine 10:585-598, 1991. SAS Institute: SAS Procedures Guide. Version 8. 1999.
Scanlon, D.P. and Chernew, M.: HEDIS[R] Measures and Managed Care Enrollment. Medical Care Research and Review 56(Suppl. 2):56-84, 1999.
Scanlon, D.P., Chernew, M., McLaughlin, C. et al.: The Impact of Health Plan Report Cards on Managed Care Enrollment. Journal of Health Economics 21(1): 19-41, January 2002.
Spector, P.E.: Summated Rating Construction. SAGE Publications, Inc. Newbury Park, CA. 1992.
Wholey, D.R., Christianson, J.B., Finch, M., et al.: Evaluating Health Plan Quality 1: A Conceptual Model. The American Journal of Managed Care 9 (2): SP53-SP64, June 2003.
Zaslavsky, A.M., Shaul, J., Zaborski, L., et al.: Combining Health Plan Performance Indicators into Simpler Composite Measures. Health Care Financing Review 23 (4):101-115, Summer 2002.
Todd Caldis, Ph.D., J.D.
The author is with the Centers for Medicare & Medicaid Services (CMS). The research in this article was supported in part by the Agency for Healthcare Research and Quality under Grant Number 1 R03 HS11515-01. The statements expressed in this article are those of the author and do not necessarily reflect the views or policies of the Agency for Healthcare Research and Quality or CMS.
Table 1 Descriptive Statistics for HEDIS[R] Measures Care Measure MEAN S.D. MIN Child Immunizations Diptheria-Tetanus-Whooping Cough (DTP) 78.76 13.31 14.19 Measles-Mumps-Rubella 87.01 8.69 32.34 Oral Polio Vaccine (OPV) 82.61 12.10 21.23 Pneumonia Vaccine (HIB) 80.74 11.99 22.22 Hepatitis B Vaccine 75.13 15.07 6.71 Chickenpox Vaccine (VZV) 63.80 11.11 19.36 Adolescent Immunizations Measles-Mumps-Rubella 58.96 20.68 0.77 Hepatitis B Rate 34.32 19.47 0 Chickenpox Vaccine (VZV) 23.98 15.95 0 Women's Health Breast Cancer Screening Rate 73.37 7.15 40.04 Cervical Cancer Screening 71.72 9.24 31.24 Prenatal Care 84.59 13.42 2.31 Check-Up After Delivery 72.38 13.64 16.17 Coronary Care Control Blood Pressure 38.94 9.57 8.47 Beta Blocker After Heart Attack 85.03 11.00 44.55 Cholesterol Screening 68.97 11.55 25.74 Cholesterol Control 44.33 17.16 0 Diabetes Care Blood Sugar Testing (HbA1c) 75.02 11.66 13.20 Poor Control of Blood Sugar 55.22 16.61 0 Eye Exams 45.32 15.07 1.54 Cholesterol Screening 68.97 11.22 11.77 Cholesterol Control 36.74 11.55 0 Monitor Diabetic Nephropathy 36.08 14.41 6.05 Asthma Care Asthma Medication Management (Age 5-9) 57.53 13.75 0 Asthma Medication Management (Age 10-17) 54.78 13.97 0 Asthma Medication Management (Age 18-56) 59.40 14.74 0 Mental Health Follow-Up After Hospitalization--7 Days 46.86 16.61 0 Follow-Up After Hospitalization--30 Days 69.41 16.61 0 Practitioner Contact--Antidep Medication 58.74 10.23 5.50 Antidep Med Acute Phase 42.13 10.45 7.81 Antidep Med Continuation Phase 21.23 10.67 0 Care Measure MAX N Child Immunizations Diptheria-Tetanus-Whooping Cough (DTP) 95.48 357 Measles-Mumps-Rubella 98.01 360 Oral Polio Vaccine (OPV) 97.57 357 Pneumonia Vaccine (HIB) 98.56 358 Hepatitis B Vaccine 96.25 356 Chickenpox Vaccine (VZV) 92.95 360 Adolescent Immunizations Measles-Mumps-Rubella 94.60 353 Hepatitis B Rate 81.51 350 Chickenpox Vaccine (VZV) 75.90 350 Women's Health Breast Cancer Screening Rate 88.66 368 Cervical Cancer Screening 91.19 368 Prenatal Care 99.99 354 Check-Up After Delivery 98.56 350 Coronary Care Control Blood Pressure 90.86 257 Beta Blocker After Heart Attack 99.99 233 Cholesterol Screening 93.50 279 Cholesterol Control 82.72 270 Diabetes Care Blood Sugar Testing (HbA1c) 97.13 367 Poor Control of Blood Sugar 99.99 355 Eye Exams 90.53 366 Cholesterol Screening 88.55 368 Cholesterol Control 76.12 360 Monitor Diabetic Nephropathy 96.58 365 Asthma Care Asthma Medication Management (Age 5-9) 93.28 263 Asthma Medication Management (Age 10-17) 96.80 285 Asthma Medication Management (Age 18-56) 96.36 337 Mental Health Follow-Up After Hospitalization--7 Days 90.97 301 Follow-Up After Hospitalization--30 Days 99.99 301 Practitioner Contact--Antidep Medication 88.00 304 Antidep Med Acute Phase 81.73 302 Antidep Med Continuation Phase 72.38 300 NOTES: HEDIS[R] is Health Employers Data Information Set. S.D. is standard deviation. SOURCE: Caldis, T., Centers for Medicare & Medicaid Services, 2007. Table 2 Factor Pattern for All Measures Factor Analysis Factor 1 Four Factor 2 Factor 3 Care Measure Categories Immunizations Asthma Percent Child Immunizations Diptheria-Tetanus-Whooping Cough (DTP) -17 106 -2 Measles-Mumps-Rubella -17 99 5 Oral Polio Vaccine (OPV) -8 101 0 Pneumonia Vaccine (HIB) 1 96 -3 Hepatitis B Vaccine -1 92 6 Chickenpox Vaccine (VZV) 15 42 1 Adolescent Immunizations Measles-Mumps-Rubella 26 72 -6 Hepatitis B Rate 30 50 2 Chickenpox Vaccine (VZV) 39 48 -4 Women's Health Breast Cancer Screening Rate 61 33 -15 Cervical Cancer Screening 48 54 -1 Prenatal Care 49 35 3 Check-Up After Delivery 50 43 6 Coronary Care Control Blood Pressure 25 41 31 Beta Blocker After heart Attack 55 31 5 Cholesterol Screening 61 30 6 Cholesterol Control 67 30 2 Diabetes Care Blood Sugar Testing (HbA1c) 57 33 15 Poor Control of Blood Sugar 75 8 19 Eye Exams 76 20 0 Cholesterol screening 72 18 3 Cholesterol Control 74 17 4 Monitor Diabetic Nephropathy 86 0 -3 Asthma Care Asthma Medication Management (Age 5-9) 16 1 85 Asthma Medication Management (Age 10-17) 12 -2 91 Asthma Medication Management (Age 18-56) 17 3 84 Mental Health Follow-Up After Hospitalization-7 Days 89 -1 0 Follow-Up After Hospitalization-30 Days 90 -1 3 Practitioner Contact-Antidep Medication 88 -20 10 Antidep Med Acute Phase 87 -19 14 Antidep Med Continuation Phase 64 -14 31 SOURCE: Caldis, T., Centers for Medicare & Medicaid Services, 2007. Table 3 Factor Loading Patterns for Category of Care Scales Child Adolescent Women's Care Measure Immunization Immunization Health Percent Child Immunizations Diptheria-Tetanus- Whooping Cough (DTP) 98 Measles-Mumps-Rubella 92 Oral Polio Vaccine (OPV) 99 Pneumonia Vaccine (HIB) 96 Hepatitis B Vaccine 91 Chickenpox Vaccine (VZV) 49 Adolescent Immunizations Measles-Mumps-Rubella 90 Hepatitis B Rate 78 Chickenpox Vaccine (VZV) 80 Women's Health Breast Cancer Screening Rate 76 Cervical Cancer Screening 89 Prenatal Care 70 Check-Up After Delivery 81 Coronary Care Control Blood Pressure Beta Blocker After Heart Attack Cholesterol Screening Cholesterol Control Diabetes Care Blood Sugar Testing (HbA1c) Poor Control of Blood Sugar Eye Exams Cholesterol Screening Cholesterol Control Monitor diabetic Nephropathy Asthma Care Asthma Medication Management (Age 5-9) Asthma Medication Management (Age 10-17) Asthma Medication Management (Age 18-56) Mental Health Follow-Up After Hospitalization--7 Days Follow-Up After Hospitalization--30 Days Practitioner Contact--Antidep Medication Antidep Med Acute Phase Antidep Med Continuation Phase Proportion Total Variance Explained 99 100 100 Alpha Statistic (Summated Scales) 0.95 0.87 0.86 Coronary Diabetes Asthma Care Measure Care Care Care Percent Child Immunizations Diptheria-Tetanus- Whooping Cough (DTP) Measles-Mumps-Rubella Oral Polio Vaccine (OPV) Pneumonia Vaccine (HIB) Hepatitis B Vaccine Chickenpox Vaccine (VZV) Adolescent Immunizations Measles-Mumps-Rubella Hepatitis B Rate Chickenpox Vaccine (VZV) Women's Health Breast Cancer Screening Rate Cervical Cancer Screening Prenatal Care Check-Up After Delivery Coronary Care Control Blood Pressure 37 Beta Blocker After Heart Attack 45 Cholesterol Screening 66 Cholesterol Control 81 Diabetes Care Blood Sugar Testing (HbA1c) 86 Poor Control of Blood Sugar 80 Eye Exams 63 Cholesterol Screening 75 Cholesterol Control 80 Monitor diabetic Nephropathy 61 Asthma Care Asthma Medication Management (Age 5-9) 91 Asthma Medication Management (Age 10-17) 98 Asthma Medication Management (Age 18-56) 92 Mental Health Follow-Up After Hospitalization--7 Days Follow-Up After Hospitalization--30 Days Practitioner Contact--Antidep Medication Antidep Med Acute Phase Antidep Med Continuation Phase Proportion Total Variance Explained 100 100 100 Alpha Statistic (Summated Scales) 0.70 (1) 0.88 0.95 Mental Mental Care Measure Health I Health II Percent Child Immunizations Diptheria-Tetanus- Whooping Cough (DTP) Measles-Mumps-Rubella Oral Polio Vaccine (OPV) Pneumonia Vaccine (HIB) Hepatitis B Vaccine Chickenpox Vaccine (VZV) Adolescent Immunizations Measles-Mumps-Rubella Hepatitis B Rate Chickenpox Vaccine (VZV) Women's Health Breast Cancer Screening Rate Cervical Cancer Screening Prenatal Care Check-Up After Delivery Coronary Care Control Blood Pressure Beta Blocker After Heart Attack Cholesterol Screening Cholesterol Control Diabetes Care Blood Sugar Testing (HbA1c) Poor Control of Blood Sugar Eye Exams Cholesterol Screening Cholesterol Control Monitor diabetic Nephropathy Asthma Care Asthma Medication Management (Age 5-9) Asthma Medication Management (Age 10-17) Asthma Medication Management (Age 18-56) Mental Health Follow-Up After Hospitalization--7 Days 88 Follow-Up After Hospitalization--30 Days 89 Practitioner Contact--Antidep Medication 15 Antidep Med Acute Phase 94 Antidep Med Continuation Phase 72 Proportion Total Variance Explained 100 100 Alpha Statistic (Summated Scales) 0.88 0.80 (1) (1) Summated scale excluded the variable with the smallest factor loading. SOURCE: Caldis, T., Centers for Medicare & Medicaid Services, 2007. Table 4 Factor Loading Patterns--Single Factor Solutions for Composite Models Factor Model 1 Factor Model 2 All, but Mental Care Measure All Categories Health II Percent Child Immunization 86 85 Adolescent Immunization 78 78 Women's Health 85 83 Coronary Care 80 81 Diabetes Care 82 83 Mental Health I 66 61 Mental Health II 33 -- Asthma Care 21 24 Proportion of Variance Explained 100 100 Alpha Statistic 0.84 0.86 Factor Model 4 Factor Model 3 All, but Asthma and Mental Care Measure All, but Asthma Health II Percent Child Immunization 87 86 Adolescent Immunization 78 78 Women's Health 83 83 Coronary Care 79 83 Diabetes Care 83 82 Mental Health I 64 62 Mental Health II 31 -- Asthma Care -- -- Proportion of Variance Explained 100 100 Alpha Statistic 0.87 0.9 SOURCE: Caldis, T., Centers for Medicare & Medicaid Services, 2007. Table 5 Validating Regression--HEDIS[R] Composite Regressed on CAHPS[R] Composites Coefficient Standard Variable Estimate Error t Value Significance Getting Needed Care 0.14 0.06 2.33 0.02 Getting Care Quickly 0.53 0.07 7.27 <.01 Physician Communication -0.47 0.12 -3.79 <.01 Staff Courtesy -0.24 0.13 -1.9 0.06 Customer Service 0.09 0.05 1.84 0.07 Claims Processing 0.09 0.03 2.62 <.01 NOTES: R-squared statistic is 0.36. Adjusted R-squared statistic is 0.35. SOURCE: Caldis, T., Centers for Medicare & Medicaid Services, 2007.
|Printer friendly Cite/link Email Feedback|
|Publication:||Health Care Financing Review|
|Date:||Mar 22, 2007|
|Previous Article:||Risk adjustment and public reporting on home health care.|
|Next Article:||Medicare's quality improvement organization program value in nursing homes.|