Psychometric evaluation of the Medicare Advantage and prescription drug plan disenrollment reasons survey.
Of the 59 million Medicare beneficiaries in 2017, 42 million (71.1 percent) were covered under stand-alone prescription drug plans (PDPs) or Medicare Advantage (MA) managed care plans. The Centers for Medicare & Medicaid Services (CMS) publicly reports comparative information on Medicare plan performance to help consumers make more informed plan choices and to help plans monitor and improve the quality of care they provide. (1) An important indicator of health plan quality is the rate of voluntary disenrollment, which CMS has reported since 2000. Voluntary disenrollment is strongly related to consumers' negative experiences and dissatisfaction with their plans. (2-5)
To provide insight into reasons for disenrollment and fill the information gap left by excluding disenrollees from the main Medicare CAHPS survey, (6) CMS worked with our study team in 2010 to develop the Medicare Advantage and Prescription Drug Plan Disenrollment Reasons Survey. This survey, which assesses reasons for voluntary disenrollment from MA plans and PDPs, was piloted nationally between November 2010 and July 2011 and has been fielded continuously since December 2012. In this article, we describe the development of composite measures (7) of beneficiary reasons for disenrollment and evaluate their psychometric properties. We assess interunit (plan level) and internal consistency reliability (at the beneficiary and plan levels) and the criterion validity (at the beneficiary and plan levels) of the composites used to summarize beneficiary reasons for disenrolling from MA plans and PDPs. Because this information is meant to help beneficiaries select plans and guide plans in their quality improvement efforts, it is important to demonstrate that scores on the survey measures provide a reliable and valid assessment of beneficiaries' reasons for disenrollment.
2 | METHODS
2.1 | Instrument development
Considerable formative work was conducted to develop the survey. This work included a review of the published literature and an environmental scan to compile a list of candidate items, extensive input from key informants (regional case managers in CMS's Office of the Medicare Ombudsman), and a round of one-on-one interviews with MA and PDP disenrollees to test the wording and flow of the survey. Interviewees generally understood the content of the survey, though findings from the interviews suggested the need for small wording changes, a set of items to discern nonvoluntary disenrollees (who were not the target of the survey) from voluntary disenrollees, and the need to make multiple references to the name of the plan from which the beneficiary disenrolled so that respondents were not confused about the referent plan.
2.2 | Data source
This study used data from the 2015 and 2016 Medicare Advantage and Prescription Drug Plan Disenrollment Reasons Survey. The unit of assessment and reporting of results of this survey is a health or drug plan contract, which we refer to as "plans" in this paper. There are two MA versions of the survey, one for plans that include Part D prescription drug coverage (MA-PD plans) and one for plans that do not (MA-Only). The MA-PD survey contained 69 items, 21 concerning reasons for voluntarily disenrolling. The MA-Only survey included 51 questions, 13 concerning reasons for voluntarily disenrolling. The PDP survey contained 54 items, 14 concerning reasons for voluntarily disenrolling. For our analyses, we grouped the MA-PD and MA-Only responses into a single category which we refer to collectively as MA. Each version of the survey contains an item measuring beneficiaries' overall ratings of their plans on a 0-10 scale, where 0 is the "worst possible plan" and 10 is the "best possible plan." The survey instruments are available on CMS's website. (8)
The survey is designed to represent the population of voluntary disenrollees from MA plans and PDPs during a calendar year. For the 2015 and 2016 surveys, monthly random samples were drawn to obtain approximately 233 cases from each MA plan's population of voluntary disenrollees and 465 from each PDP per year. The plan-level sampling design was augmented with a supplementary sample to maintain a minimum sampling rate across each of the entire populations of disenrollees sufficient to maintain accuracy of national overall and subgroup disenrollment estimates. Target sample sizes were based on analyses to determine the number of cases required to achieve reliable plan-level estimates (i.e., interunit reliability > 0.70) of reasons for disenrollment.
2.3 | Survey procedures
Sampled beneficiaries who voluntarily disenrolled from their plans were mailed a prenotification card and then a survey approximately within two months of their disenrollment. Nonrespondents to the first mailing were sent a second survey approximately 4 weeks after the first. In 2015, 132 631 surveys were fielded and 51 044 responses were obtained (raw response rate: 38.5 percent). In 2016, 152 800 surveys were fielded and 60 509 responses were obtained (raw response rate: 39.6 percent). Respondents who screened out of the survey as self-reported involuntary disenrollees (mostly movers, n = 10 462 in 2015 and 12 934 in 2016) or who did not complete any core items (n = 1780 in 2015 and 1887 in 2016, approximately 3 percent of survey respondents) were removed, leaving 38 802 respondents in 2015 and 45 688 respondents in 2016. We excluded responses from plans that consolidated at the end of a survey year or were not in operation the following year, had <11 respondents, or had no substantive responses. These restrictions resulted in an analysis sample of 36 563 disenrollees in 2015 and 44 665 disenrollees in 2016. For the analyses of 2016 data requiring case mix adjustment of composite scores or individual reasons for leaving, we further restricted the sample to those plans with at least 30 respondents, yielding 43 979 disenrollees from 378 MA plans (average number of disenrollees per plan = 91; range = 30-331) and 36 PDPs (average number of disenrollees per plan = 265; range = 76-772).
2.4 | Data analysis
We used multilevel factor analysis, implemented in Mplus version 6.12, (9) to determine how to group reasons for disenrollment for reporting. Multilevel factor analysis made it possible to simultaneously analyze item groupings at the beneficiary and plan levels. (10) We began by using multilevel exploratory factor analysis (MEFA) to investigate the factor structure of items from the 2015 Disenrollment Reasons Survey. We then conducted multilevel confirmatory factor analysis (MCFA) on the 2016 data to provide evidence of the generalizability of the best-fitting factor structure.
For the MEFA, a succession of one to five factors was extracted for both the beneficiary and the plan levels (models with greater than five factors did not converge likely because of overfitting). To facilitate consistent description of composite measures at the beneficiary and plan levels, we only considered models that had the same number of factors at the beneficiary and plan levels. Geomin (oblique) factor rotation was used to identify simple structure and appreciable factor loadings ([greater than or equal to]0.30), and fit indices were criteria for the number of factors to retain and the grouping of items. Although the MEFA model results were our primary guide for determining the specification of the MCFA models, we also sought consistent definitions of composite measures of the same construct in the MA and PDP survey contexts. For that reason, we omitted from the factor models the one item about cost (co-payment for doctor visits went up) that was applicable only to MA disenrollees. We conducted our MEFA analysis on the 2015 MA data, which included responses from both MA-PD and MA-Only disenrollees. Because all PDP items were included in the MA-PD survey, we did not think it was necessary to repeat the MEFA analysis using PDP data. Separate MCFA models were then tested on 2016 data from MA and PDP disenrollees.
In the MCFA context, we examined factor loadings with the criterion that they should be [greater than or equal to]0.40. (11) In both the MEFA and the MCFA context, we evaluated overall model fit using Bentler's comparative fit index (CFI) (12) and the root mean square error of approximation (RMSEA). (13) A CFI > 0.95 and an RMSEA < 0.06 are considered to indicate excellent fit. (14)
Based on the results of the MCFA models, we computed composite measures (scales) of reasons for disenrollment from MA plans and PDPs. We estimated the internal consistency reliability (a) of the scales at the beneficiary and plan levels using multilevel factor analytic methods described by Geldhof et al. (15) Alpha coefficients [greater than or equal to]0.70 are typically considered to indicate acceptable internal consistency reliability. (16) Internal consistency of a composite at the beneficiary level implies that a beneficiary who indicates a problem with one item in a composite is likely to indicate problems with other items in the composite. Internal consistency of a composite at the plan level indicates that a plan that has a lot of disenrollees endorsing one item in a composite is likely to have a lot of disenrollees endorsing other items in the composite.
To estimate the interunit (plan-level) reliability of each composite measure, we first estimated the plan-specific means and sampling variances of each composite measure for plans with at least 30 respondents, using the current version of the CAHPS Analysis Program as described in detail elsewhere. (17) Briefly, the program combines the individual survey items that comprise each composite, accounting for variation in individual item response and adjusting for case mix, producing adjusted estimates of the means and sampling variances of the composites for each plan. Case mix adjustment variables included age, education, self-rated general health status (poor, fair, good, very good, and excellent), self-rated mental health status (poor, fair, good, very good, and excellent), dual eligibility for Medicaid, an indicator of disenrollment during the annual Medicare Open Enrollment Period (vs at other times of the year), (18) eligibility for a low-income subsidy (an indicator of income below 150 percent of the federal poverty level), and whether the beneficiary received assistance in completing the survey or had a proxy respondent. Missing values for the case mix adjusters were imputed with the plan mean when there were at least 100 nonmissing responses from the plan, and with the MA or PDP mean otherwise. No more than 3.5 percent of values for any case mix adjustment variable required imputation. We then estimated Fay-Herriot (19) mixed-effects models to these plan-level adjusted means and sampling variances to estimate plan-level model variances and hence estimate interunit reliability [[tau].sup.2]/[[tau].sup.2] + [V.sub.p] for each plan and composite, where [[tau].sup.2] is plan-level model variance and [V.sub.p] is the sampling variance of the mean score at plan p. Those plan-level reliabilities were averaged to summarize composite interunit reliability. We also used these models to estimate the number of completed surveys per plan necessary to achieve a reliability of 0.70.
We used mixed-effects linear regression to assess the beneficiary-level and plan-level association of each composite measure, separately, with the overall plan rating. We ran these regression analyses separately for MA plans and PDPs with at least 30 respondents. For these analyses, the 0-10 overall plan rating measure was linearly transformed to a 0-100 scale, and composite measures were scored as the percentage of items endorsed multiplied by 100. Each model regressed the plan rating variable on the plan-specific mean value of the composite and the beneficiary-specific differences from each beneficiary's plan's mean value to capture plan-level and beneficiary-level effects, respectively. Additionally, all models included the case mix adjustors mentioned previously and random intercepts by plan to accommodate within-plan correlation.
3 | RESULTS
3.1 | Rates of endorsement of individual reasons for disenrollment
Table 1 shows the percent of MA and PDP respondents who endorsed each of 21 (MA) or 14 (PDP) reasons for disenrollment (seven items included in the MA survey did not apply to PDP disenrollees). Two items were endorsed by more than 30 percent of MA disenrollees: "found plan that costs less" (46 percent in 2015, 39 percent in 2016) and "preferred provider not in plan" (32 percent in 2015, 31 percent in 2016). Two items were endorsed by fewer than 10 percent of MA disenrollees: "did not know whom to contact about filling a prescription" (6 percent in 2015, 7 percent in 2016) and "customer service not courteous or respectful" (8 percent in 2015, 8 percent in 2016). The median rate of item endorsement by MA disenrollees was 18-19 percent. Three items were endorsed by more than 30 percent of PDP disenrollees: "found plan that costs less" (71 percent in 2015, 66 percent in 2016), "monthly premium went up" (44 percent in 2015, 41 percent in 2016), and "prescription co-payment went up" (31 percent in 2015, 33 percent in 2016). Three items were endorsed by fewer than 10 percent of PDP disenrollees, with the lowest endorsement rate for the item, "customer service not courteous or respectful" (4 percent in 2015 and 2016). The median rate of endorsement among PDP disenrollees was 13-14 percent. Patterns of endorsement were roughly similar between the MA and PDP samples, though PDP disenrollees were more likely to endorse financial reasons whereas MA disenrollees were more likely to endorse reasons related to patient experience.
3.2 | Factor analyses
Preliminary exploratory modeling revealed that one item, "prescription co-payment went up," was problematic, in that its loadings did not support simple factor structure. This item loaded moderately on multiple factors, but did not load strongly on any factor. Because this prevented the convergence of a 5-factor model, we removed the item from the factor analysis. With that item omitted, all five MEFA models converged. Fit indices for these models are shown in Table S1. Based on these fit indices, a model with five within (beneficiary) and five between (plan) factors was the preferred solution (RMSEA = 0.02; CFI = 0.99). Standardized factor loadings and correlations from this model are presented in Table 2.
At the beneficiary level, three items unambiguously loaded strongly (standardized loading [greater than or equal to] 0.72) onto a factor (F1) labeled "Financial Reasons for Disenrollment." Five items unambiguously loaded strongly ([greater than or equal to]0.76) onto a second factor (F2) labeled "Problems with Prescription Drug Benefits and Coverage." Three items unambiguously loaded strongly ([greater than or equal to]0.82) onto a third factor (F3) labeled "Problems Getting Information and Help from the Plan." A fourth item loaded moderately (0.57) on F3, but we chose to group this item with a different factor (F4; labeled "Problems Getting Needed Care, Coverage, and Cost Information") for the practical reason that it and the other three items that loaded on that factor (loadings for these three items ranged from 0.53 to 0.92) appeared only in the MA survey. Two items loaded unambiguously strongly ([greater than or equal to]0.80) onto a fifth factor (F5) labeled "Problems with Coverage of Doctors and Hospitals." Finally, two items, "Did not know whom to contact about filling a prescription" and "Hard to get information about cost and coverage of prescription drugs," cross-loaded about evenly onto F2 and F3. For the CFA model, we grouped these two items with F3 (Problems Getting Information and Help from the Plan) because their content fits better with that factor.
The factor structure of the items was similar at the plan and beneficiary levels with two important exceptions. First, a set of nine items that split into two factors--Problems Getting Information and Help from the Plan and Problems Getting Needed Care, Coverage, and Cost Information--at the beneficiary level all loaded onto a single factor at the plan level. However, because four of these items do not have counterparts in the PDP survey, we divided these items into two groups (consistent with the factor structure at the beneficiary level) for the CFA modeling. Second, we ignored the fifth plan-level factor because no items loaded strongly on it.
Based on this exploratory factor modeling, we specified a 5-factor model of reasons for disenrollment from MA plans and a 3-factor model of reasons for disenrollment from PDPs. Standardized factor loadings from these MCFA models are shown in Table 3. In each model, 3 or 4 standardized factor loadings at the plan level were >1. This sometimes happens when factors are strongly correlated and is not necessarily indicative of a problem with the model. (20) In the final model, we fixed those values to 1.
The 5-factor model of reasons for disenrollment from MA plans fit the data acceptably (CFI = 0.91; RMSEA = 0.04), and factor loadings were uniformly high at both the beneficiary (average standardized loading, excluding fixed loadings, is 0.87) and the plan (average standardized loading, excluding fixed loadings, is 0.91) levels. Table S2 shows the factor correlations at the beneficiary and plan levels. At the beneficiary level, the "financial reasons" factor was positively correlated with all but one of the other factors (average correlation = 0.25; the exception was "problems with coverage of doctors and hospitals"). At the plan level, however, the "financial reasons" factor was negatively correlated with all other factors (average correlation = -0.56). Such negative correlations might indicate that the more expensive plans tend to have better service, broader networks, and the like. The average correlation among the four factors not pertaining to financial reasons for disenrollment was 0.55 at the beneficiary level and 0.72 at the plan level, again suggesting that nonfinancial quality tended to be consistently high or low. The highest correlations were between "problems getting information and help from the plan" and "problems getting needed care, coverage, and cost information," at both the beneficiary (r = 0.85) and the plan (r = 0.97) levels.
The 3-factor model of reasons for disenrollment from PDPs fit the data well (CFI = 0.98, RMSEA = 0.02). Standardized factor loadings, shown in Table 3, were high at the beneficiary level (average standardized loading, excluding fixed loadings, is 0.83) and at the plan level (average standardized loading, excluding fixed loadings, is 0.82). Table S3 shows the factor correlations at the beneficiary and plan levels. As in the MA model, the "financial reasons" factor was positively correlated with the "prescription drug benefits and coverage" and "getting information about prescription drugs" factors at the beneficiary level (correlations of 0.10 and 0.16, respectively) but negatively correlated with those factors at the plan level (correlations of -0.23 and -0.42, respectively). The "prescription drug benefits and coverage" factor and the "getting information and help from the plan" factors were strongly positively correlated at both the beneficiary level and the plan level (correlations of 0.76 and 0.68, respectively).
3.3 | Internal consistency reliability of composite measures
With one exception, we computed composite measures of reasons for disenrollment from MA-PD plans and from PDPs exactly as suggested by the factor models. Although the item, "prescription co-payment went up," was problematic in the MEFA models because it had a moderate correlation with multiple factors (financial reasons, problems with prescription drug benefits and coverage, and problems getting information about prescription drugs) in the MA model, it has strong face validity as a financial reason for disenrollment, and thus, we grouped it there for the remainder of our analyses. We comment on this decision in the Discussion.
Internal consistency reliability of these composite measures at the beneficiary level was 0.73 or higher for all measures except the financial reasons measure ([alpha] = 0.64 in the MA sample; [alpha] = 0.55 in the PDP sample). Corrected item-total correlations exceeded 0.30 for all items except one, "could no longer afford plan" (r = 0.29 among PDP disenrollees). The weaker correlation for the latter was not surprising since unlike the other items it suggests a change in personal circumstances rather than plan policies. Omitting this item from the financial reasons composite did not substantially improve internal reliability among the PDP sample; therefore, we retained the 4-item composite. Internal consistency reliability of the composites at the plan level was 0.82 or higher for all measures. Table S4 provides more detail on the internal consistency of the composites.
3.4 | Interunit reliability of composite measures
The observed interunit (plan-level) reliability for the MA composite measures of reasons for disenrollment (Table 4) was above 0.75 for four of the five measures: financial reasons; getting information about prescription drugs; getting needed care, coverage, or cost information; and coverage of doctors or hospitals. For these four measures, the number of completed surveys needed to achieve a reliability of 0.70 ranged from 18 (financial reasons) to 69 (problems getting information and help from the plan). The observed interunit reliability for the "prescription drug benefits and coverage" composite was low (0.59), and the number of completed surveys needed to achieve a reliability of 0.70 was therefore high (161). Table 3 also shows interunit reliability estimates for the PDP measures. Whereas interunit reliability for the "financial problems" composite was very high (0.94), it was low (0.63 and 0.64) for the other two composites. The number of completed surveys needed to achieve 0.70 reliability was 40 for the "financial problems" composite and nearly 400 for the other two composites.
3.5 | Criterion validity
At the beneficiary level, there was a strong negative relationship between each of the five composite measures of reasons for disenrollment and beneficiaries' overall rating of the MA plan from which they disenrolled (all P's < 0.001; see the left half of Table 5). The strongest of these relationships involve "getting information and help from the plan" (beneficiary level: B = -0.605, SE = 0.006, P< 0.001; plan level: B = -1.278, SE = 0.048, P < 0.001) and "getting needed care, coverage, or cost information" (beneficiary level: B = -0.431, SE = 0.004, P < 0.001; plan level: B = -0.788, SE = 0.022, P < 0.001). These regression coefficients can be interpreted as follows (using the first as an example): A 10 percent increase in the proportion of items endorsed from the "getting information and help with the plan" composite was associated, at the beneficiary level, with a 10*0.61 = 6.1-point reduction in the overall rating on a 0-100 scale. At the plan level, there was a strong negative relationship between scores on 4 of the 5 composites and beneficiaries' overall rating of the MA plan from which they disenrolled (all P's < 0.001). The one exception was the financial reasons composite, scores on which were positively related to the overall MA plan rating at the pian level (P < 0.001). To assess the predictive utility of the one disenrollment-reason item from the MA survey that was not included in a composite (i.e., co-payment for doctor visits went up), we estimated the case mix-adjusted association between that item and the overall MA plan rating and found a statistically significant negative relationship at the beneficiary (B = -0.115, SE = 0.004, P < 0.001) level, but not at the plan level (B = -0.072, SE = 0.045, P = 0.107) between this stand-alone item and the criterion variable.
Among PDP disenrollees, there was a strong negative association at the beneficiary level between each of the three composite measures of reasons for disenrollment and disenrollees' overall rating of the PDP (all P's < 0.001; see the right half of Table 5). The strongest of these associations at the beneficiary level involves "getting information and help from the plan" (b = -0.597, SE - 0.014, P < 0.001). Associations at the plan level were statistically significant for all but the financial composite (b = -0.087, SE = 0.061, P = 0.15).
4 | DISCUSSION
As of 2016, Medicare beneficiaries, on average, could choose from 31 PDP and 20 MA-PD plan benefit packages (PBPs) offered by plans within their service area. (21) The number of plans, variety of cost-sharing structures and levels, and formulary designs make choosing a plan challenging for many beneficiaries. (22) For example, although beneficiaries can usually identify the plan features that are most important to them, many have difficulty comparing plans and consistently fail to identify plans with the lowest cost. (23,24) Moreover, beneficiaries are reluctant to switch plans once they have enrolled. (25) Currently, the Medicare Plan Finder is the only source of comprehensive plan information. CMS's aim in developing the Medicare Advantage and Prescription Drug Plan Disenrollment Reasons Survey was to help plans identify reasons why their members disenroll and provide consumers with easy-to-understand, valid, and reliable information as a useful supplement to existing measures of beneficiaries' experiences with their MA plans and PDPs. (26) Users of the Medicare Plan Finder are able to drill down from information about a plan's disenrollment rate to view data on reasons for disenrollment.
The factor analyses and assessment of internal reliability support grouping the survey items into five composite measures of reasons for disenrolling from MA-PD plans and three composite measures of reasons for disenrolling from PDPs. One item (copayment for prescription drugs went up) loaded on the financial reasons factor but was additionally associated with other factors in the model., perhaps because the financial items on the survey are tied, necessarily, to the services that people receive from their plans. We ultimately incorporated this item into the financial composite because of its face validity as an indicator of financial strain and because the related nonfinancial problems are well covered by other survey items.
Because composites are used primarily to compare plans rather than beneficiaries, their interunit reliability is of particular interest. We found that the average interunit reliability for plans with 30 or more survey completes exceeded 0.75 for five of eight composites. The three exceptions were the MA and PDP versions of the "problems with benefits and coverage" composite and the PDP version of the "getting information and help from the plan" composite. For the "problems with benefits and coverage composite," 25 percent of MA plans and 24 percent of PDPs achieved plan-level reliability of 0.70 or higher. For the "getting information and help from the plan" composite, 29 percent of PDPs achieved reliability of 0.70 or higher. At current average sample sizes, differences among plans for these composites are not big enough for reliable discrimination among most plans but would still be useful for discriminating the best from the worst performers. Future research might investigate whether combining data from two consecutive survey years improves the reliability of comparisons on these composites.
The internal consistency reliability of most composites was high at both the beneficiary and the plan levels, supporting the grouping of items into five composites at both individual and plan levels. An exception was the financial reasons composite, whose reliability at the beneficiary level was somewhat below the 0.70 standard in the MA sample and well below that standard in the PDP sample. The items that make up this composite are linked not so much by their consistency as by their common effect. Thus, for example, three different plans might be perceived by beneficiaries as costly, one because of its high premiums, another because of its large deductibles, and a third because of its of high copays. Such reports on financial burden would manifest low internal reliability, but the high interunit reliability of the financial composite suggests that consumers tend to agree on the greater financial burden at some plans.
There was substantial evidence for the criterion validity of the measures. At the beneficiary level, higher scores on each of the composites were associated with lower overall ratings of the plan from which beneficiaries disenrolled; that is, the more reasons a beneficiary endorsed, the lower they rated the plan from which they disenrolled. This was also the case at the plan level with one exception: the financial reasons composite. The positive plan-level association between scores on the financial reasons composite and overall MA plan ratings and the lack of a plan-level association between scores on the financial reasons composite and overall PDP ratings suggests that high scores on this composite--unlike the other four composites--are not necessarily indicative of a problem with plan quality. In other words, plans that some beneficiaries found to be too expensive were not considered to be poor-quality plans (in fact, the opposite was true in the MA plan context).
A couple of limitations of this study should be considered when interpreting its findings, including the relatively low response rate and that there can be up to a 3-month lag between when beneficiaries disenroll and when they complete the survey. The survey response rate is, however, typical for surveys of the Medicare population, (27) and evidence suggests that response rate is typically not a good proxy for nonresponse bias. (28) These limitations notwithstanding, our results indicate that the Medicare Advantage and Prescription Drug Plan Disenrollment Reasons Survey provides a reliable and valid assessment of beneficiaries' reasons for disenrolling from MA-PD plans and PDPs, information that, alongside information on plan disenrollment rates, can inform consumer choice and guide quality improvement.
Joint Acknowledgment/Disclosure Statement: This study was funded by Centers for Medicare & Medicaid Services (CMS) contract HHSM-500-2017-00052G.
Although prior approval and notification by CMS is not required, CMS was provided with an advanced copy of the manuscript as a courtesy.
Steven C. Martino https://orcid.org/0000-0002-1514-4133
Marc N. Elliott https://orcid.org/0000-0002-7147-5535 Alan M. Zaslavsky https://orcid.org/0000-0003-1072-6043
(1.) Centers for Medicare & Medicaid Services. Medicare plan finder. http://www.medicare.gov/find-a-plan/questions/home.aspx. Accessed September 21, 2018.
(2.) Lied TR, Sheingold SH, Landon BE, Shaul JA, Cleary PD. Beneficiary reported experience and voluntary disenrollment in Medicare managed care. Health Care Financ Rev. 2003;25(1):55-66.
(3.) Nelson L, Brown R, Gold M, Ciemnecki A, Docteur E. Access to care in Medicare HMOs, 1996. Health Aff. 1997;16(2):148-156.
(4.) Rector TS. Exhaustion of drug benefits and disenrollment of Medicare beneficiaries from managed care organizations. JAMA. 2000;283(16):2163-2167.
(5.) Sainfort F, Booske BC. Role of information in consumer selection of health plans. Health Care Financ Rev. 1996;18(1):31-54.
(6.) Goldstein E, Cleary PD, Langwell KM, Zaslavsky AM, Heller A. Medicare Managed Care CAHPS: a tool for performance improvement. Health Care Financ Rev. 2001;22(3):101-107.
(7.) Shwartz M, Restuccia JD, Rosen AK. Composite measures of health care provider performance: a description of approaches. Milbank Q. 2015;93(4):788-825.
(8.) Centers for Medicare & Medicaid Services. The Medicare Advantage and Prescription Drug Plan Disenrollment Reasons Survey. http://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/mapdp_disenrollmentsurvey.html. Accessed September 21, 2018.
(9.) Muthen LK, Muthen BO. Mplus: Statistical Analysis with Latent Variables: User's Guide. Los Angeles, CA: Muthen & Muthen; 2010.
(10.) Byrne B. Structural Equation Modeling with Mplus: Basic Concepts, Applications, and Programming. New York, NY: Routledge; 2012.
(11.) Peterson RA. A meta-analysis of variance accounted for and factor loadings in exploratory factor analysis. Mark Lett. 2000;11(3):261-275.
(12.) Bentier PM. On the fit of models to covariances and methodology to the bulletin. Psychol Bull. 1992;112(3):400-404.
(13.) Steiger JH, Lind JC. Statistically based tests for the number of common factors. Paper presented at the annual Spring meeting of the Psychometric Society. Iowa City, IA; 1980.
(14.) Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):l-55.
(15.) Geldhof GJ, Preacher KJ, Zyphur MJ. Reliability estimation in a multilevel confirmatory factor analysis framework. Psychol Methods. 2014;19(1):72-91.
(16.) NunnallyJC, Bernstein IH. Psychological Theory, 3rd edn. New York, NY: McGraw-Hill: 1994.
(17.) Agency for Healthcare Research and Quality. Analyzing CAHPS survey data. https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/analysis/index.html. Accessed September 21, 2018.
(18.) There are restrictions on who can disenroll outside of the Open Enrollment Period. We control for this factor because we are interested in capturing effects that do not reflect these differences in regulations.
(19.) Fay RE, Herriot RA. Estimates of income for small places: an application of James-Stein procedures to census data. J Am Stat Assoc. 1979;74(366):269-277.
(20.) Joreskog KG. How large can a standardized coefficient be? http://www.ssicentral.com/lisrel/techdocs/HowLargeCanaStandardizedCoefficientbe.pdf. Accessed September 21, 2018.
(21.) Cubanski J, Damico A, Hoadley J, Orgera K, Neuman T. Medicare part D: a first look at prescription drug plans in 2018. http://files.kff.org/attachment/Issue-Brief-Medicare-Part-D-A-First-Look-at-Prescription-Drug-Plans-in-2018. Accessed September 21, 2018.
(22.) Hsu J, Fung V, Price M, et al. Medicare beneficiaries' knowledge of Part D prescription drug program benefits and responses to drug costs. JAMA. 2008;299(16):1929-1936.
(23.) Abaluck J, Gruber J. Choice inconsistencies among the elderly: evidence from plan choice in the Medicare Part D program. Am Econ Rev. 2011;101(4):1180-1210.
(24.) Zhou C, Zhang Y. The vast majority of Medicare Part D beneficiaries still don't choose the cheapest plans that meet their medication needs. Health Aff. 2012;31(10):2259-2265.
(25.) Polinski JM, Bhandari A, Saya UY, Schneeweiss S, Shrank WH. Medicare beneficiaries' knowledge of and choices regarding Part D, 2005 to the present. J Am Geriatr Soc. 2010;58(5):950-966.
(26.) Martino SC, Elliott MN, Cleary PD, et al. Psychometric properties of an instrument to assess Medicare beneficiaries' prescription drug plan experiences. Health Care Financ Rev. 2009;30(3):41-53.
(27.) Elliott MN, Landon B, Zaslavsky AM, et al. Medicare Prescription Drug Plan enrollees report less positive experiences than their Medicare Advantage counterparts. Health Aff. 2016;35(3):456-463.
(28.) Groves RM, Peytcheva E. The impact of nonresponse rates on non-response bias: a meta-analysis. Public Opin Q. 2008;72:167-189.
Additional supporting information may be found online in the Supporting Information section at the end of the article.
Steven C. Martino PhD (1) | Marc N. Elliott PhD (2) | Alan M. Zaslavsky PhD (3) | Nate Orr MA (2) | Andy Bogart MS (2) | Feifei Ye PhD (1) | Cheryl L Damberg PhD (2)
(1) RAND Corporation, Pittsburgh, Pennsylvania
(2) RAND Corporation, Santa Monica, California
(3) Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
Steven C. Martino, PhD, RAND Corporation,
4570 Fifth Avenue, Pittsburgh, PA
TABLE 1 Percentage of Medicare Advantage (MA) Plan and Prescription Drug Plan (PDP) disenrollees endorsing each reason for disenrollment, 2015 and 2016 Percent endorsing Percent endorsing 2015 2016 MA PDP MA PDP Financial reasons Monthly premium went up 26.6 43.9 21.5 40.5 Prescription co-payment went up 19.8 31.2 21.9 32.5 Co-payment for doctor 18.5 (a) 22.8 (a) visit went up Found plan that costs less 45.9 71.4 39.0 65.9 Could no longer afford plan 19.8 27.0 18.4 23.7 Problems with prescription drug benefits and coverage Change in drug formulary 9.7 16.0 11.3 16.9 Plan refused to pay for a prescribed medication 11.7 15.5 12.7 17.1 Problems getting a 11.3 13.5 12.5 14.2 prescribed medication Difficult to get brand 8.9 10.4 11.0 11.6 name medications Frustration with approval process for off-formulary medications 13.5 14.5 13.7 12.6 Problems getting information and help from the plan Did not know whom to contact about filling a prescription 6.2 6.0 6.7 5.9 Hard to get info. about cost, coverage of prescription drugs 9.9 9.8 10.3 9.3 Unhappy with how plan handled 19.4 10.8 18.8 9.0 question or complaint Could not get 20.6 11.4 21.0 9.8 information or help needed from plan Customer service not 7.7 4.3 7.6 3.5 courteous or respectful Problems getting needed care, coverage, or cost info. Frustration with approval process for care, tests, treatment 24.4 (a) 23.4 (a) Problems getting needed care, tests, or treatment 23.0 (a) 23.2 (a) Problems getting plan 14.3 (a) 12.6 (a) to pay claim Hard to get info. about cost, 14.4 (a) 14.8 (a) coverage of health services Problems with coverage of doctors and hospitals Preferred provider not in plan 31.7 (a) 30.7 (a) Clinic or hospital wanted to go to not covered by plan 22.9 (a) 23.1 (a) (a) Item not included in PDP version of survey. TABLE 2 Standardized factor loadings from a 5-factor multilevel exploratory factor analysis of reasons for disenrollment from Medicare advantage plans, 2016 data F1 F2 F3 F4 F5 Items Within-plan factor loadings Monthly premium went up 0.76 0.07 0.01 -0.01 -0.01 Found plan that costs less 0.91 -0.04 -0.01 0.01 -0.05 Could no longer afford plan 0.72 0.04 0.01 0.10 0.00 Change in drug formulary 0.16 0.76 0.00 -0.04 0.02 Plan refused to pay for a -0.06 0.94 -0.05 0.11 -0.07 prescribed medication Problems getting a -0.10 0.97 -0.01 0.08 -0.07 prescribed medication Difficult to get brand 0.05 0.78 0.04 0.01 0.07 name medications Frustration with approval -0.02 0.86 0.05 0.09 0.02 process for off-formulary meds. Did not know whom to contact 0.06 0.58 0.42 -0.09 0.03 about filling a prescription Hard to get information about 0.08 0.55 0.49 -0.10 0.04 cost, coverage of Rx drugs Unhappy with how plan -0.05 0.01 0.86 0.17 -0.06 handled question or complaint Could not get information -0.07 -0.01 0.83 0.19 0.00 or help needed from plan Customer service not -0.05 0.01 0.82 0.03 -0.04 courteous or respectful Frustration with approval 0.04 0.06 0.02 0.92 0.00 process for care, tests, treatment Problems getting needed -0.04 -0.01 0.04 0.84 0.10 care, tests, or treatment Problems getting plan 0.09 0.01 0.27 0.53 0.02 to pay claim Hard to get information 0.11 0.02 0.57 0.23 0.19 about cost, coverage of health services Preferred provider -0.07 0.03 -0.07 0.09 0.80 not in plan Clinic or hospital wanted 0.00 -0.02 0.08 -0.02 0.92 to go to not covered by plan Between-plan factor loadings Monthly premium went up 0.95 -0.04 -0.02 0.00 0.00 Found plan that costs less 0.91 -0.05 0.02 0.00 -0.09 Could no longer afford plan 0.92 0.13 0.04 -0.04 -0.02 Change in drug formulary 0.03 1.01 -0.04 -0.12 -0.13 Plan refused to pay for -0.13 0.97 0.13 0.00 -0.06 a prescribed medication Problems getting a -0.10 0.82 0.09 0.15 0.02 prescribed medication Difficult to get brand 0.16 1.02 -0.17 0.02 0.09 name medications Frustration with approval -0.05 0.90 0.07 0.09 0.04 process for off-formulary meds. Did not know whom to -0.03 0.64 -0.10 0.38 0.14 contact about filling a prescription Hard to get information 0.04 0.48 0.01 0.74 -0.18 about cost, coverage of Rx drugs Unhappy with how plan handled -0.02 0.04 0.04 0.94 0.01 question or complaint Could not get information -0.09 0.07 0.02 0.85 0.08 or help needed from plan Customer service not -0.06 -0.07 -0.12 1.01 -0.05 courteous or respectful Frustration with approval 0.02 0.06 0.08 0.71 0.29 process for care, tests, treatment Problems getting needed -0.01 0.02 -0.10 0.70 0.39 care, tests, or treatment Problems getting plan 0.03 0.00 0.57 0.67 0.04 to pay claim Hard to get information -0.01 -0.01 0.33 0.86 0.01 about cost, coverage of health services Preferred provider -0.10 0.11 0.00 -0.01 0.87 not in plan Clinic or hospital -0.01 -0.09 0.07 0.06 0.92 wanted to go to not covered by plan Factor (F) correlations F1: financial reasons - -0.28 0.05 -0.55 -0.69 F2: prescription drug 0.36 - 0.04 0.63 0.28 benefits and coverage F3: getting information 0.18 0.50 - 0.22 0.17 and help from the plan F4: getting needed care, 0.02 0.46 0.64 - 0.59 coverage, or cost info. F5: coverage of -0.11 0.21 0.30 0.43 - doctors and hospitals Note. Comparative fit index = 0.99; root mean square error of approximation = 0.02. Loadings in bold are above 0.30. Correlation coefficients below the diagonal are at the beneficiary (within) level; coefficients above the diagonal are at the plan (between) level. TABLE 3 Standardized factor loadings from multilevel confirmatory factor analysis models of 5-factor (Medicare Advantage) and 3-factor (Prescription Drug Plan) models of reasons for disenrollment, 2016 data Standardized factor loadings Factor/items Level 1 Level 2 (Beneficiaries) (Plans) Financial reasons Monthly premium went up 0.75/0.76 0.97/0.56 Found plan that costs less 0.82/0.58 1.00/1.00 Could no longer afford plan 0.87/0.64 0.82/0.45 Problems with prescription drug benefits and coverage Change in drug formulary 0.77/0.71 0.63/0.71 Plan refused to pay for 0.93/0.90 0.98/0.83 a prescribed medication Problems getting a 0.95/0.96 1.00/1.00 prescribed medication Difficult to get brand 0.88/0.84 0.84/0.90 name medications Frustration with 0.95/0.96 1.00/1.00 approval process for off-formulary medications Problems getting information and help from the plan Did not know whom to 0.85/0.88 0.94/0.99 contact about filling a prescription Hard to get information 0.88/0.91 0.91/1.00 about cost and coverage of prescription drugs Unhappy with how plan 0.92/0.95 0.98/0.99 handled question or complaint Could not get information 0.92/0.95 0.99/0.99 or help needed from plan Customer service not 0.75/0.80 0.92/0.97 courteous or respectful Problems getting needed care, coverage, or cost information (a) Frustration with approval 0.93 0.97 process for care, tests, or treatment Problems getting needed 0.88 0.98 care, tests, or treatment Problems getting 0.74 0.80 plan to pay claim Hard to get information 0.88 0.94 about cost and coverage of health services Problems with coverage of doctors and hospitals (a) Preferred provider not in plan 0.96 0.95 Clinic or hospital wanted to 0.85 0.99 go to not covered by plan Note. Loadings for the Medicare Advantage (MA) model are shown on the left of the slash; loadings for the prescription drug plan (PDP) model are shown on the right of the slash. For the MA model, N = 35 072 disenrollees and 412 plans; Comparative fit index (CFI) =0.91; root mean square error of approximation (RMSEA) = 0.04; For the PDP model, N = 9593 disenrollees and 38 plans; CFI = 0.98; RMSEA = 0.02. (a) MA model only. TABLE 4 Interunit reliability of reason composites, 2016 data Mean observed Average N N per plan interunit per plan for reliability reliability (a) = 0.70 Medicare advantage Financial reasons 0.92 91 18 Problems with prescription drug benefits and coverage 0.59 89 161 Problems getting information and help from the plan 0.76 91 69 Problems getting needed care, coverage, or cost information 0.84 91 41 Problems with coverage of doctors and hospitals 0.90 91 24 Prescription drug plan Financial reasons 0.94 265 40 Problems with prescription drug benefits and coverage 0.63 265 393 Problems getting and help from the plan 0.64 265 379 (a) Based on plans with N [greater than or equal to] 30. TABLE 5 Beneficiary- and plan-level regression models predicting overall rating of plan from composites and individual items entered individually into the model, 2016 data MA Beneficiary level Plan level B SE e SE Composite measures Financial reasons -0.095 0.005 0.236 0.033 Problems with prescription -0.318 0.006 -0.861 0.071 drug benefits and coverage Problems getting information -0.605 0.006 -1.278 0.048 and help from the plan Problems getting needed -0.431 0.004 -0.788 0.022 care, coverage, or cost info. Problems with coverage -0.138 0.004 -0.351 0.025 of doctors and hospitals Item not included in a composite Co-payment for a -0.115 0.004 -0.072 0.045 doctor visit went up PDP Beneficiary level Plan level B SE B SE Composite measures Financial reasons -0.174 0.009 -0.087 0.061 Problems with prescription -0.366 0.009 -0.266 0.132 drug benefits and coverage Problems getting information -0.597 0.014 -0.812 0.227 and help from the plan Problems getting needed NA NA NA NA care, coverage, or cost info. Problems with coverage NA NA NA NA of doctors and hospitals Item not included in a composite Co-payment for a NA NA NA NA doctor visit went up Note. Analyses include only those individuals in plans with [greater than or equal to] 30 survey completes. Ns for the beneficiary-level Medicare Advantage (MA) models ranged from 330 111 to 32 993. N for the beneficiary-level prescription drug plan (PDP) models = 9085. N for the plan-level Medicare Advantage (MA) models = 378. N for the plan-level prescription drug plan (PDP) models = 36. Each model regressed the overall plan rating on the plan-specific mean value of the composite or individual item and the beneficiary-specific differences from their plan's mean value to capture plan-level and beneficiary-level effects, respectively. Additionally, each model was adjusted for age, education, self-rated general health status, self-rated mental health status, dual eligibility for Medicaid, an indicator of December disenrollment, eligibility for low-income subsidy, and proxy assistance with completing the survey. All models also included random intercepts by plan to accommodate within-plan correlation. The dependent measure was linearly transformed to a 0-100 scale for analysis. Composite measures were scored as the percentage of items endorsed multiplied by 100. Responses to single item not included in a composite were coded as 100 if the item was endorsed, 0 otherwise. Models assessing associations of the composite measures with the dependent measure included composites individually; the model assessing the association between the individual MA items and the dependent measure did not include the composites as predictors. SE = standard error. P values for all regression coefficients < 0.001 except for the coefficient for the plan-level association of the individual MA item (co-payment for a doctor visit went up) with the overall MA plan rating (P = 0.11) and for the plan-level association of the financial composite with the overall PDP rating (P = 0.15). [Correction added on 9 May 2019, after first online publication: in Table 5, the value '0.236' under 'Plan level B' column should be positive.].
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||RESEARCH ARTICLE|
|Author:||Martino, Steven C.; Elliott, Marc N.; Zaslavsky, Alan M.; Orr, Nate; Bogart, Andy; Ye, Feifei; Dambe|
|Publication:||Health Services Research|
|Date:||Aug 1, 2019|
|Previous Article:||Effects of improvements in the CPS on the estimated prevalence of medical financial burdens.|
|Next Article:||Survey mode effects and insurance coverage estimates in the redesigned Gallup well-being index.|