Response error and the Medicaid undercount in the current population survey.
Medicaid is an important part of our nation's safety net providing health insurance coverage to low-income children and adults, people with disabilities, and the elderly. Touching the lives of many Americans, the program covered an estimated 72.5 million people in 2013. (1) Administrative and survey data provide important information about Medicaid enrollees; however, accurately measuring Medicaid participation in surveys has proven difficult for decades. Dating to the 1990s, research has shown estimates of Medicaid coverage based on the Current Population Survey Annual Social and Economic Supplement (CPS ASEC) have consistently produced an undercount of beneficiaries compared to Medicaid enrollment records. (2-8) For instance, the 2000 CPS ASEC estimated 32% fewer Medicaid participants compared with Medicaid administrative records. This discrepancy between survey data and administrative records is known as the "Medicaid undercount."
This issue has important policy implications because it affects estimates of the uninsured population. For instance, individuals undercounted in survey data for Medicaid are considered uninsured if they report no other insurance coverage. Accurate and reliable estimates of the uninsured population serve as the basis for public policy decisions and inform programs focused on expanding insurance coverage. (9) Understanding the Medicaid undercount is increasingly important to assess survey estimates of the uninsured population especially given increased Medicaid enrollment following implementation of the Patient Protection and Affordable Care Act (ACA).
We contribute to the Medicaid undercount literature in several ways. First, we extend past work by measuring the Medicaid undercount in the 2007-2011 CPS ASEC using 2006-2010 Medicaid Statistical Information System (MSIS) data. Second, with substantially more linkages than previous research, we use linked 2011 CPS ASEC and 2010 MSIS data to evaluate two components of measurement error affecting the undercount: false-negative error and false-positive error (Figure S1). Some persons have Medicaid coverage but report no coverage in the CPS ASEC (false-negative error), lowering survey estimates of Medicaid coverage. Some report Medicaid coverage in the CPS ASEC but cannot be linked to MSIS enrollment data (false-positive error), increasing survey estimates of Medicaid coverage.
We examine two hypotheses related to how respondents interpret questions about health insurance on the CPS ASEC: Whether they interpret the question as asking about their current coverage status or if they have difficulty remembering past coverage, and whether respondents confuse different programs or only report certain forms of insurance. We also examine the effect of other household members' insurance coverage and describe demographic and socioeconomic characteristics associated with response error. We then provide adjusted rates of Medicaid coverage and the uninsured correcting for survey response errors. We discuss possible reasons for differences between our estimates and previous research, and we discuss the implications of our findings for uninsured population estimates.
Following years of research, the 2014 CPS ASEC health insurance module was redesigned to address the most problematic features, some of which contributed to the persistent Medicaid undercount. (10) The 2014 CPS ASEC asked respondents which months they were uninsured rather than whether they were uninsured at any point during the prior year. This question design aligns more closely with surveys such as the American Community Survey (ACS) and the National Health Interview Survey which ask about health insurance coverage at the time of the survey. Whereas the CPS ASEC has consistently produced a Medicaid undercount, the ACS has been shown to produce an overcount of Medicaid recipients. (11)
By focusing on the years preceding the CPS ASEC health insurance module redesign, our research will serve as a baseline for understanding resulting changes in survey response error for Medicaid reporting and changes in the Medicaid undercount in subsequent years under the redesigned health insurance module.
2 | LITERATURE REVIEW
Discrepancies between Medicaid enrollment data and CPS ASEC estimates of Medicaid coverage were documented in the 1990s. (2,3) Using the 1990-2000 CPS ASEC, Klerman et al (4) evaluated matched CPS ASEC and Medi-Cal data and found that 30% of adults and 25% of children did not report their Medicaid coverage.
In 2007, multiple institutions partnered on the SNACC Medicaid Undercount Project to better understand the national Medicaid undercount using linked 2000-2005 CPS ASEC data and Medicaid enrollment data.* SNACC researchers (12) determined that the CPS ASEC Medicaid undercount ranged between 31% and 38% from 2000 to 2005. They found reporting error, not editing and imputation procedures, was the main cause of the undercount, in particular false-negative response error. SNACC researchers found that people in poverty were less likely to report false-negative errors, and individuals enrolled in Medicaid longer were less likely to report false-negative errors.
Kincheloe et al (9) reviewed research on the Medicaid undercount and false-negative response error and identified several associated factors. Respondents enrolled in Medicaid may mistakenly report Medicare or another form of public or private coverage. Some respondents with dual Medicare and Medicaid coverage may only report having Medicare and not Medicaid. The scope of benefits and beneficiaries' health status also affects the accuracy of survey reports of Medicaid coverage. Persons with only partial benefits may not report Medicaid coverage. Similarly, healthier persons not using health services often fail to report Medicaid coverage compared to less healthy persons more frequently seeking health care. Automatic enrollment in some states for those with income below poverty thresholds may also increase underreporting since persons automatically enrolled may be unaware they have Medicaid.
Klerman et al (8) explored two explanations for understanding the Medicaid undercount in the CPS ASEC, the point-in-time versus recall conjectures. The point-in-time explanation suggests that respondents either disregard or do not understand insurance coverage question instructions asking respondents to report on the previous calendar year and instead report their current coverage. Cognitive testing of the CPS ASEC health insurance questions confirmed that some respondents focused on current coverage rather than the previous year. (13) SNACC (12) researchers also found that those enrolled in Medicaid when surveyed are less likely to report false negatives.
The recall conjecture argues that some respondents have difficulty keeping track of insurance coverage dates. Respondents cannot always recall whether they had Medicaid the previous year, particularly coverage early in the year. To explain the undercount, Klerman et al (8) found more support for the recall than the point-in-time conjecture. Too few people have different Medicaid coverage status when interviewed than during the prior year to explain the undercount. Instead, the undercount is related to recency of Medicaid coverage with recent coverage yielding more accurate reports.
Respondents reporting for other household members may also contribute to reporting error. Cognitive testing revealed some respondents did not have accurate knowledge of other household members' coverage, while others failed to include certain household members in their reports. (13) Pascale, Roemer, and Resnick (14) found support for a shared coverage hypothesis to explain response error in the 2001 CPS ASEC. In their analysis of matched CPS ASEC and MSIS data, respondents were most accurate reporting Medicaid coverage for other household members when the respondent and other household members shared coverage.
Most Medicaid undercount research has focused on false-negative error rather than false-positive error since it is a larger component of the undercount. SNACC (12) researchers determined false-positive reports result when persons incorrectly report having Medicaid, or when no linked MSIS record can be identified to verify the CPS ASEC report. The false-positive error rate in the CPS ASEC for 2000-2005 ranged between about 20% and 25% of all reports of Medicaid enrollment. Whereas false-negative error resulted mostly from respondent reporting, a majority of false-positive error resulted from CPS ASEC editing and imputation procedures, and about 40% resulted from respondent reporting. SNACC researchers found the rate of false positives higher among those in poverty compared to those not in poverty.
We expand on this rich literature and contribute to understanding the Medicaid undercount by updating Medicaid undercount numbers using 2007-2011 CPS ASEC data. We link more MSIS cases to the CPS ASEC compared to previous research allowing us to more accurately measure and analyze false-negative and false-positive errors. Based on prior research, we answer three main research questions regarding response error. First, how is false-negative reporting associated with the point-in-time and recall conjectures? Second, to what extent do program name confusion and selective reporting of health insurance affect false-negative error and false-positive response error? And third, how does shared Medicaid coverage in the household affect false-negative error and false-positive response error? We also describe demographic and socioeconomic characteristics associated with response error. These findings deepen our knowledge of Medicaid coverage and estimates of the uninsured population based on the CPS ASEC.
3 | DATA AND METHODS
For our analysis, we use 2006-2010 Medicaid administrative records and 2007-2011 CPS ASEC survey data on health insurance coverage. For reference, the tables show 2000-2005 Medicaid data and 2001-2006 CPS ASEC data based on the SNACC (12) Phase V Medicaid undercount report; however, discussion of the results focuses on our new years of analysis.
The CPS ASEC is a supplement that samples 98 000 households and collects additional data on poverty, migration, and work experience. Data on Medicaid enrollment are from the MSIS and are submitted electronically by states quarterly to the Centers for Medicare and Medicaid Services (CMS). The MSIS data contain information for Medicaid enrollees including number of days enrolled by month and Medicaid coverage type.
Several differences exist between the MSIS and CPS ASEC universes. States report MSIS data in fiscal quarters, each Medicaid enrollment record receives a MSIS case number, and multiple records may exist for persons with different case numbers across states. The CPS ASEC includes survey data with persons as the unit of analysis. The MSIS data include the institutionalized population, whereas the CPS ASEC sample does not include persons in institutionalized group quarters. ([daggaer]) MSIS data include information on full and restricted benefits, but the CPS ASEC does not collect data on the level of Medicaid benefit eligibility. Finally, MSIS includes point-in-time data on enrollment while prior to the 2014 redesign, the CPS ASEC only collected retrospective information on health insurance coverage for the previous calendar year. ([double daggaer])
We calculate raw and adjusted measures for the Medicaid undercount. The raw measures use the CPS ASEC and MSIS data universes without adjustments. We then adjust both datasets as outlined below to make them more comparable. We use raw and adjusted measures to evaluate the Medicaid undercount and adjusted measures to analyze false-negative reporting and false-positive reporting.
For the adjusted MSIS measure, we remove persons residing in institutional group quarters, aggregate the state MSIS data into one national file, and unduplicate by persons. We exclude enrollees reported in MSIS with only partial benefits (e.g, emergency services or family planning) or who are only covered through the State Children's Health Insurance Program (SCHIP). States' SCHIP reporting in MSIS is not consistent for separate CHIP (S-CHIP) programs and Medicaid expansion (M-CHIP) programs thus making it problematic from an administrative records perspective. For the CPS ASEC-adjusted measure, we exclude persons reported as covered only through SCHIP programs. These participants may not consider themselves to be covered by Medicaid in the same way as those with full benefits. SCHIP is also administered as a separate program from Medicaid in many states, and it is likely that many participating families are not aware of the difference. (15)
We evaluate false-negative error and false-positive error by linking individuals in the CPS ASEC to their MSIS record. In each data-set, individuals are assigned an anonymized unique person identifier called a Protected Identification Key (PIK) via probabilistic record linkage techniques. (16,17) Only observations with PIKs are linkable (Table S1). Therefore, CPS ASEC records with a PIK are re-weighted to compensate for observations not assigned a PIK. Since there is bias in PIK assignment, the CPS ASEC weighting adjustment factors are calculated based on PIK rates by age group, relative poverty ratio, health insurance status, and whether health insurance status was imputed.
We use two logistic regression models to evaluate the characteristics of people who report false negatives and false positives in the 2011 CPS ASEC. The dependent variable for the false-negative regression analysis is "1" if a person reported no Medicaid coverage in the CPS ASEC but was enrolled according to 2010 MSIS data. The dependent variable is "0" if the person reported Medicaid coverage and was enrolled in 2010 MSIS data with full benefits. In the false-positive regression, the dependent variable is "1" if a person reports Medicaid coverage in the 2011 CPS ASEC, but according to 2010 MSIS, they are not receiving partial benefits, full benefits, or coverage through the SCHIP expansion program. The dependent variable is "0" if a person reports Medicaid coverage in the CPS ASEC and has full or partial benefits in MSIS data. Both analyses exclude imputed and edited survey responses since we are interested in the characteristics of persons who report their insurance status as opposed to those whose coverage is edited or imputed.
To address the point-in-time versus recall conjecture research question, we measure for whether the respondent was enrolled in MSIS data in 2011, the year the CPS ASEC was conducted. A higher likelihood of false-positive reporting by newly enrolled respondents not enrolled during the previous calendar year would support the point-in-time hypothesis. For false-negative reports, we measure whether the respondent was enrolled with full benefits in 2011, and for false-positive reports, we measure whether the respondent was enrolled with full or partial benefits in 2011.
We assess the impact of recall issues on false-negative error using a measure of Medicaid coverage duration, measured as the number of days a person is enrolled in Medicaid during 2010. We assess the impact of recall issues on false-positive error by observing whether persons had Medicaid coverage in MSIS for 2009. For false-positive error, the hypothesis predicts that people enrolled in Medicaid 2 years before the survey are more likely to report false positives.
One source of false-positive error is the inability to link persons reporting Medicaid coverage with their MSIS enrollment data. A person may be correct in reporting coverage, but if the MSIS record has no PIK because it lacked necessary information for PIK assignment, the MSIS data cannot be linked to the survey data. For 2000-2005, it was estimated that between 39.4 and 62.1% of false-positive errors in the CPS ASEC were incorrectly identified as a consequence of unlinkable MSIS records. (12) For 2010, that estimate drops to about 18% as a result of an increased PIK rate in MSIS. Regression results are interpreted in light of this caveat. In particular, false-positive responses with coverage in 2009 and 2011 may be more likely to have an unlinkable MSIS record.
For our second research question, to account for possible confusion between Medicaid and other health insurance, both models measure CPS ASEC reports of enrollment in Medicare and any other type of insurance. To address our third research question on shared coverage, we measure whether anyone else in the household also had Medicaid enrollment data in MSIS.
We also describe demographic and socioeconomic factors associated with false-negative error and false-positive error. We include Hispanic origin, race, sex, age, education level, foreign-born status, and logged household income in both models.
4 | RESULTS
4.1 | Medicaid undercount
Figure 1 and Table 1 present results for measurement of the Medicaid undercount in the CPS ASEC. We include SNACC Medicaid undercount results for 2000-2005 for comparison. The raw undercount measures ranged from 39% to 33% from 2006 to 2010, similar to previous years of analysis. The adjusted measure of the Medicaid undercount ranged from 31% to 22% from 2006 to 2010 with 2010 the lowest for the entire time series.
The percent difference between raw and adjusted measures of the undercount increased over time reaching a high of 11.5% in 2009 and 11.2% in 2010. The statistically significant decline in the adjusted measure and corresponding increase in the gap between raw and adjusted measures likely result from more duplicate persons identified in MSIS data through improved PIK assignment (Table S1). There may also be an increased use of partial benefits relative to full benefits, and removing these lowers the adjusted undercount and increases the difference with the raw undercount.
4.2 | Measurement error descriptives
Table 2 presents rates for false-negative survey error. We show SNACC results from 2000-2005 for comparison. For 2000-2010, there were between 40% and 45% of false-negative reports for Medicaid in the CPS ASEC. This declined from 45% in 2006 to 39% in 2009 and 40% in 2010. Only about 60% of CPS ASEC respondents linked to MSIS data correctly classified themselves as enrolled in Medicaid. The results indicate the persistence of false-negative error in the CPS ASEC.
Consistent with previous research, for 2006-2009, the majority of Medicaid false-negative reports in the CPS ASEC resulted from reported insurance status (between 73% and 79%) while the remainder resulted from data imputation procedures. Among the false negatives, about 60% reported having other insurance coverage. Conversely, about 40% of the Medicaid false negatives (8 million in 2010) erroneously contributed to estimates of the uninsured.
Table 3 presents results for the CPS ASEC false-positive respondents. The false-positive error rate for 2000-2010 ranged from 2.6% to 4.7%. Among all CPS ASEC records with a PIK not matched to MSIS records, 3% to 5% reported Medicaid coverage in the survey. Of all Medicaid reports between 2000 and 2010, between 21% and 27% were identified as false-positive errors. While false-negative error mostly resulted from respondent reporting, about 60% of false positives resulted from edit and imputation procedures. CPS ASEC health insurance edits are intended to improve response accuracy by using other interview information to assign edited values. Medicaid coverage can only be assigned through edits; it cannot be taken away. Compared to individual edits, imputations are intended to improve better survey estimates overall. Future research should disentangle these different processes.
4.3 | Regression results
Table 4 shows logistic regression results modeling false-negative reports of Medicaid coverage in the CPS ASEC. Of the weighted sample, 42% are coded as false negatives, and 58% correctly reported Medicaid coverage.
We find support for both point-in-time and recall conjectures. Medicaid enrollment in the survey year is associated with a lower likelihood of a false-negative report, supporting the point-in-time conjecture that some report their Medicaid status at the time of interview rather than the previous year. Longer Medicaid enrollment is associated with lower odds of a false-negative report, supporting the recall conjecture. Compared to those enrolled 60 days or fewer, those enrolled the entire year were least likely to provide false-negative responses (80% lower odds).
We also find support that people may be confusing Medicaid with Medicare and other insurance programs, or their dual-enrollment is leading them not to report Medicaid. Reporting some other form of insurance makes a person nearly six times as likely to provide a false-negative report. Reporting Medicare is associated with about 50% higher odds of a false-negative response. We did not find evidence that shared Medicaid coverage in the household is associated with false-negative reporting error. The odds ratio is close to 1 and not significant.
Hispanics have about 34% higher odds of a false-negative Medicaid report compared to non-Hispanics. For race, persons reported in the CPS ASEC as either Black alone or Asian alone have a greater likelihood of a false-negative response compared to persons reported as White alone. Odds ratios are not significant for other race groups. ([section])
Fewer women than men report false negatives. Compared to persons aged 18-24, persons aged 45-64 are less likely to have false-negative reports, and persons aged 65 and over are more likely to report a false negative. ([paragraph]) This finding can be understood in the context of the higher odds of reporting a false-negative response when also reporting Medicare given that age and Medicare eligibility are highly correlated with most of those over age 65 enrolled in Medicare. There is no statistical difference in reporting a false negative between those aged 25-44 and persons aged 18 to 24. Foreign born is associated with a greater likelihood of a false-negative Medicaid response.
Relative to those with less than a high school diploma, persons with a high school degree, some college, or a bachelor's degree are more likely to give false-negative Medicaid reports. However, persons with a graduate degree are not statistically different from those without a high school diploma. Persons with higher family incomes are less likely to give false-negative reports than persons with lower family incomes.
Next, we discuss regression results on false-positive reporting (Table 4). Of the weighted sample, about 16% are coded as false-positive reports without Medicaid enrollment, and about 84% are coded as having Medicaid enrollment in MSIS.
Among those reporting Medicaid coverage, those also reporting Medicare coverage are no more or less likely to give false-positive reports. However, those reporting other forms of insurance are less likely to give false-positive reports than those who do not. Persons enrolled in Medicaid in the previous year and persons enrolled in Medicaid in the survey year were less likely to have false-positive reports (90% lower odds).
When Medicaid coverage is shared in the household, the respondent is less likely to give a false-positive report. False-positive reports are more likely when the respondent is the only one reporting Medicaid coverage in the household.
Hispanics are twice as likely as non-Hispanics to have false-positive reports. For race, relative to those reporting White alone, persons reporting Asian alone or Native Hawaiian or Pacific Islander alone have a higher likelihood of a false-positive report with approximately three and six times greater odds, respectively. (**)
Similar to results on false negatives, among persons who report Medicaid coverage in the CPS ASEC, women are less likely than men to give a false-positive response with about 29% lower odds. Persons aged 45-64 are about 60% more likely to give false-positive reports, and persons aged 65 and older are twice as likely compared to persons age 18-24. In contrast to the false-negative results, there is no foreign-born effect. Persons with a graduate degree are more likely to provide a false-positive response compared to persons with less than a high school degree, though the results are not significant for other education levels. And as logged family income increases, so does the likelihood of false-positive reports.
4.4 | Adjusted insurance rates
When respondents incorrectly report they do not have Medicaid coverage, they lower estimates of Medicaid coverage, whereas false-positive reports of Medicaid coverage may incorrectly add to Medicaid enrollment estimates. When we adjust the Medicaid coverage rates accounting for false-negative Medicaid reporting and false-positive Medicaid reporting, the estimated Medicaid coverage rate increases from 15.9% to 18.7%. Recall, however, a small proportion of false positives may appear as such because the corresponding MSIS enrollment data could not be linked to the CPS ASEC response, so this adjustment is an estimate, and the actual coverage rate may be higher.
When respondents report false negatives for Medicaid and also do not have any other form of government or private insurance, then those persons incorrectly increase the uninsured population. For the 2011 CPS ASEC, 8.1 million persons were incorrectly classified as uninsured (Table S2). Conversely, persons who report only Medicaid coverage when no enrollment data support that response may incorrectly reduce the uninsured population. For the 2011 CPS ASEC, this amounted to an upward bound of 5.8 million persons not included in the count of the uninsured. Similarly adjusting the 2011 CPS ASEC estimate of the uninsured population, the percent uninsured declines from 16.3% to 15.6%. However, some who report false positives for Medicaid may have health insurance but report the wrong type of coverage. For example, some enrolled in SCHIP programs may incorrectly report Medicaid coverage. Others may only appear to be false positives because their MSIS data could not be linked to the CPS ASEC. The adjusted estimated uninsured rate may actually be lower than calculated.
5 | CONCLUSION
The Medicaid undercount measured from 2000 to 2005 persists when analyzing 2006-2010. Based on the regression analysis presented here, false-negative and false-positive reporting errors are not randomly distributed across persons in the CPS ASEC. We found evidence to support both the point-in-time and recall conjectures with both processes affecting false-negative reporting which suggests that these are related processes rather than competing explanations. Respondents with a longer duration of coverage and those currently enrolled are both less likely to be false negatives, and we would expect to see an overlap between these two factors. Enrollment in Medicaid in the survey year and the duration of Medicaid enrollment in the previous calendar year both lower the likelihood of false-negative reports. We also found effects of enrollment in other insurance programs on response error. We found evidence to support the shared coverage hypothesis for false-positive responses but not for false-negative response error; however, the latter may be related to our different operationalization of shared coverage from Pascale, Roemer, and Resnick. (14) Future research should explore shared coverage related to overlapping coverage and whether the respondent is reporting for dependent children. We also find evidence of variation in false-negative and false-positive survey response errors according to race and ethnicity, age, sex, education, and foreign-born status. Additional research is needed to examine more closely these relationships.
Future work should investigate the utility of MSIS administrative records for assisting CPS ASEC editing and imputation procedures. This could facilitate reductions in response error and increased confidence in CPS ASEC estimates of Medicaid coverage. However, this would require more timely delivery and processing of Medicaid enrollment data.
The CPS ASEC is an important survey instrument providing important insights to the U.S. population. Given that the Medicaid undercount in the CPS ASEC has persisted over time, this suggests the importance of appropriate steps to align more closely survey estimates of Medicaid coverage with administrative record totals from Medicaid enrollment data. Survey error reduces accuracy of estimates of Medicaid coverage, other coverage, and the uninsured population.
To achieve more accurate estimates of insurance coverage, the Census Bureau has regularly conducted research and testing on question design for health insurance in the CPS ASEC. Beginning in 2014, the CPS ASEC asks respondents which months they were uninsured rather than whether they were uninsured at any point during the prior year. (10) This redesign will likely reduce response error to the health insurance question and improve estimates of the Medicaid covered population and yield improved estimates of the uninsured population. (18) Future analysis on the Medicaid undercount and survey response error will compare the Medicaid undercount and response error preceding the redesign with the post-redesign CPS ASEC. We expect response error and the Medicaid undercount to decline compared to the results we have presented here. We will also measure any changes to the Medicaid undercount in the years since Medicaid expansion under the ACA to determine the extent to which Medicaid expansion may shape response error and CPS ASEC estimates of Medicaid enrollment and the uninsured. Together, this will further our understanding of survey estimates of health insurance coverage and the uninsured.
Joint Acknowledgment/Disclosure Statement: This research was conducted under the employment of the U.S. Census Bureau. This article is released to inform interested parties of research and to encourage discussion. The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. The information in this article has been approved for release by the Disclosure Review Board.
(*) Researchers from the University of Minnesota's State Health Access Data Assistance Center, CMS, Assistant Secretary for Planning and Evaluation, National Center for Health Statistics, Administration for Healthcare Research and Quality, and the Census Bureau partnered to research the CPS ASEC Medicaid undercount. The project was named the SNACC Medicaid Undercount Project to represent collaborating agencies. SNACC reports can be retrieved from: https://www.census.gov/did/www/snacc/
([dagger]) CPS ASEC includes Armed Forces personnel and families in civilian housing or on military bases.
([double dagger]) The 2011 CPS ASEC asked about Medicaid coverage during the past calendar year, "At any time in 2010, (was/were) (you/anyone in this household) covered by Medicaid?" If "yes," the questionnaire asks who was covered.
([section]) Using another year, regression results for Asian alone were not significant. Because of a small number of observations this result may not be robust.
([paragraph]) We exclude minors because of the education variable.
(**) Regression results differed for different years suggesting they may not be robust because of few observations.
James M. Noon [iD] http://orcid.org/0000-0002-1478-346X
(1.) Burwell SM. 2014 Actuarial report on the financial outlook for Medicaid. CentersforMedicareand Medicaid Services; 2014. https://www.medicaid.gov/medicaid/financing-and-reimbursement/downloads/medicaid-actuarial-report-2014.pdf. Accessed August 7, 2018.
(2.) Lewis CH, Elwood M, Czajka J. Counting the Uninsured: A Review of the Literature. Washington, DC: The Urban Institute; 1998.
(3.) Card D, Hildreth AKG, Shore-Sheppard LD. The measurement of Medicaid coverage in the SIPP. J Bus Econ Stat. 2004;22(4):410-420.
(4.) Klerman JA, Ringel JS, Roth B. Under-Reporting of Medicaid and Welfare in the Current Population Survey. Santa Monica, CA: RAND; 2005.
(5.) Call KT, Davidson G, Davern M, Blewett LA, Nyman R. The Medicaid undercount and bias to estimates of uninsurance: new estimates and existing evidence. Health Serv Res. 2008;43(3):901-914.
(6.) Davern M, Call KT, Ziegenfuss J, Davidson G, Beebe TJ, Blewett LA. Validating health insurance coverage survey estimates: a comparison between self-reported coverage and administrative data records. Public Opin Q. 2008;72(2):241-259.
(7.) Davern M, Klerman JA, Baugh DK, Call KT, Greenberg GD. An examination of the Medicaid undercount in the Current Population Survey: preliminary results from record linking. Health Serv Res. 2009;44(3):965-987.
(8.) Klerman JA, Davern M, Call KT, Lynch V, Ringel JD. Understanding the Current Population Survey's insurance estimates and the Medicaid "undercount". Health Aff. 2009;28(6):991-1001.
(9.) Kincheloe J, Brown ER, Frates J, Call KT, Yen W, Watkins J. Can we trust population surveys to count Medicaid enrollees and the uninsured? Health Aff. 2006;25(4):1163-1167.
(10.) Pascale J, Boudreaux M, King R. Understanding the new Current Population Survey health insurance questions. Health Serv Res. 2016;51(1):240-261.
(11.) Boudreaux MH, Call KT, Turner J, Fried B, O'Hara B. Measurement error in public health insurance reporting in the American Community Survey: evidence from record linkage. Health Serv Res. 2015;50(6):1973-1995.
(12.) SNACC Phase V. Phase V Research Results: Extending the Phase II Analysis of Discrepancies between the National Medicaid Statistical Information System (MSIS) and the Current Population Survey (CPS) Annual Social and Economic Supplement (ASEC) from Calendar Years 2000-2001 to Calendar Years 2002-2005. Minneapolis, MN: State Health Access Data Assistance Center; 2010. http://www.shadac.org/publications/snacc-phase-v-report. Accessed August 7, 2018.
(13.) Pascale J. Measuring health insurance in the U.S. Statistical Research Division Research Report Series, Survey Methodology #2007-11. Washington, DC: U.S. Census Bureau; 2007.
(14.) Pascale J, Roemer Ml, Resnick DM. Medicaid underreporting in the CPS: results from a record check study. Public Opin Q. 2009;73(3):497-520.
(15.) SNACC Phase II. Phase II Research Results: Examining Discrepancies between the National Medicaid Statistical Information System (MSIS) and the Current Population Survey (CPS) Annual Social and Economic Supplement (ASEC). Minneapolis, MN: State Health Access Data Assistance Center; 2008. http://www.census.gov/did/www/snacc/docs/SNACC_Phase_II_Full_Report.pdf. Accessed August 7, 2018.
(16.) Fellegi IP, Sunter AB. A theory for record linkage. J Am Stat Assoc. 1969;64:1183-1210.
(17.) Wagner D, Layne M. The person identification validation system (PVS): applying the Center for Administrative Records Research and Applications' (CARRA) record linkage software. Center for Administrative Records Research and Applications Working Paper #2014-01. Washington, DC: U.S. Census Bureau; 2014.
(18.) Brault M, Medalia C, O'Hara B, Rodean J, Steinweg A. Changing the CPS health insurance questions and the implications on the uninsured rate: redesign and production estimates. SEHSD Working Paper 2014-16. Washington, DC: U.S. Census Bureau; 2014.
Additional supporting information may be found online in the Supporting Information section at the end of the article.
How to cite this article: Noon JM, Fernandez LE, Porter SR. Response error and the Medicaid undercount in the current population survey. Health Serv Res. 2019;54:34-43. https://doi.org/10.1111/1475-6773.13058
James M. Noon MA [iD] | Leticia E. Fernandez PhD | Sonya R. Porter PhD
U. S. Census Bureau, Washington, District of Columbia
James M. Noon, MA, U. S. Census Bureau, Washington, DC 20233.
TABLE 1 Measuring the Medicaid undercount in the CPSASEC CY 2000 CY 2001 CY 2002 Undercount Raw measure (a) 34.4% 34.9% 37.9% Adjusted measure (b) 31.7% 31.5% 37.8% Difference between 2.7% 3.4% 0.2% raw and adjusted MSIS enrollment Raw total 45 050 000 48 550 000 53 550 000 Adjusted total 38 150 000 40 450 000 45 950 000 CPS enrollment Raw total 29 550 000 31 600 000 33 250 000 Adjusted total 26 050 000 27 700 000 28 600 000 CY 2003 CY 2004 CY 2005 Undercount Raw measure (a) 37.1% 36.8% 37.8% Adjusted measure (b) 32.3% 30.9% 31.6% Difference between 4.7% 5.9% 6.2% raw and adjusted MSIS enrollment Raw total 56 650 000 59 350 000 61 250 000 Adjusted total 45 600 000 47 700 000 49 200 000 CPS enrollment Raw total 35 650 000 37 500 000 38 100 000 Adjusted total 30 850 000 32 950 000 33 650 000 CY 2006 CY 2007 CY 2008 Undercount Raw measure (a) 38.9% 37.3% 34.9% Adjusted measure (b) 31.0% 29.4% 26.3% Difference between 7.9% 7.8% 8.6% raw and adjusted MSIS enrollment Raw total 62 814 724 63 251 758 65 763 995 Adjusted total 48 791 594 48 813 182 50 520 559 CPS enrollment Raw total 38 369 832 39 684 984 42 830 746 Adjusted total 33 673 128 34 450 753 37 249 946 CY 2009 CY 2010 Undercount 33.9% 33.2% Raw measure (a) 22.4% 22.0% Adjusted measure (b) 11.5% 11.2% Difference between raw and adjusted MSIS enrollment 72 377 229 72 740 815 Raw total 53 813 902 55 576 733 Adjusted total CPS enrollment 47 847 441 48 580 103 Raw total 41 755 208 43 348 345 Adjusted total Notes: Data for Calendar Years 2000-2005 from SHADAC (2010). Numbers are rounded for 2000-2005 according to SHADAC report. (a) The "Raw measure" uses both the CPS ASEC and MSIS data universes without adjustments to either data series. (b) The "Adjusted measure" includes adjustment made to both the CPS ASEC and MSIS data universes to make them more comparable. TABLE 2 False-negative survey errors for Medicaid reporting in the CPS ASEC CY 2000 CY 2001 CY 2002 Correctly classified 19 090 000 20 550 000 21 350 000 as enrolled Incorrectly 14 360 000 15 450 000 17 250 000 classified as not enrolled (false-negative errors) Percent enrolled 42.9% 42.9% 44.7% who reported no enrollment Source of enrollment status among false-negative errors Reported 11 030 000 11 960 000 13 240 000 Imputed 3 340 000 3 500 000 4 000 000 Percent of false 76.8% 77.4% 76.8% negatives that were reported (not imputed) "Other Insurance" status of false-negative population: Insured 8 600 000 9 190 000 10 370 000 Uninsured 5 760 000 6 260 000 6 880 000 Percent false 40.1% 40.5% 39.9% negatives who are uninsured CY 2003 CY 2004 CY 2005 Correctly classified 23 270 000 24 380 000 24 830 000 as enrolled Incorrectly 17 680 000 18 870 000 18 670 000 classified as not enrolled (false-negative errors) Percent enrolled 43.2% 43.6% 42.9% who reported no enrollment Source of enrollment status among false-negative errors Reported 13 730 000 13 980 000 14 350 000 Imputed 3 900 000 4 880 000 4 320 000 Percent of false 77.9% 74.1% 76.9% negatives that were reported (not imputed) "Other Insurance" status of false-negative population: Insured 10 580 000 11 170 000 10 870 000 Uninsured 7 100 000 7 700 000 7 800 000 Percent false 40.2% 40.8% 41.8% negatives who are uninsured CY 2006 CY 2007 CY 2008 Correctly classified 23 800 091 24 538 855 26 397 133 as enrolled Incorrectly 19 271 550 18 942 205 18 616 298 classified as not enrolled (false-negative errors) Percent enrolled 44.7% 43.6% 41.4% who reported no enrollment Source of enrollment status among false-negative errors Reported 14 903 989 14 899 526 14 542 359 Imputed 4 367 561 4 042 679 4 073 939 Percent of false 77.3% 78.7% 78.1% negatives that were reported (not imputed) "Other Insurance" status of false-negative population: Insured 11 534 054 11 663 882 11 458 087 Uninsured 7 737 495 7 278 323 7 158 211 Percent false 40.1% 38.4% 38.5% negatives who are uninsured CY 2009 CY 2010 Correctly classified 29 522 452 29 746 502 as enrolled Incorrectly 18 699 087 20 135 208 classified as not enrolled (false-negative errors) Percent enrolled 38.8% 40.4% who reported no enrollment Source of enrollment status among false-negative errors Reported 14 666 776 14 632 048 Imputed 4 032 311 5 503 160 Percent of false 78.4% 72.7% negatives that were reported (not imputed) "Other Insurance" status of false-negative population: Insured 11 247 313 12 084 859 Uninsured 7 451 773 8 050 349 Percent false 39.9% 40.0% negatives who are uninsured Notes: Enrollment status is edited only to assign enrollment, not to take it away, so it never causes false-negative errors. Re-weighted data for Calendar Years 2000-2005 from SHADAC (2010). Numbers are rounded for 2000-2005 according to SHADAC report. TABLE 3 Medicaid false-positive survey errors for Medicaid reporting in the CPS ASEC CY 2000 CY 2001 CY 2002 Linkable records not 244 050 000 243 600 000 242 900 000 in MSIS CPS adjusted 26 050 000 27 700 000 28 600 000 enrollment Total false-positive 6 460 000 6 420 000 6 200 000 errors False-positive 2.6% 2.6% 2.6% error rate Percent reporting 24.8% 23.2% 21.7% enrolled but do not have MSIS enrollment data Source of enrollment status among false-positive errors: Reported 2 460 000 2 740 000 2 400 000 Imputed 2 560 000 2 520 000 2 480 000 Edited 1 420 000 1 160 000 1 320 000 Reported, % 38.2% 42.7% 38.7% Imputed, % 39.8% 39.3% 40.0% Edited, % 22.0% 18.1% 21.3% CY 2003 CY 2004 CY 2005 Linkable records not 241 750 000 243 650 000 245 250 000 in MSIS CPS adjusted 30 850 000 32 950 000 33 650 000 enrollment Total false-positive 6 380 000 7 460 000 7 680 000 errors False-positive 2.6% 3.1% 3.1% error rate Percent reporting 20.7% 22.6% 22.8% enrolled but do not have MSIS enrollment data Source of enrollment status among false-positive errors: Reported 2 540 000 2 500 000 2 640 000 Imputed 2 520 000 3 140 000 3 060 000 Edited 1 320 000 1 840 000 1 980 000 Reported, % 39.8% 33.4% 34.4% Imputed, % 39.5% 42.0% 39.8% Edited, % 20.7% 24.6% 25.8% CY 2006 CY 2007 CY 2008 Linkable records not 248 313 514 250 089 263 250 486 590 in MSIS CPS adjusted 33 673 128 34 450 753 37 249 946 enrollment Total false-positive 8 664 802 8 494 168 9 101 359 errors False-positive 3.5% 3.4% 3.6% error rate Percent reporting 25.7% 24.7% 24.4% enrolled but do not have MSIS enrollment data Source of enrollment status among false-positive errors: Reported 3 286 602 3 427 260 3 472 635 Imputed 3 231 220 2 873 591 3 155 255 Edited 2 146 980 2 193 317 2 473 469 Reported, % 37.9% 40.3% 38.2% Imputed, % 37.3% 33.8% 34.7% Edited, % 24.8% 25.8% 27.2% CY 2009 CY 2010 Linkable records not 249 528 631 249 691 437 in MSIS CPS adjusted 41 755 208 43 348 345 enrollment Total false-positive 10 167 982 11 618 694 errors False-positive 4.1% 4.7% error rate Percent reporting 24.4% 26.8% enrolled but do not have MSIS enrollment data Source of enrollment status among false-positive errors: Reported 3 901 083 4 411 684 Imputed 3 715 733 4 475 359 Edited 2 551 166 2 731 651 Reported, % 38.4% 38.0% Imputed, % 36.5% 38.5% Edited, % 25.1% 23.5% Notes: Data for calendar years 2000-2005 from SHADAC (2010). Numbers are rounded for 2000-2005 according to SHADAC report. TABLE 4 Weighted logistic regression results: survey response error in 2011 CPS ASEC False-negative False-positive response response Variables Odds ratios Odds ratios Demographic and socioeconomic characteristics Hispanic origin (not Hispanic 1.34 (***) 2.30 (***) omitted) Race (White alone omitted) Black alone 1.21 (**) 1.35 American Indian or Alaska 1.34 1.87 Native alone Asian alone 1.35 (*) 2.60 (**) Native Hawaiian or Pacific 1.63 6.26 (**) Islander alone Two or more races 0.84 0.74 Female (Male omitted) 0.86 (**) 0.71 (*) Age (18-24 omitted) 25-44 0.94 1.19 45-64 0.71 (***) 1.60 (*) 65+ 1.30 (*) 2.06 (*) Education (less than high school omitted) High school degree 1.33 (***) 1.30 Some college 1.19 (*) 1.21 Bachelor's degree 1.43 (**) 1.19 Graduate degree 1.33 2.83 (*) Foreign born (U.S. born 1.27 (**) 0.80 omitted) Insurance coverage Reported Medicare 1.50 (***) 0.71 coverage Reported enrolled in another 5.88 (***) 0.63 (*) insurance program Number of days enrolled in Medicaid in 2010 (1-60 omitted) 61-180 0.55 (***) 181-364 0.29 (***) All year 0.20 (***) Enrolled in Medicaid in 0.02 (***) previous year (2009) Enrolled in Medicaid in 0.39 (***) 0.04 (***) survey year (2011) Household variables Shared coverage 1.01 0.38 (***) Logged household income 0.96 (***) 1.06 (*) -2 log likelihood 24 839 027 12 883 186 Weighted N 18 202 244 13 974 634 (*) P < 0.05, (**) P < 0.01, (***) P < 0.001. Source: 2011 CPS ASEC, MSIS administrative records.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||RESEARCH ARTICLE|
|Author:||Noon, James M.; Fernandez, Leticia E.; Porter, Sonya R.|
|Publication:||Health Services Research|
|Date:||Feb 1, 2019|
|Previous Article:||Alive or dead: Validity of the Social Security Administration Death Master File after 2011.|
|Next Article:||An empiric approach to identifying physician peer groups from claims data: An example from breast cancer care.|