The impact of specialized benefits counseling services on Social Security Administration disability beneficiaries in Vermont.

A major reason for the high unemployment rate among Social Security Administration (SSA) disability beneficiaries is fear and misunderstanding about the impact of employment on their benefits (United States General Accounting Office, 1996; Johnson-Lamarche and Baird, 1997). Beneficiaries often do not work or are underemployed because they fear a loss of benefits, especially healthcare coverage, or they are unaware of and/or misunderstand available work incentives built into the SSA disability programs (United States General Accounting Office, 1996; Johnson-Lamarche and Baird, 1997).

One response to this problem has been to help SSA disability beneficiaries to understand their benefits and make optimal decisions about employment through specialized benefits counseling. This refers to specialized expert staff that provide benefits screening, benefits advisement and benefits management for SSA disability beneficiaries regarding employment (Golden et al., 2000). Recently, SSA's State Partnership Initiatives, the Ticket to Work and Work Incentives Improvement Act, Benefits Planning and Outreach grants, and other Federal-State Vocational Rehabilitation programs (Golden et al., 2001) have promoted specialized benefits counseling services nationwide. Despite the extensive implementation of benefits counseling services, however quantitative outcome evaluation efforts have until recently been minimal, as measured by the current lack of published studies on their effectiveness. Like other states, Vermont has used its SSA-funded State Partnership Initiative to create a specialized benefits counseling service. The purpose of this paper is to report preliminary employment outcomes related to Vermont's program.

Method

Overview

To evaluate Vermont's specialized benefits counseling program, we used a quasi-experimental study. Vocational Rehabilitation consumers who received specialized benefits counseling were compared with two matched comparison groups comprising concurrent and historical participants in the Division of Vocational Rehabilitation system who did not receive benefits counseling. We compared earned income for each group over four years.

The Benefits Counseling Intervention

The Vermont Work Incentive Initiative created specialized benefits counseling services in 1999. Six benefits counselors were co-located in Division of Vocational Rehabilitation (DVR) offices and Community Mental Health Center offices throughout Vermont. The benefits counselor positions were classified and organized as Vermont DVR staff and were compensated at a pay grade level equivalent to a vocational rehabilitation counselor. Benefits counseling is a newly emerging field in the rehabilitation arena. Therefore no formal benefits counseling certification or accreditation exists for these positions. As a result the project recruited and hired individuals from a variety of prior related backgrounds including public vocational rehabilitation, the state social welfare agency and the community mental health system. The benefits counselors were required to have a BA plus a minimum of 18 months experience in human service field. Applicants with over three years experience in related fields could apply without a BA. The staff provided benefits counseling services to persons receiving disability benefits from the Social Security Administration (SSA) under the Social Security Disability Insurance (SSDI) program and/or the Supplemental Security Income (SSI) program.

The benefits counseling services were individualized, longitudinal, and of variable intensity. The benefits counseling process consisted of three major practice domains identified by Golden et al (2000): benefits screening, advisement, and management. First, benefits screening involved researching and verifying the participant's current benefits status (including all benefits received) and determining what benefits issues might impact the participant's employment goals. Second, benefits advisement was the process of explaining to the participant how employment would impact the benefits he or she receives and assisting the participant to make informed choices based on this information. This also included helping the participant take advantage of the various work incentives available under these programs (e.g., impairment-related work expenses, trial work periods, State Medicaid buy-in programs). Finally, benefits management was the process of assisting the participant to develop and implement a plan to manage his or her benefits through the transition to employment. The benefits counselor also provided ongoing information and support to the participant as his or her employment and benefits status changed and evolved.

In all cases the benefits counselors worked in conjunction with the participant's vocational rehabilitation counselor and/or their community mental health case worker or supported employment specialist. But in contrast to the roles of these latter professionals, the benefits counselor role was a specialization exclusively limited to benefits issues. Through this specialization, the benefits counselors achieved a level of benefits expertise well beyond that of a generalist vocational rehabilitation counselor or mental health case manager. The vocational rehabilitation counselors and/or community mental health staff arranged for or provided all the other vocational supports (e.g. vocational counseling and guidance, job placement, on site support etc.). As such, benefits counseling is best described as a specialized supporting role in the vocational rehabilitation process.

The duration and extent of benefits counseling varies by individual according to the beneficiary's needs and preferences, but benefits counseling is available throughout the duration of the project. Follow-up information is gathered from enrollees at least quarterly, and enrollees commonly seek and receive follow-up benefits counseling services as new benefits-related questions arise or as their personal situations change (e.g., pay raises, marriage). For the group in our study, the amount of benefits counseling varied considerably, ranging from 15 minutes to more than 84 hours, with an average of approximately 8 hours per consumer.

Study Participants

The three study groups for this evaluation were drawn from a common sampling pool of current and former Division of Vocational Rehabilitation (VR) consumers. (Approximately 95% of benefits counseling enrollees in Vermont are current or former consumers of VR services.) Group 1, the "benefits counseling" group, was compared to two groups of VR consumers who did not participate in the benefits counseling program. Group 2, the "contemporaneous" comparison group, were consumers receiving VR services concurrent with the VR services received by the benefits counseling enrollees. Group 3, the "historical" comparison group, were consumers receiving VR services approximately two-and-a-half years earlier than the benefits counselees (representing the earliest group for whom baseline outcomes data were available). Each individual in the benefits counseling group was matched, according to procedures described below, to one similar consumer in the contemporaneous comparison group and one similar consumer in the historical comparison group. The purpose of the contemporaneous comparison group was to estimate effects of the program relative to non-participants at roughly the same time period during which the project was being implemented (thus attempting to control for the influence of changes in the State economic climate and other temporal factors). The purpose of the historical group was to provide an additional comparison, which would estimate effects of the program relative to non-participants from a time period earlier than implementation of the project (attempting to replicate the contemporaneous comparison, but with a timeframe of comparison less susceptible to contemporaneous treatment spillover effects on non-participants).

Through May 2002, the benefits counseling program had 672 enrollees between the ages of 18 and 60 who were SSA disability beneficiaries (SSDI, SSI, or dual eligibles) and had received VR services. As a result of the pair-wise matching procedures, the contemporaneous and historical comparison groups had the same size (N = 672). Table 1 shows a comparison of the three groups in terms of disability type, beneficiary type, and demographics.

Matching was carried out using five variables: (a) experience as a VR consumer; (b) experience as a Social Security Administration disabled beneficiary while a VR consumer; (c) primary VR disability of record (physical disability, mental illness, developmental disability, or other); (d) closest start date of the person's VR services (date of certification of eligibility); and (e) maturation in the VR service system (length of time since certification of eligibility and initiation of VR services) in aligning the time points of comparison.

The rationale for the use of these matching variables was to obtain comparison groups drawn from populations as similar as possible to project enrollees. Variables (a) and (b) were matched upon because all benefits counseling enrollees included in this analysis were simultaneous VR consumers and SSA disability benefit recipients. Disability type (c) is a variable representative of, and correlated with, many of the individualized employment challenges faced by project participants. Variable (d), marking the point in time that individuals started VR services, was used to control for statewide macroeconomic and historical effects. Because the duration of exposure to VR services is related to job readiness among VR consumers, participants and comparison group members were matched on their degree of exposure to those supports in variable (e). Finally, we controlled for prior work history, one of the strongest predictors of future earnings for individuals, directly in the regression model (rather than through matching). This was accomplished through statistical adjustments for group differences in baseline earnings and in proportions of SSA beneficiary types (SSDI, SSI, or dual SSDI/SSI).

There were three steps to the matching process. First, the sampling pool for the comparison group members was identified. For each benefits counseling participant, a list was generated of all non-participant VR/SSA consumers with the same disability category (for the 672 participants, there was a total sampling pool of 4,622 potential VR matches). Second, pairwise matching was done on the basis of VR start date. For the contemporaneous comparison group, each one-to-one match was drawn from the pool by selecting a non-participant VR consumer with the closest VR start date to that of the project participant. For the historical comparison group, each match was drawn from the pool by selecting a non-participant VR consumer with the most similar start date to that of the participant's, but minus 2.5 years in order to draw a VR/SSA sample from an earlier time period (and the earliest for which outcome data were available). Finally, pseudo-enrollment dates were assigned to all comparison group members in a manner to parallel the length of time between the benefits counseling participant's VR start date and that participant's benefits counseling enrollment date. All outcomes were compared on a timescale relative to an individual's enrollment or pseudo-enrollment date. The goal of this three-stage matching process was to control for broad disability type, time of entry into VR services, and maturation in the VR system.

The matching was relatively successful, at least for the contemporaneous group. The average absolute difference in VR start dates (and thus in enrollment/pseudoenrollment dates) between participants and comparison group members was only four days for the contemporaneous group and only five days (plus 2.5 years) for the historical group. Though not a target of the matching, it should be noted that the historical comparison group differed from the other two samples in having a lower proportion of SSDI-only participants, who differ from the other beneficiaries by having the most substantial work histories. The differing SSDI proportion for the historical group appears to be the result of less successful matching as we attempted to compare VR cohorts separated by a two-and-a-half-year time-span. The effects of beneficiary type, however, were controlled for statistically in the regression model rather than through matching. As Table 1 shows, the three study groups were quite similar on the demographic variables of racial minority status, sex, and age.

Measuring Earnings

The primary outcome for this analysis is earnings from work based on administrative records of Vermont's Unemployment Insurance program. This information is submitted by employers to the Vermont Department of Employment and Training as quarterly tax reports subject to State Unemployment Insurance laws and the Federal Employees program. Both public and private sector workers are included in this system. Omissions include earnings from self-employment, out-of-state work, and for the following employee groups: elected officials, nonprofit religious, charitable and educational organizations, unpaid family members, farm workers (with some exceptions), and some railroad employees. (Based on self-reports of working project enrollees, 5% are self-employed and 6% are working out of state, comprising the main categories of non-covered earnings.) While this data set does not include all earnings, it covers the overwhelming majority of wage earnings in Vermont, and is used in this study as an economic indicator variable for group comparisons. The earnings obtained from Unemployment Insurance records are in quarterly increments.

Prior to the analysis, two conversions were applied to the earnings data. First, all monetary amounts were converted to 1997 dollars to adjust for inflation effects across time. Second, the calendar times associated with each earnings record were converted on a person-by-person basis to time relative to the individual's date of enrollment in the benefits counseling program or, for comparison group members, relative to date of pseudo-enrollment. Thus, for an individual with an enrollment or pseudo-enrollment date of May 15, 2000, earnings reports for the 1st, 2nd, and 3rd calendar quarters of 2000 would be translated into reports for the 1st quarter prior to the quarter of enrollment, the quarter of enrollment, and the 1st quarter following the quarter of enrollment, respectively. For the group comparisons, all records for the 1st quarter following the quarter of enrollment were only compared to other records for the 1st quarter following the quarter of enrollment. These two temporal conversions allowed group comparisons of intervention effects over time for a program with rolling enrollments, and an intervention with multiple baselines by individual.

Data Analysis

To analyze the differences in group earnings outcomes, we used mixed-effect linear models (Gibbons & Hedecker, 2000; Jennrich & Schluchter, 1986; Laird & Ware, 1982; Littell et al., 1996). Such models are an extension of traditional regression techniques and are especially suitable for the examination of longitudinal data and multilevel data, where individuals may have begun and ended their participation at different points in time, and where autocorrelation is assumed for repeated measures observations. The outcome variable for the model was earnings. Comparison group (intervention, contemporaneous, or historical) and time (8 baseline quarters and 8 post-enrollment quarters) were treated as explanatory variables, and group-by-time interaction effects (for 3 groups x 16 quarters) were included in the model. Age, sex, disability type (physical, psychiatric, other), and benefit type (SSI, SSDI, and dual eligible) were treated as covariates. Pre-existing differences between the earnings levels of participants and eligible non-participants were taken into account by examining changes in the earnings gaps among groups. That is, the size of any earnings advantage for the intervention group over the comparison groups at baseline was compared to that following intervention. The goal was to determine whether a significant increase in the difference among groups in the outcome variable was associated with the introduction of the intervention.

Results

Figure 1 shows the longitudinal plots of quarterly earnings over four years, two years before and two years after the initiation of benefits counseling, or before and after the pseudo-enrollment date for comparison group participants. The Group 1 (intervention) participants clearly improve after the initiation of benefits counseling. Their average quarterly earnings range between $530 and $660 during the pre-intervention period, and they increase steadily after the intervention begins to $1,049 and $1,102 by the 7th and 8th quarters. Average earnings for Group 2, the contemporaneous comparison group, also appear to improve slightly after the pseudo-enrollment point, finishing at $651 and $653 for the 7th and 8th quarters. Average earnings for Group 3, the historical comparison group, trend upward throughout the four years, increasing from $373 to $589, without an obvious turning point.

[FIGURE 1 OMITTED]

Table 2 shows that virtually all group comparisons favor the benefits counseling group. Most importantly, the pre-vs. post-intervention changes in group means, which are equivalent to group-by-time interaction effects, indicate that Group 1 improves significantly more than Group 2 or Group 3. Thus, even though the three groups start at different levels of earnings and increase over the four years, the improvements for the benefits counseling participants are significantly greater over time than the improvements demonstrated by the two comparison groups. The increase in the earnings gap between the intervention group and the contemporaneous comparison group was an adjusted average of more than $225 per person per quarter, and the increase in the earnings gap between the intervention group and the historical comparison group was an adjusted average of more than $190 per person per quarter. This same pattern of results was obtained in more conservative tests of applying the model separately to the intervention/contemporaneous comparison alone, and then to the intervention/historical comparison alone. (In each of the latter pair-wise comparisons N = 1,344, instead of N = 2,016 for the full model.) When the dataset was split by early enrollees versus later enrollees (first half of the enrollment period versus the latter half), the pattern held for both comparisons, but was statistically non-significant for the historical comparison: for early enrollees, the increases in mean earnings gaps over the contemporaneous and historical comparison groups in the first year after enrollment were $361 (p < .0001) and $296 (p = .0025) respectively, and for the later enrollees, $274 (p = .0010) and $86 (p = .2316) respectively. The adjusted averages include zero earnings for the approximately 50% of intervention recipients who were unemployed, so adjusted average increases for employed individuals only are roughly double those amounts.

The model in Table 2 also shows the effects of a number of significant covariates. Younger participants and males show greater earnings, as do those with physical and other disabilities rather than psychiatric disabilities, and those with SSDI rather than SSI or dual eligibility status.

Discussion

The main finding of this analysis was that significant increases in mean earnings for Social Security disability beneficiaries are associated with the receipt of benefits counseling services, even after controlling for pre-existing earnings advantages, and even after controlling for key demographic predictors of earnings such as age, sex, disability type, and Social Security beneficiary type. The significance of this finding is that benefits counseling as an employment intervention adds value in terms of earnings outcomes for participants, above and beyond whatever increments might be attributed to services from Vocational Rehabilitation and/or the local Community Mental Health Agency. In other words, the earnings of all three groups increased with time and with longer exposure to vocational services, but benefits counseling enrollees achieved a greater increase in earnings in relation to the new intervention.

Benefits counseling is best conceptualized as a gateway service that provides consumers the information and support they need to make use of the available work incentives built into federal and state benefits programs. The experience of the contemporaneous comparison group supports this conceptualization. During the time period the benefits counselors were providing services, SSA improved a number of the work incentives built into the SSI and SSDI programs, primarily as a result of the 1999 Ticket to Work, Work Incentives Improvement Act. While earnings for both the intervention and contemporaneous groups came from the same time period following the introduction of these improvements, the benefits counseling enrollees experienced a much greater increase in earnings. This suggests that benefits counseling was the key service in helping the person with a disability understand and utilize the improved SSA work incentives and therefore increase their earnings.

In examining other predictors, the earnings of women, older persons, and, at least in the overall comparison, persons with a mental illness lag behind those of other individuals, even in the presence of substantial vocational supports. SSDI beneficiaries consistently showed greater earnings than either SSI or Dual-SSDI/SSI beneficiaries. From a purely programmatic perspective, this result would seem counter-intuitive. Taken as a whole, Social Security work incentive rules and programs are more favorable for SSI beneficiaries who want to increase their earnings but don't want to incur a net income penalty. The greater responsiveness of the SSDI-only group to the benefits counseling intervention, however, suggests that the more substantial work histories of these individuals are better predictors of intervention responsiveness and future earnings than are current structural differences in Social Security work incentive regulations toward different types of beneficiaries. SSDI-only beneficiaries as a group appear to offer the greatest potential payoff in terms of increased earnings resulting from the removal of regulatory work disincentives.

This study has a number of limitations. The greatest of these is the lack of random-assignment, which leaves open the possibility of selection bias. (Treatment group members had all voluntarily moved from the nonparticipant comparison pool through self-referral or through referrals from other public service providers.) Given that the nature of the treatment was a substantial and non-blind public benefit, which comparison group members would lack, random assignment was not acceptable to state-level stake-holders in the project. While this study attempted to control for self-selection effects through the use of careful matching and statistical adjustments, such confounds cannot be completely eliminated without random assignment of individuals to treatment and control conditions.

As mentioned earlier, the Social Security Administration introduced a number of new work incentives for disability program recipients during the course of the benefits counseling intervention. While the influences of most of these changes are controlled for in the contemporaneous comparison, one exception is an SSI waiver program introduced in Vermont in April of 2001. This waiver was a work incentive available only to benefits counseling enrollees in Vermont and allowed the retention of earned assets above the usual $2,000 resource limit, greater income retention from certain types of temporary unearned income (e.g., unemployment insurance benefits, worker's compensation benefits, or private disability benefits), and offered a temporary suspension of Continuing Disability Reviews (CDRs) for SSI-only recipients. A possibility exists for some confounding between effects of the benefits counseling intervention and those of the SSI waiver. It is highly unlikely, however, that such a confound is responsible for the results obtained in this study, given that the greatest earnings increases obtained by project enrollees have consistently come from SSDI-only participants.

A final limitation lies in the nature of the earnings data used in this analysis. Vermont Unemployment Insurance records do not include several categories of earnings. Thus, the subset of earnings examined in this analysis, while a large majority of all wage earnings, must be considered a strong economic indicator rather than a precise or complete tally of total wage earnings. In particular, it is possible that the outcome patterns obtained in this study might not hold for out-of-state workers, or more likely, might not hold for the small number of self-employed vocational rehabilitation consumers.

Despite these limitations, this study provided the first empirical evidence that individualized benefits counseling services are associated with significant increases in mean quarterly earnings for Social Security Administration disability beneficiaries. Future research should include larger samples, random assignment, and alternative outcome variables such as public benefits utilization.

Table 1
Comparison of Study Group Characteristics

Sample                      Benefits      Contemporaneous    Historical
Characteristic              Counselees    Comparison         Comparison

N                            672              672             672
Primary VR Disability
  Mental Illness             48%              48%             48%
  Physical Disability        34%              34%             34%
  Other                      18%              18%             18%
Percent SSI                  29%              24%             40%
Percent SSDI                 50%              52%             41%
Percent Dual Beneficiary     21%              24%             19%
Racial Minority              1.5%             3.9%            2.4%
Percent Female               50%              47%             50%
Age in Years                 M = 41           M = 41          M = 40
                             SD = 10          SD = 12         SD = 12

Note. Disability percentages are the same across groups as a
consequence of the matching procedures used.

Table 2
Regression Model Results for Group Differences in Earnings Outcomes.

(N = 2016, or 672 Individuals In Each of 3 Treatment/Comparison
Groups.)

Variables                                        Parameter       S.E.

Age (Grouped by 10-year interval)                -94.87         (8.37)
Gender (Female=1)                                -97.23         (17.83)
Disability Type (Physical vs. Others)             68.23         (26.32)
Disability Type (Mental vs. Others)              -94.38         (24.70)
Benefit Type (Dual vs. SSDI)                     -245.12        (23.32)
Benefit Type (SSI vs. SSDI)                      -263.72        (21.51)
Group (Intervention Group
  vs. Contemp. Group) *                          222.08         (25.55)
Group (Intervention Group
  vs. Historical. Group) *                       314.61         (23.57)
Overall Pre. vs. Post Mean Difference **         -198.26        (19.72)
Pre. vs. Post Mean Difference
  for Intervention Group **                      -338.52        (36.09)
Pre. vs. Post Mean Difference
  for Contemp. Group **                          -110.60        (36.18)
Pre. vs. Post Mean Difference
  for Historical Group **                        -145.66        (29.78)
Intervention Group vs. Contemp. Group
  Mean Diff. at Pre-Period **                    108.12         (26.85)
Intervention Group vs. Contemp. Group
  Mean Diff. at Post-Period **                   336.04         (43.49)
Change in Group (Intervention vs. Contemp.)
  Mean Difference from Pre. to Post **           227.92         (51.12)
Intervention Group vs. Historical. Group
  Mean Diff. at Pre-Period **                    218.18         (29.85)
Intervention Group vs. Historical. Group
  Mean Diff. at Post-Period **                   411.04         (36.25)
Change in Group (Intervention vs. Historical)
  Mean Difference from Pre. to Post **           192.86         (46.79)

Variables                                        Probability

Age (Grouped by 10-year interval)                  <.0001
Gender (Female=1)                                  <.0001
Disability Type (Physical vs. Others)              0.0096
Disability Type (Mental vs. Others)                0.0001
Benefit Type (Dual vs. SSDI)                       <.0001
Benefit Type (SSI vs. SSDI)                        <.0001
Group (Intervention Group
  vs. Contemp. Group) *                            <.0001
Group (Intervention Group
  vs. Historical. Group) *                         <..0001
Overall Pre. vs. Post Mean Difference **           <.0001
Pre. vs. Post Mean Difference
  for Intervention Group **                        <.0001
Pre. vs. Post Mean Difference
  for Contemp. Group **                            0.0022
Pre. vs. Post Mean Difference
  for Historical Group **                          <.0001
Intervention Group vs. Contemp. Group
  Mean Diff. at Pre-Period **                      0.0001
Intervention Group vs. Contemp. Group
  Mean Diff. at Post-Period **                     <.0001
Change in Group (Intervention vs. Contemp.)
  Mean Difference from Pre. to Post **             <.0001
Intervention Group vs. Historical. Group
  Mean Diff. at Pre-Period **                      <.0001
Intervention Group vs. Historical. Group
  Mean Diff. at Post-Period **                     <.0001
Change in Group (Intervention vs. Historical)
  Mean Difference from Pre. to Post **             <.0001

* Group effects were obtained from least square mean comparison in
the same model.

** Parameters were estimated from the same model but separate
estimate statement.

References

Gibbons, R.D., and Hedecker, D. (2000). Applications of mixed-effect models in biostatistics. Sankhya B, 62 (1), 70-103.

Golden, T.P., O'Mara, S., Ferrell, C., and Sheldon, J. (2000). A theoretical construct for benefits planning, assistance and outreach. Ithaca, NY: Cornell University's Program on Employment and Disability.

Golden, T.P., O'Mara, S., Ferrell, C. and Sheldon, J.R. (2001). Benefits planning assistance and outreach: Supporting career development and employment of individuals with disabilities. Spring 2001 edition (SSA Publication 63-003), Ithaca, NY: Cornell University's Program on Employment and Disability.

Jennrich, R.I. and Shluchter, M. (1986). Unbalanced repeated measures models with structured covariance matrices. Biometrics. 42. 805-820.

Johnson-Lamarche, H., and Baird, P. (1997). The barriers to employment faced by persons with disabilities: Problems and solutions (Report to the Governor and the 1997 Vermont General Assembly, January 1997), Waterbury, VT: Vermont Department of Aging and Disabilities.

Laird, N.M., and Ware, H. (1982). Random-effect models for longitudinal data. Biometrics. 38. 963-974.

Littell, R.C, Milliken, G.A., Stroup, W.W, and Wolfinger, R.D. (1996). SAS system for mixed models. Cary, NC: SAS Institute Inc. United States General Accounting Office. (1996). SSA disability: Program redesign necessary to encourage return to work. Report to the Chairman, Special Committee on Aging, United States Senate; GAO/HEHS-96-62, April 24, 1996, Washington, DC: Author.

Author Note

This study was supported by grant funding from the Social Security Administration, the Substance Abuse and Mental Health Services Administration, the Centers for Medicare and Medicaid Services, and the Robert Wood Johnson Foundation. We would like to express thanks to J Morrow of the Vermont Division of Vocational Rehabilitation for preparation of the sample dataset and feedback on design considerations, as well as to the core team of Vermont benefits counselors who made the study possible: Linda Bird, Randy Collyer, Richard Giddings, Alydia Payette, Hans Puck, and Tom Rousseau. Please address correspondence concerning this article, including requests for analysis details, to Time Tremblay.

Tim Tremblay

Vermont Division of Vocational Rehabilitation

James Smith

Vermont Division of Vocational Rehabilitation

Haiyi Xie

Dartmouth College

Robert Drake

Dartmouth College