Printer Friendly

Strengths and challenges of intervention research in vocational rehabilitation: an illustration of agency-university collaboration.

The benefits of intervention research in which new technologies, strategies, or procedures are developed and tested under field conditions have long been heralded in the psychological and rehabilitation research literatures (Frank & Elliott, 2000; Rappaport & Seidman, 2000). The processes, however, can be challenging, as the degree of control researchers have over recruiting participants or introducing the independent variables can be limited by the particular context in which the intervention is introduced (Devine, Wright, & Joyner, 1994).

Many individuals with disabilities rely on Vocational Rehabilitation (VR) services for assistance in attaining relevant employment, educational, and independent living goals. Therefore, VR counselors have a unique opportunity to facilitate active consumer involvement in the rehabilitation process and foster the development of relevant personal competencies (Balcazar & Keys, 1994). Although most professionals working in rehabilitation agree that active consumer participation and collaboration in the rehabilitation process is desirable, there is little agreement on how to best facilitate their involvement and promote independent functioning. Rehabilitation psychologists can play an important role in developing effective intervention strategies to promote consumer involvement. The purpose of the present study was to examine an intervention research process designed to promote goal attainment among VR consumers. The intervention was implemented by VR counselors in collaboration with university researchers.

Tucker, Parker, Parham, Brady, & Brown (1988) argued that meaningful involvement of VR consumers in their own rehabilitation requires counselors to develop empowering relationships with consumers, and reinforcing their personal power to select the vocational goals and services they need. Dowdy (1996) asserted that the purpose of VR services is to empower individuals with disabilities to achieve gainful employment consistent with their strengths, resources, priorities, concerns, and abilities.

Balcazar and Keys (1994) identified goal-setting and help-recruiting skills as two competencies that counselors can help consumers develop to promote and maintain independence and self-reliance. Counselors often find that consumers of rehabilitation services do not have clear vocational goals and struggle to find what it is they really want to do with their lives (Frank & Elliot, 2000; Roessler, 1980). Another common issue is that even when consumers have a clear goal in mind, they do not have a clear understanding of the specific steps they need to take in order to achieve it (Waterman, 1991). Rubin and Roessler (1995) concluded that meaningful participation in the rehabilitation planning process should not only help consumers identify potential goals but also help them evaluate the relevance and practicality of each goal in order to prioritize which goals to pursue. They added that "consumers should also understand what specific counseling and/or training steps are needed to reach their goal(s), and follow through on the plan" (p. 269).

Previous research indicates that goal attainment is optimally realized when consumers are actively involved in the goal-setting process and when goals are defined in concrete terms (Webb & Glueckauf, 1994). Goal setting is necessary to lay the foundation for participation and critical reflection about the direction of rehabilitation efforts (Balcazar & Keys, 1994). In effect, when consumers are actively involved in the goal-setting process, they often take greater control over their rehabilitation and in other aspects of their lives as well (Hope & Rice, 1995). This construct was later supported in a qualitative study (Majumder, Walls, & Fullmer, 1998) in which 104 VR clients were asked about their degree of involvement in the goal-setting process. The researchers reported that consumers with greater involvement appeared to achieve better employment outcomes. However, there is little empirical research in the area of consumer involvement in the goal-setting process in the rehabilitation literature.

Several researchers have pointed out that the ability to recruit help from others and increase self-sufficiency (therefore reducing dependency on VR counselors)--particularly in the context of pursuing complex rehabilitation goals--is critical to success (Fowler & Wadsworth, 1991; Frank & Elliot, 2000). An effective way to promote independence is to teach consumers to seek help from multiple sources rather than just their counselor (Balcazar, Keys, & Garate-Serafini, 1995). Moreover, developing strong help-recruiting skills helps consumers reduce their social isolation and gain access to a vast source of potential support (Balcazar, Fawcett, & Seekins, 1991). Research suggests that consumers' ability to recruit help and maintain an active support network also increases their effectiveness to attain desired goals (Ghikas & Clopper, 2001). Bolton and Akridge (1995) reviewed 15 experimental evaluations of ten different skill-training interventions with VR consumers and found that participants typically receive substantial benefits from the training interventions. In fact, they recommend that such programs become more widely used in rehabilitation facilities.

Empirical research on the evaluation of skill development programs for people with disabilities is more extensive. For example, Balcazar, Fawcett and Seekins (1991) developed a training program that helped college students with physical disabilities develop sound goal-setting and help-recruiting competencies. Results indicated that participants showed consistent improvement in help-recruiting skills and were able to attain 75% of their stated goals as a result of the training. Comparable results were observed in similar studies involving a group of inner-city African-American high-school students (Balcazar, Majors, et al., 1991), a group of adjudicated male adolescents with disabilities (Balcazar, et al., 1995), and a group of low-income Latino and African-American youth with disabilities transitioning from high school (Taylor-Ritzler, Balcazar, Keys, Hayes, Garate-Serafini, and Ryerson-Espino, 2001).

Related to the role of the VR counselor in supporting client progress over time, Rubin and Roessler (1995) noted that consumers often seek services not only to begin or resume a career but also to maintain it. For example, consumers frequently encounter barriers they cannot overcome without help, which often results in failure to secure or retain employment or to accomplish educational goals (Rubin & Roessler, 1995). Gibbs (1990) reported that of 2,500 VR consumers whose cases were successfully closed in the state of Virginia, 25% lost their jobs within 90 days and 85% were unemployed one year after case closure. The role of the VR counselor in providing support and feedback to consumers, as well as helping them solve problems as they arise, is critical. The availability of follow-along support services is important to help consumers identify the accommodations necessary to maintain their goals such as staying in school and remaining employed.

Unfortunately, there is an implicit expectation in the counseling process that consumers are responsible for following through with agreed upon plans, and VR counselors often report that increased caseloads prevent them from keeping regular contact with many of their consumers (Balcazar & Keys, 1994). In fact, Lott, Guarino and Millington (2001) reported that counselor training and on-the-job experience are becoming increasingly irrelevant to rehabilitation outcomes due to counselors' large caseloads which puts tremendous time constraints on the other roles and functions of a rehabilitation counselor's job. Together, these factors often prevent regular ongoing contact with counselors that could aid consumers in overcoming barriers and avoiding major setbacks in attaining and maintaining their goals.

This intervention required VR counselors to help consumers attain their goals using skills development training that emphasized goal setting, action planning, and help-recruiting skills. In addition, a follow-along support procedure was implemented, designed to enhance VR counselors' support of their clients over time. Several hypotheses were tested: First, trained participants will demonstrate more help-recruiting skills than those who do not receive training. Second, participants who receive training and follow-along support from their counselors will experience significantly more goal attainment than participants who receive either training or supervision alone, as well as participants in the active-treatment control group. Third, participants who receive either training or follow-along support from their counselors will attain more goals than participants in the active treatment control group. Fourth, participants who received training and/or follow-up support from their counselors will experience more employment success than the participants in the active-treatment control group.

Method

Participants. Ten VR counselors, one job placement specialist and one office supervisor from six VR district offices in a mid-western state volunteered to participate in this study. They worked in teams of two people per site and were asked to recruit consumers from their regular caseloads to participate in this study. A total of 89 VR consumers were initially recruited to participate in the project and were randomly assigned to four experimental conditions. Fifty-four consumers were available for follow-up contact at least one year (13 to 22 months) after the initiation of the study. Demographic characteristics of the 54 participants are summarized in Table 1.

Chi-square analyses comparing gender, race, living arrangements, type of disability, and marital status of the individuals who dropped from the study (35) and those who participated (54), showed that the two groups were similar on these characteristics. In addition, a one-way analysis of variance indicated that dropouts and participants were of similar age and education backgrounds. Many of the individuals who dropped out did so because they moved to other areas in the state or out-of-state (60%), had lacked interest in participating (25%), or were hospitalized at the time of follow-up data collection (12%). One person committed suicide.

Comparisons of participants' characteristics across the four experimental groups indicated no significant differences among the groups with respect to race, VR case status, age, education level, marital status, living arrangements, source of financial support, or diagnosis. However, the two training groups included a significantly larger number of females (71.4%) compared to males (38.6%)([chi square] = 5.93; p < .01). In addition, a larger proportion of consumers in the sample were unemployed at baseline in the training groups (67%) than the other groups. The 24 participants who were employed at baseline were equally distributed across conditions ([chi square] = 3.64, ns). Finally, there were slightly-but not significantly--more individuals with lower levels of functioning (based on the severity of the disability or presence of multiple disabilities) in the training groups (59%) than in the other two groups (50%).

Dependent Variables

Goal Attainment. A modified version of the Goal Attainment Scale (GAS, Kiresuk & Sherman, 1968) was used to assess participants' progress in reaching their goals. The GAS-which rates goal attainment on a scale from 1 to 5--has been used widely in evaluations of clinical treatment outcomes (e.g., Hogue, 1994), with good concurrent validity and inter-rater reliability (Emmerson & Neely, 1988; Garwick, 1974; Kiresuk, Smith, & Cardillo, 1994). For purposes of this study, the scale was modified to account for progress towards the goal (adding items 3 to 5). The resulting scale had the following categories: (1) deterioration; (2) goal dropped, no intent to pursue it; (3) goal on hold, with future intent to pursue it; (4) active pursuit of goal, little or no progress; (5) active pursuit of goal, some progress; (6) accomplish less than desired goal, action may continue; (7) goal attained as expected, no further actions; and (8) accomplished more than expected success. After members of the research team had scored each participant's progress toward their goals, the VR counselors were asked to confirm the reported progress in order to assess the reliability of the scaling process. The resulting inter-rater reliability for the goal attainment scores was .90.

In order to obtain a goal score for each consumer that could be compared across participants (given differences in types of goals and individual differences in experience, level of functioning, and motivation), the goal attainment scores were weighted using a formula (1). The weights were developed on the basis of 30 VR counselors' opinions regarding the relative importance of three factors: the degree of difficulty of each goal, the subjective importance of the goal to the consumers and the individual level of functioning. The degree of difficulty of each goal was rated by a group of experts on a 3-point scale (relatively easy = 1 point; moderately difficult = 2 points; difficult = 3 points). The goals were sorted into these categories independently of the individuals who set them. The level of importance of each goal was self-determined by the participants also using a 3-point scale (relatively unimportant = 1 point; moderately important = 2 points; and very important = 3 points). Finally, the VR counselors assigned participants to two levels of functioning (high functioning = 1 point; and low functioning = 2 points).

Employment outcomes. The number of individuals per group who were employed at follow-up was identified using individual interviews. The counselors also verified the information provided by the participants regarding their employment status.

Help-recruiting skills. Role plays were used to assess participants' help recruiting skills at the beginning and the end of the intervention. In the role plays, participants were asked to act out a scene with a researcher in which they applied for a job, or asked for financial aid to attend a college or trade school. These role plays were audio taped and scored using a checklist of target behaviors included in the training (see Balcazar et al., 1995 for a detailed list of the behaviors).

Design

This study used a quasi-experimental design where participants were tested prior to participating in the program and again after 22 months to determine whether there were any changes on relevant goal attainment, help-recruiting behaviors and employment outcomes. Two 2x2 factorial designs were used to evaluate the effectiveness of counselor-provided training (yes/no) and counselor-provided follow-along support (yes/no) on goal attainment and employment outcomes. Participants were randomly assigned to one of four experimental conditions: (1) training plus follow-along support from counselors (n = 14), (2) training only (n = 16), (3) follow-along support from counselor only (n = 10), and (4) an active treatment control group who received currently available VR services (n = 14).

Procedures

During meetings at six VR district offices, counselors were invited to participate as on-site trainers for the project. Ten VR Counselors, one Office Supervisor, and a Job Development Specialist volunteered to participate. Counselors were asked to select approximately ten of their consumers to participate in the study. It was recommended that they select consumers who were preparing to look for employment. Consumer recruitment took approximately 6 months. Some of the difficulties in recruiting participants included consumers not being ready for transition to employment, problems in transporting participants to the training sites, and low consumer motivation.

Prior to their initial interview, participants reviewed the objectives of the project and gave their informed consent to participate. During the initial interview, research assistants helped all participants complete a list of personal and/or rehabilitation goals and audio taped two role plays used for baseline assessment of help-recruiting skills. During the closing interview at the end of the project, trained research assistants completed the Goal Attainment Scale based on information provided by each participant. Latter they reviewed the goal attainment scores with the clients' respective counselors. The research assistants also taped two additional role plays during the closing interview to evaluate changes in help-recruiting skills. Each role pay was scored using a checklist that listed the verbal responses that were included in the training. Two independent raters then scored 50(23%) of the total number of role plays (216) to assess the inter-rater reliability of help-recruiting data. The average reliability was 95% (range 85% to 100%).

Training of counselors and consumers. The 12 VR counselors who participated in this study received two 7-hour training sessions on the content of the intervention and the experimental procedures. Counselors were trained to teach their clients how to set goals for themselves, how to develop an action plan, and how to recruit help while pursuing their goals (e.g., learning to handle rejections and looking for alternative sources of help). Counselors also reviewed effective follow-up strategies. The training sessions, which were held at the university and led by the primary author, covered the contents of a training manual (Balcazar, Fawcett, White, & Keys, 1992) and a detailed review of all of the activities to be conducted during the investigation. Following the initial formal training, project staff met with the counselors once a month to review client progress and discuss methods to enhance clients' help-recruiting, action planning and goal pursuit and attainment. University staff members refrained from taking charge of the classes and were present during the training sessions to support counselors as needed. Several troubleshooting sessions with the counselors were held to explore solutions to the barriers expressed by consumers and to encourage them to sustain their efforts.

The consumer training for the intervention was planned and conducted by each counselor in small groups and took approximately 15 hours over a period of about three months. In some cases, the two participating counselors in a field office held joint training sessions with all of their selected consumers. Only the consumers assigned to the "training only" or the "training plus follow-along counselor support" groups participated. A total of 30 consumers were trained. Individual counselors scheduled the training sessions at their convenience and on the basis of consumers' availability (e.g., three times a week for one hour, two times a week for two-hours, etc.). All training sessions were conducted in the offices where the counselors worked.

The content of the goal setting and skills training was conducted in two phases. The first phase included topics like (a) identifying personal strengths (e.g., skills, talents, and experiences) and challenges (e.g., having a disability, being poor and/or a member of a minority group); (b) setting personal goals (e.g., in areas such as employment, education, health, social involvement, and independent living); and (c) developing action plans to pursue the goals. During this phase, a strong emphasis was placed on identifying potential helpers and mentors. The second phase of training concentrated on developing participants' help-recruiting skills, which included steps for opening a meeting with a potential helper, making the request for help, handling rejections, and closing the meeting. The training manual provided the definitions of and rationale for the targeted behaviors, examples of effective help-recruiting behaviors, and multiple role-play exercises. The training sessions were highly interactive. Consumers participated in many role-play scenarios to help them practice their newly learned skills and to retain their interest and attention. They were also encouraged to suggest scenarios based on their personal experiences with vocational, educational, or independent living issues.

Follow-Along Counselor's Support. The 24 participants in the "training plus follow-along support" and "follow-along support only" groups received monthly phone calls from their counselors for a period of about 16 months. In addition, counselors scheduled face-to-face meetings as needed. In the phone calls and meetings, counselors inquired about the participants' progress toward attaining their goals, discussed any current or anticipated difficulties or problems, and celebrated their successes. When consumers reported problems, their counselors discussed additional actions needed to address the issue. The counselors encouraged their consumers to take charge of their own rehabilitation process and also assisted them as requested to help remove unanticipated barriers and provide support and resources (Roessler, 1980).

Active Treatment Control Group. Participants in this condition continued to receive currently available VR services and supports according to their individual rehabilitation programs. They were treated like other consumers in their counselors' workload. It should be noted that participating counselors had an average workload of 95 consumers (range from 80 to 125). Due to the large number of consumers, counselors managed their consumers through a network of vendors and service providers, with little opportunity to actually counsel their consumers and no opportunities for direct training. The consumers' progress was usually monitored through reports from the vendors, with little communication between counselors and consumers.

Results

Table 2 summarizes the paired samples t-test scores for changes in help-recruiting skills before and after the intervention for each of the four experimental groups. As expected, there were significant skill improvements in the groups that received training (p<.001), while the control group remained unchanged. Surprisingly, the follow-up only group, which did not receive training, also showed significant improvements at posttest (t (9) = -2.99, p < .01). Therefore, our first hypothesis was only partially confirmed. In addition, the improvements in all participants' skills, although significant, were below mastery levels. Effect sizes of the help-recruitment skills and goal attainment results were computed by subtracting the average scores of two groups and then dividing the difference by the average of the two groups' standard deviations. Pedhazur and Schmelkin's (1991) recommend this method for computing and interpreting effect sizes. In the area of help-recruitment skills, consumers who only received training services and those who received both training and follow-along support demonstrated comparably large effect sizes (1.2 and 1.1 respectively) in their improvement from pretest to posttest, followed by consumers who received follow-along services only (.57). These results indicate that the training had a strong effect on the help-recruitment skills of VR consumers.

Participants set an average of 3.5 goals (range 3 to 4) in areas like employment, education, health, and social life. The average weighted goal attainment scores showed no significant differences between the groups because all participants were able to attain some of their goals. The cumulative weighted goal attainment scores were used to compare goal attainment among the four groups. These data are summarized in Table 3. A one-way ANOVA performed to compare between group differences yielded a significant treatment effect [F(3,50) = 4.24, p < .01]. Tukey HSD Post Hoc Tests revealed that the group that received both training and follow-along counselor support performed significantly better than the training-only group (Mean difference = 61.56, p < .04) and the active treatment control group (Mean difference = 78.03, p < .008). These findings partially confirmed our second hypothesis, given that the mean difference between the training plus follow-along support and the follow-along support-only group was not significant. The third hypotheses could not be confirmed because the Tukey HSD Post Hoc Tests also showed no significant differences between the training-only group and the control group (Mean difference = 16.47, ns) and the follow-along support and the control group (Mean difference = 28.88, ns).

Our analysis of the effect sizes of the goal attainment results show that when compared to the active control group, there was a meaningful effect of follow-along services (.50) and a moderate effect of training (.29). Follow-along services appear to have a compounding effect when training services were also provided, as indicated by the large effect size (.96) when comparing the group that only received training services to the group that received training and follow-along services. Although not quite as pronounced, the differences in goal attainment by consumers who received follow-along services only and those who received both follow-along services and training was still strong (.79). Our analyses of effect sizes lead us to conclude that while both training and follow-along services are beneficial to consumers, follow-along services are the more critical to success in attaining personal goals.

Table 4 summarizes the employment data for the four groups at baseline and follow-up. The follow-up employment data were collected approximately 22 months after the beginning of the intervention. Given the small sample size and the categorical nature of these data, a non-parametric test (Wilcoxon Signed Ranks Test) was used to determine that overall, there were significant differences in employment before and after the intervention (Z = 3.162, p < .002). The training-only group showed significant employment gains from baseline to follow-up (Z = 2.000, p < .04), while the training plus counselor follow-along support group showed a trend toward improvements in the desired direction (Z = 1.732, p < .08). The groups that did not receive training showed no significant employment gains.

Discussion

This study is one of the first to experimentally examine the long-term effects of an innovative goal-focused, competence-building approach to rehabilitation counseling. It demonstrates the value of direct contact between counselor and consumer, the usefulness of promoting consumer empowerment through competence development and collaborative goal setting and the benefit of collaboration between VR agencies and universities. Overall, the results of this collaboration demonstrated the benefits of the intervention in improving goal attainment for the participants compared with members of the active treatment control group. It should be noted that the group of consumers for whom both training and follow-along counselor's support was provided demonstrated the biggest improvements on the dependent variables. The significant benefits accruing from the training only and the follow-along support conditions tended to be of lesser magnitude.

Although the two groups that participated in the training program showed significant acquisition of behaviors, the group that only received follow-along support also improved significantly. This finding suggests that the counselors' training efforts and their on-going support were beneficial to consumers. Unfortunately, none of the trained participants demonstrated a mastery level acquisition of at least 80% of the behaviors taught. It should be noted that the training process was often challenging for the counselors and participants. While the participants made notable improvements in their help recruiting skills, they may not have had long enough training to master the skills. Transportation to the training sites was also often problematic, since local transportation services were unreliable and in some cases non-existent. Counselors were often frustrated by the irregular attendance of the consumers and the time it took them to complete the curriculum. They also expressed concern about the time involved in preparing and conducting the training sessions.

The attrition rate of participants at follow-up (39% of the original sample dropped out), reduced the group sizes, limiting the power of the data analysis. Although all the groups started with at least 20 individuals, they ended up with only 10 to 16 participants per group. This is a limitation of the study since the small samples make it difficult to generalize findings to a larger population. However, this attrition rate in studies that collect long-term follow-up data is not unusual (Wagner et al., 1991). In fact, very few studies collect long-term follow-up data-over a period of a year or more--on rehabilitation and employment outcomes (Cameto, Marder, Wagner, & Cardoso, 2003; and the report from the Social Security Administration, 2001 are good examples of long-term evaluations). Another aspect of the sample of this study is that it was composed primarily by Caucasian individuals with disabilities from a major urban and suburban area in the Midwest. Although there were likely differences in the service culture of the different offices represented in this study, all counselors were employed by the same state agency. Future studies should include a more diverse sample of both counselors and consumers to examine the effectiveness of this type of training in different social, cultural, geographical, and organizational contexts.

One important issue found while implementing the project was that the supervisors and colleagues did not always support the participating counselors. They were doing much more work with their consumers than other counselors in their office, but received no additional incentives. The only incentive available for the counselors in the system at the time was related to successful closures (status 26, meaning consumers have 90 days of continuous employment), which were related to pay raises and promotions. On the other hand, one counselor indicated that the opportunity to get to know some of her consumers at a personal level allowed her to serve them better. In turn, consumers became more committed and involved in their rehabilitation. Previous studies have advocated for helping consumers develop competencies, allowing them fuller participation and greater control of their rehabilitation process (see Balcazar & Keys, 1994; Roessler, 1980). A cost-benefit analysis of the intervention was not conducted, which could have helped VR administrators with decisions about the replication of similar procedures. Future research could usefully address this issue.

With regard to the attainment of personal and rehabilitation goals, the intervention appeared to focus the participants' efforts on pursuing their goals. Goal attainment was significantly higher for the training plus follow-along support group, compared with the training-only and the active treatment control group. However, it is noteworthy that the members of the active treatment control group did engage in a meaningful level of goal achievement. This degree of goal attainment may have been influenced by the initial goal setting and follow-up reporting on goal attainment with project research assistants as part of the data collection. It may also indicate the positive impact for control group participants of receiving existing supports and services in accordance with their rehabilitation plans. Future research may consider using a waiting list control group to help differentiate the impact of measuring goal attainment from that of receiving the active treatment, i.e., standard VR services.

We did not expect that the follow-along group would outperform the training-only group on the goal attainment measure. The performance of the follow-along group indicated that monthly follow-up contact with a counselor is beneficial in and of itself. In retrospect, given the variety of functions that these follow-up calls can serve, this finding should not be surprising. During these contacts, counselors may help consumers rethink and modify their actions, plan to adjust to changing situations (Balcazar & Keys, 1994), or they may help consumers identify, analyze, and develop plans for the removal of personal or environmental barriers to goal attainment (Balcazar, et al., 1995; Jones, Perkins, & Born, 2001; Roessler, 1980). Clearly, one of the benefits of a research design that includes long-term follow up is the opportunity it presents to assess the impact of ongoing goal-focused contact over time. These findings support recent considerations about the utilization of telehealth systems--employing both computers and/or phone lines to communicate with chronically ill individuals on a regular basis. Telehealth systems can also be useful to professionals by improving communication within a network of service providers (Liss, Glueckauf, & Ecklund-Johnson, 2002).

The findings regarding job placements at follow-up were similarly unexpected. Although the participants who received training were most successful in obtaining employment, the individuals in the active treatment control group also faired relatively well. This finding was particularly surprising since most of the training was focused on the specific skills needed to initiate contact with a potential employer and to present oneself effectively during the interview process. On the other hand, the counselors were competent professionals who were able to achieve some success using standard methods. It is also possible that the counselors transferred some of what they were learning in the training and in the problem solving sessions with their selected consumers to other consumers in their case load. A research design including a control group of consumers receiving training from an untrained counselor could test this possibility.

Another topic for future research is to explore whether consumers who receive training remain employed longer than participants in the active treatment control group. We hypothesize that teaching consumers to improve their help-recruiting skills can increase their independence by expanding the sources of help available to them and increasing their degree of control. A more detailed examination of the consumers' support networks and the way they utilize such support over time should be part of future research in this area (Carling, 1995). Also relevant to assess in future research would be the participants' ability to obtain jobs once they have received training in help recruitment.

This study exemplifies both the strengths and challenges of conducting intervention research in VR settings as an agency-university collaboration. Strengths include the opportunity for evaluating training materials and procedures developed by university staff based on current best-practices and tested under actual field conditions. This evaluation made the critical comparison between current practice and a new way of working with consumers. Then if effective in impact and cost, these methods could be adapted into regular VR practice. Simultaneously, the process had several challenges. For example, participants' recruitment was difficult, took longer than expected and was affected by the characteristics of the caseloads and needs of the counselors. Each counselor made the determination about whom to include in the project. We assigned individuals randomly to the various conditions based on the names forwarded by the counselors. Future studies may want to assign participants to treatment conditions before they are assigned to specific counselors.

Another challenge was the different ways in which the training sessions were conducted. We promoted the uniformity of the training process with common workbooks and other training materials, common training of counselors and staff support. Nonetheless, there was some variability from site to site. Logistical difficulties (e.g., unreliable transportation) led to irregular attendance, often frustrating the counselors. The training process was time consuming, and as mentioned, the VR system in the state did not have an appropriate incentive structure to support counselors' engagement in the training process. We later discussed this issue with state administrators and proposed some policy changes. Unfortunately, limited funding for VR in federal and state budgets has increased the pressure on counselors to do more with less. As VR caseloads continue to increase, services dwindle. Although most aspects of the rehabilitation counselors' role preclude their direct involvement in research, this study suggests that it is not only possible but also desirable to promote such participation. Some of the counselors involved in this study disclosed that the experience allowed them to serve and get to know their clients better. Agency-university collaborations can foster the development of valuable new approaches to service delivery but they are also a challenge to the methodological rigor of the research. We need to promote organizational cultures and incentives to encourage service providers to increase participation in the development and evaluation of innovations. We also need to develop research methods and procedures that are minimally intrusive and cost-effective. This study confirms that there are rehabilitation counselors and university researchers who are willing to collaborate and that such collaborations can yield useful knowledge about the effectiveness of innovative interventions.
Table 1
Demographic Characteristics of the Participants (n = 54)

Gender
 Males 26 (48%)
 Females 28 (52%)
Race
 Caucasian 43 (80%)
 African Americans 8 (14%)
 Hispanic 2 (4%)
 Asian 1 (2%)
Education
 Less than High school 9 (17%)
 High school or GED 34 (63%)
 Post-Secondary 11 (20%)
Diagnosis
 Developmental Disabilities 10 (19%)
 Learning Disabilities 17 (31%)
 Physical Disabilities 19 (35%)
 Psychiatric Disabilities 8 (15%)
Age Range
 15 to 47 years (X = 29.6)
Marital Status
 Single 42 (78%)
 Married 5 (9%)
 Other 7 (13%)
Living Arrangements
 With parents 41 (76%)
 Spouse/Partner 7 (13%)
 Alone 6 (11%)
Source of Financial Support
 Family/Friends 30 (55%)
 Welfare/SSDI 13 (24%)
 Job 9 (17%)
 Combination 2 (4%)
 Employment Status at Baseline
 Unemployed 30 (55%)
 Employed 24 (45%
Area of Residence
 Large City 23 (43%)
 Suburb of a large city 25 (46%)
 Rural area 6 (11%)

Table 2
Paired Samples t-tests for Changes in Help-Recruiting
Skills Before and After the Intervention by Group

 Std.
 Error
Pairs N Means SD Mean

1. Training + Follow-up Pre 14 29.12 8.48 2.27
 Post 39.77 11.87 3.17

2. Training only Pre 16 26.21 11.16 2.79
 Post 39.75 11.11 2.78

3. Follow-up only Pre 10 32.58 13.49 4.27
 Post 41.30 17.30 5.47

4. Control Pre 14 31.67 8.45 2.26
 Post 31.35 8.85 2.36

All groups Combined Pre 54 29.10 11.11 1.21
 Post 37.86 12.45 1.69

 Sig.
Pairs t df (2-tailed)

1. Training + Follow-up Pre 4.08 13 .001
 Post

2. Training only Pre 4.06 15 .001
 Post

3. Follow-up only Pre 2.99 9 .01
 Post

4. Control Pre 0.13 13 ns
 Post

All groups Combined Pre 5.20 53 .001
 Post

Table 3
Cumulative Weighted Goal Attainment Scores by Group

 Training

 Yes No

 Yes Mean: 250.25 1/2 201.10
Follow-Along SD: (69.49) (60.27)
Support 188.68 (2) 172.21 (1)
 No (59.19) (55.46)

(1) p < .01 (using the Tukey HSD test)

(2) p < .05

Table 4
Employment Status of Participants by Group

Group Participants' Employment Status

 Before the Intervention

 Employed / Unemployed

Training plus 5 (36%) (3) 9
Follow-along
Support

Training Only 5 (31%) (2) 11

Follow-along 7 (70%) 3
Support Only

Active Treatment 7 (50%) 7
Control Group

TOTALS: 24% (45%) (1) 30

Group Participants' Employment Status

 At Follow-up

 Employed / Unemployed N

Training plus 8 (57%) (3) 6 14
Follow-along
Support

Training Only 9 (56%) (2) 7 16

Follow-along 8 (80%) 2 10
Support Only

Active Treatment 9 (64%) 5 14
Control Group

TOTALS: 34 (63%) (1) 20 54

(1) p < .002 (using the Wilcoxon Signed Ranks Test)

(2) p < .04

(3) p < .08


(1) Weighted Goal Attainment Score = Goal attainment score x [1.5(degree of difficulty score + level of importance score) + 2.5 (level of functioning score)].

References

Balcazar, F. E., Fawcett, S. B., & Seekins, T. (1991). Teaching people with disabilities to recruit help to attain personal goals. Rehabilitation Psychology, 36, 31-42.

Balcazar, F. E., Fawcett, S. B., White, G. W., & Keys, C. B. (1992). The mentoring handbook: Setting goals and getting support from mentors. 3rd ed. Lawrence, KS: Research and Training Center on Independent Living, University of Kansas.

Balcazar, F. E., & Keys, C. B. (1994). Teaching self-help to vocational rehabilitation clients. Directions in Rehabilitation Counseling, 5, 3-11.

Balcazar, F. E., Keys, C. B., Garate-Serafini, J. (1995). Learning to recruit assistance to attain transition goals: A program for adjudicated youth with disabilities. Remedial and Special Education, 16, 237-246.

Balcazar, F. E., Majors, R., Blanchard, K. A., Paine, A., Suarez-Balcazar, Y., Fawcett, S. B., Murphy, R., & Meyer, J. (1991). Teaching minority high-school students to recruit helpers to attain personal goals. Journal of Behavioral Education, 1, 445-454.

Bolton, B. & Akridge, R. L. (1995). A meta-analysis of skills training programs for rehabilitation clients. Rehabilitation Counseling Bulleting, 38, 262-273.

Cameto, R., Marder, C., Wagner, M., & Cardoso, D. (2003, December). Youth Employment. Data Brief from the National Longitudinal Transition Study, Vol 2, 2.

Carling, P. J. (1995). Return to community: Building support systems for people with psychiatric disabilities. New York: The Guilford Press.

Devine, J., Wright, J., & Joyner, L. (1994). Issues in implementing a randomized experiment in a field setting. In K. J. Conrad (Ed.), Critically evaluating the role of experiments. New Directions for Program Evaluation, 63, 27-40.

Dowdy, C. A. (1996). Vocational rehabilitation and special education: Partners in transition for individuals with disabilities. Journal of Leaning Disabilities, 29, 137-147.

Dunn, H. E., & Herman, S. H. (1982). Social skills and physical disability. In Daniel M. Doleys, R. L. Meredith, & Antony R. Ciminero (Eds.), Behavioral Medicine: Assessment and treatment strategies. New York: Plenum Press.

Emmerson, G. J., & Neely, M.A. (1988). Two adaptable, valid, and reliable data collection measures: Goal attainment scaling and the semantic differential. The Counseling Psychologist, 16, 261-271.

Frank, R. G. & Elliot, T. R. (2000). Handbook of Rehabilitation psychology. Washington DC: American Psychological Association.

Fowler, C. & Wadsworth, J. (1991). Individualism and equality: Critical values in North American culture and the impact on disability. Journal of Applied Rehabilitation Counseling, 22, 19-23.

Garwick, G. (1974). A construct validity overview of goal attainment scaling (ProgramEvaluation Project Report, 1969-1973). Minneapolis, MN.

Ghikas, P. A. & Clopper, M (2001). Case studies in rehabilitation. Thorofare, NJ: Slack.

Gibbs, J. (1990). Alternative measures to evaluate the impact of vocational rehabilitation services. Rehabilitation Counseling Bulletin, 34, 33-43.

Gosse, V. F., Sullivan, A. M., Ross, A. S., Simmonds, A. J. et al. (1992). Evaluation of the Goal Oriented Adult Learning (GOAL) program. Psychosocial Rehabilitation Journal, 15, 97-100.

Hartman, D., Borrie, M., J., Davison, E., & Stolee, P. (1997). Use of goal attainment scaling in a dementia special carre unit. American Journal of Alzheimer's Disease, 12, 111-116.

Hogue, T. E. (1994). Goal attainment scaling: A measure of clinical impact and risk assessment. Issues in Criminological & Legal Psychology, 21, 96-102.

Hope, R. C. & Rice, B. D. (1995). Report from the study group on strategies to enhance job retention and career advancement in rehabilitation. Hot Springs, AR: Arkansas Research and Training Center in Vocational Rehabilitation, University of Arkansas.

Jones, C., Perkins, D. V., & Born, D. (2001). Predicting work outcomes and service use in supported employment services for persons with psychiatric disabilities. Psychiatric Rehabilitation Journal, 25, 53-59.

Kiresuk, T. J. & Sherman, R. E. (1968). Goal attainment scaling: A general method for evaluating comprehensive community mental health programs. Community Mental Health Journal, 4, 443-453.

Kiresuk, T. J., Smith, A. & Cardillo, J. E. (1994). Goal attainment scaling: Applications, theory and measurement. Hillsdale, NJ: L. Erlbaum Associates.

Kiresuk, T. J., Smith, A. & Cardillo, J. E. (1994). Evaluating comprehensive community mental health programs. Community Mental Health Journal, 4, 443-453.

Liss, H. J., Glueckauf, R. L. & Ecklund-Johnson, E. P. (2002). Research on Telehealth and chronic medical conditions: Critical review, key issues, and future directions. Rehabilitation Psychology, 47, 8-30.

Lott, A. C., Guarino, A. J., & Millington, M. J. (2001, August). Education Level and Successful outcome in rehabilitation counseling. Poster presented at the annual convention of the American Psychological Association, San Francisco, CA.

MacKay, G., & Lundie, J. (1998). GAS released again: Proposals for the development of goal attainment scaling. International Journal of Disability, Development & Education, 45, 217-231.

Majumder, R. K., Walls, R. T., & Fullmer, S. L. (1998). Rehabilitation client involvement in employment decisions. Rehabilitation Counseling Bulleting, 42, 162-173.

Pedhazur, E. J. & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Rappaport, J. & Seidman, E. (2000). Handbook of community psychology. New York: Plenum.

Riggar, T. F., & Patrick, D. (1984). Case management and administration. Journal of Applied Rehabilitation Counseling, 15, 29-33.

Rubin, S. E. & Roessler, R. T. (1995). Foundations of the Vocational Rehabilitation Process, Fourth edition. Pro-ed, Austin, Texas.

Roessler, R. T. (1980). Factors affecting client achievement of rehabilitation goals. Journal of Applied Rehabilitation Counseling, 11, 169-172.

Social Security Administration (2001, January). Income of disabled worker beneficiaries. Office of Policy, Research, Evaluation and Statistics, SSA, Washington DC. (No.13-11706).

Taylor-Ritzler, T., Balcazar, F., Keys, C., Hayes, E., Garate-Serafini, T., & Ryerson-Espino, S. (2001). Promoting attainment of transition-related goals among low-income ethnic minority students with disabilities. Career Development for Exceptional Individuals, (24) 147-168.

Tucker, C., Parker, J., Parham, G., Brady, B., & Brown, J. (1988). Perceived importance of counselor characteristics among vocational rehabilitation counselors and clients. Journal of Applied Rehabilitation Counseling, 19, 28-31.

Wagner, M., Newman, L., D'Amico, R., Jay, E. D., Butler-Nalin, P., Marder, C., et al. (1991). Youth with disabilities: How are they doing? The first comprehensive report from the national longitudinal transition study of special education students. SRI International. Washington, DC.: US department of Education, Office of Special Education Program.

Waterman, J. (1991). Career and life planning: A personal gyroscope in times of change. In J. Kummerow (Ed.), New directions in career planning and the workplace. Palo alto, CA: Consulting Psychologist Press.

Webb, P. E., & Glueckauf, R. L. (1994). The effects of direct involvement in goal-setting on rehabilitation outcome for persons with traumatic brain injuries. Rehabilitation Psychology, 39, 179-188.

Wehman, P. & Moon, M. S. (1988). Vocational rehabilitation and supported employment. Baltimore: Paul Brookes Publishing Company.

Author Notes

This project was funded in part by a sub-contract from the University of Kansas, with a grant from the Rehabilitation Services Administration, US Department of Education (Grant Award #H-128A-02004). The opinions expressed here are those of the authors and not the US Department of Education. Special thanks to Richard Meldrum and to Drs. Stephen B. Fawcett and Glen White for their support in the implementation of this contract.

Fabricio E. Balcazar

University of Illinois

Cecile Lardon

University of Alaska Fairbanks

Christopher B. Keys

DePaul University

Curtis Jones

University of Illinois at Chicago

Margaret Davis

DePaul University

Fabricio E. Balcazar, Department of Disability and Human Development, University of Illinois at Chicago, 1640 West Roosevelt Road, Chicago IL, 60608. Email fabricio@uic.edu
COPYRIGHT 2005 National Rehabilitation Association
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2005, Gale Group. All rights reserved. Gale Group is a Thomson Corporation Company.

Article Details
Printer friendly Cite/link Email Feedback
Author:Davis, Margaret
Publication:The Journal of Rehabilitation
Geographic Code:1USA
Date:Apr 1, 2005
Words:7167
Previous Article:Barriers to the accommodation request process of the Americans With Disabilities Act.
Next Article:Striking Back at Stroke: A Doctor-Patient Journal.
Topics:


Related Articles
Work satisfactoriness of former clients with severe handicaps to employment.
Vocational and medical rehabilitation: the impact of health care policy and funding on service provision.
Rural and urban vocational rehabilitation: counselors perceived strengths and problems.
Organizational transformation: a bold journey, not a guided tour.
An Individualized Job Engagement Approach for Persons with Severe Mental Illness.
Trends in vocational rehabilitation: 1978, 1988, 1998.
Assessing readiness of clients with disabilities to engage in job seeking behaviors. (Readiness of Clients).
Person-centered planning: a gateway to improving vocational rehabilitation services for culturally diverse individuals with disabilities. (Person...
The role and function of certified vocational evaluation specialists.
A synergistic program evaluation model partnership between a state agency and a university program.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters