A meta-analysis of rehabilitation outcomes based on education level of the counselor.
Evaluation of counselor performance in the state and federal vocational rehabilitation programs continues to use the number of closures and employment status at closure as two powerful outcome measures (e.g. Estrada-Hernandez & Wadsworth, 2005). This practice continues despite concerns about the limitations expressed by numerous authors regarding the use of status 26 closures as a primary means of evaluation (e.g. Freedman & Fesko, 1996; Gilbride, Thomas, & Stensrud, 1998). Therefore researchers often look at other issues related to employment outcome, such at expenditures per closure (Szymanski & Danek, 1992), the time spent providing services prior to placement in employment (Wheaton & Berven, 1994), or the number of individuals with severe disabilities successfully rehabilitated (Szymanski & Danek, 1992; Wheaton & Berven, 1994). It may be that other results are important for evaluating counselor performance, as suggested by the 5 factor counselor performance measures of Phillips, Butler and Thomas (1988) or the client quality of life perspective of Bishop and Feist-Price (2002); however, legislation, advocacy groups, and the rehabilitation field have all recognized employment as a crucial if not the sole outcome of the rehabilitation relationship. No matter how outcomes are measured, the people rehabilitation are accountable to are asking for better overall outcomes for individuals with disabilities; therefore the field has been asked to empirically determine the value of the education level of rehabilitation counselors (Bolton, 1990; Millington, Miller, Asner-Self & Linkowski, 2003). This study used meta-analysis to examine the differences in employment outcomes for clients based on the educational level of the counselor. Findings may help establish a minimum training level for rehabilitation counselors in the state and federal system.
The National Council on Rehabilitation Education (NCRE) has explicitly sought to document the impact of rehabilitation education on improving the quality of services to persons with disabilities (Bolton, 1990). Quality has many definitions in rehabilitation; ideally, rehabilitation seeks to discover which treatment, given by whom, is most effective under what set of circumstances for a particular individual with a specific problem, and how that treatment came about (Paul, 1969). However, as Cronbach and Snow (1977) eloquently stated in their discussion of interactions, it is nearly impossible to answer all of these implied questions in a single set of studies. Each separate question must be addressed empirically in isolation of the others. This study attempts to answer the "by whom?" question.
The debate on effectiveness of psychological intervention can be traced back prior to the 1954 amendments. The value of professional training was questioned in Eysenck's (1952) landmark study, which implied that severe neurotic patients recovered better spontaneously than they did when receiving systematic psychotherapy. This argument moved into the rehabilitation field with Truax and Lister's (1970) series of articles that supported the hypothesis that counselor aides were at least as effective (and seemingly more so) than those with master's degrees in rehabilitation when working with vocational clients in a rehabilitation center. In the counseling field, Durlak (1979) showed similar findings in comparing studies evaluating professional therapists with paraprofessionals with no formal training. In the rehabilitation field, Stubbin's (1982) study suggested that counselor's with bachelor's degrees rehabilitate people with disabilities just as well as do master's level counselors.
The concern about the quality of rehabilitation services, has been reflected in numerous legislative pieces, including the 1984 Rehabilitation Amendments (EL. 99-221) and the 1990 Americans with Disability Act (EL. 101-336). Implicit in the law is the need to determine empirically the extent to which competent individuals, appropriately trained and certified for the positions they hold, are practicing in the rehabilitation service delivery system (Sharpson, Wright & Leahy, 1987). This emphasis on empirically based competence began as a result of findings as early as 1966 when Muthard, Miller and Obermann reported that less than 30% of practicing rehabilitation counselors could be identified as fully qualified by academic training and experience (Shapson, Wright & Leahy, 1987). This concern continues today as indicated by 2002 Office of Special Education and Rehabilitative Services (OSERS) statistics which indicate that almost half (47.4%) of new rehabilitation counselors hired by states do not meet the Comprehensive System of Personnel Development (CSPD) minimum qualification standards for that state (Chan, 2004). Such low percentages of qualified rehabilitation workers continues to be a concern in many states and settings, as it appears the numerous expected openings for rehabilitation counselors will be filled by individuals who do not meet the minimal qualifications to be considered "qualified rehabilitation professionals".
The large percentage of practicing rehabilitation counselors without the highest standard of rehabilitation education can be seen despite the legislative mandates of the 1992 Rehabilitation Act amendments. Many states minimum qualification standards do not require new rehabilitation counselors to possess Certified Rehabilitation Counselor (CRC) status or even a master's degree. While many states have plans in place to become compliant with the legislative mandate, there are still many states only requiring a bachelor's degree (Chapin, 2004).
Indeed the most recent published survey of states indicated that only 13 state CSPD qualification standards required a master's degree as the minimal hiring requirement for rehabilitation counselors, and none required counselors to possess CRC certification (Froehlich, Garcia, & Linkowski, 1998). In addition to stated hiring practice, almost half of those hired do not meet the minimum standards set by the state (Chan, 2003). State and Federal agencies as well as the private sector apparently make numerous exceptions concerning qualified rehabilitation professionals, such as counselors who have specialized caseloads, speak another language or have other skills that allow them to be hired without a master's degree in rehabilitation or a related field (Flowers, Strong, Turner, Moore & Edwards, 1998). The lack of value associated with the master's degree can also be seen in rehabilitation settings such as substance abuse counseling and forensic counseling employment situations wherein the bachelor's degree is often the minimum requirement (Chapin, 2004).
This trend to hire counselors without a master's degree in rehabilitation is surprising given that graduates of master's in rehabilitation counseling (MRC) programs generally reported higher skill-attainment levels than practitioners with other types of training (Wright, Leahy & Riedesel, 1987). Further, McCarthy and Leierer (2001) found rehabilitation client's greatest concern regarding their rehabilitation counselor was the counselor's credentials and educational achievement. Additionally, a series of articles by Szymanski and colleagues (1989a, 1989b, 1991, & 1992) appeared to show that a higher level of rehabilitation education improves employment outcomes for clients with disabilities. These findings helped push the rehabilitation field into legislation, calling for only "qualified rehabilitation professionals" to serve those with disabilities being included in the 1992 Rehabilitation Act Amendments. Put together, there appears to be no question in regard to who is the most qualified to provide rehabilitation services to persons with disabilities. However, a close look at outcome studies indicates that the individual comparisons between the education levels of the counselor did not empirically support the superiority of the master's degree in rehabilitation.
The Szymanski and Danek (1992) study in particular has been one which others have cited as demonstrating that counselors with master's degrees have better outcomes than those with bachelor degrees. A closer look at this study, along with the others that have been cited as indicating replication (Cook & Bolton, 1992; Szymanski & Parker, 1989a) finds that the study does not show statistical significance for employment outcomes for clients with or without severe disabilities. Remarkably, the Szymanski and Danek (1992) statistics when broken down indicate a higher average closure rate for clients who have severe disabilities for both counselors with related master's degree and counselors with bachelors degree over those counselors who had a master's in rehabilitation. The 30 counselor's with a master's in rehabilitation in the study had a mean of 12.67 (5.28) competitive closures as compared with 16.11 (7.40) competitive closures for those counselors with a related master's and 13.07 (7.31) for counselors with a bachelors or unrelated master's. Competitive closures for the three groups of counselors for clients that did not have severe disabilities were statistically insignificant, although slightly favoring those with a master's in rehabilitation (with means of 9.86 vs. 9.38 vs. 9.75 competitive closures respectively).
Cook and Bolton (1992) and Szymanski and Parker (1989a), while sometimes having slightly different comparisons such as master's in rehabilitation and related master's combined, also failed to find statistical significant differences between the groups. The articles generally argued for case closure rate percentage as a key variable, suggesting that the percentage of closed cases that were closed as competitive is a more appropriate measure of counselor success. While this may be the case for specialized caseloads or if comparing across states, it does not seem to be the way most rehabilitation counselors or programs are judged. The other argument set forth by these articles was the aptitude treatment interaction (ATI) design that showed those with a master's did better in the early years (the first three) of employment compared to other degrees and worse in later years (more than 14) of employment. These findings were again based on percentage of competitive closures as a function of total closures. Others (e.g. Bolton, 1990) have suggested that these articles showed the benefits of a counselor having a master's in rehabilitation and pushed the wording of qualified rehabilitation professional into the Rehabilitation Act Amendments of 1992. Qualified rehabilitation professional is generally defined as a counselor with a master's degree in rehabilitation counseling. It appears the benefit of the degree has yet to be proven empirically. Indeed when the results for these studies are examined, it can be seen in Table 1 that none of the classic studies (Cook & Bolton, 1992; Szymanski & Danek, 1992; Szymanski & Parker, 1989a) effect sizes achieved statistical significance as the confidence interval contains zero for each of the studies.
In reality it appears these articles should have cast doubt upon who is the most qualified rehabilitation professional. Observers wishing to make an argument against rehabilitation master's degree have plenty of fodder from these studies. Indeed many appear to have not put much faith in the studies as can be seen by the relative few states that have changed their entry level hiring requirements and practices for rehabilitation counselors to one that requires a master's in rehabilitation. Perhaps, silently, many states and legislative bodies have not pushed for a minimum hiring requirement because they do not feel there is a great enough value in the degree to justify the higher costs that would be associated with hiring only master's level counselors as entry level rehabilitation counselors.
The rehabilitation field recognizes that outcome research can show differences effectively, so the search for the most effective method for obtaining accurate outcome results has become paramount. McAweeney, Forchheimer, and Tate (1997) examined the most effective way to test phenomenon in rehabilitation. They concluded that in this era, when outcomes are critical, researchers in rehabilitation cannot afford to conduct studies wherein significant methodological short-comings compromise the validity of the results. Their conclusion was that, while both null hypotheses testing and assessing statistical power have their place in research, rehabilitation researchers should focus on the calculation and interpretation of statistical power and effect size. This sentiment has been echoed by others in the field (e.g. Lustig & Strauser, 2005).
Following the referendum of McAweeney et al. (1997), the present study examines effect sizes and statistical power in findings of outcome research focusing on differences in rehabilitation counselor's educational degree. These differences are found using meta-analytic techniques first described by Smith and Glass (1977.
Meta-analysis or research synthesis is a quantitative method for aggregating similar studies in order to test hypotheses (Wampold, 2001). The results of a meta-analysis embody an overall estimate of effectiveness, as in the case of rehabilitation counselors assisting rehabilitation clients into employment. The estimate or effect magnitude signifies a standard measure that becomes a generalizable indicator of effectiveness of a particular degree.
This meta-analysis was designed to examine effects of education level on rehabilitation counselors' successful employment outcome variables. Journal articles from 1980 to 2004 were reviewed to find studies that explored this issue. The Journal of Applied Rehabilitation Counseling, the Journal of Rehabilitation, the Journal of Rehabilitation Administration, Rehabilitation Counseling Bulletin, and Rehabilitation Education were included in the analysis on the assumption that they were likely to contain relevant research on persons with disabilities and their outcomes in a rehabilitation setting. To identify studies for this meta-analysis, the first two authors examined every study published in the included journals, and conducted a standard WebSPIRS 5.0 online literature search with selected databases. To be included in this meta-analysis, a study had to (a) involve a population of individuals with disabilities, (b) be an employment outcome based measurement, (c) differentiate outcomes for individuals based on the educational degree of their counselor and (d) contain the necessary statistics to conduct a meta-analysis. The determination of appropriate outcomes was at the discretion of the literature.
This search uncovered nine studies that dealt with outcomes and the degree of the rehabilitation counselor. Some of these studies were not usable in this meta analysis because they did not address the question empirically (e.g. Stubbins, 1982) or because the study combined groups such as those with master's in rehabilitation and those with related master's (e.g. Cook & Bolton, 1992). There remained only three relevant studies that included the necessary statistics, the search then expanded to dissertations and other journals for studies. The social science citation index was then used by tracking all citations of the nine articles selected with the belief that any studies on this topic would reference one or more of these articles. However, no further appropriate studies were found.
The dearth of outcome-based studies made the use of meta-analytic techniques more difficult. This difficulty was due in part to the lack of meta-analysis-friendly data in studies that distinguished between levels of qualification of the rehabilitation professional. Attempts were made to make such data more interpretable. This process involved one case of unclustering the results of a cluster analysis (Wheaton & Berven, 1994) and other cases interpreting tables and graphs included in the published studies to identify means and standard deviations not fully included in the text (Szymanski & Parker, 1989a). In cases of deciphering graph results, two raters independently scored the graphs and identified means and standard deviations. Where large discrepancies occurred, any difference of more than one data point, a third rater was used. The averages of rater's scores were then used as means and standard deviations for those studies. The resulting meta-analysis then had three studies, with eleven comparisons, to answer the question concerning differences based on the education level of the rehabilitation counselor.
The meta-analysis was conducted by using the mean differences between rehabilitation counselors with master's degrees in rehabilitation and rehabilitation counselors with any other type of degree. For each study an estimated effect size as well as an estimate of the variance of this estimate was calculated on each dependent variable related to employment outcome (e.g. number of closures by the counselor in a year, time to reach closure, expenditures per closure). Dependent variables for the studies that did not address outcomes (e.g. caseload size) were not included in the analysis. A sample effect size for each dependent variable was obtained by calculating the difference in the means of those with a master's in rehabilitation as compared to those without. An unbiased estimate of the effect size and the standard error were then calculated (Ahn & Wampold, 2001; Hedges & Olkin, 1985). Next, to calculate a single effect size for each study, the weighted effect size for each dependent variable was combined (Wampold, 2001). The aggregate effect size for all the studies were then combined after being weighted by the inverse of the variance, this yielded the aggregated effect size estimate of d (Ahn & Wampold, 1997; Hedges & Olkin, 1985). The variance and confidence intervals were also calculated using procedures described by Wampold (2001). Finally, a test for homogeneity was conducted to determine whether the effect sizes all estimate the same population (Lipsey & Wilson, 2001).
The meta-analysis compared master's degrees in rehabilitation counseling to the combined group of all other degrees, which included bachelor's degrees and master's degrees in related and unrelated fields. Meta-analysis reveals that the overall influence of education on rehabilitation outcomes (Table 1) is significant. Level of education's overall research synthesis produced an effect size of +0.20 with a variance of 0.0040. The 95% confidence interval [(d 1.96 SD(d)] of lower (0.07) and upper bounds (0.32) does not contain a zero, indicating this effect size is statistically significant at the .05 level. Thus, the null hypothesis that there is no difference based on education level can be rejected.
One of the criticisms of meta-analysis is that aggregation occurs over studies that may be dissimilar in many different ways. Consequently, it is argued that the studies are not answering the same research question, in that they are not estimating a common population parameter (Wampold, 2001). This question of whether the studies are similar can be answered empirically by using a test of homogeneity: the test statistic Q-value (Hedges & Olkin, 1985). Comparing the Q value with a chi-squared distribution with k-1 degrees of freedom (df= N- 1) allows one to arrive at a decision as to whether homogeneity exists. The research synthesis of education level for the present meta-analysis resulted in homogeneity. The Q-value of 2.08 compared to [chi square](3) = 7.81 ([alpha] = 0.05) resulted in a decision of failure to reject the homogeneity hypothesis. Therefore, homogeneity or a common population parameter is determined. The studies in the present meta-analysis are comparing similar populations thus allowing the groups to be studied together.
A breakdown of this meta-analysis can be conducted to compare counselors with different types of degrees. Three comparisons, based on the literature, were done, rehabilitation counselors with master's degrees in rehabilitation counseling, were compared against both those with related master's degrees and against rehabilitation counselors with either undergraduate degrees, or master's degrees unrelated to rehabilitation. Finally, rehabilitation counselors with master's degrees in rehabilitation counseling, or related masters, together were compared against counselors with bachelor or unrelated master's degrees. Table 2 shows these individual comparisons. Only one of the three comparisons proved statistically significant: the comparison of the master's in rehabilitation counseling versus the group of unrelated master's and bachelors degrees. The effect size, d, of .29 shows a small effect size, which accounts for close to 2% of the variance in outcomes.
Surprisingly, the comparison of rehabilitation counselors with master's degrees in rehabilitation and those with related master's versus the group of those with either bachelor's or unrelated master's degrees (d =. 19) is not significant and explains just under 1% of the variance in outcomes for clients between these two groups of rehabilitation counselors. The other comparison which did not show significance was that for rehabilitation counselors possessing master's degrees in rehabilitation versus those with master's degrees in related fields, while not statistically significant, the effect size of. 13 indicates clients are better off with counselors with master's in rehabilitation counseling rather than other master degrees.
This meta-analysis indicates that clients with rehabilitation counselors with master's degrees in rehabilitation counseling have better employment outcome results than do clients with rehabilitation counselors with other degrees. The current results indicate that both Truax and Listers' (1970) and Stubbins' (1982) call for paraprofessionals to serve as entry-level workers with clients with disabilities would be detrimental in terms of desired outcomes. This analysis also supports the continued inclusion of those with a master's degree in rehabilitation into legislation that calls for the most qualified individuals to serve persons with disabilities.
A better understanding of how this translates into practical outcomes is needed; therefore while statistical significance is meaningful, the practical significance of the effect size of education level is more important. Cohen (1988) suggested the way to interpret the size of the effect involves comparing it with a benchmark, specified by the following standards:</p> <pre> 1. Large effect: 0.80 2. Medium effect: 0.50 3. Small effect:
0.20 </pre> <p>The present effect size (ES) magnitude of 0.20 would be classified as a small effect. The finding of an effect size equal to .20 indicates that educational level of the rehabilitation counselor accounts for about 1% of the variance in employment outcomes for people with disabilities, using r =E[S.sup.2]/E[S.sup2] +4, (Lipsey & Wilson, 2001). It is seen to be about the average effect size when compared to other clinical psychology effects (Cooper, 1997). What this analysis does show is the education attained by their rehabilitation counselors can explain a statistically significant amount of the variability in why people with disabilities have successful outcomes.
Another way to interpret effect size magnitude is to examine the overlap of the control and treatment distributions. The master's degree in rehabilitation counseling was viewed as the treatment while other education degrees were viewed as controls. The effect size magnitude represents the mean of the treatment group in the control group distribution (Smith & Glass, 1977) using a binomial effect size display (Rosenthal & Rubin, 1982). The cumulative normal distribution of the effect magnitude represents the proportion of the control group population who are worse off than the average person in the treatment population. The value of the standard normal cumulative distribution for the present aggregated effect size magnitude (0.20) is 0.58. In other words, the average client with a counselor who has a master's in rehabilitation is more successful than 58% of those clients who have a counselor with another degree.
Often the statistical analysis may be the reason why research is not used in practice. The landmark studies on the education level of the rehabilitation counselors all used difficult concepts to comprehend. ATI design,. 10 significance level and y intercepts based on the experience level of the counselor are innovative and empirically solid designs, however they may not translate well into practical significance. This meta-analysis indicates an effect size of .20 which may be viewed as two tenths of a standard deviation. The standard deviation on the number of employment closures per year for a counselor is about 17 (pooled SD =17.11) according to Cook and Bolton (1992). The effect size of .20 is equivalent to a z score of .5793 according to standard z-score tables of standard distributions (e.g. Aiken, 2003). Using this z-score and the standard deviation of 17 closures it can be calculated that the average counselor with a master's degree in rehabilitation will have about 4 (3.97) more closures each year as compared to counselors who do not have a master's in rehabilitation counseling. According to the OSERS there were 11,755 state rehabilitation counselors in 2003. The studies in this meta-analysis, along with Chan (2004), suggest that approximately 50 to 70% of state rehabilitation counselors do not have a master's degree in rehabilitation. Using the conservative estimate of 50%, this study suggests that an additional 20,000-24,000 more individuals with disabilities would be have an outcome of employment every year if all counselors had a master's degree in rehabilitation. Similar cost savings of $225,000 per counselor could also be calculated in this manner. These findings suggest that finding ways to recruit and retain qualified rehabilitation counselors would have a strong impact on the employment outcomes of individuals with disabilities. Rehabilitation therefore may wish to emphasize the practical significance of these findings in order that small scale studies do not erode the importance of a master's degree in rehabilitation improperly.
It is imperative to note that hiring qualified rehabilitation professionals is an increasing problem, there are expected to be 3,812 openings for rehabilitation counselors in 2002-2007 and just 1606 MRC graduates going into state agencies (Chan, 2004). This 58% expected short fall may have various reasons. There have been cuts in the number of long term training grants to institutions which train rehabilitation counselors. Only about a third of the 5232 individuals graduating with a master's in rehabilitation counseling will be entering the state and federal system (Chan, 2003). Human resource directors may be hiring below CSPD standards due to that lack of an alternative with this low number of qualified counselors applying to state agencies. The shortage of qualified applicants is also likely due to state agencies average starting salary of $32,443, as compared to the average starting salary for similar master's degrees of $36,583. Thus rehabilitation counselors in state agencies are making only 88% of what their peers in similar fields are making in their first position (Chan, 2004). The lack of respect for the masters degree can also be seen in the salaries for rehabilitation counselors with the degree in comparison to those without a masters degree in their first 4 years with a state vocational rehabilitation agency that sees the average salary of those with a masters at $35,671 compared to those without a masters at $35,285, a statistically insignificant differences for those who have studied 2-4 years in a rehabilitation education program (Chan, 2004).
Also noteworthy is a look at the individual study effect sizes as seen in Table 3. Despite their legislative significance, the early articles concerning rehabilitation counselor education did not achieve statistical significance overall, or for 26 closure rates. This points to the limitations of significance testing in rehabilitation as pointed out by McAweeney et al. (1997). It also suggests the need to aggregate studies to more appropriately determine the practical significance of the findings.
One important finding from this analysis is the small effect size that a master's degree has on client outcome. Kosciulek and Szymanski (1993) noted that the majority of rehabilitation research does not have enough power to detect small effect sizes when a difference does exist. This can also be the case of the many small samples of counselors typically seen in a vocational rehabilitation office or even region. It is reasonable to assume that administrator and legislative bodies could often believe that no difference exists between counselors with a master's in rehabilitation and those without due to smaller sampling sizes which result in less power. A state with 100 counselors would have a less than 50% chance of showing the significance of the master's degree in rehabilitation due to Type II error.
This study is an initial attempt to look at the rehabilitation literature and meta-analyze the published studies of a particular state of affairs. Not surprisingly there were quite a few difficulties that limit the overall interpretability and generalizablity of the results.
First, data used for the question of level of education effect on outcomes were not in most cases easily transferable from the published literature to a meta-analysis. Numerous studies were discarded due to their lack of common statistical data that has been called for by major journals (namely, means and standard deviations for all groups studied). Therefore, some studies that addressed this issue and may have given more understanding to the results were not used, limiting the overall impact of the results.
Secondly, the number of studies used in this meta-analysis was small. The rehabilitation literature has recently made great improvements in using experimental design to test hypothesis in rehabilitation settings. It appears this move to research-based articles will only continue to help the rehabilitation field to be seen as a legitimate science supporting effective treatments. We agree with Kosciulek and Szymanski (1993), among others, for the call to have true experimental design for the research done in rehabilitation settings. However, this change takes time to be seen in the literature and limited the number of usable studies. Still this meta-analytic study is valid due to the very large sample sizes in each of the studies.
What is a sufficient number of studies to be used in a meta-analysis? Some meta-analyses have used over 500 studies, others much fewer. If meta-analysis had been applied to the treatment of acute myocardial infarction (heart attack due to blood clots), it would have shown after three studies that streptokinase (an enzyme that dissolves clots) was an effective treatment, saving many lives (Hunt, 1997). Authorities in meta-analysis have suggested that as few as two studies or five studies would be sufficient (Lipsey & Wilson, 2001). In this meta-analysis, more studies would be beneficial. For one, all of the studies used in this meta-analysis were ex-post-facto and not true-experimental designs. However, in this analysis the three studies all had very large sample sizes, giving more power to each of the studies and thus more power to this meta-analysis. One way to understand the increase in power is by using Cohen's power tables (1988), as recommended by Kosciulek and Szymanski (1993). By combing the three studies the power to detect a small effect size is increased from the .17-.33 it was for each of the individual studies to a power of .56 for the overall meta-analysis. If we assume a medium effect size difference exists between counselors with master's degrees in rehabilitation and those without, the power to detect this difference increases with the meta-analysis to greater than 99. A power of 99 indicates the difference between counselors educational level would be detected more than 99% of the time the difference was present. This is much greater than the power of 86 that was present in the Szymanski and Parker (1989a) study with a sample size of 100. The increased power of meta-analysis allows more confidence in the findings of the studies. In this analysis numerous studies could not be used because they did not reflect true experimental designs with a control group or, at the very least, pre- and post-intervention measures that could help appropriately determine the effect of employment outcome variables in question.
A criticism could be raised that the included studies of the effect of education level all contain inherent Type I error due to the similarity in their methodologies. Normally, a meta-analysis would correct for this with the number of studies in the analysis. In this particular case, since the literature is rather thin, this is a valid critique. It may be that counselors with MRCs are more likely to be in metropolitan environments that lead to higher closure rates or that those without MRC degrees have gotten into the field by chance or circumstance. These are valid criticisms of the individual studies whose importance is lessened but not eliminated in a meta-analysis. The authors agree that more research should be done not only in regard to the degree held by the rehabilitation professional but also with regard to the components of that education that result in desired outcomes for persons with disabilities.
Finally, given that studies with statistically significant results are more likely to be published (Atkinson, Furlong, & Wampold, 1982), and studies published in rehabilitation journals are likely to be conducted by those who favor the rehabilitation field, a critical eye should be used when interpreting the results. Of course, the selection criteria did not favor studies that supported one hypothesis over those supporting another for inclusion in the meta-analysis.
These results lend empirical support to the value of graduate training in rehabilitation as a means of producing competent, professionally-committed practitioners to work in the field of vocational rehabilitation (Shapson, Wright & Leahy, 1987). They also indicate that master's degrees should be the minimally acceptable educational attainment for entry level work as a rehabilitation counselor. The findings suggest that those "most qualified" to serve persons with disabilities are counselors with master's degrees in rehabilitation counseling. This does not mean that master's degrees in rehabilitation are the only way to achieve educational levels that results in positive outcomes for rehabilitation clients.
Further research needs to be conducted. The 'by whom' question has been empirically addressed, the "what type of treatment' and 'how does it come about' is still being debated. The call to uncover the aspects of rehabilitation education that make for successful counselors was made by Thomas (1990); this issue needs to be addressed. Something about rehabilitation interventions by appropriately trained counselors improves employment outcomes: there is still a need to determine what that is. Future research can include a meta-analysis of the interventions addressing the types of interventions that lead to successful outcomes. Moreover, because the ultimate indicator of former clients' vocational success is sustained employment (Bolton, 1990) it is imperative that longitudinal follow-up assessment strategies be implemented (Bolton, 1981).
Rehabilitation researchers can also concentrate on individual elements of the rehabilitation education programs, the rehabilitation counselor, the rehabilitation center, the employer, and the person with the disability that contribute towards successful transition into competitive employment by focusing on qualitative (or theory development) and not just quantitative methodologies. Qualitative processes can be applied such as structured observations and interviews of successful rehabilitation counselors and of their successful clients who have transitioned into competitive employment. From the details gathered during the interviews and observations, the researcher can develop theories about the elements or triangulation of elements that contribute towards successful placement. The development of the theories would have far reaching implications such as increased successful placement of individuals with disabilities, explanations or predictions that later can be tested with quantitative methodology, and enhanced educational preparation of future rehabilitation counselors.
The rehabilitation field has learned a lot, but there are still miles to go. Traversing those miles can continue by enhancing our understanding of intervention techniques through advanced ways of understanding phenomenon such as meta-analytic techniques, a focus on qualitative methodology and theory development. The rehabilitation field has miles to go, but we will not navigate those miles unless we continue to take the next step.
References marked with an asterisk (*) indicate studies included in the meta-analysis.
Ahn, H., & Wampold, B. E. (2001). Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. Journal of Counseling Psychology, 48, 251-257.
Aiken, L. R. (2003). Psychological testing and assessment (11th ed.). New York: Allyn Bacon.
Atkinson, D. R., Furlong, M. J., & Wampold, B. E. (1982). Statistical significance, reviewer evaluations, and the scientific process: Is there a (statistically) significant relationship? Journal of Counseling Psychology, 29, 189-194.
Backer, T. E. (1977). New directions in rehabilitation outcome measurement. Washington, D.C.: Institute for Research Utilization.
Bishop, M. & Feist-Price, S. (2002). Quality of life assessment in the rehabilitation counseling relationship: Strategies and Measures. Journal of Applied Rehabilitation Counseling, 33(1), 35-41.
Bolton, B. (1981). Follow up studies in vocational rehabilitation. Annual Review of Rehabilitation, 2, 58-82.
Bolton, B. (1990). Research methodology for investigating the relationship between rehabilitation counselor education and client outcomes. Rehabilitation Education, 4, 79-81.
Chan, T. (2003). Recruiting and retaining professional staff in VR agencies: Some preliminary findings from the RSA evaluation study. Washington, DC: American Institutes for Research.
Chan, T. (2004). Qualified personnel recruitment and retention: Challenges and opportunities. Paper presented at the National Conference on Rehabilitation Education, Washington, D.C.
Chapin, M., H. (2004). Employers' preferences for bachelors' and masters' level rehabilitation graduates. College Student Journal, 38, 362-269.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Cook, D. W., & Bolton, B. (1992). Rehabilitation counselor education and case performance: An independent replication. Rehabilitation Counseling Bulletin, 36, 37-43.
Cooper, H. (1997). Some finer points in meta-analysis. In M. Hunt (Ed.), How science takes stock: The story of meta-analysis (pp. 169-182). New York: Russell Sage.
Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and Instructional Methods: A Handbook for Research on Interactions. New York: Irvington Publishers.
Durlak, J. A. (1979). Assessment of paraprofessionals' on-the-job performance in an outpatient child guidance clinic. Journal of Community Psychology, 7, 348-362.
Estrada-Hernandez, N., & Wadsworth, J. S. (2005). External validity of the longitudinal study of the vocational rehabilitation services program sample. Rehabilitation Education, 19, 37-46.
Eysneck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting and Clinical Psychology, 64, 1020-1028.
Flowers, C. R., Strong, R., Turner, T. N., Moore, C. L., & Edwards, D. W. (1998). Agencies and preservice educational programs: Are complimentary needs being met? Rehabilitation Counseling Bulletin, 41,217-230.
Freedman, R. I., & Fesko, S. L. (1996). The meaning of work in the lives of people with disabilities: Consumer and family perspectives. Journal of Rehabilitation, 62 (3), 49-55.
Froehlich, R. J., Garcia, J., & Linkowski, D. C. (1998). Minimum hiring requirements for rehabilitation counselors in states: A comparison across federal regions. Rehabilitation Education, 12, 193-204.
Gilbride, D., Stensrud, R., & Johnson, M. (1994). Current models of job placement and employer development: Research, competencies, and educational considerations. Rehabilitation Education, 7, 215-239.
Gilbride, D., Thomas, J., & Stensrud, R. (1998). Beyond status code 26: Development of an instrument to measure the quality of placements in the state-federal program. Journal of Applied Rehabilitation Counseling, 29 (1), 3-7.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.
Kosciulek, J. F., & Szymanski, E. M. (1993). Statistical power analysis of rehabilitation counselor research. Rehabilitation Counseling Bulletin, 36, 212-219.
Hunt, M. M. (1997). How science takes stock: The story of meta-analysis. New York: Russell Sage Foundation.
Leahy, M., Szymanski, E., & Linkowski, D. (1993). Knowledge importance in rehabilitation counseling. Rehabilitation Counseling Bulletin, 3 7, 130-145.
Lipset, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
Lustig, D., & Strauser, D. (2005). Editors comment: Effect size and rehabilitation research. Journal of Rehabilitation, 70(4), 3-4.
McAweeney, M., Forchheimer, M., & Tate, D. (1997). Improving outcome research in rehabilitation psychology: Some methodological recommendations. Rehabilitation Psychology, 42, 125-135.
McCarthy, H., & Leierer, S. J. (2001). Consumer concepts of ideal characteristics and minimal qualifications for rehabilitation counselors. Rehabilitation Counseling Bulletin, 45, 12-23.
Millington, M. J., Miller, D. J., Asner-Self, K. K., & Linkowski, D. (2003). The business perspective on employers, disability and vocational rehabilitation. In E. M. Szymanski & R. M. Parker (Eds.), Work and Disability: Issues and strategies in career development and job placement. Austin, TX; Pro-Ed.
Muthard, J. E., Miller, L. A., & Obermann, C. E. (1966). Programmatic research on methods of continuing education for rehabilitation counselors (Mimeograph). Iowa City, Iowa: University of Iowa.
Paul, G. L. (1969). Behavior modification research: Design and tactics. In C. M. Franks (Ed.), Behavioral Therapy: Appraisal and Status. New York: McGraw Hill.
Phillips, R., Butler, A., & Thomas, K. R. (1988). Rehabilitation Counselor Performance Measures: A Comparative Study. Rehabilitation Counseling Bulletin, 32, 41-49.
Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166-169.
Scofield, M. E., Berven, N. L., & Harrison, R. P. (1981). Comprehensive credentialing and the future of rehabilitation. Journal of Rehabilitation, 47(1), 31-35.
Shapson, P. R., Wright, G. N., & Leahy, M. J. (1987). Education and the attainment of rehabilitation competencies. Rehabilitation Counseling Bulletin, 31, 131-145.
Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760.
Stubbins, J. (1982). The entry level for rehabilitation counselors: A reassessment. Rehabilitation Counseling Bulletin, 25, 243250.
Szymanski, E. M. (1991). Relationship of level of rehabilitation counselor education to rehabilitation client outcome in the Wisconsin Division of Vocational Rehabilitation. Rehabilitation Counseling Bulletin, 35, 23-37.
* Szymanski, E. M., & Danek, M. M. (1992). The relationship of rehabilitation counselor education to rehabilitation client outcome: A replication and extension. Journal of Rehabilitation, 58(1), 49-56.
* Szymanki, E. M., & Parker, R. M. (1989a). Competitive closure rate of rehabilitation clients with severe disabilities as a function of counselor education and experience. Rehabilitation Counseling Bulletin, 32, 292-299.
Szymanski, E. M., & Parker, R. M. (1989b). Relationship of rehabilitation client outcome to level of rehabilitation counselor education. Journal of Rehabilitation, 55(4), 32-36.
Szymanski, E. M., Parker, R. M., & Butler, A. J. (1990). Sensitivity of outcome measures in relating state vocational agency counselor performance to level of counselor education. Rehabilitation Education, 4, 93-107.
Truax, C., & Lister, J. (1970). Effectiveness of counselors and counselor aides. Journal of Counseling Psychology, 17, 331334.
Wampold, B. E. (2001). The great psychotherapy debate." Models, methods, and findings. Mahwah, NJ: Lawrence Erlbaum Associates.
* Wheaton, J. E., & Berven, N. L. (1994). Education, experience, and caseload management practices of counselors in a state vocational rehabilitation agency. Rehabilitation Counseling Bulletin, 38, 44-58.
Wright, G. N., Leahy, M. J., & Riedesel, P. S. (1987). Rehabilitation education research: The importance and attainment of professional competencies. Rehabilitation Education 1, 9-17.
Michael P. Frain
Florida Atlantic University
James M. Ferrin
David A. Rosenthal
The University of Wisconsin
Bruce E. Wampold
The University of Wisconsin
Michael: P. Fraini Assistant Professor, Dean College of Education, Florida Atlantic University, 47,273, Boca Raton, FL 33431. Email: firstname.lastname@example.org
Table 1 Overall Comparison of Counselors with Master's Degree in Rehabilitation (MRC) Versus Counselors with all Other Degrees (AOD) 95% confidence Comparison d SD(d) interval (d) Q-value MRC vs. AOD 0.20 0.06 0.07-0.32 2.08 Note. AOD = All other degrees includes bachelors, related and unrelated master's degrees. Table 2 Comparisons by Degree of rehabilitation counselor 95% Education confidence level Aggregated interval comparison Studies d SD(d) for d Q-value MRC vs. RM Szymanski 1992 Wheaton 1994 0.12 0.14 -0.16-0.40 0.04 MRC vs. BA Szymanski 1992 Wheaton 1994 0.29 0.13 0.04-0.54 0.15 MRC + RM vs. BA Szymanski 1992 Cook 1992 0.19 0.10 -0.02-0.39 0.49 Wheaton 1994 Note: BA includes counselors with bachelor's degrees and unrelated master's degree. Table 3 Individual Studies Examining Rehabilitation Outcomes based on Education of Counselor Study 95% Confidence Comparison (first author) d SD (d) Interval for d MRC Szymanski, 1992 0.10 0.22 -0.33-0.53 vs. RM Wheaton, 1994 0.14 0.19 -0.23-0.52 MRC Szymanski, 1992 0.22 0.19 -0.15-0.59 vs. BA Wheaton, 1994 0.33 0.17 0.01-0.66 MRC vs. Szymanski, 1992 0.18 0.18 -0.17-0.52 All other degrees Wheaton, 1994 0.30 0.15 0.01-0.59 combined Szymanski, 1989 0.09 0.13 -0.17-0.35 MRC+RM Szymanski, 1992 0.16 0.16 -0.15-0.47 vs. BA Wheaton, 1994 0.25 0.15 -0.05-0.55 Cook, 1992 0.03 0.29 -0.55-0.60 Note: MRC = Masters of Rehabilitation Counseling; RM = Related Masters (e.g. counseling psychology); BA = Bachelors and unrelated Masters
|Printer friendly Cite/link Email Feedback|
|Author:||Wampold, Bruce E.|
|Publication:||The Journal of Rehabilitation|
|Date:||Jan 1, 2006|
|Previous Article:||Tilting at windmills: changing attitudes toward people with disabilities.|
|Next Article:||Work Opportunities for Rewarding Careers (WORC): insights from implementation of a best practice approach toward vocational services for Mental...|