The relationship of rehabilitation counselor education to rehabilitation client outcome: a replication and extension.

The research reported in this article was supported by grant number H133C90067 from the National Institute on Disability and Rehabilitation Research to Research Associates of Syracuse. The authors wish to thank James Jeffers, John Kelly, Bob Burns, and John Irons of the Maryland Division of Vocational Rehabilitation; and Randall Parker, William Emener, and Kath Meadows for their assistance and support of this research. The Relationship of Rehabilitation Counselor Education to Rehabilitation Client Outcome: A Replication and Extension

The relationship between rehabilitation counselor education and rehabilitation client outcome in state vocational rehabilitation agencies has been an issue of ongoing interest (Ayer, Wright, & Butler, 1968; Rehabilitation Brief, 1989). The Rehabilitation Act has supported training of rehabilitation counselors at the master's degree level since 1954 (Rubin & Roessler, 1987; Wright, 1980), and The Act currently requires that vocational rehabilitation (VR) services be delivered by qualified personnel (Rehabilitation Act Amendments, 1986). However, over the past two decades while the complexity of rehabilitation counseling has increased, some VR agencies have de-emphasized counselor education (Hershenson, 1988; Pankowski & Pankoswki, 1974). During the same period the literature has reflected discussions about whether rehabilitation counseling, as practiced in state VR agencies, is a profession or a job title (Patterson, 1989) and debates about the meaning of the term qualified in reference to rehabilitation personnel (Graves, Coffee, Habeck, & Stude, 1987; Walker & Myers, 1988; Wright, 1982). The current study addresses some of these issues through an examination of the relationship of rehabilitation counselor education to client outcome in the Maryland Division of Vocational Rehabilitation (DVR).

Despite its importance to both rehabilitation counseling theory and state-federal rehabilitation policy, the utility of research on rehabilitation counseling outcomes has been obscured by inherent complexity and potential for flaws in methodology and interpretation (Szymanski, Parker, & Butler, 1990). Although some studies demonstrated relationships between counselor characteristics or behaviors and client perceptions or outcomes (e.g., Ayer, Wright, & Butler, 1968; Jenkins, West, & Anderson, 1975; Rubin, Bolton, Krauft, Bozarth, & Richardson, 1974; Szymanski, in press; Szymanski & Parker, 1989a), other studies were unable to detect such relationships (e.g., Abrams & Tucker, 1989; Danek, 1979; Emener, 1980). We will, therefore, precede the description of the current study with a brief discussion of past research.

Past Research

The methodology of the current study is built on the foundation of past studies, both in rehabilitation counseling and in other related disciplines. Research methodology remains a challenge for researchers in any area of behavioral and social science (Rosnow & Rosenthal, 1989), and we continue to build on and refine the methodology of our predecessors (W. Emener, personal communication, August, 1988). Thus, although we point to the limitations of previous research, it is important to remember that we continue to build on its foundation.

Szymanski, Parker, and Butler (1990) explained the research on the relationship of rehabilitation counselor education to rehabilitation client outcome has often been complicated by the following methodological limitations: (a) inadequate outcome measures and lack of consideration of different outcome patterns for clients with severe disabilities, (b) low statistical power, and (c) failure to account for the interactive relationship of counselor education and work experience in relation to client outcome. These limitations are discussed and illustrated by past research.

Inadequate Outcome Measures and Different Outcome Patters for Clients with Severe Disabilities

Construct validity of outcome measures (i.e., dependent variables) has been a problem throughout the history of counseling research (Newman & Scott, 1988). In research design, construct validity deals with the congruence between research operations and conceptual definitions (Cook & Campbell, 1979). To address construct validity, researchers must be sure that dependent variables actually measure the relationship or effect of interest. Some constructs, like the effect of counselor education on client outcome, are complex and difficult to fully explicate with available VR outcome measures. Szymanski, Parker, and Butler (1990) demonstrated that variables normally used to measure rehabilitation counselor performance (e.g., number of Status 26 closures) were often inappropriate for measurement of the relationship between counselor education and client outcome, because they did not include available information related to quality of rehabilitation outcomes. Such information on outcome quality includes work status of rehabilitated clients (e.g., competitive employment, sheltered employment, homemaking) and severity of client disability (i.e., severely disabled vs not severely disabled or severity status not known).

Consideration of both work status at closure and severity of client disability is particularly important in light of federal closure figures that indicates substantial discrepancies in work status at closure between clients with severe disabilities and their counterparts with non-severe disabilities. In federal fiscal year 1985, 73.3% of rehabilitated clients with severe disabilities were closed in competitive employment as contrasted with 89.7% of rehabilitated clients with non-severe disabilities (RSA, 1988). For fiscal year 1988, percentages of competitive rehabilitations were 76.9 for clients with severe disabilities and 91.6 for clients with non-severe disabilities (L. Mars, personal communication, June 28, 1990). Obviously, dependent variables that do not include or otherwise account for client work status and severity of client disability miss important dimensions of quality and intensity of rehabilitation counseling services, thus raising questions of validity.

Emener's (1980) dependent variables included numbers of closures in different closure categories, including Status 26 (rehabilitated) closures, and success rate, which was defined as the ratio of Status 26 closures to the total caseload. Abrams and Tucker (1989) used rehabilitation rate, which was the ratio of number of Status 26 closures to the sum of the numbers of Status 26, 28, and 30 closures (i.e., closures that occurred after the individual had been determined eligible for VR services). Thus, it is apparent that both Emener's and Abrams and Tucker's outcome measures missed dimensions of quality of rehabilitation outcomes (e.g., work status at closure) and severity of client disability. Danek's (1979) study included only clients with severe hearing impairments, thus addressing the severity issue; however, the use of the Status 26 closure as an outcome measure omitted the consideration of quality of outcomes that would have been included with work status at closure.

Low Statistical Power

Statistical power means the extent to which a statistical test using adequate outcome measures will be able to detect relationships or differences that actually exist among the comparison groups (Cohen, 1988; Lipsey, 1990). Statistical power is related to a variety of factors including (a) the effect size (i.e., the observed size of the difference or strength of the relationship among the variables); (b) the alpha level (i.e., the researcher's pre-established risk that differences, if found, are due to chance), and (c) the number of participants in the study (Cohen, 1988).

Effect size can be problematic in counselor effectiveness research, because client characteristics and other sources of variance, which are difficult to fully control, serve to attenuate the size of the observed relationships (Szymanski, Parker, & Butler, 1990). And, although small observed relationships or effect sizes are common in treatment effectiveness research, most such research is underpowered for finding anything but large effects (Lipsey, 1990).

For small observed effect sizes, like those to be expected in research on the relationship of rehabilitation counselor education to client outcome, at least 300 participants are usually required to achieve a statistical power of .70 (i.e., a 70% probability of finding effects that actually exist) with an alpha level of .05 (Rosenthal & Rosnow, 1984). Although they all used alpha levels of .05, Abrams and Tucker (1989) used only 192 participants, Emener (1989) used 60, and Danek (1979) used 114. Thus, they did not have adequate statistical power to find small effects.

Failure to Account for the Interactive Relationship of Counserlor Education and Work Experience

Szymanski, Parker, and Borich (1990) demonstrated that failure to consider a disordinal relationship of counselor education and work experience in relationship to client outcome could obscure relationships among the variables. Although both Emener (1980) and Abrams and Tucker (1989) included work experience as a variable, neither examined its effect conjointly with level of counselor education in relation to client outcome. Thus, neither design could have detected a disordinal interaction; and if such an interaction existed, the validity of conclusions drawn from the statistical tests could have been compromised.

The preceding discussion has demonstrated that three of the previous research studies in this area had serious research design problems. Their designs lacked sufficient statistical power to find the relationships of interest; and, since their outcome measures were not adequate, the relationships would probably have remained obscured even with high statistical power.

The current study was designed as a replication, with some additions, of Szymanski and Parker's (1989b) study of New York State VR counselors, which indicated relationships between level of counselor education and rehabilitation outcomes for clients with severe disabilities. Replication is fundamental to the advancement of knowledge due to the nature of scientific inquiry and the inherent limitations of single studies (Rosnow & Rosenthal, 1989). The methodology employed in this study was built on the foundation and lessons of previous studies. The role of this foundation is reflected in the following words of one of rehabilitation counseling's leading educators and researchers: "Research is a the systematic search for truth - ... a longitudinal process that requires the cooperative efforts of people over decades" (W. Emener, Personal communication, July 2, 1990).

Method

Procedure

The dependent variables, which were computed for each counselor for the 1988/1989 fiscal year, were: (a) competitive closure rate, which was the ratio of competitive employment Status 26 (rehabilitated) closures to all other closures, including other Status 26 closures and closures from applicant and extended evaluation statuses; (b) number of non-competitive closures, which was the total number of closures minus the number of closures in Status 26 closures with a work status of competitive employment; and (c) case service dollar expenditures for non-competitive closures, which was the total case service dollar expenditures for the non-competitive closures previously described. Dependent variables were computed separately for clients with severe disabilities and those whose disabilities were not classified as severe. The latter group included clients whose disabilities were classified as non-severe and those clients closed as ineligible for vocational rehabilitation services (Status 08) whose disability status was not known. The resulting six dependent variables were competitive closure rate for persons with severe disabilities (CCRS), number of non-competitive closures for persons with severe disabilities (NCCSD), net cumulative expenditures per counselor for non-competitive closures of clients with severe disabilities (NCEXSD), competitive closure rate for persons whose disabilities are not classified as severe (CCRN), number of non-competitive closures of clients whose disabilities are not classified as severe (NCCN), and net cumulative expenditures per counselor for non-competitive closures of clients whose disabilities are not classified as severe (NCEXN).

The independent variable, level of rehabilitation counselor education, was categorized in the following three levels: (a) master's degree in rehabilitation counseling (MRC), (b) related master's degree (e.g., counseling) (RM), and (c) unrelated bachelor's or master's degrees (UBM). Two counselors with bachelor's degrees in rehabilitation were excluded from consideration, because they did not constitute a large enough group to be considered separately and would confound the results if they were included in any of the other groups. Years of counselor work experience with Maryland Division of Vocational Rehabilitation (DVR), which was recorded in continuous form, was used as a moderator variable.

Client data was aggregated by counselor number and variables of interest were computed using the SPSS aggregate and compute procedures (Norusis, 1988). Counselor data was then matched to client data by counselor number.

Participants

Participants were rehabilitation counselors employed by Maryland DVR and the clients whose cases they closed during the fiscal year from 10/1/88 to 9/30/90. During that period, 13,502 cases were closed. Cases that were closed from referral rather than applicant status were eliminated from consideration on the premise that they had not actually applied for services or had any involvement with a counselor. The distribution of the resultant client population of 9030 individuals by closure status and work status at closure is displayed in Table 1. [TABULAR DATA 1 OMITTED]

A central administrative office within DVR distributed a one page questionnaire to all caseload carrying counselors to obtain information on level of counselor education and years of DVR experience. Follow-up of non-respondents consisted of another written request, including a duplicate questionaire and a telephone call approximately one month after the initial request. There were 127 returns and five non-respondents after follow-up. Selection was therefore not considered to be a threat to validity. The counselor number by which cases were assigned was used for follow-up and joining of counselor and client information.

The data base included caseloads of counselors who left or had been reassigned to a new caseloads. In addition, some of the respondents had not been employed long enough to have closures or had been reassigned to new caseloads. After exclusion of caseloads for which there was not a match of counselor and client data, the resultant participant population was 100 counselors (79%) and their 6995 clients. Thirty counselors had master's degrees in rehabilitation counseling (MRC), 23 had related master's degrees (RM), and 47 had unrelated bachelor's or master's (UBM) degrees. Years of work experience ranged from less than one year to 26 years; and mean year of work experience were 8.33 for MRCs, 13.31 for RMs, and 8.36 for UBMs with respective standard deviations of 6.74, 6.60, and 7.18.

Research Design and Data Analysis

A quasi-experimental design was used. Although counselor education was not normally a factor in client assignment (B. Burns, personal communication, July 2, 1990), random assignment of clients to counselor was not assumed. Variables were not manipulated; therefore, statistical control (Bolton & Parker, 1987; Cook & Campbell, 1979) was used to account for differences among the levels of counselor education in years of experience and in number and percentage of clients with severe disabilities on the caseload. The latter control was accomplished by the computation of the dependent variable CCRS.

The following rival null hypothesis was tested as a preliminary measure: There are no differences in total number of closures and number of closures of persons with severe disabilities among counselors with different levels of education.

The following experimental null hypotheses were tested: 1. There are no differences in counselor performance with clients

with severe disabilities (i.e., CCRS, NCCSD,

NCEXSD) among counselors with different levels of education. 2. There are no differences in counselor performance with client

whose disabilities are not classified as severe (i.e.,

CCRN, NCCN, NCEXN) among counselors with different

levels of education.

It was expected that counselors with master's degrees in rehabilitation counseling would have higher competitive closure rates, lower numbers of non-competitive closures, and lower cumulative expenditures for non-competitive closures of clients with severe disabilities than their counterparts with different levels of educational preparation. Differences among counselor education levels were not expected for these three variables applied to clients whose disabilities had not been classified as severe.

The rival hypothesis was tested with oneway analysis of variance. The aptitude treatment interaction (ATI) statistical design (Borich, 1986; Pedhazur, 1982) was used to test the experimental hypotheses. The use of the ATI design was based on Szymanski and Parker's (1989a, 1989b) demonstration of a disordinal interaction of level counselor education and work experience in relation to some dependent variables. Borich, Godbout, and Wunderlich's (1975) ATILIN1 program was used for pairwise comparisons among counselors with different levels of education of the regressions of each dependent variable on years of counselor DVR work experience. The SPSS regression program (Norusis, 1988) with a non-additive multiple regression model with product vectors (Pedhazur, 1982) accounting for the interaction of level of counselor education and counselor years of DVR work experience variable. Three planned comparisons (R2) for each dependent variable. Three planned comparisons were made for each dependent variable; each educational level was compared with each other educational level (i.e., MRC vs RM, MRC vs UBM, RM vs UBM).

Four secondary dependent variables were included in the analyses to assist in the interpretation of the results. These four variables, which were contained in the composites of the major dependent variables, were (a) number of sheltered employment closures of clients with severe disabilities (NSHLS), (b) number of homemaker closures of clients with severe disabilities (NHOMS), (c) number of Status 08 closures of clients with severe disabilities (N08S), and (d) cumulative expenditures for non-competitive closures of all clients (NCEXT). The secondary analyses were planned and conducted to explain the nature of significant differences obtained on the primary dependent variables.

Statistical power was estimated prior to data analysis with Rosenthal and Rosnow's (1984) statistical power tables, N of 100, and a small effect size based on Szymanski and Parker's (1989b) results. An alpha level of .05 would have resulted in power of less than .50 for most comparisons, in other words less than a 50% chance of finding differences or relationships that actually exist. Therefore, an alpha level of .10 was used for all data analysis. Such adjustments of alpha are recommended in situations of fixed sample size (Cohen, 1988).

Results

Oneway analyses of variance for the rival hypothesis failed to reveal significant group differences ion total number of closures: F (2,97) = 1.61, p = .21. Similar results were found for the number of clusores of persons with severe disabilities: F (2,97) = 1.38, p = .26.

Regression equations and means for each dependent variable are reported in Table 3. Effect sizes ranged from .04 for CCRS to .09 for NSHLS. Interaction of educational level and work experience were evident through visual inspection of the unstandardized regression coefficients for most dependent variables. Thus, the choice of the ATI statistical design was supported.

ATI analyses were used to examine the pairwise differences among the regression equations for each dependent variable. ATI results differ from those of traditional analyses; specific F and p values are not reported, rather ATI results are the specific range of years of experience over which the groups differ at the pre-set significance level (.10 for this study). The results of the ATI analyses are reported in Table 3. Regions of significant difference among the educational levels were found for the regressions of primary dependent variables CCRS, NCEXSD, and NCEXN on years of counselor DVR work experience. Regions of significant difference were also found for the regressions of secondary variables NSHLS, NO8S, and NCEXT. [TABULAR 2 and 3 DATA OMITTED]

The comparison of MRCs with UBMs revealed that MRCs had (a) higher competitive closure rates of persons with severe disabilities from the beginning of agency service through 3 years of experience, (b) lower cumulative expenditures for non-competitive closures of persons with severe disabilities from 4.3 through 12.5 years of experience, (c) higher cumulative expenditures for non-competitive closures of clients whose disabilities were not classified as severe from the beginning of agency service through 19 years of experience, (d) lower numbers of persons with severe disabilities closed in sheltered employment from 7.3 through 26 years of agency service, and (e) lower cumulative expenditures for all non-competitive closures from 4.2 through 26 years of experience. No other regions of significant difference were found for the comparison of MRCs with UBMs.

The comparisons of MRCs with RMs revealed only one region of significant difference. MRCs had fewer Status 08 closures of persons with severe disabilities from 4.1 through 10 years of agency service.

The comparison of RMs with UBMs revealed that RMs (a) had lower cumulative expenditures for non-competitive closures of clients whose disabilities were not classified as severe from 6.7 through 14.9 years of experience and (b) lower cumulative expenditures for all non-competitive closures from 4.2 through 26 years of experience. No other regions of significant difference were found for the comparison of RMs with UBMs.

Discussion

The results of the oneway analyses of variance of the total number of closures and the number of closures of persons with severe disabilities did not allow rejection of the rival null hypothesis. Although a tendency existed, similar to that found by Ayer, Wright, and Butler (1968), for non-MRCs to process more cases than MRCs; the differences were not significant. Thus, numbers of closures could not be assumed to account for differences found among the comparison groups on the other dependent variables.

The results of this study demonstrate that Maryland DVR counselors with master's degrees in rehabilitation counseling have higher rates of competitive outcomes and are more cost efficient than their colleagues with unrelated bachelor's and master's degrees in their service to people with severe disabilities. The results also demonstrate that both counselors with master's degrees in rehabilitation counseling and those with related master's degrees are more cost efficient in overall service delivery than their colleagues with unrelated bachelor's and master's degrees.

The study results related to service delivery to clients with severe disabilities essentially replicated Szymanski and Parker's (1989b) New York State results. Although there were some differences in regions of significance, the effect sizes were remarkably similar for the competitive variables. Szymanski and Parker (1989b) found R2s of .03, .07, and .05 respectively for CCRS, NCCSD, and NCEXSD. Effect sizes found in this study were .04 for CCRS, .08 for NCCSD, and .05 for NCEXSD.

The secondary analyses in the study further quantified the cost-benefit result and provided some explanation of the nature of group differences. Both MRCs and RM's were found to be more cost-efficient than UBMs, and these differences did not dissipate with time. In addition MRCs had fewer sheltered employment closures of persons with severe disabilities than UBMs and fewer Status 08 closures of persons with severe disabilities than RMs.

We suspect that the omission of earlier years from some regions of significance reflects the time it takes for caseloads to reach maturity. In other words, during the early years of a counselor's tenure with the agency, caseload sizes may be small and group differences obscured. The Szymanski and Parker (1989b) study revealed a similar phenomena. The CCRS variable was designed to eliminate some of the effect of the different caseload sizes, thus, possibly accounting for early regions of significance of this variable in both previous and current studies.

The effect sizes, which are the amounts of variance in the dependent variables accounted for by relationships with the independent variables, are in Cohen's (1988) small effect size range. However, recall from the earlier discussion that uncontrolled variance in the design (e.g., client characteristics, service variables) attenuates effect size, so that what may really be a medium or a large effect can be seen as a relatively small effect. Variance due to client characteristics is known to be a potent factor in rehabilitation outcome studies (Bolton, 1987; Vandergoot, 1986; Walls & Tseng, 1987). Such variance most likely serves to reduce the observed effect size in studies of the relationship of rehabilitation counselor education to rehabilitation client outcome. Thus, it is likely that the actual relationship between rehabilitation counselor education and rehabilitation client outcome in Maryland DVR is larger than that observed in this study.

The research design addressed potential threats to validity from inadequate preoperational explication of constructs (Cook & Campbell, 1979, p. 64-65) through construction of three complementary dependent variables designed with considerations of quality of rehabilitation counseling service. The replication of the New York results extends the external validity of the study to a limited degree within the state VR agency framework. However, the study results are not generalizable beyond the state federal VR system. It will be important for future research efforts to address this limitation.

Additional threats to validity resulted from use of the .10 alpha level and three pairwise comparisons, which elevated the probability of Type 1 error, i.e., the possibility that the observed differences resulted from chance. However, the alternative alpha level of .05 would have resulted in such low statistical power (less than .50) that the results would have been invalid. In situations like this, replication is recommended to continue to establish the validity of the pattern of results. It should be noted that this study is a partial replication and extension of earlier research; thus, the threat to validity imposed from a slight elevation of Type 1 error is not as serious as it would have been without the earlier study.

A further limitation resulted from the quasi-experimental nature of the design of this study, which did not allow proof of cause and effect. Thus, it is possible that level of counselor education served as a marker variable for another variable not included in the design (e.g., commitment to rehabilitation counseling). If this is the case, one would not expect as positive results from providing MRC training to currently employed counselors as one would expect from hiring new MRCs as opening arise. Nonetheless, because there is evidence of a relationship between level of rehabilitation counselor education and outcome for clients with severe disabilities, both strategies should be explored.

The intent of the state-federal vocational rehabilitation system is to provide positive outcomes for eligible persons with disabilities. However, not every contingency in the system can be controlled to produce such outcomes. For example, it is obviously not possible or desirable to control client characteristics in order to increase positive VR outcomes. Nonetheless, it is possible to control counselor characteristics, particularly counselor education, through hiring criteria. It is also possible to control the effect of counselor characteristics through differential assignment of clients with severe disabilities to counselors with master's degrees in rehabilitation counseling.

In conclusion, the results of this research show a relationship between rehabilitation counselor education and rehabilitation outcome for clients with severe disabilities in Maryland DVR. Counselors with master's degrees in rehabilitation counseling achieve higher rates of competitive outcomes and are more cost efficient than their colleagues with unrelated bachelor's and master's degrees.

The results of this study replicate those of Szymanski and Parker's (1989b) research on the New York State VR agency and are similar to those of Szymanski's (in press) research on the Wisconsin agency. The complexity of this type of research, which was discussed in this article, has prevented some earlier studies from revealing similar results. However, at this time, an emerging trend in the research suggests policy implications for state rehabilitation agencies. For example, the prospect of rising federal and state deficits and a federal tax increase will certainly increase taxpayer pressure on public programs to demonstrate greater cost efficiency. This research suggests that hiring master's level rehabilitation counselors to provide services to clients with severe disabilities could be a cost effective management practice and could also provide these clients with more competitive employment closures.

References

Abrams, J. M., & Tucker, C. M. (1989). Counselor variables that predict job performance among state-employed vocational rehabilitation counselors. Rehabilitation Education, 3, 193-199. Ayer, M. J., Wright, G. N., & Butler, A. J. (1968). Counselor orientation: Relationship with responsibilities and performance. Wisconsin Studies in Vocational Rehabilitation, Monograph X. Madison WI: University of Wisconsin, Regional Rehabilitation Research Institute. Bolton, B. (1987). Outcome analysis in vocational rehabilitation. In M. J. Fuhrer (Ed.), Rehabilitation outcomes: Analysis and measurement (pp. 57-69). Baltimore: Brookes. Bolton, B., & Parker, R. M. (1987). Research in rehabilitation counseling. In R. M. Parker (Ed.), Rehabilitation counseling: Basics and beyond (pp. 157-187. Austin, TX: Pro-Ed. Borich, G. D. (1986). Trait-treatment interactions in the classroom: Researching the effects of instructional treatments on different types of learners. Schweizerische Zeitschrift fur Kaufmannisches Bildungswesen, 5, 128-139. Borich, G. D., Godbout, R. C., & Wunderlich, K. W. (1976). The analysis of aptitude-treatment interactions: Computer programs and calculations. Austin, TX: Oasis Press. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin. Danek, M. M. (1979). Rehabilitation outcomes and caseload management as a function of counselor expertise in hearing impairments (Doctoral dissertation, University of Maryland, 1979). Emener, W. G. (1980). Relationships among rehabilitation counselor characteristics and rehabilitation client outcomes. Rehabilitation Counseling Bulletin, 23, 183-192. Graves, W. H., Coffey, D. D., Habeck, R., & Stude, E. W. (1987). NCRE position paper: Definition of the qualified rehabilitation professional. Rehabilitation Education, 1, 1-7. Hershenson, D. B. (1988). Along for the ride: The evolution of rehabilitation education. Rehabilitation Counseling Bulletin, 31, 204-217. Jenkins, W. M., West, D. N., & Anderson, R. M. (1975). Rehabilitation college clients and counselor effectiveness. Education Catalyst, 34-39. Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage. Newman, J. L., & Scott, T. B. (1988). The construct problem in measuring counseling performance. Counselor Education and Supervision, 28, 71-79. Norusis, M. J. (1988). SPSS/PC + V2.0 Base Manual. Chicago: SPSS. Pankowski, M. L., & Pankowski, J. M. (1974). Why a master's degree in rehabilitation counseling? Journal of Applied Rehabilitation Counseling, 5(3), 147-152. Patterson, J. B. (1989). Rehabilitation counselors: In the public eye. In K. Williams & J. Spann (Eds.), US Today: The VR counselor and rehabilitation today (pp. 7-17). Dunbar, WV: West Virginia Research and Training Center. Pedhazur, E. J. (1982). Multiple regression in behavioral research: Explanation and predication (2nd ed.). New York: Holt, Rinehart, & Winston. Rehabilitation Services Administration (1988). Report comparing services and outcomes for severely disabled and non-severely disabled persons rehabilitated in fiscal year 1985 (RSA-IM-88-24). Washington, DC: U.S. Department of Education, Office of Special Education and Rehabilitative Services. Rosenthal, R., & Rosnow, R. L. (1984). Essentials of behavioral research: Methods and data analysis. NY: McGraw-Hill. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284. Rubin, S. E., Bolton, B., Krauft, C. C., Bozarth, J. D., & Richardson, B. K. (1974). Rehabilitation counselor interview styles and client outcome. (Series 1, monograph no. 18). Fayetteville, AR: University of Arkansas, Arkansas Rehabilitation Research and Training Center. Rubin, S. E., & Roessler, R. T. (1987). Foundations of the vocational rehabilitation process (3rd ed.). Austin TX: Pro-Ed. Szymanski, E. M. (in press). The relationship of level of rehabilitation counselor education to rehabilitation client outcome in the Wisconsin Division of Vocational Rehabilitation. Rehabilitation Counseling Bulletin. Szymanski, E. M., & Parker, R. M. (1989a). Competitive closure rate of vocational rehabilitation clients with severe disabilities as a function of counselor education and experience. Rehabilitation Counseling Bulletin, 32, 292-299. Szymanski, E. M., & Parker, R. M. (1989b). Relationship of rehabilitation client outcome to level of rehabilitation counselor education. Journal of Rehabilitation, 55(4), 32-36. Szymanski, E. M., Parker, R. M., & Borich, G. D. (1990). Aptitude treatment interaction designs in research on the relationship between rehabilitation counselor education and rehabilitation client outcome. Rehabilitation Education, 4, 83-92. Szymanski, E. M., Parker, R. M., & Butler, A. J. (1990). Sensitivity of client outcome measures in relating state vocational rehabilitation agency counselor performance to level of counselor education. Rehabilitation Education, 4, 93-107. Vandergoot, D. (1987). Review of placement literature: Implications for research and practice. Rehabilitation Counseling Bulletin, 30, 243-272. Walker, M. L., & Myers, R. W. (1988). A counter proposal: Defining the qualified rehabilitation professional. Rehabilitation Education, 2, 49-57. Walls, R. T., & Tseng, M. S. (1987). Measurement of client outcomes in rehabilitation. In B. Bolton (Ed.), Handbook of measurement and evaluation in rehabilitation (2nd ed.) (pp. 183-201). Baltimore: Brookes. Williams, F. (1986). Reasoning with statistics: How to read quantitative research (3rd ed.). Fort Worth: Holt, Rinehart, & Winston. Wright, G. N. (1980). Total rehabilitation. Boston: Little, Brown. Wright, G. N. (1982). Contemporary rehabilitation counselor education. Rehabilitation Counseling Bulletin, 25, 254-256.