Determining vocational levels of people with disabilities in Japan: A statistical prediction approach.

Determining Vocational Levels of People with Disabilities in Japan:

A Statistical Prediction Approach

The purpose of this paper was to determine the appropriateness of statistical techniques in predicting vocational placement levels of people with physical disabilities in Japan. Two hundred and fifty-three rehabilitation clients were administered the Japanese version of the General Aptitude Test Battery (GATB) and the Wechsler Adult Intelligence Scale (WAIS). The data were analyzed by multiple discriminant analysis using GATB and WAIS scores as predictors and actual vocational placement levels as criterion. Using the discriminant function derived, 79.5% of the clients in the validation sample were accurately classified, representing a significant improvement over the base rate prediction of 62.2%. It can be concluded that prediction formulas are moderately successful in predicting work performance levels of people with disabilities. Also, a substantive interpretation of the discriminant function suggested the existence of different vocational ability requirements for different vocational placement levels. Implications for vocational evaluation were discussed.

Vocational evaluation is an integral part of the vocational rehabilitation process. Pruitt (1986) defined vocational evaluation as:

"A comprehensive and systematic process that utilizes work (real or simulated) to assess the vocational potential of people with disabilities. The process also incorporates other data such as test scores, medical data, information on education and work experience and the needs and interests of the disabled persons in setting vocational goals and service requirements" (p. 311).

Essentially, information obtained from vocational evaluation is used to assist clients with disabilities in identifying their functional competencies and limitations, formulating realistic vocational goals, and determining rehabilitation programming needs (Power, 1984).

In countries such as the United States, technologies for the practices of vocational evaluation are fairly well developed. According to Nadolsky (1971), these technologies can be classified in terms of evaluation and observation technologies, and interpretation technologies. Vocational evaluation approaches (e.g., psychometric assessment, job analysis, work samples, and situation assessment) developed specifically to facilitate the gathering of vocational potential information of people with disabilities could generally be classified as evaluation and observation technologies. On the other hand, vocational evaluation tools--such as the Dictionary of Occupational Titles (DOT) (U.S. Department of Labor, 1982), the Selected Characteristics of Occupations Defined in the Dictionary of Occupational Titles (SCODOT) (U.S. Department of Labor, 1981) and DOT related publications, computerized job-matching programs, and statistical prediction formulas, etc.--developed to facilitate the synthesis of assessment data, are considered interpretation technologies (Hattori, 1982).

The DOT and the SCODOT, which contains detailed job analysis and worker traits information of 12,000 jobs in the United States, provides vocational evaluators perhaps the most efficient means to organize and synthesize the voluminous evaluation data gathered during evaluation. For example, a residual employability profile can be developed from assessment data to delineate the work capacity of a client with disability, in terms of the exact job attributes (physical capacities, vocational aptitudes, educational requirements, interests, etc.) defined by the DOT. The resulting profile can then be compared systematically to jobs listed in the entire DOT to identify appropriate job placement alternatives for persons with disabilities (Sink & Field, 1981).

Although the DOT approach is very efficient, the accuracy of vocational evaluation recommendations made on the basis of job analysis and worker-trait requirements information provided by the DOT is increasingly being challenged. Recently, Bose, Grzesik, Geist, and Bryant (1986) reported in their study that the job analysis information of many jobs listed in the DOT is no longer valid due to rapid technological changes (e.g., automation). Another job-matching approach, of course, is to develop statistical prediction formulas on the basis of data obtained from assessment battery, to predict training and work success in specific employment areas for people with disabilities (McCarron & Dial, 1976). The McCarron-Dial System exemplifies this statistical approach to vocational evaluation in the United States. The statistical prediction approach has also been the practice of many rehabilitation professionals outside of the United States (Hattori, 1982).

In Japan, evaluation and observation technologies in vocational evaluation are also fairly well developed. For example, many psychometric assessment instruments and work samples have been developed for the purposes of assessing vocational potential of people with disabilities (Hattori, 1982). Also, some popular vocational and psychological testing instruments in the United States (e.g., the Wechsler Adult Intelligence Scale and the General Aptitude Test Battery) have already been adapted and validated for local use (Kodama, Shinagawa, & Indou, 1958; Ministry of Labor, Japan, 1969).

However, interpretation tools such as the DOT are less readily available in Japan. As job requirements in Japan are not always identical to those in the United States, the use of the American DOT may not be appropriate. Although the Ministry of Labor in Japan (1986) has recently initiated a project to develop job analysis information for occupations in Japan, it is unlikely that these data will be available any time soon. Alternative interpretation technologies for determining job placement direction of people with disabilities must be developed.

Because people with physical disabilities in Japan tend to have a more defined range of job placement options than their counterparts in the United States (Hattori, 1982), the development of prediction formulas, as interpretation aids for vocational evaluators to predict vocational placement levels of disabled persons, appears to be more appropriate and practical than the DOT approach. Therefore, the purpose of this study was to demonstrate the appropriateness of the statistical prediction approach in predicting the vocational placement levels of people with physical disabilities in Japan.

Method

Participants

Participants of this study were 253 clients with physical disabilities served by the National Rehabilitation Center for the Disabled (NRCD) and the National Vocational Rehabilitation Center (NVRC) in Japan. Ninety-six clients were selected from the work adjustment training program operated by the NRCD. They were considered sheltered workshop bound clients. The other 157 were competitive employment bound clients selected from the NVRC vocational training program. Placement of these participants in the two training programs was based on their comprehensive vocational evaluation data (i.e., biographical data, interview, group and individual testing, work sample testing, and behavioral observation over an extended period of time). Only clients, who were in their respective placement programs long enough to be judged as appropriately placed, were included in this study.

Criterion and Predictor Variables

The criterion variable in this study was defined as group membership in either the work adjustment training or vocational skills-training programs (i.e., sheltered employment bound vs. competitive employment bound clients). The predictor variables used to predict group membership in different training programs were: Age; General Aptitude Test Battery (GATB - Japanese version) subscale scores including G - General Learning Ability, V - Verbal Aptitude, N - Numerical Aptitude, S - Spatial Aptitude, P - Form Perception, Q - Clerical Aptitude, and K - Coordination (Ministry of Labor, Japan, 1969); and the Wechsler Adult Intelligence Scale (WAIS - Japanese version) scores including the Verbal IQ (VIQ), Performance IQ (PIQ), and Full Scale IQ (FSIQ) (Kodama, Shinagawa, & Indou, 1958).

Data Analysis

Half of the subjects in this study were randomly assigned to a validation group and half to the cross-validation group. Forty-eight clients from the work adjustment group and 79 clients from the vocational training group constituted the validation sample. The cross-validation sample consisted of 48 work adjustment clients and 78 vocational training clients. A multiple discriminant function analysis was used to determine the ability of the assessment data to predict group membership (vocational levels) of the subjects. The predicted group membership was then compared to actual group membership to determine the classification accuracy of the discriminant function. The discriminant function was further applied to the cross-validation sample to determine the classification error rates ("shrinkage") of the discriminant analysis under cross-validation. The Statistical Package for the Social Sciences (SPSS-PC) was used for all of the statistical computations in this study.

Results

Table 1 presents the means and standard deviations of the predictor variables for the work adjustment and vocational training groups.

Because there are only two groups (work adjustment vs. vocational training) in this study, a maximum of one significant discriminant function was derived (Canonical r = .5, Wilk's [is greater than] = .75, [X.sup.2] = (11) = 34.6, p [is less than] .001). It was found that 79.5% of the subjects in the validation sample were correctly classified. This represents a considerable improvement over the base rate of 62% (the sample size of the vocational training group [i.e., the largest group in the validation sample with n = 79] divided by 127, the size of the entire validation sample). We also found 70.6% of the subjects in the cross-validation sample were correctly classified by the discriminant function. The error rate ("shrinkage") of the discriminant function upon cross-validation is therefore equal to 8.9%. The results tend to support the overall accuracy of the discriminant function. The classification accuracy matrix for the validation sample is presented in Table 2.

Furthermore, to interpret the meanings of the discriminant function, the structure matrix (correlations between the canonical discriminant function and the predictor variables) was rotated by the varimax method. The rotated structure matrix was examined to help explain the discriminant function. Two GATB variables, General Learning Ability (GATB - G) and Motor Coordination (GATB - K), were found to be highly correlated to the discriminant function (r's = .79 and .73 respectively). They also represent two of the three highest discriminant function coefficients on the function: GATB - G (1.91), GATB - K (.85), and GATB - V (-1.46). The coordinates of the two group centroids along the function are -.74 for the Sheltered Employment Group and .45 for the Competitive Employment Group. It appears that general learning ability and psychomotor skills are the most important variables in differentiating the employment potential of the rehabilitation clients in our study.

Discussion

The discriminant analysis results indicated that 79.5% of the clients in the validation sample and 70.6% in the cross-validation sample can be correctly classified by statistical prediction. As can be observed, statistical prediction formulas are only moderately successful in predicting work performance levels of people with disabilities. The chance for misclassification will always exist in vocational evaluation. Therefore, vocational evaluators using statistical prediction formulas as an interpretation aid must exercise caution in their recommendations, especially in light of the potential high cost of misclassification to both the society and clients with disabilities (Lachenbruch, 1975). Specifically, the misclassification of a client who is capable of working competitively into a sheltered program would result in a high cost to the society (in terms of wasted productivity), while the misplacement of a sheltered level client into competitive employment would result in a high personal cost (in terms of personal frustration) to the client. Fortunately, in practice, we have the flexibility of changing clients from one training program to another at both NRCD and NVRC. Therefore, the impact of misclassifications can be minimized.

A substantive interpretation of the discriminant function suggested the existence of different vocational ability requirements for clients in different vocational placement levels. The general learning ability and motor coordination scores appear to be the most important variables in differentiating the two client groups in this study. However, it should be noted that no emotional-coping variables, which are considered important predictors of competitive employment potential (c.f., McCarron & Dial, 1976), were included in this study. The inclusion of such measures could ultimately improve the classification accuracy of the discriminant function.

Finally, we have demonstrated the utility of statistical prediction as a supplementary method for interpreting assessment data and recommending vocational placement direction of people with physical disabilities in Japan. In addition, the Japanese version of the GATB appeared to be a useful tool for determining vocational potential of physically disabled persons. It appears that the statistical prediction approach is particularly appropriate for vocational evaluators in countries where the DOT and/or equivalent interpretation tools are not readily available.

KANETOSHI HATTORI, M.Ed., is a Technical Official of the National Rehabilitation Center for the Disabled, Ministry of Health and Welfare, Tokorozawa City, Japan. FONG CHAN, Ph.D., is an associate professor in the rehabilitation psychology area, Department of Psychology, Illinois Institute of Technology in Chicago, Illinois. [Table 1 & 2 Omitted]