Cross-cultural perspectives on career assessment.

The authors present an overview of the discussion and presentations that took place concerning techniques and assessment in educational and vocational guidance at the recent symposium International Perspectives on Career Development. Two topics were examined. The 1st focused on the theoretical foundations of and psychometric issues in career assessment. The 2nd focused on the practice of career assessment and the use of career measures, particularly in cultural contexts for which the measures were not originally developed.

**********

Participants in the 2004 symposium, International Perspectives on Career Development, that was sponsored by the International Association for Educational and Vocational Guidance and the National Career Development Association explored issues and outcomes that differentiate career development practices in different nations. One of the designated discussion groups focused on issues of techniques and assessment in educational and vocational guidance. In this article, we report on the general discussion and specific presentations of participants from eight countries: Canada, China, Croatia, Portugal, South Africa, Switzerland, the United Kingdom, and the United States.

The discussion and presentations on techniques and assessment in career psychology addressed a wide range of issues that can be grouped into two main areas: the development of career measurement and the practice of career assessment. The first area explored both the theoretical foundations of career assessment and the psychometric issues in test development. The discussion on the practice of career evaluation and assessment focused on the issues of qualitative and quantitative career assessment and the use of career tests in cultural contexts for which they were not originally developed. These two areas of discussion and presentation are described below. They are followed by a summary of the group's conclusions concerning career assessment.

The Development of Career Assessment

The broader level of group discussion and individual presentation initially focused on the nature of career assessment and the definition of culture. Defining the terminology seemed a logical step in setting parameters for the group's discussion. A core issue that emerged was whether career assessment is psychological or psychosocial in nature. Glavin (2004; the third author of this article) differentiated between psychological tests, which represent an internal measurement of a specific personality trait, type, or competency, and psychosocial tests, which measure an individual's ability to adapt to the environment. He argued that career measures should be considered psychosocial tests in that they are based on assessing relationships between social factors and the individual. As such, career tests measure an individual against externally defined goals, such as career maturity, career adaptability, life role salience, and career self-efficacy beliefs. This discussion of the definition of career tests was pivotal to the ensuing discussions. At the core of this debate was Glavin's viewpoint that the constructs measured by career tests are not endogenous to all human beings. If one accepts this argument, then career constructs are not universal and, consequently, should not be generalized across cultural groups.

It could be argued that subsequent presentations offered considerable support for this differential definition of assessment. For instance, Rossier (2004) reported on the cross-cultural equivalence of several personality inventories in frequent use. Rossier's paper examined personality traits in Burkina Faso, a sub-Saharan African country, and in Switzerland. His results indicated that the structural equivalence of tests is affected by the theoretical differences on which the tests are based. More specifically, when tests are based on theories that ascribe the origins of traits as biologically based (such as McCrae & Costa's, 1999, five-factor theory), then partial and full structural equivalence can be observed across cultural groups. When tests are based on theories that are sensitive to cultural context and environmental influences, then structural equivalence is less likely to be observed. Rossier concluded that tests that are more dependent on cultural contexts are less stable across cultures. His presentation demonstrated that there is greater difficulty in using psychosocial as opposed to psychological tests with different cultural groups.

Participants discussed what was meant by culture concomitantly with their discussion of the definition of career assessment. Duarte (2004) argued that defining career tests as culture fair, culture free, or culture reduced was outdated. She proposed that ecological validity was a more useful concept than culture itself. This implies that to assess behavior in a particular culture, test development should be based on situation sampling (defining the relevant and observable aspects of a particular career construct), function sampling (refining test items in terms of how they could be operationalized within a specific cultural context), and the identification of differential variables and context information (e.g., patterns of cultural or subcultural rewards).

A paper delivered by Yuen, Gysbers, Chan, Lau, Leung, Hui, and Shea (2004) provided a good example of sensitivity to ecological validity. These authors reported on the development of a career self-efficacy measure for Chinese high school students in Hong Kong. Although the authors accepted the theoretical premise that students' beliefs about their abilities could influence their motivation in the career choice process, they attempted to measure such beliefs within the realities of the Hong Kong social context. Chinese adolescents living in Hong Kong grow up in a cultural context that restricts their freedom, limits their choices in educational and occupational structures, and expects loyalty to family and particular social groups.

There was a focus on the issue of cultural validity and cultural specificity in career assessment. Watson (2004) examined Leong's (1996) description of these terms within the multicultural context of South Africa. He concluded that most attempts to address the issue of culture in career assessment in South Africa had focused on cultural validity (i.e., the validation of the use of career tests based on Westernized career theory on national cultural groups by means of construct, concurrent, and predictive validity). Watson argued that the theme of cultural specificity, defined by Leong as an exploration of concepts, constructs, and models that are specific to a cultural group, has not been addressed in career assessment in South Africa.

Whereas Yuen and associates' (Yuen et al., 2004) paper addressed the issue of cultural specificity, other papers explored psychometric issues that arose in cultural validation studies. Sverko and Babarovic (2004), for instance, reported on research using a Croatian version of Holland's (1994) Self-Directed Search. Their results offer support for the cross-cultural validity of using instrumentation based on Holland's personality typology. However, this support was partial, in that the results supported the use of the model with students who are graduating from high school but not with students in midadolescence. Within the Croatian educational system, younger adolescents face critical vocational decisions. Sverko and Babarovic's findings may be partially explained by Glavin's (2004) assertion that Holland's Self-Directed Search is not really a career test but rather a personality test that results in an individual being assigned a personality type based on Holland's (1997) Realistic, Investigative, Artistic, Social, Enterprising, and Conventional (RIASEC) system. Glavin argued that Holland's theory is relatively context free. Such a viewpoint would support the earlier discussion that psychological tests can be more easily generalized to other cultures than psychosocial tests that are based more on the dynamic process of change.

Although Sverko and Babarovic's (2004) research demonstrated the cultural validation of Holland's (1994) measure, other participants reported less positive findings. Watson (2004), for instance, reviewed several factor-analytic studies that examined the construct validity of American measures with different South African population groups. This review demonstrated that the factor structure of several career measures (i.e., the Career Decision Scale, the Career Factors Inventory, the Career Decision Profile, the Commitment to Career Choices Scale and the Vocational Identity Scale) varied from the international versions across different age and cultural groups. Watson also reported on the structural variation between equivalent samples of South African and Australian adolescents. In addition, South African research has established that the structure of Holland's (1994) hexagonal model is misshapen for Black adolescents and that the specific order of Holland's RIASEC anagram could not be maintained.

Such findings question the construct validity of a measure, an issue that was discussed at length over the 2 days. Glavin's viewpoint that the equivalence of a construct needs to be examined within the culture to which it is applied was aptly illustrated by several points raised in discussion and presentations. Conceptual equivalence (i.e., whether constructs are meaningful in the culture being studied) is a psychometric issue that examines cultural specificity. Both Glavin (2004) and Watson (2004) referred to the construct of career maturity in order to illustrate this issue. Career maturity is defined as an individual's readiness to make career or educational decisions. Western culture measures this construct by examining the extent of planning, thought, and exploration an individual has undertaken in relation to a future career. As such, career maturity implies independent thought and planning, self-sufficiency, and individual achievement, all of which are valued in individualistic cultures. However, individuals in collectivist cultures grow up valuing interdependence, group goals, and group rewards. Glavin argued that such an individual may score low when given a Westernized career maturity measure. Watson supported this viewpoint by reporting on psychometric research in South Africa that demonstrated that the self-identity of Xhosa-speaking South Africans is based on the belief that "a person is a person through others" and that this belief may negatively affect their scores on Westernized career maturity measures.

The discussion on the adaptation and translation of career measures led to the critical point that conceptual/construct equivalence should precede issues of translation, such as linguistic, scale, and normative equivalence. Duarte warned career test developers of the potential trap of equating the linguistic equivalence of test items as representing equivalence with the construct dimensions of the original career measure that is being adapted. It seemed that the use of tests in most countries demonstrated the reverse order to that discussed, with conceptual/construct equivalence becoming a post hoc psychometric activity. In essence, construct equivalence equates to cultural specificity, and Glavin argued that this is the most basic of psychometric factors to consider in career assessment and also the most pressing. Similarly, Duarte warned that adapting and translating American career measures for use in other countries raise a host of questions concerning the final product (i.e., a test that is ready for use in a different country, in a different language, and for a different culture).

Part of the discussion on cultural validation and specificity focused on test developers and users. Glavin supported literature that calls for test developers to view the test development process from a "culturalistic" perspective (i.e., to consider cultural factors in the earliest stages of test development). Both Glavin and Watson suggested that career counselors and researchers needed to step out of their own reality to consider the reality of clients from other cultural groups. Watson warned against becoming culturally encapsulated, whereas Glavin suggested that problems in career assessment arise when an ethnocentric viewpoint is adopted. Duarte suggested the importance of matching observable aspects, content choices, differential variables, and context information.

The Practice of Career Assessment

The practice of career assessment in individual and group contexts was a secondary focus of the discussion group. In a sense, much of the preceding discussion has a direct impact on the practice of career assessment, particularly in terms of the appropriateness and relevance of the measures that career counselors use in the contexts within which they practice. There was considerable discussion on qualitative and quantitative career assessment. The definition of qualitative and quantitative was not limited to the nature of the measures themselves but also to their application. Glavin (2004) stated that any career assessment is essentially a tool to facilitate discussion based on a client's current life or career situation, and Duarte added the importance of considering further career development steps. Similarly, several participants argued that the exploration of vocational behavior through assessment is limited to the specific context within which the client's behavior occurs. Thus, career assessment needs to be interpreted in conjunction with an interpretation of and sensitivity to the client's context. This qualitative use of career measures is challenged by the quantitative expansion of career assessment practices. Glavin made the point that the reality of Web-based applications is that career assessment can be applied to anyone, anywhere, at any time. This quantitative aspect of career assessment raised several concerns about how the results of career assessment are processed. One of the challenges facing career assessment as it expands its market is the inherent harm of test results that are self-interpreted. This debate raised ethical issues concerning the lack of a qualitative component in career assessment interpretation.

Concluding Remarks

Watson's research and experience in South Africa provide support for Leong's (1996) suggestion that career counselors need to move toward cultural diversity using cultural specificity rather than cultural validity. The latter involves starting with an international instrument and exploring whether or not the theory on which it is based, and the constructs it measures, can be generalized across other cultures. In essence, one is starting at the end with a completed instrument and then moving backward in an attempt to measure the goodness of fit between the instrument and the culture to which it is being applied.

Cultural specificity, however, starts from the beginning and moves forward, exploring the concepts, constructs, and models that are specific to a particular group. Starting with a cultural point of origin promotes the development of culture-specific measures that are based on the values, attitudes, and belief systems of the culture within which an individual is operating. Watson (2004) cited South Africa as an example of a country that is taking steps toward the exploration of cultural diversity using the concept of cultural specificity. These steps are evidenced by the implementation of the Employment Equity Act of 1998. This law states that psychometric assessment is only defensible when it has been proven to be scientifically valid, is applied fairly to all, and is not biased against any particular population group. This act demonstrates a zero-tolerance approach toward inferior and culturally inappropriate tests. However, it also poses problems. As Watson stated, South Africa contains within it a rich variety and a large number of cultural groups. It is questionable how feasible it would be to create an instrument that could be fairly administered to all such groups. Even if it is possible, there exists such a lack of skilled test developers that it may not be practical. Therefore, cultural specificity leaves career counselors with the dilemma of knowing which is the ideal method to address cultural diversity in test development (i.e., through the creation of culture-specific instruments), yet the concept itself may be of an impractical nature given the limited resources available to other countries.

Watson (2004) made reference to the problems caused by a critical shortage of skilled test developers. This shortfall in skilled labor identifies serious challenges to the development of culture-specific measurements. Because of a lack of money and human resources, many countries are faced with a difficult dilemma. Do they invest the time and money it would take to create their own, culture-specific, career instruments or take a shortcut and adapt American-made measures? Glavin (2004) believed the time and money saved would lead to inadequate and inappropriate test measures, delivering misleading and inaccurate test results in the short run. In the long run, such problems would occur at an increasing rate until culture-specific measures were created and used.

Ultimately, it is important to assess individuals by using instruments that are based on the principles and values of their own culture. Glavin (2004) suggested that one solution is to use international instruments and have the results interpreted by a trained professional who is aware of the cultural disparities between the individual being assessed and the culture in which the measure was originally devised. In essence, this is the practice of qualitative interpretation using quantitative results. Watson, however, raised the point that this solution would not prove practical when administering an instrument to a large population. Qualitative interpretation would become too time-consuming and costly an endeavor.

Career counselors are challenged with educating themselves about the issues involved in administering and adapting international measurements. Career instruments should be developed by skilled test developers with personal knowledge of the culture within which they are operating. As career counselors, we can consult with and provide advice to international test developers and administrators. However, the actual test development should take place at the grassroots level, by skilled developers, within the culture of origin.

The prevalence of American models and their measures in psychological research is a major challenge to the international community. Observing in other cultures how practitioners deal with the interpretation and meaning of career measurement and how individuals deal with career problems can provide new perspectives in the attempt to manage established theoretical assumptions. It could also be a way to prove that similar ends can be achieved with different means. This is scientifically pertinent. Confronting theory, construct interpretation, and meaning in career measurement is a global enterprise toward building a fair society. Such confrontation can be translated into methods of research that focus more on integrating ideas on career measurement and on exploring relationships among such measures (i.e., on contextual meaningfulness).

References

Duarte, M. (2004, June). Assessment and cultural niches: Adaptation of psychological instruments and "the global (research) village." In International perspectives on career development. Symposium conducted at a joint meeting of the International Association for Educational and Vocational Guidance and the National Career Development Association, San Francisco.

Glavin, K. (2004, June). Generalizing constructs versus adapting psychological tests for global use. In International perspectives on career development. Symposium conducted at a joint meeting of the International Association for Educational and Vocational Guidance and the National Career Development Association, San Francisco.

Holland, J. L. (1994). Self-directed search: Assessment booklet. A guide to educational and career planning. Odessa, FL: Psychological Assessment Resources.

Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed., pp. 1-15). Odessa, FL: Psychological Assessment Resources.

Leong, F. T. L. (1996). Career interventions and assessment in a multicultural world. In R. Feller & G. R. Walz (Eds.), Career development in turbulent times: Exploring work, learning and careers (pp. 275-283). Greensboro, NC: ERIC/CASS.

McCrae, R. R., & Costa, P. T., Jr. (1999). A five-factor theory of personality. In L. A. Pervin & O. P. John (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 139-153). New York: Guilford Press.

Rossier, J. (2004, June). An analysis of the cross-cultural equivalence of some frequently used personality inventories. In International perspectives on career development. Symposium conducted at a joint meeting of the International Association for Educational and Vocational Guidance and the National Career Development Association, San Francisco.

Sverko, I., & Babarovic, T. Y. (2004, June). Holland's model in Croatia. In International perspectives on career development. Symposium conducted at a joint meeting of the International Association for Educational and Vocational Guidance and the National Career Development Association, San Francisco.

Watson, M. B. (2004, June). Career assessment in a multicultural context: A South African perspective. In International perspectives on career development. Symposium conducted at a joint meeting of the International Association for Educational and Vocational Guidance and the National Career Development Association, San Francisco.

Yuen, M., Gysbers, N. C., Chan, R. M. C., Lau, P. S. Y., Leung, T. K. M., Hui, E. K. P., et al. (2004, June). Developing a career development self-efficacy instrument for Chinese high school students in Hong Kong. In International perspectives on career development. Symposium conducted at a joint meeting of the International Association for Educational and Vocational guidance and the National Career Development Association, San Francisco.

Mark Watson, Department of Psychology, Nelson Mandela Metropolitan University; Maria Eduarda Duarte, Faculty of Psychology and Education, University of Lisbon; Kevin Glavin, Department of Adult, Counseling, Health and Vocational Education, Kent State University. Correspondence concerning this article should be addressed to Mark Watson, Department of Psychology, Nelson Mandela Metropolitan University, PO Box 77000, Port Elizabeth, 6031, South Africa (e-mail: [e-mail]).