Validation of a multivariate career and educational counseling intervention model using long-term follow-up.

In this study, the author sought to validate the effectiveness of a multivariate career and educational counseling intervention model through long-term follow-up of clients seen in private practice. Effectiveness was measured by clients' commitment to and enjoyment of their chosen career paths and the relationship of these factors to adherence to program recommendations. Findings indicated that 65% of clients adhered to program recommendations. Those who adhered to program recommendations were significantly more committed to their careers; however, no significant difference was found in clients' enjoyment of their careers. Additionally, 85% of clients reported that the program met their needs, and 95% would recommend the program.

Over the years, most career counseling practitioners continue to select assessment instruments; develop techniques; and discard, add, and refine procedures until they develop a method for career decision making that provides the most information, in the most efficient process, and yields effective, meaningful results and recommendations that are accepted and implemented by their clients. Unfortunately, many career counselors do not follow up with their clients to validate the effectiveness of the recommendations, or those who do fail to publish their validated model or techniques. Although it would be desirable for practitioners to allow researchers to have access to their data to study results (Osipow & Fitzgerald, 1996), this has not often happened. This is evident by the paucity of published studies addressing the efficacy, or validation, of career counseling models (DeBell, 2002).

The Career Choice Intervention Literature

Brown et al. (2003) conducted a meta-analysis of the literature focusing on career choice intervention. They identified five intervention ingredients that were individually associated with career choice outcome and were collectively related in a linear relationship to increases in career choice effect sizes. The five interventions were the following: (a) workbooks and written exercises that record clients' goals and future plans, (b) individualized interpretations of and feedback from clients (1) assessments, (c) world-of-work information given to clients while in session, (d) clients' exposure to models of career exploration and decision making, and (e) activities designed to build support for clients' career plans. Adding one, two, or three of the five critical ingredients yielded average effect sizes of .45, .61, and .99, respectively. No study used more than three of the critical ingredients. The current study incorporated all five.

Whiston, Brecheisen, and Stephens (2003) recently conducted meta analyses of other career choice intervention literature that spanned a period of 25 years (1975 to 2000) and reported that the majority of studies were conducted prior to 1985. Of those studies, only 36% used standardized outcome measures, and many of those used excessive outcome measures (as many as 32 in one study). Most study comparisons in the literature were done on computer interventions (33%), followed by workshops (17%) and individual counseling (16%). Dagley and Salter (2004), in their summary of the 2003 annual literature review for The Career Development Quarterly, stated that there was more emphasis on research than on practice, often reflecting a total neglect of the career intervention process. Guindon and Richmond (2005), in their annual literature review of career counseling for 2004 in the same journal, stated that few articles were grounded in what has actually been shown to work in career counseling and that most career intervention articles were conceptual discussions and lacked empirical evidence.

Origins of the Study Model

To address these needs in the career counseling literature, I initiated a study (through my doctoral dissertation) to validate the effectiveness of a multivariate career and educational counseling intervention model using long-term follow-up of clients seen in private practice. The model in this study was developed over the course of 16 years in my career counseling private practice. It was developed after studying the validity and results of published tests, determining the questions that were not answered or areas that were not addressed by standardized instruments, and developing interview and informal assessment techniques to complete a model that offered a set of procedures that could be validated. As a practitioner, I believed it was important to answer the following four questions to validate this model;

1. Did the client follow the career recommendations?

2. Is there a difference in commitment to and enjoyment of the chosen career path between clients who did and did not follow the career recommendations?

3. Did the client report that the career counseling process met his or her needs?

4. Would the client recommend the service to others?

Swanson noted in 1996 that trait-and-factor theory was enjoying a resurgence, observing that it had evolved into person-environment fit. She stated that this theoretical concept was a superb example of how theory and practice can be successfully interwoven, offering a great deal of theoretical and practical flexibility. Three assumptions underlie person-environment fit theory: (a) Individuals seek out and create environments that allow for behavioral trait manifestation; (b) the better the individual and environment fit, the better the outcome; and (c) the process of person-environment fit is reciprocal--the individual shapes the environment and the environment shapes the individual.

Rounds and Tracey (1990) identified the changes that took place as trait-and-factor counseling developed into person-environment fit. First, although person-environment fit continues to be primarily rational in nature, it now involves both cognitive and affective processes. Second, it now incorporates clinical and qualitative methods of decision malting to complement the more traditional types of assessment. Third, the client is an active agent in the counseling process, and it is a reciprocal, ongoing process that the client learns.

Crites (1981) outlined six approaches to career counseling in his book Career Counseling: Models, Methods,, and Materials.. In comprehensive career counseling, used in the current study, Crites described a system he said eliminates subjective judgments and claimed that it was more reliable. In this approach, all aspects of life functioning and development are considered. If a career decision is made, the impact of that decision has an impact on the client's philosophy of life, interpersonal relationships, self-concept, and other areas too numerous to mention.

Greenwood. Career Decision-Making Model

The Greenwood Career Decision-Making Model (GCDM) is an individualized approach to career counseling that brings the client into the decision making process. This model includes the integration of four distinct domains (see Figure 1): interests, personality, aptitudes, and values. An important component of the model is a structured interview to identify career paths that arc a match for the client. Interest assessments are the Strong Interest Inventory (SII; Strong, Campbell, & Hansen, 1985) and a career planner checklist; personality assessments are the California Psychological Inventory (CPI; Gough, 1987) and the Myers-Briggs Type Indicator (MBTI; Briggs, Myers, & McCaulley, 1962); aptitude assessments are the Differential Aptitude Tests (DAT; Bennett, Seashore, & Wesman, 1987); and values assessments are five self-report measures, All the assessments are conducted in a 6-hour, 1-day session. This session is followed by a 2nd day that consists of a 3-hour structured interview in the morning, a review of the test results in the early afternoon, career exploration activities generating a list of career options in the late afternoon, and formulation of an action plan at the end of the day. A final summary report is generated after the 2-day session that provides the test results, lists the salient personality and career characteristics, recommends potential career and educational paths, and ends with an action plan. This report captures what was articulated during the 2 days and makes it possible for clients to reflect on and use the information at any time, even at a later date if they feel unprepared to act on it at the time the sessions are completed. Clients can continue to consider their options using the process taught to them and the information provided, or they can return to see the counselor, if necessary.

[FIGURE 1 OMITTED]

The GCDM is an educational process whereby the client learns about him- or herself and the world of work and how to make a match between the two. It is both a trait-and-factor and a person-environment fit model. This model is research based and offers a unique combination of interventions applied to the decision-making process. It shares characteristics with many other theories of career counseling and decision-making models that have been described by Crites (1981). Similarities with the trait-and-factor and person-environment fit theories include arriving at a diagnosis of the career problem and considering the characteristics of the client and of work environments to identify careers that are a match. Similarities with the client-centered approach include the unconditional regard of the client, the mutual responsibility for problem solving shared with the client, the goal for the client of self-understanding, and the consideration of barriers that could prevent a client from reaching his or her goal. Similarities with the psychodynamic approach include the consideration of the client's dependence, lack of information, self-conflict, and choice anxiety. Similarities with the developmental approach include its use of directive and nondirective counseling techniques and consideration of other life roles while making decisions about careers. Similarities between the GCDM and the comprehensive career counseling approach are the most numerous but the main difference between these two is in the GCDM's more specific focus on career decision making rather than on learning general problem-solving techniques and/or gaining a sense of well-being as ways to solve career decision problems.

The GCDM is both a prescriptive and a descriptive decision-making model. The rational prescriptive model is incorporated in the use of objective assessments such as the SII and the DAT that can narrow the client's career options. However, the client's arrival at a final decision is also a descriptive, individualistic, and intuitive process incorporating the information gleaned during the structured interview and from the values clarification and the personality inventories of the CPI and the MBTI

The GCDM model differs from other career decision-making models in its use of a 2-day self-exploration process to identify career options for the client to explore. One could argue that 2 long days gives the client little time to digest all of the information gathered in that time. However, what is gained is the counselor's ability to consider multiple variables while generating, with the client, a list of potential careers. The nuances of a client's profile gleaned over the 2 days are not lost and are used and factored into the career decision-making process.

In summary, what is unique about the GCDM is the combination of the following intervention strategies: (a) The counseling delivery model is conducted over 2 full days (12 to 14 hours)--the 1st day is for assessments and the 2nd day is for interviewing, feedback on assessment results, and formulating an action plan; (b) an integrated standardized battery of assessments is given to every client to identify interests, aptitudes, values, and personality characteristics; (c) the emphasis is on educating clients about themselves and the world of work and how to make the match between the two; (d) the GCDM combines methods of both the prescriptive and the descriptive decision-making models; and (e) a standardized, written, summary report of the results that includes an action plan is given to the client for future reference following the 2-day experience.

Study Research Questions and Method

The purpose of this study was to investigate the effectiveness of a multivariate career and educational counseling model through a long-term follow-up study of clients seen in a private practice setting. Effectiveness was measured by assessing clients' commitment to and enjoyment of their chosen career paths and determining the relationship of these factors with client adherence to program recommendations. The null hypothesis was that career commitment and enjoyment for those who adhered to program recommendations would not differ significantly from that of those who did not adhere to program recommendations.

Pilot Study

I conducted an earlier pilot study to validate the survey instrument in the current study. I developed a 33-question survey for the pilot study to answer the research questions. Survey items were revised, eliminated, or added based on the findings. The survey was fine-tuned so that each item was limited to eliciting information on one specific topic or idea. Internal consistency was calculated using Cronbach's alpha, and all subdomains were found to have acceptable alpha values ranging from .88 to .93. The final version of the survey contained 32 items.

Sampling

Two trained counselors used a simple computer random sampling technique with a random start to select participants who had received services at least 3 years prior to the study. Seventy-eight clients of diverse ages, educational levels, backgrounds, and experiences (see Table 1) were selected from approximately 800 names in a master log of clients served. Participants were placed into one of two groups: those who had adhered to program recommendations for 3 or more years (called Hits) and those who had not adhered to program recommendations (called Misses). An expert panel of six professionals, all with professional training in career counseling and all professors at a university, determined whether a client had adhered to program recommendations on the basis of the information gathered on telephone interview surveys.

TABLE 1

Demographics of the Sample

                 Hits    Misses      Total
Variable         n   %   n       %   n    %
Gender
Male             20  57  15      43  35  45
Female           31  72  12      28  43  55
Age
<18 years        18  69   8      31  26  33
[greater than or 33  63  19      37  52  67
equal to] 18
years
Education
High school      20  62  12      38  32  41
High school +    13  62   8      38  21  27
College          12  63   7      37  19  24
graduate
Postgraduate      6 100   0         0   6      8

Note. Hits = clients who adhered to program recommendations;
Misses = clients who did not adhere to program recommendations.

Analyses

The units of analysis in this study were the two groups of clients: hits and misses. A telephone survey developed through a pilot study and conducted by two counselors (neither of whom was the author) was used to collect the data. A t test for independent means was used to test for differences between the mean scores of the two groups because there was no relationship between the two sets of scores. This study set the p value at .05. A two-tailed test of significance was used, which some view as more compatible with the growing meta-analytic view of social science as an incremental, cumulative, and shared enterprise"(Gall, Borg, & Gall, 1996).

Percentages were used to report (a) clients who had adhered to program recommendations and stayed with their career and educational decision for 3 or more years, (b) clients who reported a commitment to and enjoyment of their chosen career paths, (c) clients who expressed that the program met their needs, and (d) clients who expressed satisfaction with their career and educational counseling experience and would recommend it to others.

Results

Most important to this study was the number of clients who had adhered to program recommendations for 3 or more years. Of the 78 participants, 51 (65.4%) met this criterion.

The t test for independent means indicated that there was a significant difference (t = -2.11, p = .038) between the hits group and the misses group in how committed clients were to their careers. The clients in the hits group were found to be significantly more committed to their career paths than those in the misses group. However, no significant difference was found between the two groups in the degree to which they enjoyed their careers (see Table 2).

TABLE 2

t Test for Independent Means Comparing Client Satisfaction (Commitment
and Enjoyment) Between Hits and Misses (N= 78)

Variable       M    SD   df  t       p >t
Commitment
Hits        1.40  0.635  76  -2.11  .038 *
Misses      1.78  0.974
Enjoyment
Hits        1.31  0.510  76  -1.95  .055
Misses      1.60  0.750

Note. Hits = clients who adhered to program recommendations;
Misses = clients who did not adhere to program recommendations.
* p <.05.

Of the participants, 68 (87.2%) indicated that they had a commitment to their career path. Of the 10 who did not express a commitment to their career path, only 2 were in the hits group (see Table 3).

TABLE 3
Summary Statistics of Clients' Commitment to Their Chosen Career Paths

Variable                f   %    Cumulative f  Cumulative %

Strongly disagree = 4   2   2.6    78            100.0
Disagree = 3            8  10.3    76             97.4
Agree = 2              19  24.3    68             87.2
Strongly agree = 1     49  62.8    49             62.8

Note. M = 1.53; SD = 0.785.

Of the participants, 73 (93.6%) indicated that they enjoyed their career paths. Of the 5 participants who indicated they did not enjoy their career path, only 1 was in the hits group (see Table 4).

TABLE 4
Summary Statistics of Clients' Enjoyment of Their Chosen Career Paths

Variable                f   %    Cumulative f  Cumulative %

Strongly disagree = 4   0   0.0     78             100.0
Disagree = 3            5   6.4     78             100.0
Agree - 2              22  28.2     73              93.6
Strongly agree = 1     51  65.4     51              65.4

Note. M = 1.41; SD = 0.612.

Of the participants, 66 (84.6%) indicated that the program met their needs. Of the 12 who indicated that the program did not meet their needs, only 2 were in the hits group (M = 1.79, SD = 0.762).

Of the participants, 74 (94.9%) indicated that they would recommend this program to others. Of the 4 participants who indicated that they would not recommend the program to others, 2 were in the hits group (M= 1.67, SD = 0.620).

Fogerty and Black (1954) compared counselor-centered career counseling with client-centered career counseling and reported that the differences in the results were more related to the experience of the counselor. In the 1998 meta-analytic study of Whiston, Sexton, and Lasoff, they found that counselors-in-training had effect sizes larger than those of experienced counselors. The current study, using one counselor, found that during the first 6 years of the study (1987-1992), there were 45% hits and 55% misses and during the second 6 years (1993-1998), there were 72% hits and 28% misses. The increase in hits over time may indicate that some of the positive results are related to greater counselor experience or it could reflect economic changes in the job market over such a long period.

Discussion

This study was undertaken to investigate the effectiveness of a multivariate career and educational counseling program. Oliver and Spokane (1988) studied the intensity of the interventions given to clients and the effect on client outcome. Their studies indicated that when they calculated the ratio of effect size to the number of sessions, individual career counseling emerged as the most effective intervention per unit of time involved. Whiston et al. (1998) also found that individual counseling produced the largest effect sizes and was clearly the most effective form of intervention. The model used in this study is an individualized, 2-day, high-intensity program.

Whiston et al. (1998) also found that client satisfaction (n = 5) had a moderate effect size of 0.29 and that counseling effectiveness (n= 3) had a large effect size of 1.0. In this study, clients who expressed a commitment to what they were doing had a large effect size of .631, but clients who expressed enjoyment of their work had a moderate effect size of .457.

In the study conducted by Oliver and Spokane (1988), they called for researchers to include the outcome measures of career decision making, satisfaction, and effective role functioning. They also called for a standard battery of measures to be used when looking at outcomes. The current study reported in all of these areas.

In 1995, Hodkinson concluded that the value of professional guidance lies in the quality of the counseling given, rather than in the client outcomes or even their completion of action plans. Those who see career guidance as providing information plus professional advice, he stated, are probably doomed to failure. In this study that used a self-exploration career-counseling model, I found that 94.9% of clients would recommend the program to others even though only 65.4% were compliant with program recommendations. This supports Hodkinson's perspective.

Study Limitations

Possibly the biggest limitation of the study was that clients were self-selected and had paid for their counseling service. Paying for a service may increase a client's desire to see it in a positive light. However, it could also make a client more demanding and critical, wanting and expecting more from a service. The length of time that had passed since a client received the service may also have influenced some of the study results. Some clients had received the service as long as 10 years before the study took place. Another limitation would be the use of only one counselor, also the investigator (author of the article), who counseled all the clients in this study. To reduce bias, I did not take part in the collection of data for the study. I trained two other counselors to conduct the telephone surveys.

The sample sizes for this study were 51 in the hits group and 27 in the misses group. The use of a small sample size for the misses group resulted in an 11% chance of finding a small effect size if one existed, a 43% chance of finding a moderate effect size, and an 82% chance of finding a large effect size (Cohen, 1988). If the sample sizes, particularly for the misses group, had been larger, differences may have been easier to detect. A future study with larger sample sizes is recommended to allow greater confidence in the results.

Future Research

Future research should continue the call for a standardized test battery to be used in the career counseling process and in the research. The review of the career counseling literature has indicated that research is often based on the use of a single interest inventory administered to clients. Clients have little idea of what to expect when asking for career counseling. There is controversy related to this issue in the fields with counselors calling for an individualized test battery for career counseling rather than a standardized test battery. Both objectives could be accomplished if a core battery of tests were to be identified on which the majority of career counselors could agree. Then counselors could add any specific assessments that they had found to be particularly useful to them in their experience.

Because practitioners are rarely researchers, it is difficult to find long-term follow-up career counseling research conducted in private practices that evaluates the effectiveness of an intervention. More practitioners need to make their practices available to researchers for follow-up studies of their clients. This study evaluated a 2-day, intensive, self-exploration, career counseling intervention conducted in a private practice and found that it met clients' needs and was a valid model for effective career counseling.

References

Bennett, G. K., Seashore, H. G., & Wesman, A. G. (1987). Differential Aptitude Tests--computerized adaptive edition. San Antonio, TX: The Psychological Corporation.

Briggs, K. C., Myers, I., 8c McCaulley, M. H. (1962). Myers-Bnggs Type Indicator. Palo Alto, CA: Consulting Psychologists Press.

Brown, S. D., Krane, N. E. R., Brecheisen, ]., Castelino, P., Budisin, I., Miller, M., & Edens, L. (2003), Critical ingredients of career choice interventions: More analyses and new hypotheses. Journal of Vocational Behavior, 62, 411-128.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (3rd ed.). New York: Academic Press.

Crites, J. O. (1981). Career counseling: Models, methods, and materials. New York: McGraw-Hill.

Dagley, J. C, & Salter, S. K. (2004). Practice and research in career counseling and development--2003. The Career Development Quarterly, 53, 98-157.

DeBell, C. (2002). Practice for a paradigm shift: A complete model for an integrative course. The Counseling Psychologist, 30, 858-877.

Fogerty, E. W., & Black, J. D. (1954). A follow-up after three years of clients counseled by two methods, Journal of Counseling Psychology, I, 1-8.

Gall, M. D., Borg, W. R., & Gall J. P. (1996). Educational research: An introduction (6th ed.). White Plains, NY: Longman.

Cough, H. G. (1987). California Psychological Inventory, revised edition. Palo Alto, CA: Consulting Psychologists Press.

Guindon, M. H., & Richmond, L. [. (2005). Practice and research in career counseling and development--2004. The Career Development Quarterly, 54, 90-137.

Hodkinson, P. (1995). How young people make career decisions. Education + Training, 37, 3-8.

Oliver, L. W., & Spokane, A. R. (1988). Career-intervention outcome: What contributes to client gain: journal of Counseling Psychology, 35, 447-162.

Osipow, S. H., & Fitzgerald, L. F. (1996). Theories of career development (4th ed.). Boston: Allyn & Bacon.

Rounds, J. B., & Tracey, T. J, (1990). From trait-and-factor to person-environment fit counseling: Theory and process. In W. B. Walsh &S. H. Osipow (Eds.), Career counsel-ma; Contemporary topics in vocational psychology (pp. 1-44). Hillsdale, NJ: Erlbaum.

Strong, E. K., Jr., Campbell, D. P., & Hansen, J. C. (1985). Strong Interest Inventory. Palo Alto, CA: Consulting Psychologists Press.

Swanson, J. L. (1996). The theory is the practice: Trait-and-factor/person-environment fit counseling. In M. L Savickas & W. B. Walsh (Eds.), Handbook of career counseling theory and practice (pp. 93-108). Palo Alto, CA: Davies-Black.

Whiston, S. C, Brecheisen, B. K., & Stephens, J. (2003). Does treatment modality affect career counseling effectiveness? Journal of Vocational Behavior, 62, 390-410.

Whiston, S. C, Sexton, T. L., & Lasoff, D. L. (1998). A career-intervention outcome: A replication and extension of Oliver and Spokane (1988). Journal of Counseling Psychology, 45, 150-165.

Janet I. Greenwood, Greenwood Associates, Tampa, Florida. Correspondence concerning this article should be addressed to Janet I. Greenwood, Greenwood Associates, Inc., 310 South Brevard Avenue, Tampa, FL 33606 (e-mail: [e-mail]).