Treatment Outcome of Four Rehabilitation Interventions for Persons with Serious Mental Illness.

The differential impact of participating in one of four rehabilitation interventions (clubhouse model program, consumer self-help group, social-skills training, and vocational-skills training) among persons with serious and persistent mental illness was investigated. When controlling for demographic variables, analysis of covariance revealed no differences among the four programs with respect to subjective quality of life, empowerment, mastery, satisfaction, and psychiatric hospitalization frequency. Implications for professional practice and research are examined in the context of present research findings.

The unemployment rate among persons with serious mental illness (SMI) is far greater than their disability warrants. Only 20% to 30% of people with SMI are competitively employed within one year following hospital discharge (Rogers, Anthony, Toole, & Brown, 1991). Outcome studies have found that only between 8% and 15% of people with SMI are involved in part-time or full-time employment (Anthony & Blanch, 1987; Braitman et al., 1995). In an effort to counteract high rates of unemployment and psychiatric hospital recidivism, rehabilitation interventions that provide on-going social support, education, and work-skills training have been used (Attkisson et al., 1992). Unfortunately, few studies exist regarding which treatment interventions are most effective for persons with SMI (Bond, Witheridge, Setze, & Dincin, 1985; Booth, 1993; Rosenfield & Neese-Todd, 1993). Four approaches--clubhouse models, consumer self-help groups, social-skills and vocational-skills training programs--have been described in the literature with varying degrees of success in facilitating employment and independent living.

Clubhouses are treatment programs that allow consumers with SMI to work part-time at community jobs until they were able to obtain full-time work. Clubhouses usually have an egalitarian hierarchy that promote consumer empowerment. In the clubhouse approach, members have jobs that pertain to the daily operation of the program. Characteristics that often typify clubhouses include: (a) voluntary membership where persons can re-enter at any time, (b) duties associated with maintaining the program are shared between staff and clubhouse members, and (c) vocational training designed to build self-esteem and self-confidence are emphasized more than acquiring job skills (Propst, 1992). In one of the few available studies that examined the impact of the clubhouse approach, Arns (1992) found that length of attendance was positively related to employment status as well as changes in vocational skills, physical functioning, personal care ability, interpersonal relationships, and social acceptability.

Consumer self-help groups were formed as a result of dissatisfaction with the mental health system and problems caused by societal barriers (Emerick, 1990). One application of the self-help program is the drop-in center that offers fellowship, recreational activities, and meals to persons with SMI (Bond et al., 1990; Webb, 1973). As in the case of clubhouse models, there is limited information as to the efficacy of this approach in comparison to other interventions (Ryback, 1971; Toro et al., 1988).

Because of social interaction problems that consumers with SMI experience (Mueser, Wallace, & Liberman, 1995; Penn & Mueser, 1996), rehabilitation programs designed to improve social skills can also facilitate employment and independent living. Studies on the impact of social-skills training for persons with SMI have found both effective (e.g., Mueser et al., 1995; Penn & Mueser, 1996) and partially effective interventions (Harpin, 1981; Hogarty et al., 1986; 1991). Typically, this intervention consists of teaching persons with SMI to develop more effective communication skills through role-play, supervision, and homework assignments (Accordino & Herbert, 1997). Social-skills training is effective in reducing symptoms such as delusions, hallucinations, depression, and impulsivity (Attkisson et al., 1992; Liberman, DeRisi, & Mueser, 1989).

Prevocational-skills training and transitional and supported employment comprise vocational-skills training (Rutman, 1994). Prevocational-skills training includes information on job interview preparation and the interview process, job application forms, resumes, job lead sources, and job retention skills (Herbert, 1987). Transitional employment, a type of vocational training, requires consumers to work in paid or unpaid, part-time, short-term (less than 6 months) employment. Supported employment requires consumers to obtain competitive employment for at least 20 hours per week with support from an on-site job coach (Powell et al., 1991).

Many outcome studies lack appropriate research methodology and statistical designs necessary to evaluate program effectiveness. This criticism has been raised in program evaluation of clubhouse models (Neese-Todd & Weinberg, 1992), consumer self-help programs (Jacobs & Goodman, 1989; Kaufmann, Freund, & Wilson, 1989), social-skills training (Penn & Muesser, 1996), and vocational-skills training (Bond & McDonel, 1991; Cook & Rosenberg, 1994). In addition, few studies have examined differential outcomes as a result of various treatment interventions (Bond et al., 1985; Booth, 1993; Rosenfield, 1992; Rosenfield & Neese-Todd, 1993; Speer & Newman, 1996). Past studies of program effectiveness tend to use functional assessments that address consumer perspectives and/or contain selective objective behavioral criteria such as psychiatric hospitalization. Consumer-oriented outcome variables that have received increasing attention in the literature include perspectives on quality of life, empowerment, mastery, and program satisfaction.

OUTCOME VARIABLES AND REHABILITATION INTERVENTIONS

A review of the literature concerning outcome-based research and treatment interventions of persons with SMI have used a variety of criteria to determine program effectiveness. Certain outcome criteria such as subjective quality of life, consumer satisfaction, and psychiatric rehospitalization have witnessed a great deal of interest while other promising outcome measures using empowerment and mastery indicators are limited. For instance, subjective quality of life is a general feeling of well-being and overall satisfaction with current aspects of one's life (Lehman, 1983, 1988). In comparison to the general public, persons with SMI report having lower quality of life (Lehman, Ward, & Linn, 1982; Rosenfield, 1992; Van Dongen, 1996). This result was attributable to problems associated with inadequate living arrangements, poor interpersonal relationships, and susceptibility to crime (Lehman, 1983); with employment (Fabian, 1992; Lehman, 1983; Rosenfield, 1992), self-esteem (Ams, 1992), and personal finances (Fabian, 1992). Consumer satisfaction, perhaps one of the two most popular outcome measures, reveals that consumers are, for the most part, very satisfied with mental health treatment services provided (e.g., Dobrzykowski, 1997; Lebow, 1982, 1983; Lehman & Zastowny, 1983; Pratt & Gill, 1996). The other popular outcome measure, psychiatric hospitalization, is often used because it can predict subsequent hospitalization (e.g., Goeke, 1993; Marino, 1984; Mintz, O'Brien, & Clancy, 1976; Setze & Bond, 1985). This criterion is a popular benchmark in determining treatment intervention effectiveness (e.g., Booth, 1993; Harrington, 1993; Marino, 1984). The outcome measure of empowerment, which involves helping consumers take control of their lives and make decisions independently, represents a major goal of psychosocial rehabilitation programs (Rappaport as cited in Rosenfield, 1992). Despite its reported importance, few studies have used this outcome measure and, of those available, data suggest that empowerment correlates positively with other quality of life indicators (e.g., Nelson, Wiltshire, Hall, Peirson, & Walsh-Bowers, 1995; Rosenfield, 1992). Similarly, empirical contributions regarding mastery are limited. However, Ams and Linney (1995), found that individuals who reported higher mastery were less likely to be re-hospitalized.

Although the five aforementioned variables offer promise as indicators of successful rehabilitation, their relationship between vocational and nonvocational outcomes is limited and inconclusive (Drake et al., 1994; Drake, McHugo, Anthony, Becker, & Clark, 1996; Fabian, 1989; 1992). The primary purpose of this study was to investigate the effectiveness of four treatment interventions (clubhouse model program, consumer self help group, social-skills training, and vocational-skills training) using multiple outcome measures (subjective quality of life, empowerment, mastery, satisfaction, and psychiatric hospitalization frequency). A secondary purpose examined the impact of consumer demographic variables on treatment outcome. In particular, a number of studies have found that when demographic variables are grouped together, they tend to exert a noticeable influence on rehabilitation outcomes (e.g., Ams & Linney 1995; Berzins, 1977; Fabian, 1989; Goeke, 1993; Lehman, Rachuba, & Postrado, 1995; Rogers et al., 1991). A final purpose examined the reliability (internal consistency) of a relatively new experimental instrument that offered promise as a comprehensive outcome measure, the Psychosocial Rehabilitation Outcomes Toolkit. Results from the study would help mental health counselors gain a better understanding of the relative, merits of these four treatment interventions, demographic variables related to outcome, and utility of the experimental measure for professional practice.

METHOD

This study used a cross-sectional correlation design of four comparison groups. Participants included persons with SMI who were already enrolled in clubhouse programs (n = 26), self-help program (n = 15), social-skills training (n = 78), or vocational-skills training (n = 28). The four treatment programs were part of services provided by a community mental health agency located in Western Pennsylvania. A description of the treatment programs and participants follow.

Clubhouse Program

In contrast to other service programs, the clubhouse program included more males (n = 17) than females (n = 9) with 96% maintaining regular staff contact. The clubhouse provided one-on-one tutoring by respective members to improve vocational and social skills. There were several options within this program. Persons could volunteer in one of three employment settings: (1) administrative unit--members managed operational duties such as answering telephone calls, publishing a monthly newsletter, and providing orientation tours; (2) employment unit--members performed office work (typing letters, mailing), tutored members in the part-time transitional employment (TE) on academic and vocational issues; and (3) food service--members performed jobs such as bus person, cashier, cook helper, food stocker, and waiter as part of preparing agency meals. Most participants (58%) were working at the time of the study. Clubhouse participants were assigned the following diagnoses: schizophrenia and other psychoses--40% (n = 11), major depression or other mood disorders--32% (n = 8), bipolar disorders--16% (n = 4), and other axis I disorders--12% (n = 3).

Self-help Program

Similar numbers of females (n = 6) and males (n = 9) participated in the self-help program with the majority of participants maintaining regular staff contact (93%). This program provided support in daily coping skills, academic training (reading and writing), and recreational activities (arts and crafts, bingo, and bowling). Volunteer employment opportunities were available in food service catering and receptionist work. Because the self-help program did not retain any records regarding consumer attendance or services provided, no written documentation regarding individual treatment goals or plans was made. The majority of persons in the self-help program were not working (67%) at the time data were collected. Participants were assigned the following diagnoses: schizophrenia and other psychoses 40% (n = 6), major depression or other mood disorders--33% (n = 5), bipolar--13% (n = 2), other axis I disorders--7% (n = 1), and conditions not attributable to a mental disorder (v-codes)--7% (n = 1).

Social-skills Program

An equivalent number of females (n = 36) and male (n = 42) participated in the program with the majority (91%) having regular staff contact. The program addressed interpersonal communication, social awareness, coping with mental illness, stress management, and personal hygiene issues. These topics typically required one hour each day and consisted of lectures and group discussions. The remaining time participants could volunteer at the agency's food service program or other businesses within the community. Participants were assigned the following diagnoses: schizophrenia and other psychoses--51% (n = 40), major depression or other mood disorders--24% (n = 19), bipolar-11% (n = 9), personality disorders-5% (n = 4), other axis I disorders-4% (n = 2), organic mental disorders-3% (n = 2), and conditions not attributable to a mental disorder (v-codes)-3% (n = 2).

Vocational-skills Program

There was an equal number of females and males (14 each) who participated in this program with a majority (89%) having regular staff contact. This program provided prevocational-skills training and supported employment. Consumers first participated in prevocational-skills training for a period not exceeding 6 weeks; training length varied based on each person's functional abilities. Persons then could enter one of three for-profit, supported employment units: (a) business unit-consumers sorted mail and prepared packages for shipping, (b) RenCare-consumers were trained to provided therapeutic peer support to residentes with SMI who were in personal care boarding homes, and (c) cartridge factory-consumers learned how to recondition used computer laser-printer cartridges. All consumers were paid a minimum hourly wage of $4.75. The majority of participants were working (96%) at the time survey data were collected. Participants were assinged the following diagnoses: schizophrenia and toehr psychoses-68% (n = 19), major depression or other mood disoders-21% (n = 6), bipiolar disorder-7% (n = 2),

Measures

We used the Psychosocial Rehabilitation Outcomes Toolkit, a 20-item experimental scale developed by the International Association of Psychosocial Rehabilitation Services (IAPSRS, 1995). The Toolkit contains four scales (Empowerment, Mastery, Program Satisfaction, Quality of Life); each include five items that use the following anchoring system: 1 = Strongly Disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly Agree. Scores range from 4 to 20 with higher scores indicating positive outcome. Examples of scale items include: Item 1, "If I were given the opportunity I am sure I could work." (Mastery Scale); Item 2, "Overall, I am satisfied with my life." (Quality of Life Scale); Item 5, "The services offered at this program are relevant to my needs." (Program Satisfaction Scale); and Item 15, "My opinions and ideas count in the development of my rehabilitation plan." (Empowerment Scale).

One-week test-retest and internal consistency reliability estimates from a nationwide study of 13 community mental health programs conducted by Arns (1997) were as follows: quality of life r =.82, [Alpha] = .68; mastery r = .80, [Alpha] = .71; empowerment r = .64, [Alpha] = .63; and program satisfaction r = .79, [Alpha] = .78, respectively. With the exception of the empowerment scale, test-retest findings (one-week period) suggested moderate reliability levels. It should be noted that a major limitation of the test-retest reliability estimates for the empowerment scale could be attributed to low sample size (n = 11) (personal communication, Paul G. Arns, September 22, 1997).

Procedure

To reduce experimental bias, a college student trained to administer the Toolkit collected data. Any participant associated with the four treatment programs did not know this individual. Ostensibly, this would reduce the influence of perceived power by the test administrator; an aspect that tends to reduce participant bias (Gill, Pratt, & Librera, 1998). Self-help group members, however, expressed concern about giving any information to an unknown person and only permitted an agency administrator to collect their data. Consequently, we decided to comply with this request rather than omitting data from the study. Only consumers who received services from one program were eligible to participate.

Data Analysis

In order to investigate the differential impact of each program, a hierarchical regression analysis was used. This procedure statistically removes the influences of confounding or unwanted variables on the dependent variables (Cohen & Cohen, 1983). Independent demographic variables were entered in a predetermined sequence based on causal priority. Given the relative contribution of demographic variables found in previous studies (e.g., Arns & Linney, 1995; Fabian, 1989; Lehman et al., 1995; Rogers et al., 1991) the sequence of independent variables was as follows:

* Gender (male or female)

* Race (White or NonWhite)

* Education (number of years of education)

* Employment status (working or not working)

* Diagnosis (schizophrenia/other psychoses or other axis I or axis II disorders)

* Length of enrollment in program

* Previous number of jobs since enrollment

* Total hospitalization frequency since enrollment

This combination of independent variables was used to predict dependent outcome variables of subjective quality of life, empowerment, mastery, consumer satisfaction, and psychiatric hospitalization frequency since program enrollment. It should be noted that hospitalization frequency was not included in the same regression equation where it served as a dependent variable. The statistical test (F test) used in this study assessed whether the additional proportion of outcome variance accounted for by participating in one of four rehabilitation programs was statistically significant beyond that accounted for by demographic variables alone. In conducting the hierarchical regression analysis the adjusted R2 value was used as an indication of the population since it accounts for the number of parameters in the regression model (Neter, Kutner, Nachtsheim, & Wasserman, 1996).

In order to assess whether consumers of the four community mental health programs reported different levels of subjective quality of life, empowerment, mastery, program satisfaction and psychiatric rehospitalization, analyses of covariance (ANCOVA) were conducted to determine differences among programs on each dependent variable. Demographic variables (gender, race, education, employment status, diagnosis, length of program enrollment, previous jobs since enrollment, and psychiatric rehospitalizations since enrollment) were used as covariates.

Power analysis. Recommended power levels are between .70 and .90 (Cohen, 1988). Using Cohen and Cohen's (1983) formula, a power of .80, alpha level of .05, an hypothesized "small" effect size (.09), and number of independent variables (program and demographic variables totaling 11), determined that at least 136 participants would be needed. Although data were collected from 147 participants, because of incomplete data, the sample size consisted of n = 112 for dependent variables of quality of life, empowerment, mastery, and program satisfaction, and n = 113 for hospitalization frequency. Using this sample size, the a priori power estimate was .70. Dummy coding was used for program variables to improve power by reducing the number of independent variables (Cohen & Cohen, 1983). The self-help program served as baseline for the self-help, social, and vocational programs and was not entered into the regression analysis. Finally, to ensure stability of results, we applied the 10 to 1 number of participants to criterion minimum ratio role (Nunnally, 1967). Given that there were 11 independent variables and no les,; than 112 participants in any of the hierarchical regression analyses, results from this analysis were considered stable.

Internal consistency. With the exception of the Ams (1997) study, there appears to be no other research investigation that has examined the reliability of individual scales. Using the internal consistency estimate formula by Cronbach (1951), the four scales contained in the Toolkit (quality of life, mastery, empowerment, and program satisfaction) were examined.

RESULTS

Internal consistency estimates (Cronbach's alpha) for the research instrument (Toolkit), revealed very low estimates for empowerment (r = .48; n = 141) and quality of life (r = .56; n = 142) scales. These estimates suggest that individual test items associated with these scales may be unreliable. Accordingly, the results of these measures will not be presented. In contrast, the reliability estimates for mastery (r = .71; n = 139) and program satisfaction (r = .74; n = 141) scales were more robust yet still considered weak.

Results from the hierarchical regression analyses indicated that participation in the four treatment programs did not appreciably result in predicted variance beyond that accounted for by the demographic variables for mastery (F [3,100] = .83), program satisfaction (F [3,100] = 1.50), and psychiatric hospitalization (F [3, 100] = 1.70). Examination of the adjusted R2 values after demographic variables were entered into the equation indicated no noticeable increase as a function of mental health program assignment. Specifically, the adjusted R2 values resulted in a minimal drop for mastery (1%) and a minimum increase of 2% for both program satisfaction and psychiatric rehospitalization.

Analyses of covariance (ANCOVA) revealed no statistically significant differences across the four programs on the three dependent variables of interest (see Table). In other words, once the variance attributed to the combination of demographic variables was accounted for, persons with SMI reported similar levels of mastery, program satisfaction, and hospital frequency. The F values for the ANCOVAS were as follows: mastery: F(11,100) = .77, p = .52, d = .18; program satisfaction: F(11,100) = 1.68, p = .18, d = .26; and total hospital frequency: F(10, 102) = 1.63, p = .18, d = .25. Effect size coefficients fell in the "small" range (i.e., .02 to .14) indicating a slight relationship between variables (Cohen, 1988).

DISCUSSION

The additional amount of predicted variance noted in the hierarchical regression analyses was quite small, indicating that the major predictor variable of interest, program treatment, was unable to explain a significant amount of variance associated with dependent variables. Further, ANCOVA results indicate that program assignment did not differentiate outcome variables once relevant demographic variables were considered. There are several possibilities to explain these findings. First, results could mean that regardless of program treatment (clubhouse, self-help, social, and vocational), the differential therapeutic benefit impact among programs is nonexistent. In essence, no matter which program individuals with serious mental illness (SMI) are assigned, self-reported benefits associated with mastery and consumer satisfaction are negligible. An examination of reported means on dependent measures indicate that persons were generally satisfied with current aspects of their lives as well as the kind of mental health treatment they received.

Second, the lack of statistical difference among programs might be that there were common factors inherent in all four programs. Specifically, there was a continuum of employment opportunities among the four programs, making them more similar than different. It also seems that it does not matter whether the employment experience is paid or unpaid, nor does it matter to what extent staff support is evident in maintaining employment and receiving therapeutic gains. For example, the Clubhouse Program provided volunteer employment opportunities and, if successful, opportunities for part-time paid employment. The Self-help Program offered volunteer employment opportunities within the psychosocial program. The Social Program offered prevocational-skills training in a variety of community volunteer work experiences. The Vocational Program incorporated prevocational-skills training with supported employment. It should be remembered that at the time data were collected, 71% of individuals participated in volunteer work. Results from this study indicate that mental health programs that offer various work experiences will result in therapeutic gains--a consistent theme noted in the literature (Arns, 1992; Attkisson et al., 1992; Bolton & Akridge, 1995; Drake et al., 1994; Rutman, 1994). It may also be that consumers require certain services at different stages in their lives. For example, consumers may benefit from a social program after discharge from an inpatient facility. When consumers are able to learn and use social skills, they could enter either a clubhouse or vocational program since both

offer vocational training. Consumers requiring more job supervision would benefit from a vocational program, while those wishing to work more independently would benefit from a clubhouse program. Consumers who have been able to hold a job for a considerable amount of time, unable to hold a job at all, or who are at retirement age would seem to benefit most from a self-help program which offers less structured treatment but more peer interaction and support. In short, each program seems to offer valuable services to consumers at different life and developmental stages. Such information should aid mental health counselors working with clients who are deciding on whether to enter into either volunteer or paid employment.

A third reason to account for the lack of statistical significance concerns the outcome measure used in this study. Despite its reported advantages as a promising research tool (IAPSRS, 1995), results from this study indicate that there are potentially serious problems with the instrument. In particular, two of the subscales, empowerment and quality of life have very poor internal consistencies. Problems with these scales, if accurate, suggest that the instrument requires additional study to establish the existence of the four scale model. Given that the only available factor analytic study is the one by Arns (1992), further investigation is warranted. Future research studies using sufficiently larger samples than the one used in this investigation should consider using a confirmatory factor analysis to support or refute the four-factor structure. In fact, after the results of this study were obtained, IAPSRS (1998) modified the Toolkit as a result of confirmatory factor analyses and has reworked existing items needed to improve the internal consistency estimates of the instrument.

One source of information that was lacking in assessing treatment outcome was how different types of mental illness and their severity might have influenced variations in the dependent variables and possibly confounding the results. The independent variable of diagnosis was divided into either one of two categories: (1) schizophrenia/other psychoses or (2) other axis I or axis II disorders. Consequently, data that contained greater refinement of participant diagnoses were not included in the analyses. Using larger data sets that allow for a finer discrimination of psychiatric diagnosis could be used in future studies. Used alone, however, diagnosis is not predictive of rehabilitation outcome (Anthony, 1994).

Despite the lack of statistical significance across treatment outcomes, it is important to recognize that consumers in all four programs reported positive outcomes. Although these findings are consistent with those reported by Lebow (1983) regarding program satisfaction, they are in contrast to other studies that reported lower levels of mastery (Rosenfield, 1987). Several reasons to account for the discrepancy between the current study and other investigations may be due to the outcome measure used. As noted earlier, problems with internal consistency associated with the Toolkit suggest that the instrument did not reliably measure the four intended constructs. In addition, the relatively small amount of predicted variance associated with program treatment may occur as a result of lower statistical power to find significant differences. It should be remembered that because of missing data, the a priori power estimate dropped from .80 to .70 and may have impeded the ability to detect statistically significant results. Another limitation of the study was the lack of randomization of participants, which impeded internal validity.

As noted earlier, this study is one of few investigations that examined multiple treatment outcomes while accounting for demographic influences. Certainly, further quantitative investigations such as this one are needed to examine further variables that predict successful outcomes. Future studies that would benefit mental health counselors should examine multiple treatment interventions, especially those that focus more on personal, social, and leisure skills. Replicating this study with persons who are newly enrolled in various treatment programs and who have not participated in any other type of community mental health treatment would also address the problem of cumulative effects of participating in various programs over time.

Given current findings from this study, the differential impact of the four treatment programs raises some important questions about their utility in community mental health treatment as well as offers valuable information to mental health counselors seeking information on the efficacy of community treatments. Moreover, the program's ability to offer benefits to consumers requires mental health counselors to employ a variety of counseling skills in each program. Hershenson (1993) discusses integral mental health counseling skills such as nourishing growth, removing sources of discomfort, improving negotiating abilities, and providing positive benefits to consumers. A combination of these skills would seem crucial for counselors to use in the four programs reviewed.

Table 1. Results of ANCOVAs, Means, and Standard Deviations of the Four Dependent Program Variables

Program                         n        M       SD

Mastery
Clubhouse Program               24      2.94     .12
Vocational Program              23      2.74     .11
Social Program                  52      2.86     .07
Self-Help Program               13      3.00     .14

Program Satisfaction
Clubhouse Program               24      3.33     .10
Vocational Program              23      3.12     .09
Social Program                  52      3.11     .06
Self-Help Program               13      3.32     .13

Total Hospital Frequency
Clubhouse Program               24      .70      .15
Vocational Program              23      .57      .14
Social Program                  53      .57      .09
Self-Help Program               13      .23      .18

REFERENCES

Accordino, M. P., & Herbert, J. T. (1997). Relationship enhancement as an intervention to facilitate rehabilitation of persons with severe mental illness. Journal of Applied Rehabilitation Counseling, 28(1), 47-52.

Anthony, W. A. (1994). Characteristics of people with psychiatric disabilities that are predictive of entry into the rehabilitation process and successful employment. Psychosocial Rehabilitation Journal, 17(3), 3-14.

Anthony, W. A., & Blanch, A. (1987). Supported employment for persons who are psychiatrically disabled: An historical and conceptual perspective. Psychosocial Rehabilitation Journal, 11(2), 5-23.

Arns, P. G. (1997, June). Using the Psychosocial Rehabilitation Outcomes Toolkit. Institute presented at the 22nd Annual Conference of the International Association of Psychosocial Rehabilitation Services, Vancouver, BC, Canada.

Arns, P. G., & Linney, J. A. (1995). Relating functional skills of severely mentally ill clients to subjective and societal benefits. Psychiatric Services, 46, 260-265.

Arns. P. G. (1992). Predicting the outcome of psychosocial rehabilitation programs. Unpublished doctoral dissertation. University of South Carolina, Columbia.

Attkisson, C., Cook, J., Karno, M., Lehman, A., McGlashan, T. H., Meltzer, H. Y., O'Conner, M., Richardson, D., Rosenblatt, A., Wells, K., & Williams, J. (1992). Clinical services research. Schizophrenia Bulletin, 18, 561-626.

Berzins, J. I. (1977). Therapist-patient matching. In A. S. Gurman & A. M. Razin (Eds.), Effective psychotherapy: A handbook of research (pp. 222-324). New York: Pergamon Press.

Bolton, B., & Akridge, R. L. (1995). A meta-analysis of skills training programs for rehabilitation clients. Rehabilitation Counseling Bulletin, 38, 262-273

Bond, G. R., & McDonel, E. C. (1991). Vocational rehabilitation outcomes for persons with psychiatric disabilities: An update. Journal of Vocational Rehabilitation, 1(3), 9-20.

Bond, G. R., Witheridge, T. F., Dincin, J., Wasmer, J., Webb, J., & De Graaf-Kaser, R. (1990). Assertive community treatment for frequent users of psychiatric hospitals in a large city: A controlled study. American Journal of Community Psychology, 6, 865-891.

Bond, G. R., Witheridge, T. F., Setze, P. J., & Dincin, J. (1985). Preventing rehospitalization of clients in a psychosociai rehabilitation program. Hospital and Community Psychiatry, 36, 993-995.

Booth, M. E. (1993). Effects of program factors on client outcomes: Evaluation of three clubhouse programs for persons with severe mental illness. Unpublished doctoral dissertation, Virginia Commonwealth University, Richmond.

Braitman, A., Counts., P., Avenport, R., Zurlinden, B., Rogers, M., Clauss, J., Kulkarni, A., Kymla, J., & Montgomery, L. (1995). Comparison of barriers to employment and employed clients in a ease management program: An exploratory study. Psychiatric Rehabilitation Journal, 19(1), 3-8.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Cook, J. A., & Rosenberg, H. (1994). Predicting community employment among persons with psychiatric disability: A logistic regression analysis. Journal of Rehabilitation Administration, 18, 6-22.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.

Dobrzykowski, E. A. (1997). The methodology of outcomes measurement. Journal of Rehabilitation Outcomes Measurement, 1 (1), 8-17.

Drake, R. E., Becker, D. R., Biesanz, J. C., Torrey, W. C., McHugo, G. J., & Wyzik, P. F. (1994). Rehabilitative day treatment vs. supported employment: I. Vocational outcomes. Community Mental Health Journal, 30, 519-532.

Drake, R. E., McHugo, G. J., Anthony, W. A., Becker, D. R., & Clark, R. E. (1996). The New Hampshire study of supported employment for people with severe mental illness. Journal of Consulting and Clinical Psychology, 64, 391-399.

Emerick, R. E. (1990). Self-help groups for former patients: Relations with mental health professionals. Hospital and Community Psychiatry, 41,401-407.

Fabian, E. S. (1989). Work and quality of life. Psychosocial Rehabilitation Journal, 12(4), 39-49.

Fabian, E. S. (1992). Supported employment and the quality of life: Does a job make a difference? Rehabilitation Counseling Bulletin, 36(2), 84-97.

Gill, K. J., Pratt, C. W., & Librera, L. A. (1998). The effects of consumer vs. staff administration on the measurement of consumer satisfaction with psychiatric rehabilitation. Psychiatric Rehabilitation Journal, 21,365-370.

Goeke, D. E. (1993). Effects of client, treatment, and community variables on hospitalization of the chronically mentally ilk An ecological perspective. Unpublished doctoral dissertation, University of Texas, Austin.

Harpin, R. E. (1981). Behavior therapy for chronically depressed patients: A clinical report. Psychological Reports, 48, 763-774.

Harrington, J. (1993). Expectations, outcomes, and patient satisfaction with mental health treatment. Unpublished doctoral dissertation, University of Michigan, Ann Arbor.

Herbert, J. T. (1987). Measurement of job placement knowledge: An experimental scale. Journal of Job Placement, 3(2), 6-11.

Hershenson, D. B. (1993). Healthy development as the basis for mental health counseling. The Journal of Mental Health Counseling. 15, 430-437.

Hogarty, G. E., Anderson, C. M., Reiss D. J,, Kornblith, S. J., Greenwald, D. P., Javna, C. D., Madonia, M. J. (1986). Family pschoeducation, social skills training, and maintenance chemotherapy in the aftercare treatment of schizophrenia: I. One-year effects of a controlled study on relapse and expressed emotion. Archives of General Psychiatry, 43, 633-642.

Hogarty, G. E., Anderson, C. M., Reiss, D. J., Kornblith, S. J., Greenwald, D. P., Ulrich, R. E, & Carter, M. (1991). Family pschoeducation, social skills training, and maintenance chemotherapy in the aftercare treatment of schizophrenia. Archives of General Psychiatry, 48, 340-347.

International Association of Psychosociai Rehabilitation Services. (1998). Measuring psychosocial outcomes: Toolkit for pilot study final report. Cambridge, MA: Human Services Research Institute.

International Association of Psychosocial Rehabilitation Services. (1995). Toolkit for measuring psychosocial outcomes. Cambridge, MA: Human Services Research Institute.

Jacobs, M. K., & Goodman, G. (1989). Psychology and self-help groups. American Psychologist, 44, 536-545.

Kaufmann, C. L., Freund, P. D., & Wilson, J. (1989). Self-help in the mental health system: A model for consumer-provider collaboration. Psychosocial Rehabilitation Journal, 13(1), 5-21.

Lebow, J. (1982). Pragmatic decisions in the evaluation of consumer satisfaction with mental health treatment. Evaluation and Program Planning, 5, 349-356.

Lebow, J. L. (1983). Research assessing consumer satisfaction with mental health treatment: A review of findings. Evaluation and Program Planning, 6, 211-236.

Lehman, A. F. (1983). The effects of psychiatric symptoms on the quality of life assessments among the chronic mentally ill. Evaluation and Program Planning, 6, 143-151.

Lehman, A. F. (1988). A quality of life interview for the chronically mentally ill. Evaluation and Program Planning, 11, 51-62.

Lehman, A. F., Rachuba, L. T, & Postrado, L. T. (1995). Demographic influences on quality of life among persons with chronic mental illness. Evaluation and Program Planning, l& 155-164.

Lehman, A. F., Ward, N. C., & Linn, L. S. (1982). Chronic mental patients: The quality of life issue. American Journal of Psychiatry, 139,1271-1276.

Lehman, A. F., & Zastowny, T. R. (1983). Patient satisfaction with mental health services: A metaanalysis to establish norms. Evaluation and Program Planning, 6, 265-274.

Liberman, R. P., DeRisi, W. J., & Mueser, K. T. (1989). Social skills training for psychiatric patients. New York: Pergamon Press.

Marino, L. L. (1984). Chronicity: The clinical utility of the concept. Unpublished doctoral dissertation, Northwestern University, Evanston, Ill.

Mintz, J., O'Brien, C. P., & Clancy, J. (1976). Predicting the outcome of psychotherapy for schizophrenics. Archives of General Psychiatry, 33, 1183-1186.

Mueser, K. T., Wallace, C. J., & Liberman, R. P. (1995). New developments in social skills training. Behaviour Change, 12, 31-40.

Neese-Todd, S., & Weinberg, J. (1992). Public academic liaison: One clubhouse approach to research and program evaluation. Psychosocial Rehabilitation Journal, 16, 147-161.

Nelson, G., Wiltshire, C., Hall, G. B., Peirson, L., & Walsh-Bowers, R. (1995). Psychiatric consumer/survivors' quality of life: Quantitative and qualitative perspectives. Journal of Community Psychology, 23,216-233.

Neter, J., Kutner, M. H., Nachtsheim, C. J., & Wasserman, W. (1996). Applied linear statistical models (4th ed.). Chicago: Irwin.

Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.

Penn, D. L., & Mueser, K. T. (1996). Research update on the psychosocial treatment of schizophrenia. American Journal of Psychiatry, 153,607-617.

Powell, T. H., Pancsofar, E. L., Steere, D. E., Butterworth, J., Itzkowitz, J. S., & Rainforth, B. (1991). Supported employment: Providing integrated employment opportunities for persons with disabilities. White Plains, NY: Longman.

Pratt, C. W., & Gill, K. J. (1996). Measuring member satisfaction in the psychiatric milieu. Psychiatric Rehabilitation Journal, 19(4), 2%35.

Propst, R. N. (1992). Standards for clubhouse programs: Why and how they were developed. Psychosocial Rehabilitation Journal, 16(2), 25-30.

Rogers, E. S., Anthony, W. A., Toole, J., & Brown, M. A. (1991). Vocational outcomes following psychosocial rehabilitation: A longitudinal study of three programs. Journal of Vocational Rehabilitation, 1(3), 21-29.

Rosenfield, S. (1987). Services organization and quality of life among the seriously mentally ill. New Directions for Mental Health Services, 36, 47-59.

Rosenfield, S. (1992). Factors contributing to the subjective quality of life of the chronic mentally ill. Journal of Health and Social Behavior, 33, 299-315.

Rosenfield, S., & Neese-Todd, S. (1993). Elements of a psychosocial clubhouse program associ-ated with a satisfying quality of life. Hospital and Community Psychiatry, 44, 76-78.

Rutman, I. (1994). How psychiatric disability expresses itself as a barrier to employment. Psychosocial Rehabilitation Journal, 17(3), 15-35.

Ryback, R. S. (1971). Schizophrenics anonymous: A treatment adjunct. Psychiatry in Medicine, 2, 247-253.

Setze, P. J., & Bond, G. R. (1985). Psychiatric recidivism in a psychosocial rehabilitation setting: A survival analysis. Hospital and Community Psychiatry, 36, 521-524.

Speer, D. C., & Newman, F. L. (1996). Mental health services outcome evaluation. Clinical Psychological Scientific Practice, 3,105-129.

Toro, P. A., Reischl, T. M., Zimmerman, M. A., Rappaport, J., Seidman, E., Luke, D. A., & Roberts, L. J. (1988). Professionals in mutual help groups: Impact on social climate and members' behavior. Journal of Consulting and Clinical Psychology, 56, 631-632.

Van Dongen, C. J. (1996). Quality of life and self-esteem in working and nonworking persons with mental illness. Community Mental Health Journal, 32, 535-559.

Webb, L. J. (1973). The therapeutic social club. The American Journal of Occupational Therapy, 27, 81-83.

Michael P. Accordino, D. Ed., CRC, is an assistant professor, Rehabilitation and Disability Studies Department, Springfield College, MA. James T. Herbert, Ph.D., CRC, is an associate professor and professor-in-charge of Rehabilitation Programs at The Pennsylvania State University, Department of Counselor Education, Counseling Psychology, and Rehabilitation Services. Email [e-mail]

Correspondence concerning this should go to Dr. Accordino at Springfield College, Rehabilitation and Disability Studies Department, 263 Alden St., Springfield, MA 01109-3797. Email [e-mail]