Effects of participation in consumer-operated service programs on both personal and organizationally mediated empowerment: results of multisite study.

INTRODUCTION

Development and funding of consumer-operated service programs (COSPs) in the mental health field have increased dramatically over the past decade [12]. COSPs, i.e., programs run by and for consumers [3], have developed along multiple paths that have resulted in a variety of program typologies, service structures, and funding streams. Many of these programs have comparable missions and goals, most notably, increased perceived empowerment among participants [4].

Consumer-operated and other types of self-help programs began proliferating in the mid-1900s with the advent of groups like Alcoholics Anonymous and other so-called "12-step" programs. Such programs were seen as a response to both the limited effectiveness of traditional approaches in treating addiction problems and, simultaneously, the power of groups to come together and offer support, solace, and assistance in a way that traditional mental health services could not. These programs, arguably the "oldest and most pervasive" of peer programs [5], evolved beyond substance abuse to address the needs of mental health consumers and now include groups such as Schizophrenics Anonymous [6]. Their growth has been so robust that a recent U.S. national survey identified a total of 7,467 such mental health mutual support groups and COSPs--a figure that exceeds the number of traditional mental health organizations [7].

Self-help programs are often described as vehicles for sharing experiences in an atmosphere of nonjudgmental acceptance and obtaining help coping with crises [8]. These characteristics were corroborated in a qualitative longitudinal study in which Ochocka and colleagues described COSPs as safe and welcoming environments and social arenas that offer opportunities to interact with peers and connect with the community at large [9]. At least one sociologist further posited the "helper-therapy" principle, i.e., the opportunity to help others and receive benefits in the process [10], which has been demonstrated empirically within the peer support field and the general population [11-12].

Weaver Randall and Salem described the "active ingredients" of self- and mutual-help programs [13]. These programs can provide venues where experiential knowledge can be shared, role models of recovery and healing are present, and social support and valued organizational roles are available [13, p. 174]. In addition, such programs provide a sense of connection, belonging, and community that is often lacking for individuals who receive services in the traditional mental health system [13]. Further, Solomon describes both social support and experiential knowledge as being critical ingredients in peer support [5].

Campbell delineates three important social movements that intersected to help shape the development and proliferation of COSPs: (1) the growth of self-help in fields other than mental health, (2) the deinstitutionalization of individuals being treated for psychiatric problems and the concomitant increase in programs for them in their communities, and (3) the rise of the patients' rights movement in psychiatry and the growing influence of the "nothing about us without us" philosophy [4].

In one of the most comprehensive narrative reviews of self-help and COSPs to date, Campbell examined 20 studies of mental health self-help programs, excluding those on programs in which the only enhancement was the addition of a peer to a team of professional providers [4]. Overall, these studies suggested that self-help and peer support programs can promote empowerment and recovery [1415]. Combined with a similar review by Solomon [5], preliminary evidence suggests that these programs decrease the need for acute mental health services and mental health hospitalizations [14,1617]; increase social support, functioning, and activities [16,1819]; decrease substance abuse [14,16]; and may benefit perceived quality of life [2021].

The majority of these studies had significant limitations in their internal validity and scientific rigor and therefore in the conclusions that could be drawn from them. Only one of the studies involved a randomized trial of the effectiveness of a COSP (i.e., a crisis hostel program [14]). Most of the studies were descriptive, had very small samples, were of questionable representativeness, were cross-sectional surveys, or were simple pre- to postevaluations of effectiveness, with the resulting potential for significant selection biases. Thus, despite the vast proliferation of COSPs and some preliminary studies about their effectiveness and active ingredients, little systematic research or empirical evidence is available about the effects of such programs on their intended outcomes.

Further complicating this examination of empowerment within COSPs is the ambiguity surrounding the use of this construct as an outcome measure. Campbell described empowerment as one of the myriad "caring" functions served by COSPs [4]. Fisher and Ahern described the contemporary values that shape most self-help programs in mental health as empowerment, personhood, and recovery-orientation (http://www.power2u.org/ articles/recovery/people_can.html). This clear focus on empowerment as a goal of COSPs appears to have emerged, at least in part, from feelings of disenfranchisement and powerlessness among mental health consumers as a result of a perceived lack of choice and control over their mental health services and treatment [22].

But despite the burgeoning use of the term empowerment within COSPs and its more ubiquitous presence in the mental health system, no consensus on its meaning appears to exist. Some researchers and authors describe empowerment as a complex multidimensional construct with psychological, social, and political components [23], while others have described the concept from a social and organizational perspective [24]. At least three measures have been developed and tested that examine empowerment among consumers of mental health services; these measures provide the operational definitions of empowerment for this study [2425].

The overall purpose of this study (the COSP-Multisite Research Initiative, which was funded by the U.S. Department of Health and Human Services [DHHS], Substance Abuse and Mental Health Services Administration [SAMHSA]), was to rigorously examine the effectiveness of COSPs on various psychological, social, and objective and subjective functioning domains among individuals who receive traditional mental health services. Our goals in this component of the study were to (1) determine whether any significant differences existed in the four measures of empowerment (three personal-level empowerment measures and one organizationally mediated empowerment measure) by experimental condition, (2) determine whether the COSP sites varied with respect to differences between the experimental and control groups or to change over time, (3) describe patterns of change in empowerment over time, and finally (4) examine whether engagement intensity in the COSP resulted in greater changes in empowerment (i.e., examine the dose-response relationship).

METHODS

Participants

Participants of the multisite study were required to have Diagnostic and Statistical Manual of Mental Disorders Axis I or II diagnosis that constituted a serious mental illness, such as schizophrenia, schizoaffective disorder, or a major depressive disorder; a diagnosis of substance abuse alone was insufficient [26]. Diagnostic information was collected at each of the eight sites either by the mental health service providers (seven sites) or diagnostic interviews conducted by clinicians on the research staff (one site). In addition to having a serious and persistent mental illness, participants must have been (1) 18 years of age or older, (2) able to provide full and informed consent, and (3) actively involved with a traditional mental health provider in the 12 months before study entry ([greater than or equal to] 4 mental health services in the past year with [greater than or equal to] 1 of those services in the 4 months before study entry). Exclusion criteria included (1) an inability to participate in the research interviews or (2) more than minimal involvement in the COSPS under study within the past 6 months (>3 visits to or meetings in the COSP under study). Individual sites were allowed to impose additional inclusion and exclusion criteria specific to their local programs as long as they adhered to the multisite inclusion and exclusion criteria. All research procedures were reviewed and approved by the institutional review boards (IRBs) at the individual research sites as well as by the IRB at the Coordinating Center at the University of Missouri.

Instruments

Common Assessment Protocol

A common assessment protocol composed of 27 scales or domains was used across all sites. To assure that the scales would yield reliable data, we selected validated scales with published data on the target population wherever available. We used scales that were culturally sensitive by having the protocol reviewed by a panel of experts with expertise in cross-cultural issues in mental health.

The common assessment protocol measured demographic variables, status variables (e.g., employment, finances, housing), clinical variables (e.g., symptoms, substance use), and subjective indicators (e.g., quality of life, hope, recovery, and social inclusion). We conducted two rounds of pilot testing of the assessment protocol and the standardized interviewer training to ensure accurate data collection.

Empowerment Measures

Making Decisions Empowerment. The Making Decisions Empowerment (MDE) scale is a 28-item instrument designed to measure subjective feelings of empowerment in which respondents rate statements on a 4-point scale (1 = "Strongly Agree" and 4 = "Strongly Disagree") [25]. Most of the items indicate greater empowerment; individual scale values for these items are reversed (subtracted from 5) so that higher scores are more favorable, and responses are then summed across items. The scale taps the domains of self-efficacy, perceived power, optimism about and control over the future, and community activism and has good consistency and internal reliability as well as good factorial and known groups' validity [25]. Examples of items include "People working together can have an effect on their community" and "I am often able to overcome barriers." For this study, we calculated the internal consistency for this instrument using baseline data as 0.81 (Cronbach [alpha]).

Personal Empowerment. The Personal Empowerment (PE) scale is a 20-item tool with two subscales, Choice and Reduction in Chance, each with a different set of response choices. The Choice subscale measures how much choice a person has in his or her life using a 4-point Likert scale (1 = "No Choice" and 4 = "A Lot of Choice") [24]. Examples of items include "How much choice do you have about how you will spend your free time?" and "How much choice do you have about where to go to get help when you have problems?" The second subscale asks respondents to rate the amount of certainty in their social, residential, and financial lives on a 5-point scale (1 = "Very Likely" or "Very Sure" and 5 = "Very Unlikely" or "Very Unsure"). Items include "How likely is it that you will get enough to eat in the next month?" and "How sure are you about how you are going to spend your time in the next month?" Most of the items indicate greater certainty about the future; individual scale values for these items are reversed so that higher scores are more favorable, and responses are then summed across items. The authors report adequate internal consistency for the overall measure [24]. Internal consistency for the two subscales in this study was 0.78 and 0.74 (Cronbach [alpha]) for the Choice and the Reduction in Chance subscales, respectively.

Organizationally Mediated Empowerment. The Organizationally Mediated Empowerment (OME) scale is a 17-item scale that inquires about involvement in an organization or a club with dichotomous (Yes/No) responses [24]. Examples of items include "In the past 4 months, have you voted in an election for officers of an organization or club?" and "In the past 4 months, have you become a volunteer on a regular basis?" The scale is scored by summing across the "Yes" responses [24]. Internal consistency for the OME scale in this study was 0.83 (Cronbach [alpha]).

Intensity of Engagement in Intervention. Intervention engagement intensity was assessed by program attendance. Although record-keeping procedures varied somewhat across sites because of structural differences, each site was required to report program attendance in a consistent format. For example, attendance was often easier to monitor and track in peer support programs than drop-in programs, which, by their nature, are more informal, spontaneous, and generally do not involve structured encounters. In addition to each site collecting program attendance data, each participant was asked (as part of the interview protocol) about the frequency of his or her attendance at a COSP. From these data, we constructed two variables to measure engagement in or intensity of the intervention; the first was defined as "binary engagement" and represented any contact at all with the experimental site or condition. It was constructed from a combination of information on program attendance and self-report data. We used both sources of data to ensure greater reliability. Thus, if the program failed to capture a participant's attendance, self-report data provided that information. For the second variable, using only data reported by the programs, we assigned each participant one of the following three scores: 0 = no use of the program; 1 = low use of the program, i.e., >0 visits to the program but <8.5 (the median value for the study); and 2 = high use of the program, i.e., >8.5 visits to the program. We used this upper cutoff to mitigate potentially spurious effects of extreme utilization values, which can occur in drop-in programs that allow numerous visits per day and are available most days of the year. For this second variable, we were unable to use self-report attendance data, which were reported on a 7-point scale that captured frequency. These data could not be combined with the program data, which consisted of numerical counts.

Scope of Empowerment Measures

As mentioned previously, one of the advantages of this study was our ability to triangulate on the measure of empowerment. The MDE scale measures the more global and personal aspects of empowerment, such as self-efficacy and a sense of control over one's life, whereas the PE subscales focus on choice, certainty, and self-determination in one's life. The OME scale provides information on one's involvement as an organizational member and community activist. Developers of the PE and OME scales argue that these constructs and their measurement are critical ingredients of empowerment. While the scales that measured personal-level empowerment were moderately correlated in this study (MDE with PE Choice: r = 0.31, MDE with PE Reduction in Chance: r = 0.42), they were less highly correlated with the OME scale because they tap different aspects of the construct, i.e., personal versus organizational empowerment (MDE with OME: r = 0.28, PE Choice with OME: r = 0.07, and OME with PE Reduction in Chance: r = 0.13).

Procedures

Sample and Recruitment

At all sites participants were recruited from one or more traditional mental health providers who partnered with the COSP; a common "induction" process was followed at each site. Participants were randomized to either the experimental or control group, with sex as a blocking variable. Additional blocking variables, such as race/ ethnicity, were used at several sites. Using these procedures, we recruited, screened, gained informed consent from, and enrolled 1,827 individuals in the study. Interventions

Experimental Arm. The experimental condition involved attendance at the COSP under study in addition to usual traditional mental health services. The eight study sites were located in various states around the country, including three in the northeast, one on the west coast, two in the South, and two in the Midwest; experimental and control groups were located at each site. The experimental COSPs loosely fell into the following categories: drop-in (n = 4), peer support and mentoring (n = 2), and education and advocacy (n = 2). As described previously, common ingredients across all COSPs included a focus on peer support, recovery education, empowerment, and tangible assistance for independent community living. A measure was conceptualized, developed, and tested during the study that allowed us to examine the extent to which programs exhibited the principles and practices identified by our study experts (e.g., choice, recovery orientation). For a full description of these programs and measurement of both the common ingredients of the programs and their fidelity, please refer to Johnsen et al. [27].

Control Arm. The control condition consisted of an array of traditional mental health services, such as psychopharmacology and medication management, case management, residential services, psychotherapy, day services, and psychosocial rehabilitation services. Mental health services such as these are delivered by many different professionals and paraprofessionals including psychiatrists, social workers, psychologists, and residential providers. These services would be considered standard interventions in many mental health systems.

Assessment Procedures

The assessment protocol was administered in a face-to-face interview after recruitment and initial screening were performed and informed consent was obtained. All informed consent forms and procedures were approved by local IRBs. Interviewers were carefully trained, monitored, and supervised. Follow-up assessments were briefer versions of the baseline protocol and took about 12 hours to complete. Randomization occurred subsequent to the baseline interview. Each site developed randomization procedures that prevented "gaming," largely through the use of computer-generated random number tables; these procedures remained under the control of the researchers and not the interviewers or program staff. Follow-up assessments were conducted at 4, 8, and 12 months.

Statistical Analyses

Statistical analyses were conducted with SPSS, version 13.0, (SPSS Inc, Chicago, Illinois) or SAS, version 9.0, (SAS Inc, Cary, North Carolina). We derived scale scores at each time point for each measure using mean substitution when <1 or 2 items were missing depending upon the scale length. Descriptive statistics and baseline differences across the sites were examined with analysis of variance.

We designed the primary analyses to examine the effectiveness of the intervention using an intent-to-treat (ITT) approach, i.e., comparing outcomes based on randomization status regardless of actual participation in the intervention. We used a mixed model approach (SAS) to examine differences in outcomes between the experimental and control groups over time. We converted time points to a continuous variable that ranged from 1 to 4, with 1 representing baseline, 2 representing the first follow-up, 3 representing the second follow-up, and 4 representing the last follow-up. Site x group x time interactions were included in the models; subject was treated as a fixed effect, and autoregressive covariance structures were used because each person's data over time were expected to be correlated. All participants with data at baseline and at least one follow-up were included. Missing data imputation at the scale level was not necessary because the mixed procedure includes all available data.

To examine the magnitude of the effects, we derived effect sizes (ESs) from the slope of the difference between the experimental and control groups overtime divided by the mean standard error.

Further, we examined whether actual use of the intervention, regardless of assigned condition, affected participants' outcomes using both measures of engagement and an as-treated (AT) approach.

We were concerned that individuals who elected to use the COSPs might be different in demographic or clinical characteristics from individuals who elected not to use the programs. If these differences were present, a selection bias that could confound the results may have existed. For example, if equal numbers of men and women were assigned to the experimental condition (COSP) but more women than men actually engaged in or attended the program, sex could confound examination of the effectiveness of the COSP. Therefore, we used a propensity score technique to create groups that were similar in characteristics and only differed in propensity of COSP use [28]. We used logistic regression to predict whether a subject with postbaseline data was among those who would have engaged in the experimental condition, whether or not they had been assigned to that condition through randomization. Baseline clinical and demographic variables served as independent variables, producing for each individual a propensity score that represented the probability that the person would self-select into the COSP. We compared engagement groups using only data from participants who scored in the middle third of the COSP-use propensity distribution; thus, our analysis included only those participants who were the least inclined to either seek or avoid COSPs. Although this approach reduced statistical power and resulted in a more conservative test, the observed differences may be more validly attributed to program effects than selection biases.

For the AT analyses, we used mixed models structured as in the ITT analyses, except they included different group variables and a subsample of participants that was restricted to the middle third of the COSP-use propensity distribution.

RESULTS

Analyses of Baseline Data

All ITT analyses presented here included the entire sample of participants, while the AT analyses included a subsample of participants as previously described.

Demographic and Clinical Characteristics

Baseline demographic data for the entire sample suggested that participants were, on average, in their early 40s (with large variability in age), from diverse racial/ethnic backgrounds, and had had substantial involvement with the mental health system prior to the study. With receipt of Social Security payments as a proxy measure, the participants appeared to have severe disabilities because of their psychiatric illnesses. In addition, fully 50 percent of the sample was diagnosed with schizophrenia or psychotic disorder. Full descriptive data appear in Table 1.

Baseline Differences

Given the reasonably large number of individuals per site and the careful randomization procedures, we expected that randomization would result in groups with no significant differences at baseline on the outcome measures but conducted baseline analyses as a check on these procedures. We found no significant differences in baseline scores for the MDE, PE Choice, PE Reduction in Chance, or OME for the entire sample by experimental condition (Table 2). Site analyses demonstrated a significant difference between groups at only two sites on the baseline measures. At site 6, those in the experimental group had higher mean MDE scores than those in the control group, while at site 7, those in the control group had higher OME scores (Table 2).

We did, however, find significant differences across sites in baseline values of the MDE, PE Choice, PE Reduction of Chance, and OME (F = 20.01, df = 7, p < 0.001; F = 12.15, df = 7, p < 0.001; F = 8.73, df = 7, p < 0.001; and F = 42.82, df = 7, p < 0.001, respectively). The site effect was not unexpected since participants may have naturally varied by site given the diverse geography and variety of traditional mental health programs from which they were recruited. In addition, sites could have imposed additional inclusion criteria (beyond these described previously for the multisite study) that could account for some of the cross-site variability. However, one site (site 8) was deemed from these analyses as well as other data to be an outlier, in that the baseline MDE and OME scores at this site were significantly higher than other sites. As will be described subsequently, these findings led us to perform sensitivity analyses within the ITT analyses that excluded this site. We found a significant site x group interaction for the MDE (F = 2.09, df = 7, p = 0.04) but no such interactions for the remaining measures. Table 2 contains baseline mean and standard deviation (SD) data by site and group.

Engagement and Attrition

Since study participants were able to continue to receive traditional mental health services and use COSPs regardless of their randomization assignment, the degree of adherence to the assigned condition was a critical factor in this study. We found that 57 percent of those assigned to the intervention condition and 15 percent of those assigned to the control condition used the COSPs, which made the ITT analytic approach a conservative one. The attrition rate from baseline to the end point of the study (12 months) was approximately 20.5 percent.

Primary Intent-To-Treat Analyses

The primary purpose of these analyses was to determine whether any significant differences existed in the four empowerment measures by experimental condition over time for the randomized sample. Results reported here are based on data for the maximum number of participants eligible for each analysis, i.e., those who had data for baseline and at least one follow-up on the given measure.

Making Decisions Empowerment

Results of ITT mixed procedure analyses for the MDE scale showed a significant time effect ([F.sub.1, 4059] = 28.13, p < 0.001), which reflected an increase in empowerment over time for study participants overall. The main effect of experimental condition over time (i.e., time x group interaction) was not significant ([F.sub.1, 4059] = 2.30, p = 0.13), but the time x group x site interaction was suggestive ([F.sub.7, 4059] = 1.44, p = 0.19), since significance values should ideally be >0.2. Sensitivity analyses confirmed that one site (site 8) was the principal source of this interaction as described previously. Results of analyses repeated with this site excluded indicated a significant time x group interaction ([F.sub.1, 3815] = 5.68, p = 0.02) and no significant time x group x site interaction ([F.sub.6, 3815] = 0.88, p = 0.51).

Personal Empowerment: Choice

Results of the mixed procedure ITT analyses for the PE Choice suggested a significant effect over time for study participants overall ([F.sub.1, 4062] = 10.75, p = 0.001). Examination of the time x group interaction provided results that just missed the cutoff for significance ([F.sub.1, 4062] = 3.53, p = 0.06); we did, however, find a significant time x group x site interaction ([F.sub.7, 4062] = 2.45, p = 0.02). Sensitivity analyses again indicated that the same site as for the MDE analyses (site 8) was a prime contributor to this interaction, but this time we found other site differences. Subsequent analysis excluding site 8 suggested not only a significant time x group interaction ([F.sub.1, 3818] = 5.41, p = 0.02) but also a significant time x group x site interaction ([F.sub.6, 3818] = 2.70, p = 0.013), which indicated that time x group effects across the remaining sites still varied significantly.

Personal Empowerment: Reduction in Chance

Results of the mixed procedure ITT analyses for the PE Reduction in Chance subscale showed neither an overall effect of time ([F.sub.1, 4025] = 1.95, p = 0.16) nor a time x group interaction effect ([F.sub.1, 4025] = 0.05, p = 0.82).

Organizationally Mediated Empowerment

OME scores were generally quite low and positively skewed in this study. On the 17-point OME scale, the baseline mean was just over 2 and the modal score was 0. Transformation of the scores to achieve more normal distributions had negligible effect, so results reported here are based on raw scores.

Results of the mixed procedure ITT analyses for the OME scale showed an overall negative effect over time ([F.sub.1, 4075] = 47.17, p < 0.001); on average, scores declined over time. Moderating this trend, we found a significant positive time x group interaction effect ([F.sub.1, 4075] = 4.75, p = 0.03). However, the time x group x site interaction ([F.sub.7, 4075] = 1.87, p = 0.07) just missed the cutoff for significance; this result was not attributable to any one site.

Relationship of Intervention Engagement Intensity to Outcomes: As-Treated Analyses

These analyses determined whether engagement or involvement intensity in the COSP resulted in greater changes in empowerment. These analyses included the subsample of participants described in the "Statistical Analyses" (p. 789) section. This AT analytic approach represents an important complement to the ITT analyses, particularly in view of the only modest engagement in COSPs by persons assigned to the experimental condition. Table 3 contains the mean and SD of the no-use, low-use, and high-use groups on each of the dependent measures for each time point.

As noted earlier, in order to minimize selection bias, we only analyzed data for participants in the middle third of the COSP use propensity distribution. Results of these analyses suggest a significant effect of intervention intensity for both the MDE and the PE Choice. The time x engagement interaction was significant with the binary measure only for the MDE ([F.sub.1, 314] = 3.97, p = 0.046), but the time x engagement interaction was significant with the intensity measure for both the MDE ([F.sub.2, 1306] = 7.03, p = 0.02) and PE Choice ([F.sub.2, 1316] = 3.15, p = 0.04). We found no significant time x engagement intensity x site interactions for these measures. Thus, these results suggest greater effects on personal empowerment from higher levels of COSP engagement intensity. We found no significant effects for the PE Reduction in Chance. The time x engagement intensity interaction for the OME was not significant with the intensity measure ([F.sub.2, 1317] = 1.93, p = 0.15), although the time x engagement intensity x site interaction ([F.sub.14, 1317] = 1.48, p = 0.11) reflected considerable variation in time x engagement intensity interactions across sites. ITT and AT results are presented in Table 4.

Additional Secondary Analyses

In addition to assessing the significance of relationships, we examined the ES changes in experimental participants over time compared with changes in control group participants over time both by site and for all sites combined. Small and variable sample sizes precluded the primary use of significance levels for site comparison.

Figures 14 display ESs--the magnitude of the difference between experimental and the control groups over the course of the study--for each measure at each site and all sites combined. For the MDE, ESs were at or above 0.20 (Cohen's cutoff for the measurement of a small ES [29]) at four of the eight sites, which represents a meaningful effect of the intervention. However, the ESs were significant and moderate at only two sites (site 3, ES = 0.55, p = 0.045; site 7, ES = 0.49, p = 0.05) (Figure 1). The overall ES for all sites combined was 0.14 (p = 0.13), and without site 8, the overall ES increased to 0.22 (p = 0.02). For the PE Choice, three of the eight sites showed a positive ES of at least 0.20, with two of the sites having significant moderate effects (site 3, ES = 0.68, p = 0.02; site 7, ES = 0.74, p = 0.003). The overall ES was 0.17 (p = 0.06) (Figure 2), and without site 8, it was statistically significant (ES = 0.22, p = 0.02). Overall, sites 3 and 7 had the most robust and consistent findings for change in personal empowerment. On the PE Reduction in Chance, only one site showed a statistically significant difference (site 1, ES = 1.03, p = 0.004) and one other site had a moderate but not significant effect (site 7, ES = 0.38, p = 0.12). Conversely, this scale had the greatest number of sites with negative ESs, with three exceeding 0.20 in absolute value (Figure 3).

[FIGURE 1 OMITTED]

Finally, for the OME, four of the eight sites demonstrated a positive ES of at least 0.20; only one site had a statistically significant ES (site 6, ES = 0.54, p = 0.04).

[FIGURE 2 OMITTED]

The overall ES was significant when all sites were examined (ES = 0.19, p = 0.03) (Figure 4) and without site 8 (ES = 0.20, p = 0.03). These variations in ES by site are consistent with the modest differences seen in the primary analyses and would have attenuated differences when experimental and control groups were examined in the aggregate.

DISCUSSION

In this study, we sought to determine whether significant differences in empowerment existed between study participants who were exposed to an experimental intervention that consisted of involvement in or attendance at a COSP compared with individuals who were not exposed to such programs. All participants were receiving traditional mental health services when recruited into the study. The eight COSPs that participated in this multisite study can be described generally as drop-in programs, peer support and mentoring programs, and education and advocacy programs. Our outcome variable, empowerment, has various definitions; here we measured it using four different scales as described previously.

We followed several analytic approaches. The parent study was conducted under a stringent experimental design that included ITT analysis within a multisite randomized clinical trial; we used this approach within the current study. We also applied post hoc observational methods, including AT analysis, to estimate the effect of actual services use, examine differences in slopes, and examine the results by site.

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

The AT results offered the opportunity for further confirmation and elaboration of our findings and are particularly important in the context of a study with somewhat low engagement rates and crossover and in light of the fact that some of the traditional services in the control arm possessed some of the active ingredients (e.g., consumer choice) present in the COSPs but to a lesser degree. To counteract these difficulties, we compared outcomes for participants who actually used the services with outcomes for those who did not, regardless of randomization status. Our AT mixed models were conservative in that they had small samples and included only those participants whose characteristics were least associated with either seeking or avoiding COSPs, thus reducing the impact of selection bias.

Results of our analyses from the strict ITT approach yielded very marginal results, perhaps because of engagement of assigned participants in the COSPs. However, our AT and secondary analyses paint a more optimistic picture of the effects of COSPs that are worthy of consideration. Results of ITT analyses with both measures of personal empowerment (MDE and PE Choice) had initial overall results that were below thresholds for significance and very small ESs, but sensitivity analyses revealed that results from one site (site 8) obscured the more generally positive results from the remaining sites. With this site removed, both measures showed significant, small, and positive effects overall, albeit with significant cross-site variation remaining for the PE Choice. Likewise, the overall effect on the OME was positive, significant, and at the threshold of a very small effect, but with significant variation across sites and in the context of a general downward trend for both groups. The distribution of scores on this last measure raises questions about the fit of the underlying construct or its operationalization with the programs or population in this study, but clearly, relatively few participants in either condition experienced an increase in the number of organizational roles or activities as measured by this instrument.

Results of the AT mixed models confirmed the inferences that emerged from the ITT analyses. COSP use was positively associated with increases in personal empowerment as measured by both the MDE and the PE Choice, results that were maintained without significant variation across the eight sites. We found no effect for the PE Reduction in Chance, and the nonsignificant effect for OME appeared to be in the context of substantial variation among the sites. Other results suggest that gains from COSPs become apparent only with higher levels of use.

Our results support the conclusion that COSPs in general have a positive additive effect on empowerment when this construct is measured subjectively as represented by the MDE and the PE Choice. These effects are significant but small in magnitude. These results are consistent with studies cited earlier [1415] as well as with the empowerment philosophy espoused by numerous writers (http:// www.power2u.org/articles/recovery/people_can.html). Certain authors have argued for measuring empowerment with an organizational focus [24] and advanced the notion that COSPs promote opportunities for participation in valued organizational roles [13]. However, we found little evidence that the programs in this study affected organizational empowerment. Thus, when empowerment was measured, either in terms of expectations for positive or negative effects on the environment or in terms of objective organizational roles, we found insufficient evidence for positive effect during the 12-month period assessed, although a longer observation period may have allowed such effects to be seen.

The variability in effects observed across sites in this study may have been attributable to the nature of the intervention at each site and can be considered a study limitation. Though the interventions can be generally described as falling into the three types mentioned earlier (drop-in, peer support and mentoring, and education and advocacy), we noted differences in local geography, in program resources, and in the way in which the interventions were carried out. We considered analyzing the data by the program category. However, when we examined the programs more carefully, both in terms of the service delivery models and the baseline values and demographics, we concluded that while programs in the same category may have delivered a generally similar category of service, the nature of the services and the contexts in which they delivered that service as well as the service recipients were too different to warrant such analyses. Future studies may wish to address this issue more carefully.

In addition to the limitations and cautions already noted, we do not know to what extent the results may apply beyond the programs included in this study. The current sample shows considerable variation and was selected in part to reflect a wide range of services. However, in the absence of information about the overall universe of COSPs, these results must necessarily be generalized with caution.

Finally, we should mention possible limitations due to measurement of our independent and dependent variables. First, the empowerment outcomes were gathered solely through self-report and, therefore, possess the limitations inherent in such assessments. Second, engagement and intensity data were reported by the individual sites or self-reported. Both of these approaches could lead to error. We do not know whether attendance at the COSPs was underreported or overreported, and correspondingly, we cannot predict the effect this might have on the outcomes we observed.

CONCLUSIONS

Interventions that are developed for, delivered by, and managed by consumers appear to have modest salutary effects on certain aspects of empowerment, especially when empowerment is measured in global terms and in terms of one's sense of self-efficacy, control, sense of power, and optimism about the future and when we examine participants who use the COSPs at a meaningful level.

Individuals with greater engagement in and attendance at the COSPs fared better in their empowerment outcomes. Overall, it would appear that some COSPs improve empowerment, while others are less effective. Further research suggests that we examine and unbundle the program ingredients that may be effective in this regard as well as the specific contextual variables that may be responsible for these differences. The extent to which such an examination can occur within the context of a randomized trial must be considered, particularly given the large number of individuals randomized to the experimental condition in this study who did not engage in the voluntary COSPs.

In the meantime, results of these analyses suggest that program administrators and policy makers should examine the critical ingredients of COSPs [27] and focus their efforts and resources on developing those aspects of these programs, both in COSPs per se and potentially in other programs as well. Clearly, for both philosophical and efficacy reasons, COSPs can compliment traditional mental health programs. However, we must better understand the critical ingredients of these programs as well as the local resources, geography, and participant mix that assist these programs in imparting benefit to consumers.

ACKNOWLEDGMENTS

This material was based on work supported by the U.S. DHHS (Program Support Center, Strategic Acquisition Service, Division of Acquisition Management purchase order HHSP233200500771P) and the Missouri Institute of Mental Health Coordinating Center from the U.S. DHHS, SAMHSA, Center for Mental Health Services (CMHS) (grant SM-52328). The SAMHSA/CMHS Cooperative Agreements to Evaluate Consumer-Operated Services were funded through seven research collaborators:

1. Mount Sinai School of Medicine (CMHS grant SM52372), Advocacy Unlimited, Connecticut Department of Mental Health and Addiction Services, and the University of Connecticut.

2. PEER Center, Inc (CMHS grant SM-52332); Florida Mental Health Institute and Florida International University; Henderson Mental Health Center; Mental Health Client Action Network; and Santa Cruz County Substance Abuse and Mental Health Services.

3. The University of Chicago Center for Psychiatric Rehabilitation (CMHS grant SM-52363), GROW of Illinois, Janet Wattles Mental Health Center, and Provena Behavioral Health.

4. Boston University, Center for Psychiatric Rehabilitation (CMHS grant SM-52352); St. Louis Empowerment Center; Places for People, and BJC Behavioral Healthcare.

5. The Edmund S. Muskie School of Public Service, University of Southern Maine (CMHS grant SM-52362); the Portland Coalition for the Psychiatrically Labeled; Catholic Charities Maine Support Recovery Services; and Shalom House, Inc.

6. Friends Connection of the Mental Health Association in Southeastern Pennsylvania (CMHS grant SM52355), Philadelphia Office of Mental Health, and the University of Pennsylvania Center for Mental Health Policy and Services Research.

7. BRIDGES (Tennessee Mental Health Consumers' Association), Michigan State University Department of Psychology (CMHS grant number SM-52367), and Vanderbilt University Department of Psychiatry.

Coordinating Center: University of Missouri School of Medicine Institute of Mental Health (CMHS grant number SM-52328), Northrup Grumann Information Technologies, American University, and University of Massachusetts School of Medicine. Federal Representatives: CMHS, SAMHSA.

The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of DHHS, SAMHSA, CMHS, or the other collaborating partners.

The authors have declared that no competing interests exist.

Submitted for publication October 2, 2006. Accepted in revised form May 16, 2007.

REFERENCES

[1.] Clay S. About us: What we have in common. In: Clay S, Schell B, Corrigan PW, Ralph RO, editors. On our own, together: Peer programs for people with mental illness. Nashville (TN): Vanderbilt University Press; 2005. p. 316.

[2.] Clay S, Schell B, Corrigan PW, Ralph RO. On our own, together: Peer programs for people with mental illness. Nashville (TN): Vanderbilt University Press; 2005.

[3.] Kyrouz EM, Humphreys K, Loomis C. American self-help clearinghouse: A review of research on the effectiveness of self-help mutual aid groups. In: Madara EJ, White BJ, editors. Self-help group sourcebook: Your guide to community and online support groups. 7th ed. Denville (NJ): Saint Claire's Health Services; 2002.

[4.] Campbell J. The historical and philosophical development of peer-run programs. In: Clay S, Schell B, Corrigan PW, Ralph RO, editors. On our own, together: Peer programs for people with mental illness. Nashville (TN): Vanderbilt University Press; 2005. p. 1766.

[5.] Solomon P. Peer support/peer provided services underlying processes, benefits, and critical ingredients. Psychiatr Rehabil J. 2004;27(4):392401. [PMID: 15222150]

[6.] Salem DA, Gant L, Campbell R. The initiation of mutual-help groups within residential treatment settings. Community Ment Health J. 1998;34(4):41929. [PMID: 9693870]

[7.] Goldstrom ID, Campbell J, Rogers JA, Lambert DB, Black-low B, Henderson MJ, Manderscheid RW. National estimates for mental health mutual support groups, self-help organizations, and consumer-operated services. Adm Policy Ment Health. 2006;33(1):92103. [PMID: 16240075]

[8.] Riessman F. Ten self-help principles. Soc Policy. 1997; 27(3):611.

[9.] Ochocka J, Nelson G, Janzen R, Trainor J. A longitudinal study of mental health consumer/survivor initiatives: Part 3-A qualitative study of impacts of participation on new members. J Community Psychol. 2006;34(3):27383.

[10.] Riessman F. The "helper-therapy" principle. Soc Work. 1965;10:2732.

[11.] Brown SL, Nesse RM, Vinokur AD, Smith DM. Providing social support may be more beneficial than receiving it: Results from a prospective study of mortality. Psychol Sci. 2003;14(4):32027. [PMID: 12807404]

[12.] Roberts LJ, Salem D, Rappaport J, Toro PA, Luke DA, Seidman E. Giving and receiving help: Interpersonal transactions in mutual-help meetings and psychosocial adjustment of members. Am J Community Psychol. 1999;27(6): 84168. [PMID: 10723537]

[13.] Weaver Randall K, Salem DA. Mutual-help groups and recovery: The influence of settings on participants' experience of recovery. In: Ralph RO, Corrigan P, editors. Recovery in mental illness: Broadening our understanding of wellness. Washington (DC): American Psychological Association; 2005.

[14.] Dumont J, Jones K. Findings from a consumer/survivor defined alternative to psychiatric hospitalization. Evaluation Center @HSRI and NASMHPD Research Institute. Outlook. 2002;Spring:46.

[15.] Van Tosh L, Del Vecchio P. Consumer operated self-help programs: A technical report. Rockville (MD): U.S. Center for Mental Health Services; 2000.

[16.] Forquer S, Knight E. Managed care: Recovery enhancer or inhibitor? Psychiatr Serv. 2001;52(1):2526. [PMID: 11141523]

[17.] Galanter M. Zealous self-help groups as adjuncts to psychiatric treatment: A study of Recovery, Inc. Am J Psychiatry. 1988;145(10):124853. [PMID: 3421346]

[18.] Powell TJ, Yeaton W, Hill EM, Silk KR. Predictors of psychosocial outcomes for patients with mood disorders: The effects of self-help group participation. Psychiatr Rehabil J. 2001;25(1):311. [PMID: 11529450]

[19.] Yanos PT, Primavera LH, Knight EL. Consumer-run service participation, recovery of social functioning, and the mediating role of psychological factors. Psychiatr Serv. 2001;52(4): 493500. [PMID: 11274496]

[20.] Chamberlin J, Rogers ES, Ellison ML. Self-help programs: A description of their characteristics and their members. Psychiatr Rehabil J. 1996;19(3):3342.

[21.] Davidson L, Chinman M, Kloos B, Weingarten R, Stayner D, Tebes JK. Peer support among individuals with severe mental illness: A review of the evidence. Clin Psychol Sci Pract. 1999;6(2):16587.

[22.] McLean A. Empowerment and the psychiatric consumer/ ex-patient movement in the United States: Contradictions, crisis and change. Soc Sci Med. 1995;40(8):105371. [PMID: 7597459]

[23.] Rappaport J. The power of empowerment language. Soc Policy. 1985;16:1521.

[24.] Segal SP, Silverman C, Temkin T. Measuring empowerment in client-run self-help agencies. Community Ment Health J. 1995;31(3):21527. [PMID: 7621659]

[25.] Rogers ES, Chamberlin J, Ellison ML, Crean T. A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatr Serv. 1997;48(8): 104247. [PMID: 9255837]

[26.] American Psychiatric Association. Diagnostic and statistical manual for mental disorders. 4th ed. Washington (DC): American Psychiatric Association; 1994.

[27.] Johnsen M, Teague G, McDonel Herr E. Common ingredients as a fidelity measure. In: Clay S, Schell B, Corrigan PW, Ralph RO, editors. On our own, together: Peer programs for people with mental illness. Nashville (TN): Vanderbilt University Press; 2005. p. 21338.

[28.] Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for casual effects. Biometrika. 1983;70:4155.

[29.] Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale (NJ): Lawrence Erlbaum; 1988.

E. Sally Rogers, ScD; (1) * Gregory B. Teague, PhD; (2) Carolyn Lichenstein, PhD; (3) Jean Campbell, PhD; (4) Asya Lyass, MA; (1) Ren Chen, MS; (5) Steven Banks, PhD (6) [dagger]

(1) Center for Psychiatric Rehabilitation, Boston University, Boston, MA; (2) Department of Mental Health Law and Policy, Louis de la Parte Florida Mental Health Institute, University of South Florida, Tampa, FL; (3) Walter R. McDonald & Associates, Inc, Sacramento, CA; (4) Program in Consumer Studies and Training, Missouri Institute of Mental Health, University of Missouri, St. Louis, MO; (5) Policy and Services Research Data Center, Department of Mental Health Law and Policy, Louis de la Parte Florida Mental Health Institute, University of South Florida, Tampa, FL; (6) Department of Psychiatry, University of Massachusetts Medical School, Worcester, MA

Abbreviations: AT = as-treated, CMHS = Center for Mental Health Services, COSP = consumer-operated service program, DHHS = Department of Health and Human Services, ES = effect size, IRB = institutional review board, ITT = intent-to-treat, MDE = Making Decisions Empowerment, OME = Organizationally Mediated Empowerment, PE = Personal Empowerment, SAMHSA = Substance Abuse and Mental Health Services Administration, SD = standard deviation.

* Address all correspondence to E. Sally Rogers, ScD; Center for Psychiatric Rehabilitation, Boston University, 940 Commonwealth Avenue, Boston, MA 02215; 617-353-3549; fax: [phone]. Email: [e-mail]

([dagger]) Dr. Steven Banks died August 10, 2007.

DOI: 10.1682/JRRD.2006.10.0125

Table 1.
Baseline participant characteristics (N = 1,827).

Characteristic                             Percent   Mean

Demographic
  Sex (% female)                              60.1    --
  Age (yr)                                     --    42.7
  Race/Ethnicity
    White                                     56.9    --
    African American                          16.5    --
    Hispanic                                   1.2    --
    Other                                      2.5    --
    Biracial/Multiracial                      22.4    --
  Currently Married                           12.6    --
  Have Any Children                           52.8    --
  Currently Employed for Pay or Volunteer     29.3    --
  Education
    0-8 yr                                     9.3    --
    9-12 yr                                   23.7    --
    High School Graduation                    25.3    --
    Any College/Vocational Training           38.7    --
    Any Postgraduate Work                      3.0    --
  Social Security Income (last 30 d)          83.6    --
  Living Arrangement
    Currently in Own Residence                57.9    --
    Currently Homeless                        10.2    --
Clinical
  Primary Axis I Diagnosis
    Mood Disorders                            44.4    --
    Anxiety Disorders                          3.7    --
    Schizophrenia/Psychotic Disorders         50.4    --
    Other                                      1.5    --
  Any Psychiatric Hospitalization             82.1    --
  Age First Psychiatric Contact (yr)           --    23.0
    (hospitalization or outpatient)
  >5 Psychiatric Hospitalizations             36.3    --
  Recent Psychiatric Hospitalization          16.0    --
  Physical Disability                         51.7    --

Table 2.
Baseline mean [+ or -] standard deviation for Making Decisions
Empowerment (MDE), Personal Empowerment (PE), and Organizationally
Mediated Empowerment (OME) by site and differences between
experimental (n = 920) and control (n = 907) groups by site and
overall. Actual n for each outcome may vary because of missing values.

         Outcome              Site         Experimental

MDE                             1       79.06 [+ or -] 8.66
                                2       75.92 [+ or -] 7.86
                                3       76.32 [+ or -] 6.62
                                4       78.77 [+ or -] 7.82
                                5       77.39 [+ or -] 8.24
                                6       81.13 [+ or -] 8.28
                                7       79.35 [+ or -] 8.69
                                8       86.32 [+ or -] 9.37
Total MDE                   All Sites   78.12 [+ or -] 8.47
PE Choice                       1       34.12 [+ or -] 4.31
                                2       32.31 [+ or -] 5.58
                                3       29.42 [+ or -] 5.65
                                4       30.81 [+ or -] 5.37
                                5       32.93 [+ or -] 5.18
                                6       32.84 [+ or -] 5.19
                                7       29.77 [+ or -] 5.56
                                8       33.49 [+ or -] 3.83
Total PE Choice             All Sites   32.05 [+ or -] 5.43
PE Reduction in Chance          1       39.83 [+ or -] 6.28
                                2       37.16 [+ or -] 6.17
                                3       39.45 [+ or -] 5.79
                                4       39.17 [+ or -] 6.46
                                5       38.96 [+ or -] 6.44
                                6       38.72 [+ or -] 6.00
                                7       39.47 [+ or -] 6.67
                                8       42.08 [+ or -] 5.22
Total PE Reduction in       All Sites   38.62 [+ or -] 6.32
  Chance
OME                             1        2.63 [+ or -] 3.05
                                2        1.40 [+ or -] 2.34
                                3        2.58 [+ or -] 2.62
                                4        1.65 [+ or -] 2.15
                                5        2.17 [+ or -] 2.30
                                6        2.52 [+ or -] 3.05
                                7        2.16 [+ or -] 2.42
                                8        6.38 [+ or -] 3.71
Total OME                   All Sites    2.15 [+ or -] 2.79

         Outcome              Site            Control

MDE                             1       81.03 [+ or -] 9.28
                                2       75.98 [+ or -] 8.19
                                3       76.79 [+ or -] 7.08
                                4       79.49 [+ or -] 7.43
                                5       79.12 [+ or -] 7.86
                                6       77.80 [+ or -] 8.59
                                7       79.93 [+ or -] 7.85
                                8       84.06 [+ or -] 9.94
Total MDE                   All Sites   78.23 [+ or -] 8.43
PE Choice                       1       32.85 [+ or -] 5.14
                                2       32.58 [+ or -] 5.66
                                3       29.82 [+ or -] 6.22
                                4       30.53 [+ or -] 5.51
                                5       32.76 [+ or -] 5.34
                                6       32.37 [+ or -] 4.38
                                7       31.36 [+ or -] 4.27
                                8       34.15 [+ or -] 5.09
Total PE Choice             All Sites   32.11 [+ or -] 5.43
PE Reduction in Chance          1       39.67 [+ or -] 6.11
                                2       37.25 [+ or -] 6.45
                                3       37.50 [+ or -] 6.27
                                4       38.92 [+ or -] 5.84
                                5       37.77 [+ or -] 6.08
                                6       38.05 [+ or -] 6.00
                                7       40.54 [+ or -] 5.52
                                8       40.17 [+ or -] 5.75
Total PE Reduction in       All Sites   38.25 [+ or -] 6.21
  Chance
OME                             1       2.90 [+ or -] 3.59
                                2       1.40 [+ or -] 2.48
                                3       2.25 [+ or -] 2.52
                                4       1.83 [+ or -] 2.24
                                5       2.39 [+ or -] 2.59
                                6       3.08 [+ or -] 3.03
                                7       3.39 [+ or -] 3.06
                                8       5.77 [+ or -] 3.51
Total OME                   All Sites   2.32 [+ or -] 2.92

         Outcome              Site          F-Statistic

MDE                             1              1.53
                                2              0.01
                                3              0.15
                                4              0.56
                                5              2.77
                                6              6.93 *
                                7              0.17
                                8              1.35
Total MDE                   All Sites          0.00
PE Choice                       1              2.31
                                2              0.37
                                3              0.14
                                4              0.16
                                5              0.06
                                6              0.43
                                7              3.70
                                8              0.53
Total PE Choice             All Sites          0.05
PE Reduction in Chance          1              0.02
                                2              0.03
                                3              3.28
                                4              0.09
                                5              2.16
                                6              0.55
                                7              1.12
                                8              2.92
Total PE Reduction in       All Sites          1.58
  Chance
OME                             1              0.22
                                2              0.00
                                3              0.53
                                4              0.41
                                5              0.45
                                6              1.54
                                7              7.07 *
                                8              0.70
Total OME                   All Sites          1.70

* Significant difference between sites at p < 0.01.

Table 3.
Mean [+ or -] standard deviation (SD) for Making Decisions
Empowerment (MDE), Personal Empowerment (PE), and
Organizationally Mediated Empowerment (OME) scales by
intervention engagement intensity for each time point.
Data presented in these analyses only for participants in
middle third of consumer-operated service program propensity
distribution.

                                      No Use
                     Time
    Outcome         Period       Mean [+ or -] SD       n

MDE *                  1       78.23 [+ or -] 8.10     365
                       2       78.42 [+ or -] 7.80     335
                       3       78.91 [+ or -] 7.93     309
                       4       79.31 [+ or -] 8.12     271

PE Choice              1       33.11 [+ or -] 4.71     368
  ([dagger])           2       32.83 [+ or -] 5.37     337
                       3       32.82 [+ or -] 5.42     315
                       4       33.24 [+ or -] 4.87     273

PE Reduction in        1       38.58 [+ or -] 6.17     366
  Chance               2       38.87 [+ or -] 6.25     333
                       3       38.97 [+ or -] 6.26     312
                       4       39.49 [+ or -] 5.93     271

OME                    1        2.27 [+ or -] 2.86     366
                       2        1.70 [+ or -] 2.25     334
                       3        1.57 [+ or -] 2.22     312
                       4        1.61 [+ or -] 2.20     272

                                     Low Use
                     Time
    Outcome         Period       Mean [+ or -] SD       n

MDE *                  1       78.53 [+ or -] 8.43     77
                       2       78.95 [+ or -] 7.22     72
                       3       78.57 [+ or -] 7.95     65
                       4       79.91 [+ or -] 9.24     69

PE Choice              1       31.94 [+ or -] 5.08     77
  ([dagger])           2       31.81 [+ or -] 6.02     73
                       3       31.74 [+ or -] 5.61     66
                       4       32.49 [+ or -] 6.24     69

PE Reduction in        1       39.16 [+ or -] 5.51     78
  Chance               2       38.86 [+ or -] 6.60     74
                       3       38.59 [+ or -] 5.96     66
                       4       38.70 [+ or -] 6.96     70

OME                    1        2.02 [+ or -] 2.96     78
                       2        1.92 [+ or -] 2.44     74
                       3        1.51 [+ or -] 2.23     67
                       4        1.52 [+ or -] 2.22     70

                                     High Use
                     Time
    Outcome         Period       Mean [+ or -] SD       n

MDE *                  1       77.14 [+ or -] 6.55     78
                       2       78.79 [+ or -] 6.76     74
                       3       79.38 [+ or -] 6.61     71
                       4       81.35 [+ or -] 6.60     64

PE Choice              1       31.11 [+ or -] 6.19     76
  ([dagger])           2       32.35 [+ or -] 5.07     71
                       3       32.28 [+ or -] 5.42     71
                       4       33.20 [+ or -] 5.05     65

PE Reduction in        1       39.02 [+ or -] 5.79     74
  Chance               2       40.21 [+ or -] 5.13     68
                       3       39.45 [+ or -] 4.95     68
                       4       39.25 [+ or -] 5.31     62

OME                    1        2.21 [+ or -] 2.79     78
                       2        2.19 [+ or -] 2.59     74
                       3        2.62 [+ or -] 3.07     73
                       4        2.66 [+ or -] 2.89     65

* Significant differences between groups ([F.sub.2, 1306] = 7.03,
p = 0.02); high use > no use, low use.

([dagger]) Significant differences between groups ([F.sub.2,
1306] = 3.15, p = 0.04); high use > no use.

Table 4.
Summary of intent-to-treat (ITT) and as-treated (AT) statistical
analyses (p-values) for Making Decisions Empowerment (MDE),
Personal Empowerment (PE), and Organizationally Mediated
Empowerment (OME) scales.

                                                      MDE
           Results                  MDE        (1 site removed *)

ITT
  Time x Group                       NS              0.02
  Time x Group x Site               0.19              NS
                                 ([dagger])
AT
  Binary Engagement (yes/no)
    Time x Group                   0.046              --
    Time x Group x Site              NS               --
  Intensity (no use, low
      use, high use)
    Time x Group                    0.02              --
    Time x Group x Site              NS               --

                                     PE            PE Choice
           Results                 Choice      (1 site removed *)

ITT
  Time x Group                      0.06             0.02
  Time x Group x Site               0.02             0.01
AT
  Binary Engagement (yes/no)
    Time x Group                     NS               --
    Time x Group x Site              NS               --
  Intensity (no use, low
      use, high use)
    Time x Group                    0.04              --
    Time x Group x Site              NS               --

                                PE Reduction
           Results               in Chance            OME

ITT
  Time x Group                       NS              0.03
  Time x Group x Site                NS              0.07
                                                   ([double
                                                   dagger])
AT
  Binary Engagement (yes/no)
    Time x Group                     NS               NS
    Time x Group x Site              NS               NS
  Intensity (no use, low
      use, high use)
    Time x Group                     NS               NS
    Time x Group x Site              NS        0.11 ([dagger])
                                                   ([double
                                                   dagger])

* Note that sensitivity analyses were conducted if significant
time x group x site interaction existed.

([dagger]) p = 0.2 was cutoff for 3-way interaction.

([double dagger]) This marginally significant time x group x site
interaction was not attributable to any 1 site; therefore, we do not
present sensitivity analysis with 1 site removed.

NS = not significant.