Test of efficacy of model family strengthening programs.

Abstract: The present study used national, multi-site data to determine the efficacy of the model programs implemented. The data (N = 1,080) for the study was the Family Strengthening Initiative funded by the Center for Substance Abuse Prevention (CSAP). Data were collected using the cross-site instrument developed by CSAP and the Program Coordinating Center (PCC) staff. All study sites utilized a longitudinal, pre- and post-test design. Measures included the constructs of family resilience, family conflict, family cohesion, and familyattachment. Findings from analysis of covariance (ANCOVA) showed differential effects of these model programs. For family attachment and family cohesion, the Nurturing Parenting Program appeared to be the most effective among all the model programs tested (p<.05). It is concluded that community-based organizations adopting these family-focused programs need to consider the outcome factors that are most relevant to their program.

**********

According to the literature, every psychological theory of child development has acknowledged the importance of family factors on adolescent behavior (Kumpfer, Alvarado, & Whiteside, 2003). In recent years, prevention strategies have given increasing attention to the importance of family factors in delaying or reducing the onset and frequency of adolescent substance use (Center for Substance Abuse Prevention, 2002). The literature highlights that implementing validated family focused interventions can benefit the public health (Mrazek & Haggerty, 1994).

The core of family-focused interventions is training parents and caregivers in theory-based child management strategies. Another key component of behavioral family interventions is educating parents in the principles of behavior. Multiple studies have evaluated the added impact of educating parents on the behavioral principles of the strategies used. For example, Erhardt and Baker (1990) and Lawes (1992) conducted longitudinal studies and found significant improvement with respect to parental knowledge of behavioral principles following family-focused interventions. Another element of family-focused interventions is to modify family functioning among high risk populations that have shown promise in reducing adolescent problem behaviors and drug use (Borduin, Henggeler, & Manley, 1995; Schmidt, Liddle, & Dakof, 1996). In a longitudinal study of Hispanic adolescents and their parents, Santisteban and colleagues (2003) investigated the efficacy of a family intervention. Results showed that participants in the family intervention showed significantly greater pre- to post intervention improvement in self reports of family cohesion than those in the control group. Moreover, efficacy of the intervention in improving family functioning was evidenced by independent observer ratings of improvement in family interactions.

The ecological systems model of human development provides a basis for understanding the parent-child relationship within families embedded in a complex social environment with diverse expectations (Bronfenbrenner, 1979, 1986). Specifically, the ecological systems theory expands our understanding of the child beyond the individual's psychological development to a view that positions the child within the context of relationships and interactions with others (Worden, 1999). The concept of mutual interdependence of these systems, first proposed by Bronfenbrenner, is a way of viewing the child as a "dynamic and evolving being that interacts with, and thereby restructures, the many environments with which he/she comes into contact" (Gardiner, Mutter, & Kosmitzki, 1998, p. 9). The family, as a developmental niche, is also an open system that interacts across its boundary with its surrounding contexts. Alluding to the ecological perspective, Germain and Gitterman (1987) assert that "neither the people served, nor their environment, can be fully understood except in relationship to each other" (p. 493).

To date, various family-focused model programs have been implemented in order to improve the family relationship and to reduce behavioral problems among children. However, in reviewing the literature, few studies have evaluated the efficacy of various family-focused model programs. The current study used national, multi-site data to determine the efficacy of the model programs implemented. A cross-site survey assessed parents prior to the intervention to establish baseline. Parents were also assessed post-intervention to determine the proximal impact of the parenting intervention provided. Four main indicators of family functioning were used.

METHOD

SAMPLE

The data source for the current study was the Family Strengthening Initiative funded by the Center for Substance Abuse Prevention (CSAP). Data were obtained from a total of thirty-two funded study sites that implemented an evidence-based model program. However, only program sites that adopted one of the following four model programs were included in data analyses: 1) Strengthening Multi-Ethnic Families & Communities (n=699), 2) Parenting Wisely (n=78), 3) Strengthening Families Program (n=75), and 4) Nurturing Parenting Program (n=65). The total sample contained 1,080 participants.

MODEL PROGRAM DESCRIPTION

1) Strengthening Multi-Ethnic Families & Communities--integration of various prevention and intervention strategies geared toward reducing violence against self, the family and the community. The program targets ethnic and culturally diverse parents of children ages 3-18 years. The goal is to reduce drug and alcohol use, teen suicide, juvenile delinquency, gang involvement, child abuse and domestic violence (Center for Substance Abuse Prevention, 2002).

2) Parenting Wisely--an interactive program designed for families at risk with children from early elementary to high school age. The Parenting Wisely program is based on social learning theory, family systems theory, and cognitive theory. Parenting Wisely seeks to help families enhance relationships and to decrease conflict through behavior management and support. It enhances child adjustment and potentially reduces delinquency, substance abuse, and involvement with the juvenile justice system (Center for Substance Abuse Prevention, 2002).

3) Strengthening Families Program--a behavioral and cognitive skills training program designed to increase resilience and to reduce risk factors for substance abuse, depression, violence and aggression, delinquency and school failure in high-risk children and their parents. Parents learn to increase desired behaviors in children by using attention and rewards, clear communication, effective discipline, substance-use education, problem solving, and limit-setting strategies (Center for Substance Abuse Prevention, 2002).

4) Nurturing Parenting Program--family-centered programs designed to build nurturing skills as alternatives to abusive parenting and child-rearing attitudes and practices. Based on a re-parenting philosophy, parents and children learn cognitive and affective activities designed to build self-awareness, positive self-concept, self-esteem, and empathy in order to learn alternatives to yelling and hitting. These skills enhance family communication and awareness of needs, replace abusive behavior with nurturing, and promote healthy physical and emotional development (Center for Substance Abuse Prevention, 2002).

MEASURES

In addition to the routine demographic variables such as age and race, information concerning participants' income, marital status, educational attainment, employment status, and household composition were obtained. Although other measures were used, provided below are the family functioning measures specific to this study. Family conflict was measured using the 10-item Conflict subscale of the widely used Family Environment Scale (FES) Form R. The FES Form R measures the real or present environment (Moos & Moos, 1994). The FES Conflict subscale was modified by adding three additional questions. The Conflict construct indicates the extent to which open expression of anger and aggression and generally conflicting interactions are characteristics of the family (e.g., "We fight a lot in our family"). The response categories for the items in this subscale were modified and adapted for this initiative. The responses were true, somewhat true, somewhat false, and false.

Internal consistency of .75 has been reported for the Conflict scale. Several studies using a variety of methodological approaches and participants provide ample evidence of the construct validity of the FES (Dancy & Handal, 1984; Forman & Forman, 1981; Moos & Moos, 1983). Additionally, the FES has been normed with white, African American, and Latino groups (Moos & Moos, 1994, p. 18). Items assessing the constructs of family resilience, family attachment, and family cohesion were modified by project staff and adapted for the current Family Strengthening Initiative as part of the core family measures contained within the instrument (see Table 1). The response choices for the family resilience items were 5-point Likert-type scales ranging from 1 (no strength) to 5 (amazing strength). The family cohesion items had a 4-point Likert-type scale which included true, somewhat true, somewhat false, and false. Finally, the family attachment scale had responses of yes, no, and don't know.

Upon initial examination of the family scales in their entirety, reliability analysis resulted in the following coefficient alpha found in Table 2.

Coefficient alpha indicates item homogeneity based on the scores of each scale. Alpha levels greater than 0.70 indicate modest reliability, which is acceptable for early stages of research. Alpha levels greater than 0.80 are considered adequate (Nunally & Bernstein, 1994). The family strengths and resilience scale and the family relations and cohesion scale exhibited good scores. However, the family attachment and family conflict scales showed unacceptable alpha based on criteria set by Nunally and Bernstein.

DATA COLLECTION

Each study site assessed parents prior to intervention to establish baseline, as well as after and post-intervention to assess the intervention's proximal impact. Only one parent from each household was assessed even if both parents attended the program sessions. The parent who provided the majority of the caregiving served as the index parent.

To address the research questions, data were collected using the cross-site instrument developed by CSAP and PCC staff. All thirty-two study sites utilized a longitudinal, pre-and post-test design. Study sites used either one-on-one administration or small group administration of the cross site instrument. For the purposes of this study, family functioning, 30 day substance use, attitudes toward drug use, and perception of harm were measured by self reports by parents participating in the program.

DATA ANALYSIS

The analyses consisted of demographics, such as gender, race, education, age, and employment status of the sample. An analysis of covariance (ANCOVA) was also performed for each of four family functioning outcome variables as the dependent variable. The four program models served as the independent variable and the family functioning variables at pre-test, age, and education were covariates. If a significant result was found, a post hoc test was conducted on the adjusted means between program models using Duncan's criterion.

RESULTS

Coefficient alpha is an appropriate reliability estimator for composite measures containing multiple components. Multiple components may be homogeneous in the sense of measuring a single latent variable, or they may be heterogeneous in the sense of measuring two or more factors or latent variables (Osburn, 2000). It has been noted that internal consistency reliability coefficients, including coefficient alpha, that are based on a single administration of a test may overestimate reliability. This may occur because such coefficients assign transient error due to differences in test administration, temporary changes in the participant, and so on, to true-score variance (Schmidt & Hunter, 1996). Although it is true that coefficient alpha may sometimes be inflated for the reasons mentioned above, a potentially more serious problem is the tendency of coefficient alpha to underestimate the true reliability when the data are multidimensional. If the items conform to classical test theory assumptions, coefficient alpha is then a lower bound to the true reliability of a measure (Osburn, 2000).

DEMOGRAPHICS

Other characteristics of the study participants include an average of 11 years (SD = 3.87) of schooling with 30.4% of the respondents unemployed but looking for work, 27.0% employed full time, 15.9% specified other for employment, and 11.4% were employed part time. Other participants were either unemployed due to disability (6.9%) or unemployed due to retirement and volunteer work (8.1%). Additionally, 46.2% of respondents reported living in the city, 22.9% lived in a suburb/ town, 11.7% lived on a reservation and 10.5% lived in a rural area. Family composition characteristics included the following: 33.9% of the families had two children under 18 years of age living at home, 20.1% only had one child, 18.6% had three children, and 18.6% had four or more children under 18 years of age living at home. Ninety percent of the participants reported being the biological relative of the child. Of this number, 74.2% were the biological mother, 16.9% were the biological father, and 6.9% were either an aunt or uncle. Of those respondents who were not a biological relative, 33.9% were a step-parent, 18.6% were a partner to a biological parent, 15.3% were a foster parent, 8.5% were adoptive parents, and the remaining 20.4% were a surrogate parent, close non relative or designated other. Finally, 41.5% of the families used English as the primary language in the home, 28.3% used Spanish, and 30.2% spoke another language.

1) Family Attachment. The ANCOVA found that there was an overall significance of family attachment scores among model programs after controlling for the baseline family attachment scores, age, and education F (6, 782) = 15.12, p<.0001. The model programs factor after adjusting all covariates was significant, p=.03. The post hoc test on the adjusted means showed that the Nurturing Parenting Program had the highest score. This score was significantly higher than the Strengthening Family Program and the Parenting Wisely Program (p<.05).

2) Family Cohesion. The ANCOVA found that there was an overall significance of family cohesion scores among model programs after controlling for the baseline family cohesion scores, age, and education [F (6, 782) = 45.06, p<.0001]. The model programs factor after adjusting for the covariates was significant (p=.05). The post hoc test on the adjusted means showed that Nurturing Parenting Program had the highest score compared to all other three model programs (p<.05). All other model programs were not significantly different from each other (p>.05).

3) Family Resilience. The ANCOVA found that there was an overall significance of family resilience scores among model programs after controlling for the baseline resilience scores and age [F(6, 818) = 35.48, p<.0001]. After adjusting all covariates, the model programs were significant (p<.02).

The post hoc test on the adjusted means showed that the Strengthening Multi-Ethnic Families and Communities Program scored significantly higher than the Nurturing Parenting Programp < .05). No other programs significantly differed from each other.

4) Family Conflict. The ANCOVA found that there was an overall significance of family conflict scores among model programs after controlling for the baseline conflict scores and age [F(6, 788) = 41.64, p<.0001]. The model programs factor after adjusting for all covariates was significant (p<.03).

The post hoc test on the adjusted means showed that the Strengthening Multi-Ethnic Family Communities Program scored significantly higher than the Parenting Wisely and Strengthening Families Program. The Nurturing Parents Program also scored significantly higher than the Strengthening Families Program (p<.05).

DISCUSSION

Family-focused programs can strengthen conditions of family functioning as measured by family resilience, family cohesion, family conflict, and family attachment. Ultimately, these improved family functions help at risk youth avoid substance abuse and other personal and social problems including school failure, crime, family violence, and abuse. There have been a host of family-focused programs for substance abuse and delinquency prevention since the 1980s. Many programs have reported to be effective (Aktan, Kumpfer, & Turner, 1996; Center for Substance Abuse Prevention, 2002; Kumpfer et al., 2003). However, a review of the literature indicated that there have been no studies that compared the efficacies of these four family-focused prevention programs. To the best of our knowledge, this study is the first effort to assess four model programs using a pre- and post-study design.

Findings from this evaluation study showed differential effects of these model programs. For family attachment and family cohesion, the Nurturing Parenting Program appeared to be the most effective among all the model programs tested. The Nurturing Parenting Program is an extensive. Its content involves one, 2-3 hour block of time per week, for 12 to 45 weeks. A previous evaluation of the Nurturing Parenting Program with 121 abusive adults and 150 abused children revealed significant improvements in parenting attitudes, personality characteristics of both parents and children, and family interaction patterns (Center for Substance Abuse Prevention, 2002). Another validation study also found similar results in the form of significant pre- and post-study changes in parenting attitudes and child reading practices. Our results confirmed these findings by showing that the Nurturing Parenting Program improves family attachment and family cohesion.

For family resilience and family conflict, Strengthening Multi-Ethnic Families and Communities Program showed the best results. This program consisted of training classes at a variety of locations including churches, schools, community agencies and other locations. A previous study of 22 parent groups (N=357) in multi-ethnic family programs showed significant improvements in parents' sense of competence, as well as child competence and behavior (Center for Substance Abuse Prevention, 2002). Key components of family resilience are Family Unity (togetherness, cohesion), Positive Mental Health, Physical Health, Emotional Strength, and Spiritual Strength. These areas are in line with improved competencies as reported in the CSAP document (Center for Substance Abuse Prevention, 2002).

CONCLUSIONS AND IMPLICATIONS

Based on these findings, we conclude that the Family Strengthening Initiative made significant contributions to the body of literature. This evaluation demonstrated that the initiative is effective in improving family functioning as measured by family resilience, family cohesion, family conflict, and family attachment. Though the determination of specific mechanisms for these improvements will require additional studies; however, the general mechanism may be that the interventions change parenting practices when parents are provided with guidance that allows them to set clear limits, criticize less, and use praise more often. Likewise, parents may also experience several benefits including reduced stress, improved communication and improved conflict resolution.

Though many family-focused prevention programs have been reported to be effective, programs do not always generate the same effective outcomes. The findings from this study suggest that community-based organizations adopting these family-focused programs need to consider the outcome factors that are most relevant to their program. Particularly, the Nurturing Parenting Program is most suitable for improving family attachment and family cohesion, while the Strengthening Multi-Ethnic Families and Communities Program is most suitable for improving family resilience and family conflict.

LIMITATIONS

Although this particular study design can examine an index change from the pretest to the posttest, there are several limitations. First, the study had no control group. Consequently, we could only compare the model effects rather than the intervention effect. Second, since this was a community-based study, samples were not randomly selected. It is unknown how much these model differences may be due to differences in samples. The ANCOVA technique we adopted only controlled for age and education. There might be other potential covariates that have not been included in the statistical model. Future studies should consider more stringent research designs: 1) by the use of an appropriate randomly assigned control group, and 2) by carefully examining the sample differences. In this case, all model programs can be compared with each other, as well as with the control group. If the intervention group is significantly different from the control group post-intervention, the difference between the two groups can be attributed to the effect of the intervention.

ACKNOWLEDGEMENT: The study described was supported by grant number UD1 SP09227 from CSAP, SAMHSA, and the U.S. Department of Health and Human Services. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of CSAP, SAMHSA, or the U.S. Department of Health and Human Services.

REFERENCES

Aktan, G. B., Kumpfer, K. L., & Turner, C. W. (1996). Effectiveness of a family skills training program for substance use prevention with inner city African-American families. Substance Use & Misuse, 31, 157-175.

Bogenschneider, K. (1996). An ecological risk/protective theory for building prevention programs, policies, and community capacity to support youth. Family Relations, 45, 127-138.

Borduin, C. M., Henggeler, S. W., & Manley, C.M. (1995). Conduct and oppositional disorders. In V. B. Van Hasselt & M. Hersen (Eds.), Handbook of adolescent psychopathology (pp. 349-383). New York: Lexington Books Press.

Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives. Developmental Psychology, 22, 723-742.

Bronfenbrenner, U. (1979).The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.

Center for Substance Abuse Prevention (CSAP). (2002). The National Cross-Site Evaluation of High-Risk Youth Programs. Monograph Series 1-4 prepared for CSAP by EMT Associates, Inc. and ORC Macro.

Erhardt, D., & Baker, B. L. (1990). The effects of behavioral parent training on families with young hyperactive children. Journal of Behavior Therapy and Experimental Psychiatry. 21, 121-132.

Gardiner, H. W., Mutter, J. D., & Kosmitzki, C. (1998). Lives across culture: Cross cultural human development. Boston: Allyn & Bacon.

Germain, C. B., & Gitterman, A. (1987). Ecological perspective. In A. Minahan (Ed.). Encyclopedia of social work (18th ed., pp. 488-499). Silver Spring, MD: National Association of Social Workers.

Glogower, E, & Sloop, E. W., (1976). Two strategies of group training of parents as effective behavior modifiers. Behavior Therapy, 7, 177-184.

Kumpfer, K. L., Alvarado, R., & Whiteside, H. O. (2003). Family-based interventions for substance use and misuse prevention. Substance Use &Misuse, 38, 1759-1787.

Lawes, G. (1992). Individual parent training implemented by nursery nurses: Evaluation of a program for mothers of pre-school children. Behavioral Psychotherapy, 20, 239-256.

McMahon, R., Forehand, R., & Griest, D. (1981). Effects of knowledge of social learning principles on enhancing treatment outcome and generalization in a parent training program. Journal of Consulting and Clinical Psychology, 49, 526-532.

Mrazek, P., and Haggerty, R. (1994). Reducing risk for mental disorders: Frontiers for preventive intervention research. Washington, DC: National Academy Press.

Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

Osburn, H.G. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psychological Methods, 5, 343-355.

Santisteban, D. A., Coatsworth, J. D., Perez-Vidal, A., Kurtines, W. M., Schwartz, S. J., LaPerriere, A., et al. (2003). Efficacy of brief strategic family therapy in modifying Hispanic adolescent behavior problems and substance use. Journal of Family Psychology, 17, 121-133.

Schmidt, F.L., & Hunter, J.E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1, 199-223.

Schmidt, S. E., Liddle, H. A., & Dakof, G. A. (1996). Changes in parenting practices and adolescent drug abuse during multidimensional family therapy. Journal of Family Psychology, 10, 1-16.

Worden, M. (1999). Family therapy basics (2nd ed.). Pacific Grove: Brooks/Cole.

CHES AREA

Responsibility IV: Evaluating Effectiveness of Health Education Programs

Resa F. Matthew, MPH is affiliated with The McFarland Institute. Min Qi Wang, PhD is a Professor in the Department of Public and Community Health at the University of Maryland. Nikki Bellamy, PhD and Evelyn Copeland, PhD are affiliated with the Center for Substance Abuse Prevention at the Substance Abuse and Mental Health Services Administration 

Table 1. The variables associated with family functioning construct.

Family Resilience

Family Supportiveness/Love/Care
Positive Family Communication
Effective Parenting Skills (reading to child, rewarding)
Effective Discipline Style (less spanking, consistent)
Family Organization (rule, chore, self responsibility)
Family Unity (togetherness, cohesion)
Positive Mental Health
Physical Health
Emotional Strength
Knowledge and Education
Social Networking (friends, community)
Spiritual Strength

Family Conflict

People in my family often insult or yell at each other.
People in my family have serious arguments.
We argue about the same things in my family over and over.
We fight a lot in our family.
Family members rarely become angry.
Family members hardly ever lose their tempers.
Family members often criticize each other.
We really get along well with each other.
In our family, we believe you don't get anywhere by raising
  your voice.
Family members sometimes get so angry they throw things.
Family members sometimes hit each other.
If there's disagreement in our family, we try hard to smooth
  things over and keep the peach.
Family members often try to one-up or out-do each other.

Family Cohesion

I'm available when others in the family want to talk with me.
I listen to what other family members have to say, even when
  I disagree.
Family members ask each other for help.
Family members like to spend free time with each other.
Family members feel very close to each other.
We can easily think of things to do together as a family.
Do you feel very close to your child in this program?
Do you share thoughts and feelings with your child?
Do you enjoy spending time with your child in this program?
Do you feel very close to your partner?
Do you discuss parenting ideas with your partner?

Family Involvement

Does your child help with family fun activities?
Does your child like to get involved in such family activities?
Do you have time to listen to your child when he/she wants to talk
  to you?
Do you and your child do things together at home?
Does your child go out with other family members to movies, sports
  events, or other things?
Do you have friendly chats with your child?
Does your child help you with chores, errands, and/or other work?
Do you talk with your child about how he/she is doing in school?

Table 2. Reliability coefficients of Family Core Measures

     Scale                  # of items   Coefficient alpha

Family Resilience               12           alpha=.93
Family Conflict                 13           alpha=.68
Family Relations/Cohesion        6           alpha=.82
Family Attachment                5           alpha=.59