Printer Friendly

Do student characteristics influence the effectiveness of the tools for getting along curriculum? An examination using a cognitive-behavioral intervention.


Cognitive-behavioral interventions (CBIs) are effective in decreasing externalizing behavior in school-aged children. To ensure that CBIs meet the needs of a diverse student population, it is important to examine whether intervention effectiveness is influenced by characteristics common to students identified with problem behaviors. In this study, we used structural equation modeling to examine whether the effectiveness of Tools for Getting Along, a school-based, universally delivered CBI, was moderated by student race, socioeconomic status, and/or levels of anger control prior to intervention, and whether post-intervention anger levels served as a mediator of the intervention effects. Findings suggested that race and socio-economic status did not moderate intervention effectiveness, but effects on externalizing behavior were moderated by levels of anger control prior to intervention. Findings also indicated that post-intervention anger control levels did not mediate intervention effects. We discuss the limitations of our study, along with implications for research and practice.

Keywords: aggression, race, social-emotional learning, social problem solving, mediation; moderation, multi-level structural equation modeling


Student problem behavior characterized as aggressive, impulsive, and/or rule-breaking has a negative effect on the overall school environment. The display of these problem behaviors by students is disruptive to classrooms and perplexing to school personnel, who must respond with efficient and effective interventions to assuage short-and long-term negative outcomes. Persistent externalizing behaviors often result in academic difficulties, high school dropout rates, social maladjustment, chronic unemployment, and an increased risk of substance abuse and adult criminality (Vaughn, Salas-Wright, DeLisi, & Maynard, 2013; Weissman, Antinoro, & Chu, 2009). Researchers have noted a complex relationship between biological and environmental variables in the development of externalizing behaviors (Dodge & Pettit, 2003; Jimenez-Barbero, Ruiz-Hernandez, Llor-Esteban, & Waschgler, 2016). Among these variables are family adversity, low-quality teacher-child relationships, and highly sensitive neurobiological stress responses (Fearon, Bakermans-Kranenburg, Van IJzendoorn, Lapsley, & Roisman, 2010; O'Connor, Dearing, & Collins, 2011; Obradovic, Bush, Stamperdahl, Adler, & Boyce, 2010). Efforts to prevent and ameliorate outcomes associated with externalizing behaviors often include well-planned, structured, and sustained school-based social and emotional learning interventions to develop positive attitudes, constructive behaviors, and effective cognitions (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). Among these are school-based cognitive-behavioral interventions (CBI; Barnes, Smith, & Miller, 2014).

School-based CBIs are effective in decreasing externalizing problem behaviors (e.g., Daunic et al., 2012; Smith, Taylor, Barnes, & Daunic, 2012), and in recent years, CBI research has moved from the determination of its effectiveness to the exploration of ways to ensure that CBIs will continue to have positive effects on an increasingly diverse school population (Wright, Ellis, Holloway, & Wong, 2014). To refine CBIs to meet the needs of the changing student population, it is important to examine whether intervention effectiveness is influenced by student characteristics common among those with problem behaviors. This information will guide refinement efforts to ensure that CBIs are effective for the students most in need. Thus, the purpose of this study was to use structural equation modeling (SEM) to examine whether student race, socio-economic status (SES), and/or levels of anger control prior to intervention moderated the effectiveness of Tools for Getting Along (TFGA), a school-based CBI. In addition, we explored whether levels of anger control, a skill associated with lower levels of externalizing behavior (Kim & Deater-Deckard, 2011), mediated the effectiveness of this intervention in decreasing externalizing behaviors and increasing social problem-solving skills.

Cognitive Behavioral Interventions

CBIs use approaches to social-emotional learning that combine behavioral modification strategies with strategies from cognitive psychology to treat various classes of disorders, including conduct disorders and aggression (Smith et al., 2012). Commonly included in CBIs are techniques grounded in social problem-solving theory, which is based on the following assumptions: (a) the ability to solve social problems is positively related to social competence and inversely related to maladaptive behaviors, and (b) training in social problem-solving skills will increase social competence and help reduce and prevent further development of maladaptive behavior in individuals with problem-solving deficits (D'Zurilla & Nezu, 2010).

Researchers have found that CBIs can be effectively delivered in schools by school staff (Barnes et al., 2014) and are sustainable in school settings because they require few specialized resources. In examining sustainability, it is also important to examine whether the intervention's effects are generalizable to a diverse sample of students. National trends show a decrease in the percentage of White students enrolled in United States public schools and suggest an increase in racial and ethnic minority student enrollment, with the Hispanic student population experiencing the largest enrollment growth (Ahmad & Boser, 2014; U.S. Department of Education, National Center for Education Sciences, 2016). Moreover, though researchers suggest that a student's emotion regulation, including anger control, is negatively correlated with externalizing behavior (Halligan et al., 2013), there is a lack of research that explains how inevitable differences in emotion regulation for a diverse student population will influence intervention effectiveness.

Student Characteristics


Although there is a well-established literature base showing that Black and Hispanic students often obtain more disciplinary referrals and are disproportionately identified as having emotional and/or behavioral disorders (EBD) than their White peers (Cavendish, Artiles, & Harry, 2015; Skiba et al., 2011), researchers have not examined racial/ ethnic differences in intervention effectiveness in ameliorating student externalizing behavior. The relationship between race and identification of students as having externalizing behaviors is complex, with many potential contributors to the overrepresentation of racially diverse students in the EBD disability category and in obtaining disciplinary referrals (McKenna, 2013; Skiba et al., 2011).

Among the factors that may play a part in the overrepresentation of racially diverse students in the EBD disability category and in high disciplinary referral rates are (a) referral subjectivity, (b) a lack of cultural competence in the school environment, and (c) a lack of research about whether race and ethnicity act as moderators of treatment outcomes (McKenna, 2013; Skiba et al., 2011). Referral subjectivity refers to both subjectivity in referring students for special education services (i.e., subjectivity in referring as well as subjectivity in the tests used to determine whether a student meets identification criteria for EBD) and subjectivity in determining which student behaviors should result in suspension, particularly when there is no clear district mandate about particular behaviors (Skiba et al, 2011). The negative effect of subjectivity on identification of racially diverse students as having externalizing behaviors is linked to a lack of cultural competence (Howard, 2015). Though the American student population has, and continues to, become more diverse, the teaching force remains primarily White and middle class (U.S. Department of Education, National Center for Education Sciences, 2016). Consequently, educators are often not from the same cultural backgrounds as their students and need to be educated in cultural competence to guard against misinterpretation of student behaviors. Though professional development and pre-service teacher education programs are now addressing this much-needed skill set, there is still much work to be done.

A third factor that may play a part in racial minority student identification as having externalizing behaviors is a lack of research that considers race and ethnicity as moderators of treatment outcomes (McKenna, 2013). In cases where students have or are objectively at-risk for developing externalizing behavior, ensuring that interventions are effective for that student population is imperative. With clear evidence that students who identify as Black or Hispanic are more likely to obtain disciplinary referrals and be identified as having EBD, it can be assumed that these students are likely targets for school-based social-emotional learning interventions such as those based on a cognitive-behavioral approach. Therefore, it is important to determine if interventions are effective for different racial groups.

Socio-Economic Status

A consistent finding in the epidemiological literature is the relationship between low SES and early-onset externalizing behaviors (Farrington, 1978; Shaw & Shelleby, 2014). For example, Dodge, Pettit, and Bates (1994) found that SES was negatively correlated with teacher-rated externalizing behaviors and peer-reported aggressive behavior. This relationship between poverty and externalizing behavior is likely influenced by the myriad of adversities faced by children living in poverty. Students from low SES backgrounds and racial and ethnic minority populations report more traumatic and stressful life events than White students. In addition, this population faces high rates of interpersonal trauma, referred to as complex trauma (i.e., physical, sexual and/or emotional abuse, neglect, and/or witnessing domestic violence; Purvis, Milton, Harlow, Paris, & Cross, 2014). Complex trauma has been linked to (a) increased referrals for special education and disciplinary action, (b) higher rates of school failure, grade retention, and school drop-out, (c) lower grades and standardized test scores, and (d) higher rates of mental health disorders (Greeson et al., 2011; Slade & Wissow, 2007). As likely participants in CBI programming in schools, it is important to determine if CBIs are effective with this population.

Anger Control

Anger is an emotion that is typically (though not always) associated with externalizing problem behaviors (Lochman, Barry, Powell, & Young, 2010). According to Lochman and colleagues (2010), anger is an appropriate and adaptive reaction to a threatening situation and can serve as a motivator to encourage a person to take action against a threat. Unfortunately, anger also causes intense physiological reactions related to the fight-or-flight response, and it may be difficult to manage. The inability to control anger has many negative implications for adjustment and can lead to negative outcomes in relationships, self-esteem, and health (Lochman et al., 2010). It is important to know whether intervention effectiveness differs based on pre-test levels of anger control as this can help in making refinements to a specific CBI intervention and/or help in determining whether it is necessary to use different types of CBI based on the level of anger control that a student possesses prior to intervention.


This study was a secondary analysis of data collected as part of a three-year efficacy trial of the Tools for Getting Along (TFGA) curriculum (see Daunic et al., 2012). TFGA is a CBI for fourth and fifth grade focused on improving anger control through instruction in social problem-solving. TFGA is delivered at the universal level (i.e., delivered to general education students in a whole-group format) but targets students who are at-risk for emotional and/or behavioral difficulties. Through direct instruction, teacher modeling, class discussion, small group activities, and self-talk, TFGA focuses on six social problem-solving steps that are aligned with Crick and Dodge's (1994) social information-processing model.

Study Purpose

The purpose of this study was to investigate if and how student characteristics (i.e., race, SES, level of anger control) moderate the effects of a school-based CBI, TFGA, on post-intervention measures of social problem solving, aggression, and externalizing behavior. In addition, we also examined whether anger control following intervention mediated the effects of TFGA on social problem solving, aggression, and externalizing behaviors.

The research questions examined in this study were:

1. How do students' level of anger control at pre-test influence the effectiveness of TFGA on social problem solving, aggression, and externalizing behaviors?

2. How do student race and/or SES influence the effectiveness of TFGA on social problem solving, aggression, and externalizing behaviors?

3. Does students' anger control level at post-test mediate the effectiveness of TFGA on social problem solving, aggression, and externalizing behavior?



Participants included 2,086 fourth- and fifth-grade students from 20 elementary schools in North Central Florida. Due to low numbers of students from Hispanic, Asian, and Native American/Pacific Islander populations in our sample, we limited our analyses to two racial groups Black and Other (made up primarily of White students with a small number of students who identified as Biracial, Asian, or Hispanic). Table 1 presents descriptive statistics of mean participant scores and standard deviations of measures of interest at pre- and post-test as well as frequencies for participant race and SES (free or reduced lunch status).


The Clinical Assessment of Behavior-Teacher Rating Form (CAB-T; Bracken & Keith, 2004) is a teacher-reported behavior measure that consists of 70 questions that constitute a range of clinical, adaptive, and educationally related scales. Likert scale item responses range from 1 (always or very frequently) to 5 (never). The CAB-T has demonstrated evidence of validity (i.e., test content, factor-analytic studies, and convergent, discriminant, and concurrent) across clinical populations such as those with conduct disorders and other disruptive behavior disorders (Bracken & Keith, 2004). Cronbach's alpha estimates for raw scores on the subscale of interest in this paper, Externalizing Behavior, at pre-test and post-test were .97 and .98, respectively (Daunic et al., 2012).

The Social Problem-Solving Inventory--Revised (SPSI-R; D'Zurilla, Nezu, & Maydeu-Olivares, 2002) is a 52-item Likert-type self-report. There are five subscales: Positive Problem Orientation (PPO), Negative Problem Orientation (NPO), Rational Problem-Solving Style (RPS), Impulsivity/Carelessness Style (ICS), and Avoidance Style (AS). Among diverse populations, researchers have found that the SPSI-R has strong internal consistency and stability over time, and there is also evidence of strong structural, concurrent, predictive, convergent, and discriminant validity (D'Zurilla et al., 2002). Cronbach's alphas obtained for years 1 and 2 of the current sample ranged from .60 (PPO) to .91 (RPS) at pre-test and from .64 (AS) to .93 (RPS) at post-test (Daunic et al., 2012).

The Reactive-Proactive Aggression Scale is a teacher-report measure developed by Dodge and colleagues (Dodge & Coie, 1987) and includes 19 items that can be answered on a five-point scale ranging from 1 ("never true") to 5 ("always true"). There are two subscales that constitute the Total Aggression scale (Reactive Aggression, Proactive Aggression). Daunic and colleagues (2012) found internal reliability scores of .90 for Reactive Aggression and .91 for Proactive Aggression.

The Anger Expression Scale for Children (AESC; Steele, Elliot, & Phipps, 2003) is a 21-item self-report measure with four subscales (i.e., Trait Anger, Anger Out, Anger-In, Anger Control). Previous analyses indicate that AESC subscales demonstrate good convergent validity with similar measures of hostility and anger (Steele, Legerski, Nelson, & Phipps, 2009). Reliability estimates from sample data using Cronbach's alpha were .81 at pre-test and .81 at post-test for Anger Control (Daunic et al., 2012).

Student race and SES. We designated race as a binary variable ("Black" compared to "Other," which included White, Hispanic, and other races) due to the predominance of Black- and White-American students and the low number of Hispanic-Americans and students of other races in our sample. We used students' free or reduced lunch (FRL) status to determine SES, and we assigned eligibility for FRL as a binary variable.


Our study presents a secondary analysis of data collected as part of an efficacy trial for the Tools for Getting Along curriculum (see Daunic et al., 2012, for description of efficacy trial). Daunic and colleagues (2012) trained teachers and guidance counselors in classrooms from schools assigned to treatment on CBI strategies and TFGA implementation for 10 hours over two days. Classrooms in the control group maintained "business as usual." Measures were administered in treatment and control classrooms within two weeks of training and again within two weeks of completing the curriculum (i.e., mid-April to early May).

Data Analytic Approach

We used structural equation modeling (SEM), a family of related procedures used for analyzing multivariate data, in this analysis. SEM has several advantages over regression models in that it can incorporate multiple independent and dependent variables in one model as well as hypothetical latent constructs that sets of observed variables might represent. The advantage of using latent variables in SEM is that measurement error is controlled in the model. SEM models also provide a way to test a complex set of hypothesized relationships among observed and latent variables.

When examining observed relationships between variables such as a person's beliefs and their behavioral outcomes, it becomes clear that these relationships may be part of a more complex chain of effects. When describing these relationships, the concepts of mediation and moderation play an important role. According to Little, Card, Bovaird, Preacher, and Crandall (2007), a mediator can be thought of as the carrier or transporter of information along a causal chain of effects, while a moderator is the change of a relationship in a system.

Research Questions 1 and 2 look at moderation relationships, while Research Question 3 looks at mediation relationships.

We used Mplus Version 6.0 (Muthen & Muthen, 1998-2010) to conduct SEM of the data to test the moderation and mediation relationships set forth in our research questions. In this analysis, we tested for a mediation effect by examining whether the confidence intervals of the indirect effect overlapped zero (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002). Due to limitations in Mplus, indirect effect standard errors were calculated with the Monte Carlo confidence interval method (Mackinnon, Lockwood, & Williams, 2004) using the Monte Carlo Method for Assessing Mediation (MCMAM) in R. We tested for moderation effects in this model by using the latent moderated structural (LMS) approach (Klein & Moosbrugger, 2000).

Due to the nested nature of this data (i.e., students were assigned to classrooms, which were assigned to a particular school), we determined intervention effects by using design-based adjustment (Sterba, 2009; Wu & Kwok, 2012) for the effects of clustering, which is one of the recommended techniques when the experimental design violates the independence of observations assumption required by traditional ANOVA designs (Heck, 2001; Muthen & Muthen, 1998-2010).


Descriptive Statistics

First, we conducted a preliminary analysis using independent samples f-tests to investigate the equivalence of treatment and control groups on pre-treatment outcome measures. Students in the treatment group had higher internalizing behavior (f (1958) = -3.26, p=0.00), anger control (f (1899) = -2.06 p = 0.04), NPO (f (1970) = -3.35, p = 0.00), RPS (t (1941) = -2.95, p = 0.00), and AS (t (1844) = 3.29, p = 0.00) scores at pre-intervention than those in the control group. Based on these results, we decided to control for pre-test outcome scores of interest in our structural model (see Figure 1).

Confirmatory Factor Analysis for Social Problem-Solving Latent Variable

Prior to analyzing the structural model, we conducted a confirmatory factor analysis of the latent social problem-solving indicators to determine if the commonly used one-factor model with five subscales (D'Zurilla et al., 2002) fit the data presented in the current study. The decision to test this model was based on the results of previous pilot analyses that suggested that a one-factor model was not a good fit for our data. We examined the fit of the model with several fit indices including three commonly used model fit indices: the Steiger-Lind root mean square error of approximation (RMSEA; Steiger, 1990), Bentley comparative fit index (CFI; Bentler, 1990), and the Standardized Root Mean Square Residual (SRMR; Hu & Bentler, 1999). A RMSEA value of 0.06 or less is considered good fit, and values of 0.08 to 0.05 are considered reasonable. CFI values of 0.95 or greater and a SRMR value of .08 or less indicate good fit (Hu & Bentler, 1999). Our analyses suggested that a one-factor model was not a good fit for our data (SRMR = 0.229; [chi square] = 1073.54, p = 0.00; RMSEA = 0.41 (CI = 0.39, 0.43), p = 0.01; CFI = 0.397; TLI = -0.01). We then conducted an exploratory factor analysis with maximum likelihood estimation in Mplus to determine whether a one-factor model with modifications ([chi square] = 1366.55; BIC = 24183.91) or a two-factor model would work best for the latent social problem-solving variable ([chi square] = 1.60; BIC = 22849.04). A decrease in the Bayesian Information Criterion (BIC) and Chi-Square ([chi square]) from a one-factor to a two-factor model suggested that the two-factor model was a better fit. A closer look at the Geomin rotated loadings provided as part of the exploratory factor analysis suggested that PPO and RPS load on one factor, and NPO, ICS, and AS load on a different factor. We interpreted these two factors to be positive social problem solving (PPO, RPS) and negative social problem solving (NPO, ICS, AS). Using available modification indices and theory, we correlated the positive and negative social problem solving latent variables and RPS with ICS (Figure 2).



Structural Model

Figure 1 illustrates the values of the relationships examined in our structural model. The model included both mediation (i.e., direct and indirect) and moderation relationships. Mplus was unable to complete analyses with mediation, design-based adjustment of clustering effects, and multiple imputations of missing data simultaneously; thus to handle missing data in this model full information maximum likelihood was used (Enders, 2013).

The specification of relationships in our model resulted in adequate model fit on all model fit indices (AIC = 80790.1; BIC = 81424.4; SRMR = 0.024; [chi square] (171) = 8415.2, p = 0.00; RMSEA = 0.067 (CI = 0.053, 0.063), p = 0.01; CFI = 0.954; TLI = .888). A correlation table that illustrates correlations between variables of interest is available upon request from the first author. The significant results of this analysis are illustrated in Figure 3, with parameter estimates for all significant relationships (including direct, and if tested, indirect and total effects) provided in Table 2.

Question 1. Results revealed that none of the indirect effects of TFGA on social problem solving, aggression, and externalizing behaviors through anger control at post-test (mediation hypothesis) were significant. In addition, a look at the direct effects of TFGA on anger control showed that TFGA did not have a statistically significant direct effect on post anger control.


Question 2. Results showed that anger control at pre-test did not moderate the effect of TFGA on positive social problem solving, negative social problem solving, reactive aggression, or proactive aggression. Conversely, there was a significant pre-test anger control by TFGA interaction effect on externalizing behavior ([beta] = -0.249 SE = -2.417, p < .05).

Question 3. The race by TFGA and FRL by TFGA interactions had no significant effect on externalizing behavior, social problem solving, or aggression measures.


TFGA is a CBI designed to help students think about the possible consequences of their choices when faced with social problems, particularly those involving anger-provoking situations (Smith et al., 2012). To this end, we hypothesized that the intervention would increase a student's ability to deal effectively with anger and improve social problem-solving skills and abilities. We, therefore, wanted to explore whether an increased ability to control anger after TFGA implementation would serve to mediate the effects of the intervention on social problem-solving and behavioral outcomes. The results of our analysis suggest that TFGA did not have a significant direct effect on anger control, and as a result, anger control did not mediate the effect of TFGA on outcome measures. Significant direct effects of TFGA on other anger subscales have been found with this sample (Daunic et al., 2012), and it is possible that the intervention's effect is mediated through other subscales represented in the AESC (e.g., trait anger, anger-in, or anger-out). Future research is needed to explore this possibility.

We found that pre-test anger control moderated TFGA effects on externalizing behavior at post-test. Specifically, the findings suggest that students in the treatment group with higher anger control at pretest had less externalizing behavior at post-test than students in the control group with higher anger control at pre-test. One possible reason for this finding may be that students with lower anger control may need a more intensive intervention than TFGA or other universally delivered interventions can provide. Particularly, for students in the sample who had low levels of anger control prior to intervention, the amount of treatment exposure may have played a role in the interaction effect. Typically, the TFGA curriculum was provided 1-2 days a week over a six-month period for 30 minutes at a time in a general education setting. Mellard, McKnight, and Jordan (2010) suggested that school professionals evaluate and adjust variables such as treatment dosage to increase intervention effectiveness. Increasing dosage specific to anger control by providing secondary or tertiary intervention, therefore, may make a significant difference in externalizing behavior outcomes.

The class-wide delivery of CBI also raises questions about making CBI strategies relevant to all participants, particularly those from different racial and cultural backgrounds. Results indicated that neither race nor socio-economic status moderated the effect of TFGA on externalizing behavior, social problem solving, or aggression measures. In light of an increasingly diverse student population, this finding is promising as it suggests that CBIs are equally effective across race and socio-economic status. The concepts of diversity and culture are complex, however, and expand beyond race and socio-economic status. Other concepts related to culture and diversity include immigration status, geographical location, gender, religious affiliation, and language (Howard, 2015). Further research is needed to explore the relationship between intervention effectiveness and these factors more fully.


One of the major limitations of a secondary analysis is that the data may not adequately reflect the new researchers' questions of interest (Bradshaw, Mitchell, O'Brennan, & Leaf, 2010). In this study, we examined the moderating effects of student demographics (i.e., race and FRL) on intervention effectiveness. Variation in rates of externalizing behavior and reactive aggression for populations that differ demographically may be related to many social factors other than race and FRL, such as poverty, family adversity, low-quality teacher-child relationships, and highly sensitive neurobiological stress responses (Fearon et al., 2010; Obradovic et al., 2010; O'Connor et al., 2011). Although we were able to include indicators of poverty (FRL) and race in our model, the original study did not collect data on other social factors. Without taking these other variables into account, an analysis of the moderating effect of demographics on intervention effectiveness is limited. Similarly, the data were based on a sample that included low numbers of students from Hispanic, Asian, and Native American/Pacific Islander populations. As a result, we analyzed race as a dichotomous variable (Black or Other), and that restricts the generalizability of our results to primarily Black- and White-American students.

Implications for Future Research

There is a need for future researchers to explore the effects of student characteristics on TFGA's (and other CBIs') effectiveness using other potential mediator and moderator variables. One variable of interest in light of our findings related to anger control is students' level of impulsivity, which has also been found to correlate positively with externalizing behavior (Halligan et al., 2013). Like anger control, impulsivity can influence a person's ability to move successfully through the social problem-solving steps that are found in TFGA. For example, impulsive people may not take the time to attend to social cues or may act on the first solution alternative they think of without debating the potential consequences. Future research on the mediating and moderating effects of impulsivity would provide important information on whether TFGA effectiveness and that of CBIs in general differ for students who are more or less impulsive.

In addition to examining the effects of student characteristics, there is a need for research that explores the mediating and moderating effects of school contextual variables on TFGA effectiveness. Researchers have found that the quality of teacher-child relationships is related to levels of externalizing student behavior, and some have targeted this relationship in an attempt to ameliorate negative student behavior (O'Connor et al., 2011). We encourage researchers to expand on this work and examine teacher variables such as teacher stress or level of professional training that may have an influence not just on the student-teacher relationship but also on intervention effectiveness, particularly when the teacher is the implementation facilitator (Lochman, 2003; Ringeisen, Henderson, & Hoagwood, 2003). In our sample, we found that teachers accounted for up to 39% of the variance in student outcomes (Daunic et al., 2012). In addition to teacher characteristics, fidelity to treatment may also influence intervention effectiveness. When an intervention does not appear to be effective, treatment fidelity data can be examined to determine if this was the case or if the intervention was not implemented as intended. In the case of TFGA, the mean observer-rated treatment fidelity indicated that most observed teachers followed the curriculum as intended (Daunic et al., 2012). Though beyond the scope of this research, it would be interesting to conduct a more complete analysis of the relation of treatment fidelity in the TFGA study to the intervention's effectiveness.

Implications for Practice

The results of this study showed that TFGA does not have differential effects for students based on race or socio-economic status. Thus, it would appear, based on this initial study, that the intervention can be equally effective for a variety of students (i.e., Black or White, those qualifying or not qualifying for FRL), yet for those implementing the curriculum, it would perhaps enhance instruction to make the instruction relevant to all participants as this increases student engagement and learning of concepts (Lawson & Lawson, 2013). One method of increasing student engagement is the use of culturally responsive pedagogy (Gay, 2010; Griner, 2012; Osher, Bear, Sprague, & Doyle, 2010). Though it is typically used in academic contexts, we urge readers to explore the use of culturally responsive pedagogy in implementing CBI.

For students in the non-clinical school population that have relatively lower levels of anger control, as determined by pre-intervention scores, there may be a need to use caution when providing these students with a universally delivered intervention only to address their behavioral needs. Universally delivered CBIs have many advantages, in that they may help prevent costly referrals to special education for students who have not yet been identified as needing further support, and they offer at-risk students an opportunity to learn social problem solving alongside typically developing peers (Daunic et al., 2012). For students who are struggling with lower levels of anger control, however, universal implementation may not adequately address their needs, and school personnel should explore the use of small group and, perhaps, individual instruction that can address explicitly specific cognitive-processing deficits.

In sum, as CBI researchers continue to refine interventions to better meet the needs of all students, it is important to examine the effects of a variety of student characteristics on intervention effectiveness. This task may prove difficult because of complex relationships among the multiple factors involved, but this study provides an initial look at the role some student characteristics play in CBI efficacy. As the field of intervention research advances, it is imperative that investigators continue to examine the relationships between student characteristics and intervention effectiveness to ensure that the social and emotional benefits of these interventions are fully realized for all students with or at-risk for problem behaviors.


Ahmad, F. Z., & Boser, U. (2014). America's leaky pipeline for teachers of color: Getting more teachers of color into the classroom. Retrieved from /05/04/88960/americas-leaky-pipeline-for-teachers-of-color/

Barnes, T. N., Smith, S. W., & Miller, M. D. (2014). School-based cognitive-behavioral interventions in the treatment of aggression in the United States: A meta-analysis. Aggression and Violent Behavior, 19(4), 311-321. doi: 10.1016/j.avb.2014.04.013

Bentler, P. M. (1990). Fit indexes, Lagrange multipliers, constraint changes and incomplete data in structural models. Multivariate Behavioral Research, 25(2), 163-172. doi: 10.1207/s15327906 mbr2502_3

Bracken, B. A., Keith, L. K. (2004). Examiner's manual: Clinical Assessment of Behavior. Odessa, FL: Psychological Assessment Resources.

Bradshaw, C. R, Mitchell, M. M., O'Brennan, L. M., & Leaf, P. J. (2010). Multilevel exploration of factors contributing to the overrepresentation of black students in office disciplinary referrals. Journal of Educational Psychology, 102(2), 508-520. doi: 10.1037/a0018450

Cavendish, W., Artiles, A., & Flarry, B. (2015). Tracking inequality 60 years after Brown: Does policy legitimize racialization of disability? Multiple Voices for Ethnically Diverse Exceptional Learners, 14(2), 1-11.

Crick, N. R., & Dodge, K. A. (1994). A review and reformulation of social information-processing mechanisms in children's social adjustment. Psychological Bulletin, 115(1), 74-101. doi: 10.1037/ 0033-2909.115.1.74

Daunic, A. R, Smith, S. W., Garvan, C. W., Barber, B. R., Becker, M. K., Peters, C. D., Taylor, G. G., Van Loan, C. L., Li, W., & Naranjo, A. H. (2012). Reducing developmental risk for emotional/behavioral problems: A randomized controlled trial examining the Tools for Getting Along curriculum. Journal of School Psychology, 50(2), 149-166. doi: 10.1016/j.jsp.2011.09.003

Dodge, K. A., & Coie, J. D. (1987). Social-information-processing factors in reactive and proactive aggression in children's peer groups. Journal of Personality and Social Psychology, 53,1146-1158. doi: 10.1037/0022-3514.53.6.1146

Dodge, K. A., & Pettit, G. S. (2003). A biopsychosocial model of the development of chronic conduct problems in adolescence. Developmental Psychology, 39, 349.

Dodge, K. A., Pettit, G. S., & Bates, J. E. (1994). Socialization mediators of the relation between socioeconomic status and child conduct problems. Child Development, 65, 649-665. doi: 10.2307/ 1131407

Durlak, J. A., Weissberg, R. R, Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students' social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. doi: 10.1111/j.1467-8624.2010.01564.x

D'Zurilla, T. J., & Nezu, A. M. (2010). Problem-solving therapy. In K.S. Dobson (Ed.), Social Problem Solving: Theory, Research, and Training Handbook of Cognitive-Behavioral Therapies (3rd ed.) (pp. 197-225). New York, NY: The Guilford Press.

D'Zurilla, T. J., Nezu, A. M., & Maydeu-Olivares, A. (2002). Social problem-solving inventory--revised (SPSI-R): Technical manual. North Tonawanda, NY: Multi-Health Systems.

Enders, C. K. (2013). Analyzing structural equation models with missing data. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (pp. 493-519). Greenwich, CT: Information Age Publishing.

Farrington, D. P. (1978). The family backgrounds of aggressive youths. The Book Supplement to the Journal of Child Psychology and Psychiatry, 1,73-93.

Fearon, R. P, Bakermans-Kranenburg, M. J., Van IJzendoorn, M. H., Lapsley, A. M., & Roisman, G. I. (2010). The significance of insecure attachment and disorganization in the development of children's externalizing behavior: A meta-analytic study. Child Development, 81, 435-456. doi: 10.1111/j.1467-8624.2009.01405.x

Gay, G. (2010). Culturally responsive teaching: Theory, research, and practice. New York, NY: Teachers College Press.

Greeson, J. K., Briggs, E. C., Kisiel, C. L., Layne, C. M., Ake III, G. S., Ko, S. J., ... & Fairbank, J. A. (2011). Complex trauma and mental health in children and adolescents placed in foster care: Findings from the national child traumatic stress network. Child Welfare, 90, 91-108.

Griner, A. C. (2012). Addressing the achievement gap and disproportionality through the use of culturally responsive teaching practices. Urban Education, 48, 585-621. doi: 10.1177/004208 5912456847

Halligan, S. L., Cooper, P. J., Fearon, P, Wheeler, S. L., Crosby, M., & Murray, L. (2013). The longitudinal development of emotion regulation capacities in children at risk for externalizing disorders. Development and Psychopathology, 25, 391-406. doi: 10.1017/ S0954579412001137

Howard, T. C. (2015). Why race and culture matter in schools: Closing the achievement gap in America's classrooms. New York, NY: Teachers College Press.

Heck, R. H. (2001). Multilevel modeling with SEM. In G. A. Marcoulides & Randall E. Schumacker (Eds.), Nezv developments and techniques in structural equation modeling (pp. 89-128). Mahwah, NJ: Lawrence Erlbaum.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55. doi: 10.1080/10705519909540118

Jimenez-Barbero, J. A., Ruiz-Hernandez, J. A., Llor-Esteban, B., & Waschgler, K. (2016). Influence of attitudes, impulsivity, and parental styles in adolescents' externalizing behavior. Journal of Health Psychology, 21,122-131. doi: 10.1177/1359105314523303

Kim, J., & Deater-Deckard, K. (2011). Dynamic changes in anger, externalizing and internalizing problems: Attention and regulation. Journal of Child Psychology and Psychiatry, 52,156-166. doi: 10.1111/j.1469-7610.2010.02301.x

Klein, A., & Moosbrugger, H. (2000). Maximum likelihood estimation of latent interaction effects with the LMS method. Psychometrika, 65, 457-474. doi: 10.1007/BF02296338

Lawson, M. A., & Lawson, H. A. (2013). New conceptual frameworks for student engagement research, policy, and practice. Review of Educational Research, 83, 432-479. doi: 10.3102/003465 4313480891

Little, T. D., Card, N. A., Bovaird, J. A., Preacher, K. J., & Crandall, C. S. (2007). Structural equation modeling of mediation and moderation with contextual factors. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling contextual effects in longitudinal studies (pp. 207-230). New York, NY: Psychology Press.

Lochman, J. E. (2003). Commentary: School contextual influences on the dissemination of interventions. School Psychology Review, 32, 174-177.

Lochman, J. E., Barry, T., Powell, N., & Young, L. (2010). Anger and aggression. In D. W. Nangle, D. J. Hansen, C. A. Erdley, & P. J. Norton, Practitioner's guide to empirically based measures of social skills (pp. 155-166). New York, NY: Springer.

MacKinnon, D. R, Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99-128. doi: 10.1207/s15327906mbr3901_4

MacKinnon, D. R, Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83-118. doi: 10.1037/1082-989X.7.1.83

McKenna, J. (2013). The disproportionate representation of African Americans in programs for students with emotional and behavioral disorders. Preventing School Failure: Alternative Education for Children and Youth, 57, 206-211. doi: 10.1080/1045988X.2012.687792

Mellard, D., McKnight, M., & Jordan, J. (2010). RTI tier structures and instructional intensity. Learning Disabilities Research and Practice, 25(4), 217-225. doi: 10.1111/j.1540-5826.2010.00319.x

Muthen, L. K., & Muthen, B. O. (1998-2010). Mplus user's guide (6th ed.). Los Angeles, CA: Muthen & Muthen.

Obradovic, J., Bush, N. R., Stamperdahl, J., Adler, N. E., & Boyce, W. T. (2010). Biological sensitivity to context: The interactive effects of stress reactivity and family adversity on socioemotional behavior and school readiness. Child Development, 81, 270-289. doi: 10.1111/j.1467-8624.2009.01394.x

O'Connor, E. E., Dearing, E., & Collins, B. A. (2011). Teacher-child relationship and behavior problem trajectories in elementary school. American Educational Research Journal, 48,120-162. doi: 10.3102/0002831210365008

Osher, D., Bear, G. G., Sprague, J. R., & Doyle, W. (2010). How can we improve school discipline? Educational Researcher, 39, 48-58. doi: 10.3102/0013189X09357618

Purvis, K. B., Milton, H. S., Harlow, J. G., Paris, S. R., & Cross, D. R. (2014). The importance of addressing complex trauma in schools: Implementing trust-based relational intervention in an elementary school. ENGAGE: An International Journal on Research and Practices in School Engagement, 1(2). Retrieved June 13,2016, from /the-importance-of-addressing-complex-trauma-in-schools -implementing-trust-based-relational-intervention-in-an -elementary-school-2/

Ringeisen, H., Henderson, K., & Hoagwood, K. (2003). Context matters: Schools and the "research to practice gap" in children's mental health. School Psychology Review, 32,153-168.

Rothbaum, E, & Weisz, J. R. (1994). Parental caregiving and child externalizing behavior in nonclinical samples: A meta-analysis. Psychological Bulletin, 116, 55-74. doi: 10.1037/0033-2909.116.1.55

Shaw, D. S., & Shelleby, E. C. (2014). Early-onset conduct problems: Intersection of conduct problems and poverty. Annual Review of Clinical Psychology, 10, 503-528. doi: 10.1146/annurev-clinpsy-032813-153650

Shonk, S. M., & Cicchetti, D. (2001). Maltreatment, competency deficits, and risk for academic and behavioral maladjustment. Developmental Psychology, 37, 3-17. doi: 10.1037/0012-1649.37.1.3

Skiba, R. J., Horner, R. H, Chung, C., Rausch, M. K., May, S. L., & Tobin, T. (2011). Race is not neutral: A national investigation of African-American and Latino disproportionality in school discipline. School Psychology Review, 40, 85-107.

Slade, E. P., & Wissow, L. S. (2007). The influence of childhood maltreatment on adolescents' academic performance. Economics of Education Review, 26,604-614. doi: 10.1016/j.econedurev.2006.10.003

Smith, S. W., Taylor, G. G., Barnes, T., & Daunic, A. P. (2012). Cognitive-behavioral interventions to prevent aggression of students with emotional and behavioral disorders. In B. G. Cook, M. Tankersley, & T. J. Landrum (Eds.), Classroom Behavior, Contexts, and Interventions (Vol. 25). Bingley, UK: Emerald Group.

Steele, R. G., Elliot, V., & Phipps, S. (2003). Race and health status as determinants of anger expression and adaptive style in children. Journal of Social and Clinical Psychology, 22,40-58. doi: 10.1521/jscp.

Steele, R. G., Legerski, J. P, Nelson, T. D., & Phipps, S. (2009). The anger expression scale for children: Initial validation among healthy children and children with cancer. Journal of Pediatric Psychology, 34, 51-62. doi: 10.1093/jpepsy/jsn054

Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173-180. doi: 10.1207/s15327906mbr2502_4

Sterba, S. K. (2009). Alternative model-based and design-based frameworks for inference from samples to populations: From polarization to integration. Multivariate Behavioral Research, 44, 711-740. doi: 10.1080/00273170903333574

U.S. Department of Education, National Center for Education Statistics. (2016). Racial/ethnic enrollment in public schools. Retrieved from

Vaughn, M. G., Salas-Wright, C. E, DeLisi, M., & Maynard, B. R. (2013). Violence and externalizing behavior among youth in the United States: Is there a severe 5%? Youth Violence and Juvenile Justice, 1, 3-21. doi: 10.1177/1541204013478973

Weissman, A. S., Antinoro, D., & Chu, B. C. (2009). Cognitive-behavioral therapy for anxious youth in school settings: Advances and challenges. In M. J. Mayer, R. Van Acker, J. E. Lochman, & F. M. Gresham (Eds.), Cognitive-behavioral interventions for emotional and behavioral disorders: School-based practice (pp. 173-203). New York, NY: The Guilford Press.

Wright, R., Ellis, M., Holloway, S. R., & Wong, S. (2014). Patterns of racial diversity and segregation in the United States: 1990-2010. The Professional Geographer, 66,173-182. doi: 10.1080/00330124.2012.735924

Wu, J., & Kwok, O. (2012). Using SEM to analyze complex survey data: A comparison between design-based single-level and model-based multilevel approaches. Structural Equation Modeling: A Multidisciplinary Journal, 19, 16-35. doi: 10.1080/10705511.2012.634703

Tia Navelene Barnes

Department of Human Development and Family Studies, College of Education and Human Development, University of Delaware

Stephen W. Smith

School of Special Education, School Psychology, and Early Childhood Studies, College of Education, University of Florida

Ann P. Daunic

School of Special Education, School Psychology, and Early Childhood Studies, College of Education, University of Florida

Walter L. Leite

School of Human Development and Organizational Studies in Educational Psychology, College of Education, University of Florida

Address correspondence to: Tia Navelene Barnes, Ph.D., University of Delaware, Department of Human Development and Family Studies, 111 Alison Hall West Newark, DE 19716. Email:

This study was supported by a four-year grant from the US Department of Education, Institute of Education Sciences (# R324B060029) submitted to CFDA 84.324A, National Center for Special Education Research.
Table 1
Descriptive Statistics


Variable Name      Descriptive         Mean (SD)    Frequency

Internal (Pre)     Internalizing      64.0 (11.1)      --

Internal (Post)    Internalizing      63.9 (10.8)      --

External (Pre)     Externalizing      73.9 (16.5)      --

External (Post)    Externalizing      72.8 (16.4)      --

Anger (Pre)        Anger Control      16.0 (4.6)       --

Anger (Post)       Anger Control      15.8 (4.4)       --

PPO (Post)         Positive Problem   16.9 (4.2)       --

NPO (Post)         Negative Problem   24.5 (7.4)       --

RPS (Post)         Rational/          60.5 (16.3)      --
                   Adaptive Scale

ICS (Post)         Impulsive/         24.1 (7.8)       --
                   Careless Scale

AS (Post)          Avoidance Scale    16.6 (5.1)       --

Reactive (Post)    Reactive            6.7 (3.1)       --

Proactive (Post)   Proactive           4.9 (2.7)       --

Race               Black                  --           378

FRL                Free or Reduced        --           861


Variable Name       Mean (SD)    Frequency

Internal (Pre)     65.0 (11.4)      --

Internal (Post)    64.2 (11.5)      --

External (Pre)     74.8 (15.7)      --

External (Post)    73.1 (17.1)      --

Anger (Pre)        15.9 (4.8)       --

Anger (Post)       15.8 (4.5)       --

PPO (Post)         16.5 (4.6)       --

NPO (Post)         23.7 (7.5)       --

RPS (Post)         56.8 (16.1)      --

ICS (Post)         24.0 (7.42)      --

AS (Post)          15.8 (4.9)       --

Reactive (Post)     6.9 (3.4)       --

Proactive (Post)    5.1 (2.8)       --

Race                   --           208

FRL                    --           775

Table 2
Significant Structural Model Parameter Estimates

Predictor      DV               Total Effect     Direct Effect

Tools for      Post            5.159 * (2.26)    5.071 * (2.27)
Getting        Externalizing
Along (TFGA)   Behavior

Post Anger     Negative              --         -1.125 * (-4.32)
Control        Problem

               Post Reactive         --         -.052 * (-2.95)

Pre Anger      Post                  --         -.249 * (-2.42)
Control &      Externalizing
TFGA           Behavior

Race           Post Reactive         --          -.453 * (2.83)

               Post                  --         -2.110 * (-2.55)

Predictor      DV                Indirect Effect

Tools for      Post            .088 (-.38, ,56) (+)
Getting        Externalizing
Along (TFGA)   Behavior

Post Anger     Negative                 --
Control        Problem

               Post Reactive            --

Pre Anger      Post                     --
Control &      Externalizing
TFGA           Behavior

Race           Post Reactive            --

               Post                     --

Note. Positive and Negative Problem Solving at post-test.
* = statistically significant at p < .05; (+) confidence
interval standard error.
COPYRIGHT 2016 West Virginia University Press, University of West Virginia
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2016 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Barnes, Tia Navelene; Smith, Stephen W.; Daunic, Ann P.; Leite, Walter L.
Publication:Education & Treatment of Children
Article Type:Report
Geographic Code:1USA
Date:Nov 1, 2016
Previous Article:Using self-regulated strategy development to help high school students with EBD summarize informational text in social studies.
Next Article:Effects of a summer learning program for students at risk for emotional and behavioral disorders.

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters