Printer Friendly

Prereferral intervention: a prescriptive approach.

Prereferral Intervention: A Prescriptive Approach

ABSTRACT: This investigation assessed effects of three increasingly inclusive versions of the

Behavioral Consultation (BC) model on problem behavior of students in mainstream classrooms

in an effort to develop an effective and efficient approach to prereferral intervention. Subjects

were 43 general educators, their 43 most difficult-to-teach students (without disabilities), and

12 school consultants, representing seven inner-city middle schools. Teachers and students were

assigned to three BC groups, representing least (BC 1), more (BC 2), and most (BC 3) inclusive

variants of BC, and one control group. Preintervention, postintervention, and follow-up

observations of student behavior indicated that more inclusive BC versions exerted stronger

effects than the least inclusive variant in reducing problem behavior. Theoretical and practical

implications for consultation-related activity are discussed. * Since the U.S. Department of Education's first child count in 1976-1977, the number of students served under the Education for All Handicapped Children Act (Public Law 94-142) and Chapter 1 has grown each year, with an increase of 712,688 children, or 16%, from 1976-1977 to 1986-1987 (see U.S. Department of Education, 1988, Table 1). This increase likely reflects attempts to provide children with disabilities an appropriate education. There is increasing suspicion, however, that too many children are being identified. Incorrect identification is undesirable for many reasons, including the unnecessary separation and stigmatization of children (e.g., Reynolds & Balow, 1972), disruption of school programs (e.g., Will, 1986), and excessive costs (e.g., Singer, 1988).

TEACHER REFERRALS

Contributing to the apparent overidentification of children as having disabilities is a large number of teacher referrals. New referrals to special education for the 1984-1985 school year, reported by 28 large urban districts (Research for Better Schools, 1986), ranged from a low of 600 (Memphis, Tennessee) to a high of 33,855 (New York City). The median number of new referrals for the districts was 2,358. On average, 54.5% of these students were certified "handicapped," thereby indicating the importance of teacher referral to eventual special education placement. Burgeoning numbers of teacher referrals and special education placements represent an important rationale for prereferral intervention.

PREREFERRAL INTERVENTION

Definition

Prereferral intervention is a teacher's modification of instruction or classroom management before referral to better accommodate a difficult-to-teach (DTT) pupil without disabilities. This activity is often "brokered" by one or more support staff, such as a special educator or school psychologist, who works indirectly with a targeted DTT student through consultation with the teacher. Implicit in this definition is a preventive intent; that is, (a) eliminating inappropriate referrals while increasing the legitimacy of those that are initiated and (b) reducing future student problems by strengthening the teacher's capacity to intervene effectively with a greater diversity of children. This preventive thrust jibes with the least-restrictive doctrine described in P.L. 94-142, requiring educators to attempt to accommodate DTT students' instructional and social needs in the most "normal" setting possible.

Popularity and Evidence of Success

The popularity of prereferral intervention is suggested by a recent survey of 49 state directors of special education (Carter & Sugai, 1989). Thirty-four claimed they either required (n = 23) or recommended (n = 11) their respective local education agencies to establish such interventions. Nevertheless, there have been few data based investigations of the effectiveness of prereferral interventions. Corroborating this view is a recent computer search of the ERIC data base (from January 1983 to September 1988). Based on numerous combinations of clusters of ERIC "key words," the search produced only three empirical investigations into the effectiveness of prereferral intervention. We know of only eight additional pertinent and published studies. At best, this infrequent research is only suggestive of the importance of prereferral intervention. Given this, respondents in the Carter and Sugai survey expressed appropriate, yet surprising, tentativeness when asked to judge its efficacy. Three-quarters of the state directors claimed that prereferral intervention was successful only "sometimes" or admitted "no basis for determining" its success--hardly overwhelming confidence in so popular a strategy.

To help find ways to strengthen general educators' capacity to accommodate DTT pupils' instructional and social needs, Special Education Programs of the U.S. Department of Education sponsored an Enhancing Instructional Program Options research initiative in 1985. Mainstream Assistance Teams (MATs), a 3-year project to develop, implement, and validate prereferral intervention in an urban school system, were funded as part of this federal initiative.

MAINSTREAM ASSISTANCE TEAMS

We embedded prereferral intervention within a larger process of teacher consultation, as have others before us (e.g., Cantrell & Cantrell, 1976; Chalfant, Pysh, & Moultrie, 1979; Graden, Casey, & Bonstrom, 1985). The model of teacher consultation employed is known as behavioral consultation (BC). It was selected because a steadily growing corpus of school-based research indicates its effectiveness in increasing pupils' academic performance and decreasing their general disruptiveness (e.g., Sibley, 1986; Tombari & Davis, 1979). Although much has been written about BC (e.g., Bergan, 1977; Kratochwill & Van Someren, 1985), proper understanding of the MAT project requires a brief description of this consultation approach.

Behavioral Consultation

Definition. BC involves a triadic network of consultant, teacher, and pupil along with indirect service; that is, consultants attempt to change students' school performance or behavior by working with teachers. Unlike alternative consultant models, BC encourages the exploration of antecedents and consequences of behavior in naturalistic settings to permit identification of variables influencing frequency, intensity, and duration of problem behavior. Behavioral consultants view the teacher, and often the student, as a problem solver who participates as a coequal in designing intervention strategies, which typically are based on empirically validated laws of behavioral change. Evaluation of planned interventions is data based; effectiveness is judged in terms of whether student or teacher behavior has met previously set goals.

Stages of BC. BC is conducted within a series of four interrelated stages: Problem identification, problem analysis, plan implementation, and problem evaluation. Consultants guide teachers through a majority of these stages in a succession of structured interviews, in which specific objectives must be accomplished before consultation may proceed to subsequent stages.

Major objectives of the first stage, problem identification, are to define the problem behavior in observable terms and obtain a reliable estimate of its frequency, intensity, or duration. In the problem analysis stage, the goal is to validate the existence of a problem, identify instructional and student variables that may contribute to a solution, and collaboratively develop an appropriate plan. During plan implementation, the consultant monitors implementation and provides corrective feedback, helping ensure that the intervention is delivered as designed. The goal of the final stage, problem evaluation, is for the consultant and the teacher to evaluate the effectiveness of the intervention, and, if it has proved ineffective, to determine modifications.

Component analysis of BC. A widespread assumption in the BC literature is that all four stages of the model are critical; none is dispensable (cf. Gresham, 1982). Nevertheless, a review of the pertinent literature indicates no previous component analysis of BC. That is, systematic attempts to determine the relative value of the various stages of BC are lacking.

The apparent absence of such effort seems to reflect a more general dearth of process-outcome research in the consultation literature (e.g., Alpert & Yammer, 1983; Medway, 1982; Meyers, Pitt, Gaughan, & Freidman, 1978; West & Idol, 1987; Witt & Elliott, 1983). This is unfortunate; process-outcome research, like component analysis, might help identify one or more stages of consultation that may be implemented in abbreviated fashion, or perhaps eliminated altogether, without loss of benefit to teachers and students. In other words, component analysis may contribute to the development of a simultaneously effective and efficient consultation process, one that effects meaningful change but requires a minimum of time, effort, and resources. The importance of economy to consultation is signaled by the frequently mentioned fact that insufficient time prevents many building support staff from engaging in consultation (e.g., Alpert & Trachtman, 1980). Many researchers have demonstrated a strong relation between degree of efficiency and teacher acceptability of interventions (e.g., Elliott, Witt, Galvin, & Peterson, 1984; Reimers, Wacker, & Koeppl, 1987; Witt, Martens, & Elliott, 1984). A consultation process that is economical as well as effective can only increase the frequency with which special educators, school psychologists, and other support personnel undertake consultation.

MAT Research in Year 1 (1985-1986)

Year 1 activity was conducted in seven middle schools in a large urban school district in the Southeast. In search of economy, we conducted an analysis of major components of the BC process, hoping to identify dispensable aspects of school consultation. Specifically, we evaluated three increasingly inclusive versions of BC and a control group. Findings appeared contradictory. Teacher ratings indicated DTT students in more- and most-inclusive versions of the model displayed greater social or academic improvement than those in least-inclusive BC and control groups. Direct observation of their classroom behavior, however, failed to produce similar between-group differences. Such inconsistency produced many, and sometimes conflicting, interpretations.

Another disquieting finding involved the second stage of BC, in which MAT consultants participated as members of a multidisciplinary team composed of the regular teacher and other building-based support staff. Its purpose was to engage in collaborative problem solving to develop effective interventions. To better understand Year 1 outcomes, we scrutinized the nature of these collaboratively developed interventions. This analysis, reported in Table 1 of a previous article (Fuchs & Fuchs, 1989), indicated that most teams developed interventions involving some type of teacher reinforcement. Disturbingly, however, a majority of these planned interventions did not call for teachers to monitor, or maintain written records, of student performance, which presumably should have been the basis for determining whether, and if so when, reinforcement was appropriate. In short, whereas some interventions were planned and implemented carefully, most reflected weak designs or were conducted inconsistently (see Fuchs & Fuchs, 1989).

MAT Research in Year 2 (1986-1987)

Poorly conceptualized or executed interventions in Year 1 argued for strengthening their design and implementation. This was addressed (a) through selection of a limited set of interventions supported by research and (b) by development of prescriptive instructions and materials to guide their use. By requiring MAT consultants and teachers to select among a small group of carefully detailed interventions, we sacrificed some consultation-teacher autonomy and collaboration to help ensure accurate implementation of judiciously chosen interventions. In keeping with this more directive approach, the importance of the multidisciplinary team diminished, and it was eliminated in Year 2.

We decided to conduct a second experimental investigation of the three BC versions. One reason was that, despite BC's widespread use, there are few experimental studies supporting its conceptual integrity. Validity studies such as component analysis are important to on-going model building and responsible educational practice. A second, more practical, reason recognizes that many school consultants probably do not implement all four stages of BC. Rather, we believe they typically limit their activity to the first two stages; that is, helping teachers to define the problem and identify possible solutions. To our knowledge, no previous study (besides the Year 1 investigation) has explored the effectiveness of this popular, albeit abbreviated, BC version by comparing it with more complete BC variants and a control group.

Although a second component analysis was planned, Year 2 research differed in important ways from the preceding year's effort. Because of the inconsistency of the teacher rating and observation data in Year 1, the research design was modified (a) by exploring the honesty of teacher ratings by comparing those communicated to consultants with those expressed anonymously; (b) by requesting teachers to complete a second, and very different, rating scale; (c) by increasing the frequency with which DTT pupils were observed; and (d) by observing DTT students' peers for comparison purposes. Finally, in an attempt to link outcome with process, data were collected on the frequency and accuracy with which interventions were implemented. This article describes the Year 2 effort to develop, implement, and validate effective and feasible prereferral interventions across multiple classrooms and schools in a large urban school district.

METHOD

Setting

Five experimental schools were identified. Two control schools were selected that matched experimental schools on location (inner city) and level (middle school), as well as these five factors: (a) proportion of Black students; (b) annual percentage of pupils referred for psychological evaluations; (c) average reading and math scores on the Stanford Achievement Test and a criterion-referenced test; (d) a composite index of a school staff's likelihood to refer students for psychological evaluations; and (e) percentage of students receiving free lunch. As shown in Table 1, experimental and control schools were comparable on these dimensions.

In contrast to other schools in the district, the seven project schools enrolled an equal percentage of Black students and were associated with a similar annual rate of referral for psychological evaluations. Moreover, pupils in project schools earned reading and math scores comparable to fourth- through eighth-grade students in the school system.

Participants

Consultants. Eight school-based support staff were associated with the five experimental schools: five special education resource teachers; two pupil personnel specialists (PPSs), a newly created position requiring the assessment skills of a psychologist, advising capacity of a counselor, and family-work experience of a social worker; and one librarian. The two PPSs were formally trained and experienced school psychologists. In return for their project involvement, consultants received a small cash stipend.

All consultants were female. One was Black; the remainder were Caucasian. They ranged in age from 28 to 48 years (M = 40.50, SD = 8.45), had between 3 and 18 years of experience as classroom teachers (M = 10.75, SD = 5.50), were in their present school from 1 to 6 years (M = 3.00, SD = 1.60), participated in 0 to 3 formal courses on teacher consultation (M = .88, SD = 1.25), and reported 0 to 40% (M = 11.80, SD = 12.90) of work spent in consultation. Their relatively infrequent consultation with classroom teachers reflected at least two facts: They were largely untrained and inexperienced in consultation, and their work time was dominated by competing job responsibilities. Special educators were required to maintain heavy direct-service caseloads; PPSs were expected to counsel students or families, conduct schoolwide job awareness and drug prevention programs, and serve as attendance officers.

Four graduate students, three in special education and one in school psychology, also served as consultants. Each special education student was assigned one experimental school; the school psychology student, two schools. In addition to conducting teacher consultation, the four graduate students were trained to provide technical assistance to school-based consultants. They endeavored to make certain that the consultants (a) understood the consultation process and substantive nature of prescribed classroom interventions; (b) had all necessary project-related materials; and (c) were proceeding with consultation in timely fashion, while collecting required student performance and fidelity of treatment data. In short, as dispensors of technical assistance, the graduate students were the on-site experts, facilitators, and monitors responsible for ensuring that project activities were completed with fidelity and timeliness.

Project Teachers. In experimental schools, consultants recruited 30 fifth- and sixth-grade teachers and one seventh-grade teacher, who met three criteria: First, they had at least one DTT pupil at risk for special education referral or retention; second, they expressed willingness to participate in project activities; and third, they were perceived by consultants as individuals with whom the consultants could work. Thus, teachers in experimental schools were not selected randomly; rather, as often happens in practice, they were identified as persons expressing a need for and willingness to participate in consultation.

In control schools, principals and project staff recruited 12 fifth- and sixth-grade teachers who, like the experimental teachers, had a DTT pupil and agreed to participate in project-related activity. On average, the 43 experimental and control teachers had taught 12.91 years and currently had 24.56 students in the classroom; 36 were Caucasian and 7 were Black; and 33 were female. Experimental and control teachers received a small cash stipend for project participation.

Difficult-to-Teach Students. All project teachers identified their most DTT nonhandicapped pupil. A majority of these 43 students was male (77%), in fifth (47%) or sixth (51%) grade, and Caucasian (60%). Their teachers reported them reading at a median 5.02 grade equivalency in the Ginn 720 reading series. A total of 30% previously had been held back at least one grade. Additionally, 53% were described as DTT primarily because of "off task" or "inattentive" behavior; 21% as a result of "poor interpersonal skills with adults"; 19% because of "poor academic work," despite an ability to perform better; and 7% because of "intrapersonal characteristics" such as low motivation or because of "insufficient academic skills."

Description of Contrasting Versions of BC

The importance of various components of the BC model was explored by creating three increasingly inclusive versions. In the least inclusive variation, (problem identification and problem analysis) (BC 1), consultant and teacher worked collaboratively to identify and analyze the problem. The consultant, however, did not assist the teacher in implementation of the intervention. Moreover, consultant and teacher did not evaluate intervention effects in a formative fashion, precluding opportunity for "fine tuning" the intervention.

The second variant of BC (problem identification, problem analysis, and plan implementation) (BC 2) included the BC 1 stages, but also required the consultant to make a minimum of two classroom visits, during which the consultant observed the teacher implement the intervention, and provided corrective feedback. Like the first version, however, this second variation of BC did not include a formative evaluation stage.

Finally, our third and most inclusive version (BC 3) incorporated the first three stages (problem identification, problem analysis, and plan implementation), as well as required consultant and teacher to formatively evaluate intervention effects.

Written Scripts

All but one stage (i.e., plan implementation) were conducted during formal interviews or meetings. Inspired by the Cantrell's Heuristic Report Form (see Cantrell & Cantrell, 1977; 1980), we recast descriptions by Bergan (1977), Gresham (1982), and Witt and Elliott (1983) of the substance covered in these formal meetings into written scripts, which guided much of the consultants' verbal behavior. Our expectation was that scripts would help consultants (a) create a rationale and overview for the meetings; (b) establish and maintain a logical and quick-paced "flow"; (c) obtain descriptions of the classroom environment, evaluations of targeted students, and important logistical information; and (d) systematically check the accuracy of key information.

We believed that in addition to promoting efficiency, scripts might enhance fidelity of treatment. In other words, assuming (a) scripts accurately reflected the BC model and (b) consultants faithfully followed them, we could be confident the model was implemented accurately and in a standard manner. (See Kratochwill, 1985, and Kratochwill & Van Someren, 1985, on the importance of conducting BC in a standard fashion.) This fidelity-of-treatment issue was especially important because most MAT consultants lacked formal consultation training and experience. Mindful of this, the four graduate students assigned to the experimental schools were instructed to encourage the school-based consultants to follow the scripts with fidelity, as well as record the degree of accuracy with which scripts in fact were used. Each BC version had its own script.

Prereferral Intervention

Teacher-Student Contract. A teacher-student contract was selected as an intervention strategy for two related reasons: First, a majority of teacher-consultant pairs independently decided to use it during Year 1 (see Fuchs & Fuchs, 1989). Second, recent surveys (e.g., Martens, Peterson, Witt, & Cirone, 1986) indicated that it was viewed positively by a large proportion of classroom teachers. The contract stipulated six dimensions of treatment: (a) type and degree of desired change in social or academic behavior; (b) classroom activity to which the contract would apply; (c) the strategy by which behavior would be monitored; (d) the nature of reward; (e) when and by whom the reward would be delivered; and (f) whether the contract could be renegotiated. Teachers were given blank copies of contracts. Attached was a message reminding them that activity, material, and token rewards were demonstrably effective in helping many DTT students improve their attitude and over behavior.

To enhance the salience of these rewards, consultants encouraged teachers to base the nature of the reward on student interests and to award reinforcers as soon as possible following a demonstration of the desired behavior. Each contract was good for only 1 day. Teachers were required use them for at least 3 weeks: every day during the first week and a minimum of two times during the second and third week. Thus, across the 3 weeks, as well as across BC groups, teachers were to use the contracts for a minimum of 9 days.

Monitoring Behavior and Academic Performance. While blocking for membership in BC 1, BC 2, and BC 3, the 31 teachers were assigned randomly to either a student- or teacher-monitoring group. Sixteen were directed to monitor student performance; 15 were trained by consultants to instruct their DTT students to monitor themselves. Also, depending on the nature of the problem behavior, monitoring procedures involved either interval recording or product inspection. Interval recording was defined as a "monitoring technique used to record whether a social behavior does or does not occur during a predetermined period or interval." Consultants recommended use of interval recording when student behavior was primarily disruptive to the teacher's or classmates' work or well-being (such as disturbing noise or inappropriate touching of others). Building on work by Hallahan and associates (e.g., Hallahan, Lloyd, Kosiewicz, Kauffman, & Graves, 1979; Hallahan, Marshall, & Lloyd, 1981), we developed directions and monitoring forms to guide students' (or teachers') use of interval recording.

Essentially, this procedure involved use of an audiotape, which played soft "beeps" both to signal the end of each recording interval and to prompt the DTT student (or teacher) to place a plus (+) or minus (-) sign in a corresponding place on the monitoring sheet. To record a plus, the student either (a) displayed the target behavior when the situation called for it, or (b) refrained from displaying it when it would have been inappropriate to do so. Before recording a minus, the student either (a) did not display the target behavior when it would have been appropriate to display it, or (b) exhibited the behavior when it was inappropriate to do so (see Fuchs, in press, for details).

Product inspection was defined as "evaluation of academic work at the end of a predetermined duration." This form of monitoring was used for behaviors primarily interfering with the student's own academic work (such as inattentiveness and frequent getting out of seat). As with interval recording, teachers and students were required to adhere to specific guidelines; and special monitoring sheets were created to facilitate record keeping (see Fuchs, in press).

Training

In two all-day sessions totaling 16 hours, consultants were trained in three areas. First, the problem-solving, collaborative, and databased nature of BC was discussed. To improve understanding of these features, consultants role played within the context of several prepared vignettes and received corrective feedback. Second, using videotapes of actual classroom conflict, they were trained to employ reliably a systematic observation procedure. Third, consultants were introduced to the teacher-student contract and the interval-recording and product-inspection monitoring procedures. They became familiar with the monitoring sheets, were taught when to use interval recording and when to use product inspection, and were informed how contracts and monitoring activities fit within the larger context of BC.

Assignment of Teachers and Scripts to Consultants

Assigning Teachers. On the second day of training, consultants were grouped by school and provided a list of teachers in their respective buildings who had volunteered for the project. To help ensure an equitable distribution of comparatively strong and weak teachers among all consultants, the school-based consultants rated each teacher's "demonstrated ability and willingness to work with difficult-to-teach students," using a 6-point Likert-type scale (6 = "very high"; 1 = "very low"). With these ratings as a guide, each of the eight school consultants then chose two general educators with whom to consult and assigned the remaining 15 experimental teachers to the four graduate students. Teachers deliberately were not assigned randomly to consultants because many school consultants worked as members of well-defined teacher teams responsible for regular planning and monitoring of school activities; to have paired them with teachers not part of their team would have been to introduce burdensome logistical complications.

Assigning Scripts. We did randomly assign experimental teachers to the three BC versions, with 10, 10, and 11 teachers assigned to BC 1, BC 2, and BC 3, respectively. On the 6-point rating scale, consultants assigned BC 1, BC 2, and BC 3 teachers average ratings of 4.50, 4.40, and 5.09, respectively, F (2, 28) = 1.10, ns. Table 2 displays teachers' class size, race, gender, and years of professional experience by the three BC variants, as well as for the 12 control teachers. As indicated in Table 2, the groups were not reliably different on any dimension, except gender; there was a greater proportion of males in BC 2 than in the other groups.

Table 2 also provides information by experimental and control groups for chronological age, grade level, and percentage Black, male, and retained at least 1 year. There was a larger proportion of Black students in BC 1 than in other groups. All student groups were comparable on the remaining dimensions. To guard against subject loss, we recruited slightly more teachers and students than the desired n of 10 per group. Because no attrition occurred, we were left with the unequal group sizes displayed in Table 2.

Random assignment of teachers to scripts meant that most consultants used one form of BC with one teacher and a contrasting variant with another. Consultants were informed that there was no compelling a priori reason to believe that one script would be more effective than another and, consequently, it would be a mistake for them to anticipate which script would prove more effective.

Following training, each consultant was given a spiral-bound notebook, including: (a) written scripts for every teacher and meeting; (b) classroom observation forms and worksheets to direct calculation of percentage of displayed targeted behavior; (c) copies of the teacher-student contract; (d) description of various reinforcement strategies teachers might incorporate into contracts; (e) an outline of the interval-recording and product-inspection monitoring procedures; (f) multiple copies of the two types of monitoring sheets; and (g) a detailed timeline encouraging consultants to conduct project activity in a timely fashion.

Consultation Procedure

Figure 1 displays sequences of salient consultation activity associated with the three versions of BC. BC 1 (least inclusive sequence) differs from BC 2 and BC 3 (most inclusive sequence) in its omission of classroom visitations. The uniqueness of BC 3 in relation to BC 2 is BC 3's potential for a third classroom visit, fourth meeting, and fifth observation. Consultants conducted all observations except the follow-up observations, described later. The first two helped validate as well as established a "baseline" for teacher-identified problem behavior. As depicted in Figure 1, Observations 3 and 4 (and Observation 5 in BC 3) were conducted following completion of the intervention and were used to assess its effect. Figure 1 also indicates that BC 1 and BC 2 called for 6 weeks of consultation, whereas BC 3 could take a maximum of 8 weeks.

Measures and Data-Collection Procedures

Measures and data-collection procedures included fidelity-of-treatment indexes, as well as direct observations of DTT students' and peers' classroom behavior; two types of teacher ratings; questionnaires for students, teachers, and consultants; and debriefing interviews with consultants and teachers. Such measures and procedures constituted a modest multimethod, multiperson approach to MAT evaluation.

Fidelity of Treatment. Each day on which the teacher-student contract was implemented, the student (or teacher) completed a monitoring sheet. As indicated, there were two types: interval recording and product inspection. Interval-recording forms required: (a) the student's name, (b) the class activity during which monitoring occurred, (c) the target behavior, (d) the duration of the recording period, and (e) a tally of minus (-) or plus (+) signs recorded in a grid of cells representing monitoring intervals, to indicate whether or not target behavior was displayed. Finally, the monitoring sheet guided students' (or teachers') calculation of the percentage of intervals during which target behavior occurred.

Product-inspection forms required specification of (a) the product (e.g., math worksheet), (b) the number of minutes permitted for work completion, (c) the evaluative criteria including quantity (i.e., amount of work to be completed) and quality (i.e., percentage of work to be completed correctly) indicators, and (d) a record of actual performance with respect to these criteria. Last, these monitoring sheets requested teachers to indicate whether the student met the amount and quality criteria.

Following completion of project activity, trained graduate students inspected DTT students' interval-recording or product-inspection monitoring forms and rated each set on these dimensions: (a) total number of monitoring sessions initiated and the number initiated during Weeks 1, 2, and 3; (b) percentage of sessions completed; and (c) percentage of monitoring forms completed with 100% accuracy. Across these dimensions, interrater agreement ranged from 88% to 100%.

Observations. The observation procedure combined features of time-interval recording and anecdotal note taking on antecedents and consequences to the target behavior (see Fuchs, in press, for details). Consultants, who conducted all observations except at follow-up, were instructed by audiotape to observe the DTT student and two randomly selected same-sex peers on a rotating basis for 2-minute intervals. During the first interval, consultants observed the DTT pupil; during the second, peer 1; third, the DTT student; fourth, peer 2; fifth, the DTT pupil, etc. Each 2-minute interval was divided into 10-second blocks for observing (8 seconds) and recording (2 seconds). Students were observed during 30-minute sessions, twice preceding intervention and twice immediately after completion. Three weeks following postobservations, they were observed once more by a trained research assistant. Thus, DTT students and peers were observed at least five times (and possibly six times in BC 3). Observations focused only on DTT pupils' target behavior.

Following 6 hours of training with interval recording, school-based and graduate student consultants demonstrated interrater agreement of .91, which was calculated by dividing the number of agreements by agreements plus disagreements on an interval-by-interval basis. Two "blind" observers were matched with consultants for 21% of preobservations and 33% of postobservations. Mean interrater agreement was .93 and .94, respectively. Disagreements were resolved through subsequent discussion. (See Fuchs, in press, for more detailed description of the observation procedure.)

Teacher Ratings. Each experimental and control teacher identified four to six problematic social or academic behaviors of their DTT student. One became the focus of prereferral as "target behavior." Using a 5-point Likert-type continuum, teachers rated each behavior in terms of severity (1 = "severe," 5 = "mild"), manageability (1 = "unmanageable," 5 = "easily managed"), and tolerability (1 = "intolerable," 5 = "tolerable"). Selection of these scales was based partly on Safran and Safran's work (1985), suggesting their importance in understanding teacher perceptions of problem behavior. Experimental teachers provided ratings before and immediately following intervention. Control teachers' preratings and postratings were obtained at the same time. In prior related research (Fuchs & Fuchs, 1989), the internal consistency (Cronbach's alpha) of these ratings at preintervention and postintervention has been .93 and .92, respectively.

Concurrent with completion of the postratings, all teachers responded to a questionnaire, which included a second postintervention rating. Teachers were informed that, unlike the previous two ratings, this rating was anonymous. Thus, they were asked not to place their names on the questionnaires. In fact, we ensured the anonymity of individuals; but by secretly marking the questionnaires, we identified respondents by group assignment. This procedure (approved in writing prior to implementation by the university's Committee for the Protection of Human Subjects) provided opportunity to compare "public" and "private" ratings, thereby helping to determine the frankness of teacher ratings obtained by consultants.

Revised Behavior Problem Checklist (RBPC). The RBPC (Quay & Peterson, 1983) was completed by teachers on their DTT pupils before and immediately following intervention. The RBPC contains 89 items, 77 of which constitute six independent scales: Conduct Disorders, representing a dimension of aggressive, noncompliant, quarrelsome, interpersonally alienated, acting-out behavior; Socialized Aggression, which measures a rejection of authority; Attention Problems-Immaturity, reflecting problems in concentration, perseverance, impulsivity, and direction-following; Anxiety Withdrawal, subsuming characteristics of anxiety, depression, fear of failure, social inferiority, and self-concern; Psychotic Behavior, relating to overt psychosis and related language dysfunctions; and Motor Tension-Excess, involving gross motor behavior and motoric tension.

Teachers rated the RBPC items on a 0 to 2 scale after consultants read them standard instructions. Reliability and validity of the RBPC appear comparable to or better than those of most behavior rating scales (Eliason & Richman, 1988): Interrater agreement ranges between .52 and .85; 2-month test-retest reliability is between .49 and .83; and there is evidence of concurrent validity with several DSM-III diagnoses, peer nominations, and direct observations (see Quay & Peterson, 1983; 1987).

Questionnaires. Following completion of the intervention the graduate student consultants administered questionnaires individually to teachers and students

in BC 1, BC 2, and BC 3, and to the school-based consultants. Items explored respondents' views on project effectiveness, the relative importance of facets of the intervention, degree of difficulty associated with implementing the interventions, and so forth. Each item was rated on a 5-point Likert-type scale.

Debriefing Interviews. Several weeks after administration of the questionnaires, school-based and graduate student consultants in the same schools were interviewed together. The five separate debriefing sessions lasted between 30 and 90 minutes, with two to four consultants participating. Consultants were encouraged to offer candid impressions of the project, including ways in which it might be enhanced. Seven randomly chosen experimental teachers also were interviewed. These sessions, too, were open-ended and meant to provide teachers a chance to speak their minds.

Data Analysis

Two points require clarification regarding data analysis. First, as mentioned, the eight school-based consultants and four graduate students worked with 16 and 15 teachers, respectively. To explore whether type of consultant (i.e., school-based vs. graduate student) exerted systematic effects on any outcomes, a series of analyses was conducted. In each analysis, the consultant factor was not significant. Thus, it was eliminated from subsequent analyses. Second, data obtained from all measures but one were subjected to statistical analysis. The exception was the questionnaire, from which only descriptive data are presented. This is because most items were dissimilar across types of respondent, precluding straightforward comparisons.

RESULTS

Fidelity of Consultation Process

As described previously, the BC 3 process is different from that of BC 2 because it includes a formative evaluation component. This requires teacher and consultant in Meeting 3 to measure the student's current performance level against a goal set in Meeting 1. Moreover, if consultant and teacher decide that this goal has not been reached, then they must change the student goal, modify the intervention, or both. In the case of a changed goal or modified intervention, the consultant conducts a third classroom visit, fourth meeting, and fifth student observation, none of which is part of BC 2.

On the other hand, if the consultant-teacher team determines that the goal has been attained, or, if not attained, that sufficient progress has been demonstrated, then consultation is terminated. In this case, BC 3 is much like BC 2. All this prompts the question, "How many consultant-teacher dyads in BC 3 determined that changes in goal or intervention were necessary?" Answer: none. Although two teachers claimed initial goals had not been met, both expressed satisfaction with student progress. Thus, aside from the fact that BC 3 participants formally compared students' current level of performance to initial goals, there was little to distinguish BC 3 members' activity from that of BC 2.

Fidelity of Classroom Interventions

Monitoring: Teacher Versus Students; Interval Recording Versus Product Inspection. Half the teachers in BC 1 and BC 2 monitored DTT students' behavior, while the other half taught the DTT students to assume this task. In BC 3, 6 and 5 teachers were involved in teacher and student monitoring, respectively. As reported elsewhere (Fuchs, in press), teacher versus student monitoring did not produce any reliable differences and will not be discussed further. In BC 1 and BC 2, 7 of 10 teacher-student pairs used product inspection; in BC 3, 7 of 11. Thus, across the three experimental groups, virtually identical proportions of teacher-student pairs used product inspection, [Chi.sup.2] (2, n = 31) = .13, ns; and for this reason, type of monitoring procedure (interval versus product inspection) was not explored further.

Monitoring: Frequency and Accuracy. Means in Table 3 indicate teachers and students in the three experimental groups complied with the requirement that contracts and monitoring should be implemented a minimum of nine times across 3 weeks. Moreover, on average, they satisfied the related request that they monitor behavior or academic performance a minimum of five, two, and two times during Weeks 1, 2, and 3, respectively. Additionally, the table shows that teachers and students completed monitoring tasks in a thorough manner and with a relatively high degree of accuracy.

A one between (BC 1 vs. BC 2 vs. BC 3) one within (number of sessions vs. percent complete vs. percent perfect vs. percent components correct) analysis of variance (ANOVA) indicated (a) no significant effect for the experimental group factor, F (2, 28) = 1.96, but (b) a significant effect for the fidelity scale factor, which was due to contrasting metrics for the different fidelity indexes. Therefore, no follow-up analyses were conducted. In addition, there was no significant interaction, F (6, 84) = 1.79.

A second one between (BC 1 vs. BC 2 vs. BC 3) one within (Week 1 vs. Week 2 vs. Week 3) ANOVA indicated no significant effect for experimental group, F (2, 28) = .18, and no significant interaction between experimental group and week of implementation, F (4, 56) = .65. There was a significant effect, however, for the week factor, F (2, 56) = 43.29, p [is less than] .001. A series of paired t tests revealed that the number of monitoring sessions implemented during Week 1 was greater than in Week 2, t (30) = 8.24, p [is less than] .001 and Week 3, t (30) = 6.86, p [is less than] .001. There was no reliable difference between Weeks 2 and 3.

Frequency with Which Contractual Goals Were Met. On average, pupils in experimental groups achieved their daily contract-related goals during a majority of the monitoring sessions: 66% (SD = 31) for BC 1; 62% (SD = 31) for BC 2; and 72% (SD = 19) for BC 3. A one between ANOVA (BC 1 vs. BC 2 vs. BC 3) indicated these proportions were not reliably different, F (2, 18) = .78. These data are based on the 21 DTT students (7 per experimental group) involved in product inspection.

Observations

Table 4 displays (a) percentages of intervals during which DTT students and peers demonstrated target behavior and (b) discrepancies between them at preobservation and postobservation and follow-up (see Figure 2). A three-way ANOVA revealed a Group x Student (DTT vs. peers) x Trial interaction, F (6, 78) = 5.42, p [is less than] .001. Scheffe analysis indicated that, from preobservation to postobservation, reductions in the DTT student-peers discrepancy were reliably greater for BC 2 (M = .30) and BC 3 (M = .28) than for BC 1 (M = .13) and controls (M = .02). No group demonstrated significant behavior change from postintervention to follow-up.

Teacher Ratings

Table 5 presents means and standard deviations of teacher ratings for BC and control groups and pretreatment trial, posttreatment trial, and anonymous trial. A two-way ANOVA indicated a significant effect for the Group x Trial interaction, F (6, 78) = 3.34, p [is less than] .01. Figure 3 displays this interaction. Scheffe analysis showed that changes between preintervention and postintervention ratings for BC 1, BC 2, and BC 3 were reliably greater than for controls (see Table 5). There was no reliable preintervention to postintervention change between BC 1, BC 2, and BC 3, or between postintervention and anonymous ratings across BC groups and between BC and control groups.

Revised Behavior Problem Checklist

Means and standard deviations of teacher responses on the RBPC are shown in Table 6. A Group x Scale x Trial Multivariate Analysis of Variance (MANOVA) produced a significant effect, F (10, 120) = 2.16, p [is less than] .05. One-way ANOVAs on each RBPC scale revealed the following: F (3, 42) = 3.95, p [is less than] .05 for Conduct Disorders; F (3, 42) = 2.01, ns for Socialized Aggression; F (3, 42) = 2.73, p = .057 for Attention Problems-Immaturity; F (3, 42) = 1.22, ns for Anxiety-Withdrawal; F (3, 42) = 1.05, ns for Psychotic Behavior; and F (3, 42) = .89, ns for Motor Excess.

For Conduct Disorders, follow-up analysis revealed that ratings became significantly more positive from preintervention to postintervention for pupils in BC 2 than for controls; for Attention Problems-Immaturity, teacher ratings became more positive from preintervention to postintervention for students in BC 1, BC 2, and BC 3 than for controls. Preintervention and postintervention responses to Conduct Disorders and Attention Problems-Immaturity items of the RBPC are illustrated in Figure 4.

Teacher, Consultant, and Student Questionnaires

Project Effective? Across BC groups, mean teacher ratings of project effectiveness ranged from 3.00 to 3.82 on a 5-point scale (see Table 7). Ratings of 3.30 to 3.64 indicate teachers judged project activities to be feasible. Consultant ratings of project effectiveness ranged from 3.20 to 3.82. Their ratings of project components like student-teacher contracts (4.10 to 4.20) and monitoring procedures (4.50 to 4.60) also were positive. Students (a) believed their behavior improved during the contracts (3.80 to 4.30), and (b) stated they would recommend that more teachers use them (3.80 to 4.50). Moreover, students believed their rewards were important (3.90 to 4.20), contracts were fun (3.90 to 4.45) and fair (4.10 to 4.82), and they worked hard on them (3.45 to 4.20).

Project Worth Doing? Table 7 suggests that, regardless of group, teachers tended to believe the project was worth doing, with mean ratings ranging from 3.90 to 4.36. They also appear more than less likely both to continue to use the intervention next year (3.20 to 3.60) and to volunteer for the project again (3.00 to 3.60). Consultants, too, seemed to believe the project was worthwhile, with ratings between 4.10 and 4.20. With ratings between 3.45 and 3.80, they also indicated that project participation contributed to their professional development.

Consultants Collaborative and Technical Assistance Helpful? Teachers assigned their highest ratings to items concerning consultants. They believed consultants were very collaborative (4.00 to 5.00) and very helpful (4.20 to 5.00). To the question, "How supportive was the technical assistance available to you?" each consultant responded with a rating of 5.

DISCUSSION

Component Analysis

More is Better. Findings from the component analysis suggest the more inclusive versions of BC 2 and BC 3 promoted more positive student change than the least inclusive variant, BC 1. Preobservations and postobservations of classroom behavior buttress this view. In comparison to DTT students in BC 1 and control groups, BC 2 and BC 3 pupils significantly reduced initial discrepancies between themselves and their peers regarding percentage of problem behavior. BC 1 and control students did not differ in this respect. Moreover, these results were unchanged at "follow-up," 3 weeks after termination of formal interventions. Corroborating these data were responses to the RBPC, indicating BC 2 teachers perceived significantly greater reductions of Conduct Disorders than did other teachers.

Adding incisiveness to the component analysis are the fidelity-of-treatment data. They (a) substantiate that students and teachers in BC 1, BC 2, and BC 3 implemented the interventions with similar frequency, thoroughness, and accuracy, and (b) strengthen the conclusion that group differences in DTT students' classroom behavior and their teachers' perceptions of conduct disorders were due to variations in the consultation process, not quantitative or qualitative differences in classroom interventions.

Implications for Theory and Practice. These findings are important theoretically because they represent preliminary validation of the BC model; that is, they represent tentative affirmation of its integrity by upholding the central assumption that its components are important and additively related. Practically speaking, however, the "more is better" conclusion is troubling. Effectiveness aside, more inclusive versions of BC require more time and energy from school personnel and, we believe, are less likely to be seen as feasible. Underscoring the seriousness of this point is that relatively few school districts and building principals permit special educators and school psychologists opportunities to consult with classroom teachers (e.g., Idol-Maestas & Ritter, 1985; Nevin, Paolucci-Whitcomb, Duncan, & Thibodeau, 1982).

On the other hand, MAT consultants reported that, on average, participation in BC 2 and BC 3 versions required 6 hours of their time per DTT pupil. In light of demonstrated effects on student behavior and teacher perceptions, this time expenditure seems small. MAT consultants' efficiency becomes even more apparent when their 6-hour estimate is compared to the fact that it typically takes school psychologists in the same district twice as long to administer, score, and write a single psychological evaluation (Director of Psychology, personal communication, June 1987). We join Idol (1988) and others in suggesting that district officials and building principals can be sold on the economy and effectiveness of certain types of consultation-related activity.

In a related vein, another finding from our component analysis indicates that, whereas more inclusive BC versions may be more effective, conducting a BC 1 version may be better than doing nothing at all. Three types of data support this view. First, analysis of teachers' preintervention and postintervention (severity, manageability, and tolerability) ratings of target behavior indicated (a) students in BC 1, BC 2, and BC 3 received significantly and dramatically more improved ratings than controls, whereas (b) there was no discernible difference in changed ratings among the experimental groups. This pattern held whether teachers expressed ratings publicly or anonymously.

Second, BC 1, BC 2, and BC 3 teachers registered similar and dramatically more positive ratings of their DTT students from preintervention to postintervention on the Attention Problems-Immaturity scale of the RBPC. Third, the importance of BC 1 (as well as BC 2 and BC 3) is indicated by questionnaire data. This "consumer satisfaction" information reflects generally positive sentiments about the project's effectiveness, value, feasibility, and fairness across respondents and BC groups. Assuming that special educators' and school psychologists' typical consultation activity resembles BC 1, findings represent both good and bad news: Whereas conventional practice probably is beneficial, it seems less effective than it could be. Results indicate that school consultants should implement more complete versions of the consultative process.

Study Limitations

Prereferral Interventions. The apparent effectiveness of MAT prereferral interventions precluded an opportunity to conduct a more complete component analysis of BC. That is, because none of the teacher-consultant teams in BC 3 decided that a modified intervention was necessary, there was little to distinguish BC 2 and BC 3 activity. For all practical purposes, then, this component analysis only pitted BC 1 against BC 2 and BC 3. The irony is that, whereas the component analysis of Fuchs and Fuchs (1989) was compromised by many weak MAT interventions, the current analysis is rendered incomplete because of relatively strong ones.

One reason for the effectiveness of the interventions in this study may be the high fidelity with which they were conducted. Fidelity of implementation, in turn, probably was enhanced by the on-site presence of the four graduate students. As already mentioned, these students provided technical assistance and served as gophers and calendar-watchers, responsible for facilitating correct and timely implementation of project activity. Would the MAT approach to prereferral intervention have worked as well without this on-site supervision? Probably not. Thus, we believe it would be a mistake for school administrators to assume that successful adoption of the MAT hinges only on acquiring the right directions and materials. Oversignt of the type exercised by the graduate students, we believe, may be important. It is also costly. To help ensure successful adoption of MAT-type activity, some districts may be required to allocate resources accordingly.

Another reason for the apparent success of the MATs may be its prescriptive nature. Not only did written scripts guide the process of consultation, but teacher-student contracts and student monitoring forms determined much of the substance of the in-class interventions. This prescriptiveness sets the MATs apart from typical practice (cf. Friend, 1985; Johnson, Pugach, & Hammittee, 1988; Reisburg & Wolf, 1988), a point to which we will return. This divergence, however justifiable, represents an additional constraint on the generalizability of findings.

Subject Selection. Another important departure from typical practice was that, rather than wait for teachers to request help, our 12 consultants actively recruited teachers with DTT pupils to participate. This unusually "proactive" consultant behavior raises the question whether the recruited teachers and students are typical of those who normally participate in consultation activity. Moreover, though consultants were assigned randomly to experimental groups, they were not assigned randomly to teachers. Rather, they carefully selected teachers from a pool of prospective consultees whom they knew well. In addition, the teachers were volunteers who, like the consultants, had been promised a small cash stipend as quid pro quo for study participation. All these facts probably represent limits on the external validity of the investigation.

Measures. Whereas observation, rating, and questionnaire data suggested that DTT student improved because of project activity, the study is still missing a "bottom line" index; namely, percentages of pupils in the three BC groups and control condition who were referred by their teachers for formal psychological evaluation. Moreover, there was little effort to ascertain the extent to which MAT effects maintained for longer than 3 weeks. Our 3-week follow-up, while probably better than no follow-up, does not substitute for long-term evaluation of MAT effectiveness.

MAT Prescriptiveness

Without minimizing the importance of the foregoing constraints and caveats, findings suggest the effectiveness of a prescriptive approach to prereferral intervention and school consultation. As indicated, MAT prescriptiveness differs from a currently popular view that consultation should be "collaborative," representing a reciprocal arrangement "that enables people with diverse expertise to generate creative solutions to mutually defined problems" (Idol, Paolucci-Whitcomb, & Nevin, 1986.). Thus, we suspect our directive approach may be viewed as an oddity by some researchers and practitioners with interest in consultation-related school activity.

Nevertheless, we did not set out to implement prescriptive prereferral interventions. During Year 1, MAT consultants received intensive training in collaborative consultation (see Fuchs, in press; Fuchs & Fuchs, 1988; 1989). Despite expectations for success, in-class interventions were largely unimpressive. Further, during project implementation and subsequent debriefing interviews, many teachers complained they had insufficient time for the give-and-take nature of collaborative problem solving. Instead, they wanted helpful suggestions. In contrast, during debriefings and on questionnaires administered after more directive Year 2 activity, teachers from the same school district expressed satisfaction with the MAT interventions and consultation process. No one described the project experience as coercive or implicitly denigrating of his or her knowledge or skill.

Our MAT experience suggests that the form and substance of consultation should be consonant with the specifics of the situation (cf. Heron & Kimball, 1988). In schools in which stress is high, expertise in consultation is low, and consultation time is nonexistent, prescriptive approaches appear better suited for success than collaborative ones. We have no doubt that, in different situations, more collaborative approaches may represent a better choice. Moreover, situations change. As teachers and support staff as well as school administrators become more experienced, confident, and positive regarding consultation-related activity, prescriptive approaches might give way to more collaborative efforts. We concur with a pragmatic view expressed by Friend (1988), Heron and Kimball (1988), Safer (1987), and others that the nature of consultation activity should be determined more by the circumstance in which consultants find themselves than by an a priori belief system. [Tabular data 1 to 7 Omitted] [Figures 1 to 4 Omitted]

DOUGLAS FUCHS and LYNN S. FUCHS are Associate Professors and PAMELA FERNSTROM and PAMELA M. STECKER are Graduate Students, Department of Special Education, George Peabody College, Vanderbilt University, Nashville, Tennessee. MICHAEL W. BAHR is an Assistant Professor, Department of Counselor Education and Counseling Psychology, Western Michigan University, Kalamazoo, Michigan.
COPYRIGHT 1990 Council for Exceptional Children
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1990 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Fuchs, Douglas; Fuchs, Lynn S.; Bahr, Michael W.; Fernstrom, Pamela; Stecker, Pamela M.
Publication:Exceptional Children
Date:Apr 1, 1990
Words:8680
Previous Article:Special education and the regular education initiative: basic assumptions.
Next Article:Are regular education classes equipped to accomodate students with learning disabilities?
Topics:


Related Articles
Survey on prereferral practices: responses from state Departments of Education.
Prereferral interventions: progress, problems, and challenges.
Redefining "prereferral" intervention as intervention assistance: collaboration between general and special education.
The challenge of implementing collaboration between general and special education.
Peer collaboration: accommodating students with mild learning and behavior problems.
A Comparison of School-Based Intervention Teams: Implications for Educational and Legal Reform.
Sociocultural context effects on teachers' readiness to refer for learning disabilities.
Intervention-based assessment: evaluation rates and eligibility findings.
Prereferral intervention processes: a survey of state practices.
Teacher assistance team social validity: a perspective from general education teachers.

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters