Printer Friendly

Effects of self-evaluation on the independent work skills of preschool children with disabilities.

Effects of Self-Evaluation on the Independent Work Skills of Preschool Children with Disabilities

ABSTRACT: This study examined the effects of a self-evaluation treatment package on the

independent work skills of preschool children with disabilities. Children learned to conduct

self-assessments and to compare their ratings with those of the teacher. Data were collected on

children's percentages of appropriate behavior, the level of teacher prompting, and the match

between child and teacher ratings. Results indicated that the treatment produced immediate and

substantial improvements in child behavior and that it was possible to systematically withdraw

each component while maintaining a high level of appropriate child behavior. * Traditionally, the primary function of schooling has been to equip students with the basic academic skills of reading, writing, and arithmetic. Today, however, schools are also considered to be a major socializing institution, conveying a "hidden curriculum" (Cartledge & Millburn, 1978)--the learning of social behaviors valued by peers, teachers, and the larger society in which the school functions. One skill necessary for children's successful functioning in school, and ultimately in society, is the ability to perform certain tasks in an independent fashion. For teachers, the degree of independence displayed by children is a crucial factor in the teacher-pupil relationship and in the teacher's management of the learning environment. For children, the ability to assess one's own behavior, to make accurate and correct judgments about the appropriateness of that behavior, and to alter one's subsequent behavior are skills that serve one not only in school but later in life. The teaching of self-assessment and self-evaluation, however, is rarely a part of school curricula (Rueda, Rutherford, & Howell, 1980).

The benefits of promoting self-evaluation in children are substantial for both children and teachers. For the child, the ability to manage his or her own behavior may help maintain newly learned skills across settings, teachers, and tasks (Baer & Fowler, 1984; Holman & Baer, 1979; O'Leary & Dubey, 1979; Rose, Lessen, & Gottlieb, 1982); and the ability to work independently minimizes dependence on the presence of the teacher (Burgio, Whitman, & Johnson, 1980). For teachers, the ability of children to function independently and to self-manage may reduce the amount of time given to nonacademic duties (e.g., classroom management and discipline) (O'Leary & Dubey, 1979); free teachers from routine supervision (Burgio et al., 1980); and assist teachers in preventing common behavior problems that interfere with or preempt opportunities to learn (Strain & Sainato, 1987).

Using a variety of training procedures, self-management and its components (e.g., self-assessment, self-monitoring, and self-evaluation) have been successfully taught to children both with disabilities (Drabman, Spitalnik, & O'Leary, 1973; Hallahan, Lloyd, Kneedler, & Marshall, 1982; Hallahan, Marshall, & Lloyd, 1981; Holman & Baer, 1979; Rhode, Morgan, & Young, 1983; Robertson, Simon, Pachman, & Drabman, 1979; Shapiro, McGonigle, & Ollendick, 1980; Smith, Young, West, Morgan, & Rhode, 1988) and without disabilities (Bolstad & Johnson, 1972; Glynn & Thomas, 1974; Thomas, 1976). Significant among the intervention strategies designed to improve the classroom behavior of school-aged children have been those that require children to (a) assess their own behavior, (b) "match" their assessments with those of the teacher, and (c) be rewarded contingently, either for accuracy of their assessments or the quality of their performance on a specific task (Drabman et al., 1973; Robertson et al., 1979; Rhode et al., 1983; Turkewitz, O'Leary, & Ironsmith, 1975). Results from these studies indicate that elementary school children were able to learn accurate self-assessment and that intervention procedures could be removed systematically without a corresponding decline in child performance.

A review of these studies prompted the development of a self-evaluation package for use with preschool-aged students with disabilities who were candidates for enrollment in regular kindergarten programs in the public schools.

The specific purpose of this study was to determine if young children would learn to evaluate their own behavior and subsequently match their self-evaluations with the teacher. The children would then be reinforced for correctly evaluating their performance. In addition, we proposed that the components of the intervention (e.g., self-assessment matching with the teacher and reinforcement) could be withdrawn in sequence while the children's appropriate behavior maintained itself. We also expected to demonstrate that children's appropriate behavior during the use of the self-evaluation package would be maintained with a low level of teacher prompting, and that the level of appropriate behavior during intervention would be similar to preintervention levels in which teacher prompts and instructions were frequent.


Subjects and Setting

This study was conducted in an integrated classroom for autistic and normally developing preschoolers in a large urban elementary school. Although there were a total of six children with disabilities and six children without disabilities enrolled in the classroom, only four children (all boys) participated in this study. Upon entry into the preschool program, two children (C1 and C4) were rated as severely autistic on the Childhood Autism Rating Scale (CARS) (Schopler, Reicher, DeVellis, & Daly, 1980), and at the same time failed to achieve a basal score on the McCarthy Scales of Children's Abilities (McCarthy, 1972). A third child, C3, was rated as moderately autistic on the CARS; he also failed to achieve a basal level on the McCarthy Scales. These three children were diagnosed by a licensed child psychiatrist as autistic, using Diagnostic and Statistical Manual of Mental Disorders-III (DSM-IIIR); see American Psychiatric Association, 1980, criteria. The fourth child, C2, entered the program as a normally developing peer whose general cognitive index on the McCarthy Scales was 100. At the time of the study, C1 (chronological age, CA, = 69 months), C4 (CA = 49 months) and C3 (CA = 66 months), had received ratings of "no autism" on the CARS after being enrolled in the program for 16, 21, and 23 months, respectively. In addition, their general cognitive index scores on the McCarthy Scales had increased to 106 for C4 and to 86 for C3. There was no change in the general cognitive index score for C1, who still failed to reach a basal level on the McCarthy Scales. The fourth child, C2 (CA = 61 months) had been enrolled in the preschool for 4 months at the onset of the study.

Subject selection was based on prebaseline observations of children during independent work time. Three children (C1, C3, and C4) displayed a low level of autistic like behaviors (stereotypy, lack of social responsiveness, perseverative speech, tantrums) during highly structured and teacher-directed activities; and they consistently exhibited high levels of off-task and disruptive (out-of-seat, talking-out, hitting) behavior. The fourth child, C2, also displayed a high rate of off-task and inappropriate behavior when not under direct supervision of the teacher.

Prebaseline observations confirmed the teacher rankings of the children's inability to work in the absence of teacher direction. In addition, teachers noted that all four children were targeted for kindergarten entry in approximately 9 months and expressed concern regarding the children's inability to work independently.

The study was conducted daily during a 20-minute (min) seatwork time in the preschool classroom. Materials available included books, puzzles, blocks, a kitchen and dress up area, and the seatwork (table) area. In the latter area, two tables were set up parallel to each other. Perpendicular to the tables and approximately 3 feet (ft) away was a set of 2 1/2-ft high shelves containing a bin for each child.

The daily schedule proceeded as follows: Calendar and circle time, lesson, free play, snack and bathroom, language lesson, grossmotor activities, seatwork, and free play. Each day all children spent a portion of their time doing developmentally appropriate workbook pages and dittos at the table area. A total of eight children, including the target children, were seated at the tables, while the other children participated in small group activities. Two of the target children sat at each table.

Seatwork time consisted of two separate tasks. Each child first completed two worksheets and then a "work job." Work jobs varied widely, from matching a set of individual letters with word cards to counting sets of a specified number of objects, to putting together word puzzles. Materials for both tasks were chosen by the teacher according to each child's ability. Teachers were asked to select tasks to which children had been previously introduced.

Behavioral Measures

Observations. Teacher behavior and target children's performance were assessed throughout the independent seatwork session by two trained observers using a 10-second (s) partial-interval recording system. Observers rotated their observations among target children every 10 s for the entire 20-min session. In addition, all target students were rated individually by the teacher at the end of the independent work session. Three basic categories of behavior were recorded and defined as follows: 1. Appropriate child behavior

a. On-task--attending to the task, with

pencil on paper, or attention focused on

the work job for more than 5 s.

b. Waiting--waiting to receive instructions,

materials, feedback, reinforcement, or

teacher attention. 2. Inappropriate child behavior

a. Off-task--not engaged in teacher-as-signed,

approved behavior for more than

5 s.

b. Disruptive--being out of seat without

permission, talking out, hitting peers, or

interfering with peer's attempts to work. 3. Teacher behaviors

a. Verbal prompts--any verbal command

or directive given to the child or the group

by the teacher.

b. Praise--any positive verbal comment by

the teacher (e.g., "good," "okay") to

the child or to the group.

c. Negative--any verbal reprimand by the

teacher or placing a child in time out.

Rating Scales. Performance measures were obtained on the four target children during the independent work time in each phase of the study. The teacher, who also acted as the intervention agent, completed an Independent Seatwork Rating Scale (developed by the first author) for each child. The teacher rated each of the following as either "yes" or "no." Did the child (1) listen to the teacher's directions; (2) sit in the chair with feet on the floor; (3) work quietly; (4) raise his hand with a question or when finished; (5) finish the worksheets; (6) work neatly; (7) put paper and pencil into the proper bin; (8) find the correct work job and go quietly back to the table; and (9) check the work when finished and put it away?

These ratings were obtained to (a) assess whether the children performed these behaviors during the seatwork period and (b) assess the "match" between the teacher's rating of child behavior and the child's self-evaluation during the same period.

Observer Training and Reliability

Interobserver agreement for observational measures between the two data collectors was calculated on a point-by-point basis. Agreement was calculated by dividing the sum of target behaviors recorded in agreement by that number plus the number recorded in disagreement, then multiplying by 100. Occurrence and nonoccurrence agreement was computed separately. Observers were required to achieve agreement percentages exceeding 90% for 5 consecutive days with each other and the first author before data collection. Subsequent reliability checks were distributed evenly over 26% of the observation sessions. Agreement percentages ranged from 93% to 100% and 92% to 100%, respectively, for the occurence of all targeted teacher and child behaviors. Agreement percentages for the nonoccurrence of teacher and child behavior ranged from 91% to 97% and 88% to 98% for the two observers, respectively.

Reliability was also calculated for the Independent Seatwork Rating Scale, which was completed by the teacher throughout the intervention phase of the study. Interrater reliability between the teacher and the first author on their mean agreement for the nine items on the rating scale was 96% with a range from 94% to 100%.

Procedure and Design

To assess the effects of the self-evaluation package on children's behavior during the independent seatwork time, a sequential withdrawal-of-treatment design was used. This design allows for the withdrawal of one component of a multiple-component treatment, and then a second (and so on) to evaluate the maintenance of behavior after experimental control has been established (Rusch & Kazdin, 1981). Conditions of the design were as follows.

Baseline 1 (B1). During this and all subsequent conditions teachers distributed materials and gave general instructions to students. Students were told to (a) finish their worksheets and raise their hands; (b) put their worksheets and pencils in their bins and pick up a "work job" from on top of the bookshelf; and (c) complete the work job, put it away, and join the other children in free play. During B1, the teacher gave initial instructions, remained present throughout table time, and answered all questions. The teacher also gave unlimited prompts and praise for appropriate behavior--typical classroom practice.

Baseline 2 (B2). A second baseline condition was necessary, given that the purpose of this study was to investigate children's ability to work independently of extensive teacher prompts. During the B2 condition, the teacher gave initial instructions, then limited each child to two teacher prompts. Additional prompts were permitted only for excessive off-task or disruptive behavior. Teachers were instructed to answer any question a child asked in an appropriate fashion (i.e., by raising one's hand).

Self-Assessment Plus Matching With the Teacher Plus Reinforcement. Before the two baseline phases, all children in the group were photographed modeling appropriate behaviors for the Independent Seatwork Rating Scale (i.e., listening to the teacher's directions, sitting appropriately, working quietly). Each child's picture was then placed on construction paper, with a caption describing the rating scale activity being demonstrated. Next to each picture, two faces were drawn: a happy face for "yes" and a frowning face for "no." The photos and drawings were inserted in clear plastic covers so that the children could check "yes" or "no" with an erasable grease pencil. The sets of photos were then placed in individual three-ring binders. At the end of table time, children were instructed to assess whether they had exhibited each of the rating scale behaviors during that session.

In addition to this self-assessment component, matching of ratings with the teacher was accomplished as follows: During the independent seatwork time, the teacher who participated in the study rated the children's appropriate behavior during the session. Children were rated as having performed a particular behavior if they were judged by the teacher to have been appropriate 80% of the session. After children completed the self-assessment, they met individually with the teacher and compared their self-assessment rating scale. Students received praise from the teacher for both appropriate behavior and accurate self-assessment. During the first treatment phase, all four children participated in matching with the teacher, but in later treatment phases the number of children participating varied.

For the reinforcement component of the package, children selected small toys if their self-assessment matched with that of the teacher on seven or more out of the nine categories on the rating scale. Reinforcement was based solely on matching. If children accurately reported inappropriate or off-task behavior and the teacher concurred, this was considered a "match" and could thereby be figured into the total score.

Baseline phases B1 and B2, the implementation of the total package, and the return to B2 were identical for all children. The reintroduction of the self-assessment intervention varied according to each child's behavior.

Baseline 2. This return to the B2 condition was identical for all children. As before, teacher prompts were limited to two per child, following initial instructions. Teachers were permitted to answer children's questions asked in an appropriate manner (e.g., by raising one's hand).

Self-Assessment Plus Matching With Teacher Plus Reinforcement. This phase was reintroduced for three children (C1, C2, and C3) following the second implementation of the B2 condition. Because the behavior of one child (C4) never returned to prior baseline levels, the study for this child ended in the second B2 condition.

Self-Assessment Plus Matching With the Teacher. Following the reintroduction of the complete treatment package for three children (C1, C2, and C3), the reinforcement component of the package was withdrawn to assess the impact of the remaining two components, self-assessment plus matching with the teacher, on target child behavior. The withdrawal of the reinforcement component occurred first with C1 and C2, and then with C3.

Self-Assessment Only. In this phase, children were required to self-assess but were not required to compare their self-assessments with teacher assessments. After completing their seatwork tasks, children took their self-evaluation books, marked "yes" or "no" next to each of the nine items, returned their books to the shelf and moved to the next activity. Once again, this phase was introduced simultaneously for C1 and C2 and then for C3.

Baseline 2. With the total withdrawal of all treatment components, C1, C2, and C3 were once again in the B2 condition. The study ended here for C2 and C3.

Self-Assessment Only. Given the deterioration of the behavior of C1 in the prior condition, self-assessment was reintroduced for this child. This phase was identical to the previous self-assessment-only phase. The study ended here for C1.


Appropriate Child Behavior

Figure 1 shows the percentages of appropriate behaviors for all four children. In B1, with constant teacher presence and monitoring, appropriate behavior averaged 79% for C1, 89% for C2, 76% for C3, and 62% for C4. C2 was absent for 2 out of 7 days of B1.

When teacher prompting was reduced to two prompts per child in B2 (a level approximating normative rates in kindergarten), appropriate behavior for all subjects decreased. Appropriate behavior for C1 dropped below 30% twice during this phase, whereas C2's appropriate behavior fell below 50% for 5 out of 9 days, and C3's below 60% for all recorded intervals for 4 out of 9 days. Appropriate behavior for C4 averaged 44% for this condition.

Introduction of the complete self-evaluation package dramatically increased appropriate behavior for C2 and C3, from B2 averages of 53% and 64% to 93% and 89%, respectively. C1's behavior improved 19%, from an average of 52% to 71%. Behaviors were maintained for 7 days after a return to baseline. On the eighth day, however, when a different intervention agent (the second teacher in the classroom) was introduced, behaviors of C1 and C2 immediately dropped to levels similar to the B2 phase and remained at these levels even when the original agent returned. C3 maintained near 90% on-task behavior throughout the agent change. For C4, introduction of the self-evaluation package resulted in a gradual change in behavior over the next 8 days, with appropriate behavior on the final 2 days of this phase, at a 90% level.

After 14 sessions in the return to B2, two children (C1 and C2) were returned to the total-treatment-package condition. One child, C3, remained in B2, due to continuing high levels of on-task behavior. In the return to the B2 condition, C4 maintained an average level of 85% appropriate behavior throughout the remainder of the study. With the return to the total treatment package, C1 and C2 again showed increases in on-task behavior, from B2 levels of 46% and 70% to 82% during the re-implementation of the treatment package. Over the same time period, C3's behavior dropped to 61%.

After 6 days of a return to the full treatment package for C1 and C2, the reinforcement component was removed for both, and C3 was reintroduced to all three components of the intervention. C1's behavior dropped slightly from 82% to 80%; C2's behavior improved from 88% to 97%, and C3's behavior in the full self-evaluation package rose from 61% in the last portion of B2 to 93%.

Following 7 days in the self-assessment plus match-with-teacher intervention phase, matching was withdrawn for two children, C1 and C2. At the same time, the reinforcement component was withdrawn for C3. Appropriate behavior remained high for both C1 and C2, with C1 improving slightly from 80% to 82%, and C2 remaining at 96% on-task behavior during self-assessment only. C3, without reinforcement, dropped slightly from 93% to 90% appropriate behavior.

After 6 days in the self-assessment-only phase, two children, C1 and C2, were returned to B2 to examine maintenance of their behavior, and the match-with-teacher portion of C3's treatment was removed. C1's behavior dropped to 71%. After 12 sessions, C1 was returned to self-assessment, where his appropriate behavior rose to a level of 86%, the highest average for C1 throughout the study. C2's on-task behavior remained stable in the final B2 phase; and C3, continuing in self-assessment, dropped slightly to 88% appropriate behavior. Finally, C3 entered another B2 phase with no evidence of deterioration in the level of appropriate behavior.

Accuracy of Child Self-Assessments: Matching With the Teacher

To determine the accuracy of children's self-assessments, an analysis was made of the match between teacher ratings and children's self-assessments on the Independent Seatwork Rating Scale. A "match" was counted if both the child and the teacher rated any item as either "yes" or "no." The mean percent agreement between child and teacher ratings for the initial, full-package phase ranged from 60% for one child, C4, to 74%, 93%, and 95% for three children (C1, C2, and C3, respectively). The second opportunity for children to self-assess and to compare these assessments with the teacher (the second full-package phase) found children's accuracy to be much improved, with C1 having a mean percent agreement of 96% and C2 and C3 matching with the teacher 100% of the time. After the reinforcement component was withdrawn from the self-evaluation package, the mean percent agreement between child and teacher ratings for C2 and C3 remained at 100%, whereas the match between self-ratings for C1 and the teacher dropped slightly to 91%.

Teacher Prompts

The frequency of teacher prompts for child target behaviors was examined to determine if the self-evaluation package reduced the amount of teacher attention necessary to maintain children's appropriate behavior (see Table 1). During the initial baseline condition (B1), teacher prompts were not restricted and ranged from a mean number of 12.2 for C2 to 25.2 for C4. In the second baseline condition (B2), teacher prompts were limited and ranged from a mean number of 3.7 to 5.1 across all subjects. (Teachers were asked to limit themselves to two prompts; however, they could answer children's questions or deal with serious disruptive behavior.) With the implementation of the total self-evaluation package, teacher prompts ranged from 5.6 for C3 to 8.9, 8.1, and 11.3 for C1, C2, and C4, respectively. During the withdrawal of the complete self-evaluation package (B2), teacher prompts ranged from 2.9 to 3.1 for all children. With the reintroduction of the self-evaluation package teacher prompts fell to 2.2 for C3 and rose to 7.2 for C1. Subsequently, the withdrawal of the reinforcement component found the mean number of teacher prompts ranging from 5.5 prompts per session for C1 to 6.6 prompts per session for C2. When children were no longer required to match with the teacher, rates of teacher prompting rose slightly for C1 (6 to 7.8) and fell slightly for C2 and C3. The final baseline condition (B2) yielded levels of teacher prompting at 6.9, 6.2, and 6 for C1, C2, and C3, respectively.


In summary, the results of this study indicate the following: * The four children's appropriate behavior was

initially maintained by high levels of teacher

behavior. * When teachers were asked to reduce their

heavy prompting of appropriate behavior to

normative kindergarten levels, all children's

appropriate behavior decreased to

unacceptable levels. * The introduction of the three-component

self-assessment package produced

immediate and substantial improvements in child

behavior. * It was possible to systematically reduce the

components of the package to the most

minimal level of complexity and effort. * Once the self-assessment repertoire was

taught, teachers did not need to resume their

intensive monitoring of appropriate behavior

for intervention effects to be maintained. In a practical sense, the use of the self-assessment package replaced the already effective, but inefficient, use of teacher prompting. If we sum the number of prompts that the teacher was giving to these four children alone, then we see that intervention to control appropriate/inappropriate behavior occurred once every 10 s. To extrapolate to the larger group, the teacher was engaging in managerial behavior once every 5 s or less. We consider this level of managerial effort to be inefficient because it must, by definition, preclude extended episodes of teaching cognitive, communicative, or social skills. By the end of the study, the teacher was engaged in approximately one-third of the behavior management effort to produce the same, if not better, outcome. It remains for further study to discover if, in fact, the added time for teaching is used toward such an end.

The specific evaluation of the self-assessment package expands prior study in several directions. First, the successful use of the package is, to our knowledge, unique to this age and disability group. Prior study methodology on self-assessment was altered in three ways to accommodate our target group. First, children were provided with a referent (pictures of themselves doing what they should be doing) for appropriate behavior. Second, children were not required to recall and report specific incidents to validate their assessments. The marking of the "Yes-No" faces thus reduced the communicative and cognitive demands often incorporated in self-assessment procedures with older children and adults. Finally, the target behavior under study was chosen because it was easily, though certainly not readily, performed by the children. An interesting avenue for further exploration is the use of the package for skill acquisition in more complex instructional domains. Currently, we have piloted the package with good results in teaching new social skills (Sainato, Goldstein, & Strain, 1989).

Efforts to reduce the components of the package were quite successful. For C1, C2, and C3 there was good maintenance of appropriate behavior above B2 levels. For C2 and C3 it was even possible to eliminate the entire package (final B2 phase) and maintain high levels of appropriate behavior. The case for C4 was mixed. Clinically, results were excellent in that he never returned to baseline levels following the initial full-package phase. Experimentally, this effect precluded a clear functional analysis of the package with C4. Though it is impossible to know the exact nature of the variables responsible for C4's high level of appropriate behavior over the second B2 phase, we noted that he was overheard making many nonprompted, nontaught "self-statements," including "I do good," and "I work neat." Interestingly, other researchers have reported similar findings in which verbal self-mediation facilitated the responding of children with autism--as well as children without disabilities--across a variety of tasks (Egel, Shafer, & Neef, 1984; Koegel, Dunlap, Richman, & Dyer, 1981).

In summary, the self-evaluation package represents a promising technology for assisting preschool children with disabilities to perform more independently and appropriately.

One of the limitations of this study is the absence of generalization data for the treatment effects on the students' seatwork behavior. Future research might investigate this aspect both within the special education classroom and the transfer of such behaviors from special to regular education. In addition to the question of generalization, additional investigations regarding the practicality of this methodology might focus on the ability of classroom teachers to implement these procedures with other disability groups and more complex target behaviors. The self-evaluation package may be initially more time consuming for teachers to implement; but once children have learned the procedures and are working more independently, teachers may be more free to provide instruction. [Figure 1 Omitted] [Tabular Data 1 Omitted]

DIANE M. SAINATO is Assistant Professor, The Ohio State University, Columbus. PHILLIP S. STRAIN is Associate Professor of Psychiatry and Special Education and Director of the Early Childhood Intervention Program, Western Psychiatric Institute and Clinic, University of Pittsburgh, Pennsylvania. DANIEL LEFEBVRE is Educational Consultant, West Hartford, Connecticut. NANCY RAPP is Developmental Specialist, Western Psychiatric Institute and Clinic, University of Pittsburgh, Pennsylvania.
COPYRIGHT 1990 Council for Exceptional Children
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1990 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Sainato, Diane M.; Strain, Phillip S.; Lefevbre, Daniel; Rapp, Nancy
Publication:Exceptional Children
Date:Apr 1, 1990
Previous Article:Early identification of developmentaly disabled and at-risk preschool children.
Next Article:Sometimes patent medicine works: a reply to Braaten, Kauffman, Braaten, Polsgrove, and Nelson.

Related Articles
1990 school mainstreaming contest winners.
Preschool mainstreaming contest winners, 1992.
Understanding early warning signs.
The impact of inclusion on language development and social competence among preschoolers with disabilities.
Teaching a child with autism to share among peers in an integrated preschool classroom: acquisition, maintenance, and social validation.
One-on-one Tx for autism trumps mainstreaming.
Learning disabilities and young children: identification and intervention: a report from the National Joint Committee on Learning Disabilities *...
Social skills intervention for a child who is blind.

Terms of use | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters