Program evaluation through the eyes of a child.

Over 20 years of evidence proves the important benefits of children's participation in high-quality early childhood programs (Ryan, 1974; Schweinhart, Barnes & Weikart, 1993). Such programs are characterized by the following conditions:

* the program is based on an understanding of child development

* the program is individualized to meet the needs of every child

* the physical environment is safe and orderly, and it contains varied and stimulating toys and materials

* children may select activities and materials that interest them, and they learn by being actively involved

* adults show respect for children's needs and ideas and talk with them in caring ways

* parents feel respected and are encouraged to participate in the program (Koralek, Colker & Dodge, 1993).

If high quality is to be maintained, however, program evaluation must be ongoing and systematic. The above indicators reflect complex dimensions of programs, and should serve as the foundation for assessing program quality. Although outside experts can evaluate programs, self-study allows early childhood educators to determine the quality of their own programs, serves as a vehicle to improve or update programs and promotes awareness of what constitutes high quality (Herr, Johnson & Zimmerman, 1993).

Early childhood educators can use a number of tools for program review. For example, the Early Childhood Environment Rating Scale (ECERS) can be used by classroom teachers or program administrators to measure seven areas related to child care quality: 1) furnishings and display, 2) personal care routines, 3) language/reasoning experiences, 4) social development, 5) fine and gross motor activities, 6) creative activities and 7) adult needs. ECERS, often used by teachers and administrators, rates items on a scale from 1 (inadequate) to 7 (excellent). Teachers using ECERS report that they gain an objective impression of their program's quality. The descriptive nature of the scale enables staff to discern what actions can be taken to improve quality, thus providing a focus for staff training (Harms & Clifford, 1980; Kontos & Stevens, 1985).

Since its publication in 1984, the Accreditation Criteria and Procedures of the National Academy of Early Childhood Programs has increasingly been used by early childhood educators to validate the quality of their programs. The quality components addressed in the criteria include the following: curriculum, administration, health and safety, physical environment, staff qualifications and development, staff-children interactions, staff-parent interactions, evaluations, staffing, and nutrition and food service. Educators can determine the extent to which their center meets the criteria through a self-study process that involves parent and staff surveys, classroom observations and gathering of program-related data.

Adding the Child's Point of View to Program Evaluation

Children are in a unique position to judge the quality of their early education programs, and hence can be an important source of information. One study demonstrated that children as young as 4 can reliably evaluate teacher quality. It is interesting to note that, based on responses, 4-year-olds' perceptions of teaching and school differed from those of kindergarten and 1st-grade students (Driscoll, Peterson, Browning & Stevens, 1990). A program's effectiveness ultimately depends on how it is viewed by children (Katz, 1992).

Both teacher and program evaluation need multiple lines of evidence (Katz, 1993; Peterson & Kauchak, 1982). While some links between traditional measures of program quality and program effectiveness exist (Harms & Clifford, 1980; Howes, Phillips & Whitebook, 1992; Phillips, 1987), Katz asserts: ". . . the important ultimate effects of a program depend primarily on how it is viewed from below" (1992, p. 67); that is, how the program is subjectively experienced by each child. This bottom-up perspective ". . . requires answers to the central question, 'What does it feel like to be a child in this environment?'" (p. 67). Katz provides a list of more specific questions to help understand children's perceptions of their experiences and interactions (Figure 1).

Directly interviewing young children may, as Katz (1992) points out, be unreliable and ". . . pose ethical issues related to provoking them to criticize their caretakers and teachers" (p. 71). Using this preliminary study, we decided to explore the feasibility of gaining preschool children's perceptions. Additionally, we make a case for including this line of evidence in program evaluation.

Gaining the Bottom-Up Perspective

We used the Western Illinois University Infant & Preschool Center as our study site. The Center is licensed by the Department of Children and Family Services and serves 38 children from 6 weeks to 6 years of age. The curriculum is designed to allow children to explore and manipulate the environment through meaningful interaction with adults, other children and materials.

Rather than using direct interviews to gain the children's perspectives, we chose two observational methods: videotaping and direct observation. Video-taping efforts focused on learning experiences/centers, while direct observation focused on individual children. Since teachers and children in this laboratory site are frequently observed and videotaped, we considered these methods to be relatively unobtrusive.

Videotaping

Because data gathered over extended time periods is of greater validity, we videotaped over a 4-week period. Videotaping was conducted in the preschool room during three half-hour time slots. We selected times based on the nature of the questions we were asking. For example, we taped arrival and departure times to help us answer the questions "Do I usually feel welcome, rather than captured?" and "Am I usually glad to be here, rather than eager to leave?" We also videotaped child-initiated activity times in order to gather data regarding the adult-child and child-child interactions. As a validity check, we asked teachers to verify that videotapes contained representative classroom experiences. Because the videotapes did capture typical arrival, departure and activity periods, we believe that valid generalizations can be made regarding children's perspectives on their experiences.

Welcomed and glad to be here. Some of the questions suggested by Katz could be answered with a yes/no response, including the two mentioned above. We concluded that the children usually did feel welcome and glad to be present, as the taped arrivals and departures showed us that:

* each child was greeted individually by the teachers

* children saw their parents being welcomed by the teachers

* children were allowed time for parental "leave-taking"

* children continued with activities after parents arrived for pick up

* children shared their day's activities with parents

* children prepared to leave in a leisurely manner.

During early morning arrival times, the teacher consistently found time to greet each family. When greeting the children, she would bend, kneel or sit down to establish eye contact. For children experiencing difficulty separating from their parents, the teacher would participate with the children in leave-taking rituals that included waving good-bye from the classroom door; at times she would even walk with the child and parent to the outside door and wave good-bye until the parent was out of sight. Afterwards, children were allowed to become gradually involved in activities at their own pace.

At the close of the day when parents returned, children typically finished their activities and put any materials away before preparing to leave. Children often invited their parents to observe and even participate in activities. One child, for example, eagerly showed his father a new computer game before getting his coat from his locker. Parents and teachers were very tolerant and accepting of these various end-of-day activities. One father brought in a set of photographs to show his son and the teacher. Another waited patiently, chatting to the teacher and other parents while his daughter performed a lengthy dance to, as she described it, "get the giggles out."

Protected and understood. In general, we were able to answer questions regarding interactions with adults and other children by analyzing segments taped during activity time. We found that children usually did feel accepted, understood and protected, as indicated by the following:

* children were encouraged to pursue their own interests

* children's creations were valued

* children's rights were protected by adults

* children's displays of affection were reciprocated

* children's needs, even when inconvenient for adults, were understood and respected.

Throughout the child-initiated activity period, children engaged in a variety of individual pursuits. The videotaped segments showed that adult support was an important factor in children feeling accepted and safe. In one episode, several boys were flying paper airplanes and comparing how far they could fly. A girl attempted to enter their play, saying "I want to fly one." The teacher responded, "Let's make one. I want to learn how, too!" A little later, they all marveled when one child's plane flew into the hallway. The teacher said, "Wow! That one went a long way!"

One afternoon during departure time, both teacher and parent took time to look for a child's lost hairbow. When it was not recovered, the distressed child said, "It's lost." The teacher responded, "Oh, no. It's not lost. It's just misplaced. We'll find it and put it in your locker." The child accepted this response and was then able to leave with her mother.

Another episode involved a 4-year-old boy who was seated at a table with a male university student. When the student told the child that he was leaving, the child immediately stood, threw his arms around the young man's neck and gave him a hug and a kiss on the cheek. The student smiled broadly, returned the hug and put his hand out for the child to "give him five." These reciprocated displays of affection were commonplace, leading us to infer that children felt a high level of acceptance and protection.

Taken seriously. In addition to being accepted, understood and protected, we observed that children were addressed seriously and respectfully, as evidenced by the following:

* children's creations evoked interest from adults

* adults accepted children's invitations to join in play

* children controlled their own play, even when joined by adults.

A noteworthy example occurred when one child, having created a table top zoo scene using model plastic animals, called to the teacher, "Get your animals." The teacher joined him and appeared to be very interested in his zoo. As the boy directed the play, telling his teacher when and how to make the animals move, she followed his directions in a serious manner.

Accepted. Observations of interactions led us to conclude that children felt they belonged and were accepted by the majority of their peers, as well as by the adults in the room. We witnessed many occurrences of the following types of interactions:

* children receiving individual attention from the teachers

* children being openly friendly toward one another

* children interacting with parents other than their own

* children issuing invitations to each other for play

* children taking interest in other children's products.

Children anticipated the arrival of their friends. On one occasion, a child seated at a computer called out "Come and see this" to his friend who was entering the room with his mother. He then moved his chair over to make room for his friend to sit. At another time, a child was showing his mother the class photo album. Another child joined them, pointing out various photographs. After her son went to another activity, the mother continued to look at the album with the second child.

Engaged and challenged. The remaining questions in Figure 1 seem to require answers that fall along a continuum, rather than simple yes/no responses. Therefore, we developed a Child Observation Form consisting of a rating scale and an accompanying system for coding indicators of a child's perspective. We found the observation form particularly helpful for analyzing a lengthy episode, through which answers to most, if not all, the questions could be inferred. We used the form [ILLUSTRATION FOR FIGURE 2 OMITTED] to analyze an activity involving ramps and balls. The teacher had built ramps to varying heights. Children rolled balls down the ramps while the teacher solicited predictions as to how far each ball would roll. After a few attempts, the teacher left the children to experiment on their own. Although a number of children participated in the activity at varying times, one child remained throughout the entire activity period. For this reason, the sample Child Observation Form is based on that child's experience.

Direct Observation

The authors selected three children for direct observation. The children's ages and time spent at the preschool differed - variables that we believed might result in children's different perceptions of similar experiences. Child A was a 3-year-old who was new to the classroom, having recently moved from the infant and toddler room. Child B was a 4-year-old who was new to the center and had not attended any other program. Child C was a 5-year-old who had attended the program since infancy. Each child was observed from arrival through lunch, one morning a week, over a three-week period. We kept detailed, sequential records of each child's activities and the context in which they took place.

Direct observation of both Child A and Child B confirmed the findings and generalizations we gained through analysis of the videotaped segments. The generalizations pertaining to the child's perceptions of and responses to the curricular activities did not hold true, however, for Child C, the 5-year-old with extensive time in the program. This particular child's home environment is rich in materials and opportunities for intellectual stimulation. Both of her parents are teachers, and the child demonstrates high verbal ability. During a one-hour activity period, she spent an average of 10 minutes each in five different activities. When extended involvement did occur (20-30 minutes), the activities in which she was engaged were "low-level" ones. For example, one day she spent 23 minutes with another child "meowing" like a cat. Although she occasionally became absorbed in teacher-initiated activities, during the observation periods she did not ever engage in self-initiated activities that were challenging or that required an extensive time commitment.

The classroom teacher who reviewed our detailed observation records verified the validity of these direct observations, saying that the recorded activities, responses, verbalizations and expressions were typical of the three children observed.

Insights into program quality. The value of obtaining a bottom-up perspective is clear from our own experience with program evaluation. Analysis of the videotapes allowed us to make generalizations about groups of children, as well as individuals within groups. We also gained more in-depth information through direct observation. As we attempted to see our program through the children's eyes, we gained a greater understanding of and appreciation for our program. By adding this perspective to our evaluation process, we gained insight into the meaning that routine events and activities hold for these children. Analysis of the videotapes and observation records led us to infer that the overwhelming majority of our children have positive experiences with the program.

As Katz points out, however, "Answers to the questions listed for the criteria proposed for each perspective can also guide decisions about necessary modifications in the services offered for each child . . ." (1992, p. 71). At our center, for example, the teacher followed up on the analysis of Child C by working extensively to create opportunities to engage and challenge this child. Through conferences with the child, and also with her parents, the teacher learned about the child's areas of interest and, consequently, was able to support the child in selecting more complex and involved projects.

Summary and Conclusions

The benefits of children's participation in high-quality early childhood programs have been well-documented. Ongoing program evaluation is a key to maintaining such high-quality programs. Typically, adults judge a program's quality. In this study, we demonstrated the feasibility of including the children's perceptions of their experiences within a program and, therefore, their perceptions of its quality. When considering the questions posed by Katz (1992) in light of the characteristics of a high-quality program (Koralek, Colker & Dodge, 1993; Schweinhart & Weikart, 1993), we found that children's responses to the questions did provide valuable information about a program's quality. Adding the bottom-up perspective to our self-study process helped us to better understand, and therefore improve, the program.

As more children are placed in early childhood programs, they can become allies in ensuring the high quality of those programs. Early childhood educators who are willing to invest the time to view their program through the eyes of children will agree that children can and should play a central and valued role in program evaluation.

Figure 1

Considering the Bottom-Up Perspective on Quality

1. Do I usually feel welcome, rather than captured?

2. Do I feel that I belong, rather than one of the crowd?

3. Do I usually feel accepted, understood and protected, rather than scolded or neglected by the adults?

4. Am I usually accepted, rather than isolated or rejected by the majority of my peers?

5. Am I usually addressed seriously and respectfully, rather than as "precious" or "cute"?

6. Do I find most activities engaging, absorbing and challenging, rather than just amusing, fun, entertaining or exciting?

7. Do I find most experiences interesting, rather than frivolous or boring?

8. Do I find most activities meaningful, rather than mindless or trivial?

9. Do I find most experiences satisfying, rather than frustrating or confusing?

10. Am I usually glad to be here, rather than eager to leave?

References

Driscoll, A., Peterson, K., Browning, M., & Stevens, D. (1990). Teacher evaluation in early childhood education: What information can young children provide? Child Study Journal, 20(2), 67-79.

Early childhood issues included in many legislative initiatives. (1994). Young Children, 49(1), 59.

Harms, T., & Clifford, R. M. (1980). The early childhood environment rating scale. New York: Teachers College Press.

Herr, J., Johnson, R. D., & Zimmerman, K. (1993). Benefits of accreditation: A study of director's perceptions. Young Children, 48(4), 35-38.

Howes, C., Phillips, D. A., & Whitebook, M. (1992). Thresholds of quality: Implications for social development of children in center-based child care. Child Development, 63, 449-460.

Katz, L. G. (1992). Early childhood programs: Multiple perspectives on quality. Childhood Education, 69, 66-71.

Katz, L. G. (1993). Five perspectives on quality in early childhood programs. Urbana, IL: ERIC.

Kontos, S., & Stevens, R. (1985). High quality child care: Does your center measure up? Young Children, 40(2), 5-9.

Koralek, D. G., Colker, L. J., & Dodge, D. T. (1993). The what, why, and how of high-quality early childhood education: A guide for on-site supervision. Washington DC: National Association for the Education of Young Children.

National Academy of Early Childhood Programs. (1984). Accreditation criteria and procedures of the National Academy of Early Childhood Programs. Washington, DC: KNACK.

Peterson, K., & Kauchak, D. (1982). Teacher evaluation: Perspectives, practices and promises. Salt Lake City, UT: Center for Professional Practice, University of Utah. (ERIC Document Reproduction Service No. 233996.)

Phillips, D. (1987). Quality in childcare: What does research tell us? Washington, DC: KNACK

Ryan, S. (1974). A report on longitudinal evaluations of preschool programs, Volume I, Longitudinal evaluations. Washington, DC: U.S. Department of Health, Education and Welfare.

Schweinhart, L. J., Barnes, H. V., & Weikart, D. P., with Barnett, W. S., & Epstein, A. S. (1993). Significant benefits: The High/Scope Perry preschool study through age 27. (High/Scope Educational Research Foundation Monograph No. 10.) Ypsilanti, MI: High/Scope Press.

Schweinhart, L. J., & Weikart, D. P. (1993). Success by empowerment: The High/Scope Perry preschool study through age 27. Young Children, 49(1), 54-58.

RELATED ARTICLE: Call for Associate Editor

Journal of Research in Childhood Education

The Association for Childhood Education International (ACEI) seeks a field Associate Editor for the Journal of Research in Childhood Education, a scholarly education journal now entering its tenth year of publication and having a circulation of approximately 5,000. Published twice yearly, the Journal features articles that advance knowledge and theory of the education of children, infancy through early adolescence.

Candidates for the associate editorship must: 1) hold current membership in ACEI; 2) be knowledgeable about and experienced in early, middle and later childhood education and education research; 3) be competent in both education writing and editing; and 4) have suitable experience in both the production and editorial facets of publication. A doctorate is preferred. Additional responsibilities involve serving on both the Research Committee and the Publications Committee, attending committee meetings and editing the "Reviews of Research" column of Childhood Education.

Job performance requires that the candidate have an affiliation with either a major university or other education institution and that the institution provide sufficient technical, equipment and communications support.

Starting date (2-year term) is with the Spring/Summer 1996 issue (Vol. 10, No. 2). Applications should be addressed to: Dr. Steven Silvern, Chair, ACEI Publications Committee, 11501 Georgia Ave., Ste. 315, Wheaton, MD 20902. For information, call ACEI Director of Publications at 800-423-3563. Appointment is by the ACEI Executive Board.

Kathy Barclay is Professor, Elementary Education and Reading, and Cecelia Benelli is Assistant Professor, Early Childhood Education, Western Illinois University, Macomb.