Challenges in implementing center-based and home-based early head start programs.

Abstract. Early Head Start services are typically offered through home- or center-based delivery models. A formative evaluation of an example of each service delivery model was conducted. The purpose was to examine the issues involved in the implementation of these two service delivery models relative to the content of services, intensity of services, opportunities for family involvement, and efficiency of program management. Findings suggest that the two delivery models differ in content and intensity of services. Both models faced challenges in program implementation and management and both struggled with family involvement and engagement. Implications for future research are discussed.

**********

A decade ago, the Advisory Committee on Services for Families with Infants and Toddlers and the Advisory Committee on Head Start Quality and Expansion recommended the implementation of an Early Head Start (EHS) program to serve the needs of low-income families with infants and toddlers. Since that time, over 600 Early Head Start programs have begun serving this target population, and 17 of these programs have participated in an extensive national evaluation (U.S. Department of Health & Human Services, 2002).

Findings from the 17 experimental Early Head Start sites have yielded optimistic results regarding the efficacy of the program (U.S. Department of Health & Human Services, 2002). These positive results showed enhanced child and family outcomes for those families who received high-quality and comprehensive services for at least one year. Specifically, the findings indicate that 3-year-old children who had completed the EHS program performed significantly better on a range of measure of cognitive, language, and social-emotional development than control group peers. The parents also performed better on many aspects of the home environments and parenting behavior than randomly assigned control families.

Caution should be used, however, when reviewing these results for programming and policy implications. First, the gains for both children and families were modest in size. Second, these modest gains were obtained from 17 model programs selected, in part, based on their histories of exemplary service provision and full adherence to Head Start performance standards. In addition, the 17 model programs received technical support from the federal level to initiate this new and very complex program. Questions arise as to whether these same results can be replicated with more typical programs that do not have the same level of technical support or evaluation scrutiny and therefore might be struggling much more with issues of program initiation and maturation.

The current literature contains few formative evaluation studies of programs in the initial phases of implementation. This is true despite the fact that formative evaluations have the potential to enhance the services delivered by operating programs and to provide insight to other grantees initiating a program (U.S. Department of Health & Human Services, 2002). All too often, outcome evaluations are conducted with programs during the formative stages when programs are still struggling with start-up issues and obstacles in implementing comprehensive services (Gilliam, Ripple, Zigler, & Leiter, 2000). Having a formative evaluation process is important for maintaining a focus on program goals, collecting information to determine levels of success in achieving goals, and setting targets for improvement.

Although not required to conduct a formal evaluation (as is mandated with the initial programs), EHS programs funded in subsequent years often opted to integrate an evaluation component. Two such programs in New England, one implementing a center-based model and the other providing home-based services, contracted with the University of Rhode Island Head Start Research Team to design and implement a formative evaluation to support program improvement efforts. The formative evaluation was collaboratively designed with a Head Start grantee and examined program delivery, content, and management. Consistent with the recommendations outlined in the Head Start Bureau's Early Head Start Evaluation Summary (U.S. Department of

Health & Human Services, 2002), the Early Head Start Performance Standards served as the basis for the content of the evaluation and the recommended multiple method approach informed the evaluation design. Additionally, the evaluation design was flexible enough to evaluate diverse Early Head Start program models (Golas, Horm-Wingerd, Caruso, & Dickinson, 2001) and enabled its application to evaluate both center- and home-based EHS programs.

Description of the Programs

The two Early Head Start Programs evaluated are both operated by Head Start grantees in New England and began their Early Head Start Programs during the second wave of funding from the U.S. Department of Health and Human Services. Although not required to conduct a formal evaluation, both agencies opted to hire independent evaluators to conduct an evaluation and aid in the continuous improvement component of the Early Head Start model.

The two programs are similar in terms of population demographics, which consist of mostly single, white mothers with a high school diploma or GED. The one major difference between families attending the center-and home-based programs was employment status. The majority of the families enrolled in the center-based program worked part- or full-time, while families enrolled in the home-based program were typically unemployed.

The center-based model had children attend the Early Head Start Center 5 days a week, with at least one home visit per month conducted by the family worker. In addition, teachers visited the home twice during the year to administer developmental assessments and to mentor parents. The center-based model was structured like a full-day care and education program for infants and toddlers and focused on enhancing child development while supporting parents in their role as the child's first teacher. Opportunities for family involvement included evening workshops on resume writing and computer skills, potluck dinners, Weight Watchers' meetings, and organized trips to local playgrounds and museums.

The home-based model required families to complete a weekly, one-hour-long home visit with an assigned home visitor. Each home visit included two components: 1) addressing family needs and concerns, and 2) instructing the family about child development and appropriate learning activities. The Early Head Start program developed a structured curriculum that was to be delivered during the weekly home visits. The curriculum included such topics as health and nutrition, guiding children's social development, discipline, and child-initiated learning activities. Families were expected to attend weekly, three-hour-long family day meetings at the Early Head Start site, facilitated by the mental health or child development coordinators. The infants and toddlers were under the care and supervision of the home visitors while the parents participated in these group meetings. The goal of the family day meeting was to facilitate discussion and brainstorm solutions to the common struggles of participating families. The primary focus of this model was the families' mental well-being and progress toward economic self-sufficiency.

Evaluation Questions

Given that these two models have the same goals, the similarities and differences in their delivery and efficacy need to be examined. The multi-method formative evaluation, collaboratively designed by the University of Rhode Island Head Start Research Team and a Rhode Island-based Early Head Start grantee, provided a mechanism to examine the relative efficacy of the two approaches in achieving the goals of EHS. The need to examine the two service delivery models is especially critical, given that the existing literature contains little information about the efficacy of center- versus home-based service delivery models. For example, the results from the study of the initial 17 model programs examined the efficacy of attending an EHS program compared with peers who did not obtain EHS services. Both program delivery models were compared to non-treated controls. No direct comparison of service delivery models was completed.

The purpose of this study was to address this void by investigating four major questions:

* Do the two delivery models provide the same content of services?

* Do the two delivery models provide the same dose, or intensity, of services?

* Do the two delivery models provide families with opportunities to participate and engage?

* Do the two delivery models provide effective mechanisms for managing Early Head Start services?

Method

Participants

The participants included the families of the infants and toddlers enrolled in the two Early Head Start programs and the staff delivering the services. At the center-based program, teachers and family workers were approached individually by the researchers, who explained the evaluation study and secured their consent to participate. At the home-based program, researchers met with home visiting staff in a group meeting to explain the evaluation and obtained their consent to participate. All Early Head Start staff assisted the researchers in recruiting children and families for participation in the evaluation research. Upon enrollment in the Early Head Start program, families were informed about the evaluation and asked to give consent for themselves and their children's participation.

Participants from the center-based program included three lead and three assistant teachers, a family worker, and the Early Head Start coordinator. The lead teachers had bachelor's degrees and were state-certified early childhood educators. The assistant teachers all had bachelor's degrees in related areas and were all working towards state certification as early childhood teachers. The teachers' role was to design developmentally appropriate curriculum and conduct developmental assessments of the children on an ongoing basis. The family worker had a bachelor's degree in a human service field and some experience working with children and families. The main role of the family worker was to support families in attaining economic self-sufficiency. The Early Head Start coordinator had a master's degree in a human service field and provided overall program oversight.

The majority of the 26 participating center-based families reported they were white, non-Hispanic, and spoke English as their primary language. Approximately half of the participants (53 percent) reported that they were married or living with someone. Eighty percent of the mothers were working, participating in job training, or enrolled in an educational program.

Participants from the home-visiting program included five Early Head Start home visitors and three Head Start coordinators. The five home visitors had college degrees in a human services field and some previous experience working with young children and their families. The home visitors scheduled weekly visits with families in their homes and delivered a curriculum including information on community resources and contacts, child development, child rearing, health, and nutrition. The home visitors also attended to emergency crises, such as spousal abuse, lack of financial resources to cover basic needs, and depression. The three Head Start coordinators had master's degrees and extensive experience in the human service field. The coordinators provided supervision and oversight of the Early Head Start home visitors as well as monitoring other programs offered by the Head Start grantee.

There were 107 participating families from the home visiting program. All families reported that they were white, non-Hispanic, and spoke English as their primary language. The majority were female-headed families, with 77 percent being single, divorced, or separated. Forty percent of the mothers were working, participating in job training, or enrolled in an educational program.

Instruments

A variety of instruments were used to collect information about the participating families and the two service delivery models. A brief demographic survey was designed by the research team and used to collect information about families participating in both programs. When possible, published scales were used to address each of the four research questions of program content, intensity of services, program participation and engagement, and program management. Many of these published scales were the same as those used in the Early Head Start Research and Evaluation Project (U.S. Department of Health and Human Services, 2002), the national-level evaluation of outcomes attained by the initial Early Head Start Grantees. To meet the demands of this local evaluation, the research team customized instruments to more closely examine program attendance and program management procedures--issues typically addressed in formative evaluation studies.

A brief description of all instruments, along with their respective respondents and frequency of distribution, is listed in Table 1. Table 1 categorizes instruments by the four major research questions--program content, intensity of services, program participation and engagement, and program management.

1. To assess the content of services delivered by the two program models, the following published instruments were used: Knowledge of Child Development (Macphee, 1981); Family Environment Scale (Moos, 1986); Family Support Scale (Dunst, Trivette, & Deal, 1988); Maternal Self-Efficacy Scale (Teti & Gelfand, 1991); Infant/Toddler Environment Rating Scale (Harms, Cryer, & Clifford, 1990); and Arnett Scale of Caregiving Behavior (Arnett, 1989).

2. To assess the dose or intensity level of services across the two service delivery models, the following instruments were designed and used by the research team: Center-based Attendance Log, Home-visit Attendance Log, and Home-visiting Family Day Attendance Log.

3. To assess the opportunities for families to participate in and engage across both service delivery models, the following instruments were administered: Family Participation Survey and the Community Contact Log.

4. Relative to the presence of effective mechanisms to manage the delivery of services in both program models, the following instruments were used: Center Director Questionnaire, Barriers to Providing Services, Assessment Checklist, Transition Checklist, Center-based Individualized Development Plan Checklist, Home-visit Plan Checklist, and the Management Plan. The Management Plan included both survey items and interviews and discussion with Early Head Start administrators and staff.

Procedure

Researchers and Early Head Start coordinators initially met to mutually develop and generate appropriate research procedure and protocol. This process included generating the research questions, selecting instruments, establishing a distribution timetable and method for the instruments, and agreeing upon roles and responsibilities of EHS staff and researchers. Consent was secured from the EHS program coordinators for their participation, including completion of the Center Director Questionnaire, Barriers to Providing Early Head Start Services, and the Management Plan. Subsequent meetings were scheduled bi-annually to review procedures and discuss emerging results. A focus of these ongoing meetings was to discuss the implication of the results for continuous improvement planning.

Next, researchers met with the Early Head Start staff of both the center- and home-based programs to review the purposes of the evaluation and to inform the staff of their integral role in the distribution and collection of instruments. Given the nature of the established relationships between EHS staff and the families, the staff were able to serve as liaisons between the researchers and the families. Staff obtained consent from participating families and facilitated the collection of data. Subsequent meetings were scheduled for researchers and EHS staff to exchange information, problem solve any issues, and discuss emerging findings.

Once family members enrolled in the EHS program and gave consent to participate in the evaluation research, they completed the Demographic Questionnaire and the Knowledge of Child Development Survey. Three months later, a second wave of instruments--including the Family Environment Scale, the Family Support Scale, and the Maternal Self-Efficacy Scale--were distributed. Six months after initial enrollment, families completed the Family Participation Survey. All surveys were distributed and collected by EHS program staff. In order to ensure confidentiality, families were provided with coded envelopes in which to seal completed materials prior to returning them to the EHS staff.

After receiving families' consent, researchers collected archival data through accessing EHS program records and the files maintained on enrolled families. Researchers kept records of family and child program attendance, community contacts, and children's developmental assessment results for both the center- and home-based programs. As part of the records review, researchers evaluated the accuracy and completeness of records and program policies.

A large part of the formative evaluation included on-site observations of the two programs. All observational tools were completed by two or more trained observers to control for observer bias. Training included practice with the instruments in Early Head Start settings until an acceptable inter-observer rating was achieved (r = .95). Observers met bi-monthly to review observations and address any questions regarding observational techniques. To obtain a more accurate picture of the Early Head Start service delivery, observational instruments were completed over the course of a year and based on multiple visits to complete each checklist.

Trained observers conducted observations in classrooms for the center-based program and during home visitor case review meetings for the home-based program. These observations allowed researchers to complete several of the program management checklists and complete the Infant Toddler Environment Rating Scale (ITERS). In addition, the observations provided a format for the collection of anecdotal and qualitative information about the programs.

Results

A combination of quantitative and qualitative analyses was used to investigate the four main research questions:

* Do the two delivery models provide the same content of services?

* Do the two delivery models provide the same dose, or intensity, of services?

* Do the two delivery models provide families with opportunities to participate and engage?

* Do the two delivery models provide effective mechanisms for managing Early Head Start services?

These questions will be used to organize the presentation of the results.

Do the two delivery models provide the same content of services?

Descriptive statistics were used to summarize the scores on the Infant/Toddler Environment Rating Scale (ITERS), the Arnett Scale of Caregiving Behavior, the Individualized Development Plan Checklist (IDPC), and the Home Visit Plan Checklist (HVPC). Using field notes recorded at staff meetings, the authors used the constant comparison technique to identify patterns and trends regarding staff concerns with the quality and content of services.

The available evidence suggests that different content was delivered in the two programs. This is not a surprising finding, given the underlying philosophies of the two service delivery models. Although enhanced child outcomes was a common goal, the center-based program implemented a child-focused approach while the home-based program concentrated on facilitating child outcomes through family-focused services.

The center-based classrooms were rated as demonstrating "minimal to good" quality, and were staffed by caregivers who demonstrated warmth and compassion. An overall rating of 3.8 (out of 7) was found, across classrooms, on the ITERS. The Arnett Scale indicated that the caregivers staffing these classrooms were warm, compassionate, and caring individuals. The average rating on the Arnett Scale was 3.0 out of 4. Based on the research team's classroom observations and records review using the IDPC, a discrepancy was found between the individualized assessment records and the goals and activities for individual children. Although the IDPCs were complete and adequately detailed, there was no indication that this information was used in planning curriculum or individualized activities for individuals or the group of children. The lowest scores on the ITERS were typically found in the learning activities section, where the classrooms received an overall rating of 3. These findings suggesting "minimal" quality in learning activities, combined with the observed disconnect between assessment and planning, became the focus of improvement efforts, as reflected in the Management Plans.

Similar to the classroom personnel, the home visitors staffing the home-based program received average ratings of 3.5 out of 4 on the Arnett Scale, indicating that they were warm and compassionate. Based on weekly observations of the home visitors' case review sessions, the HVPC, and interviews with home visit staff, it was found that only one of the two intended goals was generally met in the 1-hour-long home visit. Home visit staff continually expressed concerns over not being able to address the child development issues, as intended in the home visit curriculum. The vast majority of home visits concentrated on family issues and crises and typically there was limited coverage of child development topics. Likewise, the family day meetings were focused on family mental health issues, with little focus on child growth and development.

Do the two delivery models provide the same dose, or intensity, of services?

Basic descriptive statistics were used to illustrate families' adherence to schedules and compliance with program expectations. The center-based program included a full-day early childhood experience, 5 days per week. Records indicate that, on average, children attended approximately 80 percent of the time. In this model, which also included monthly hour-long home visits, families received direct services 80 percent of the time. Additionally, 100 percent of the families completed at least one year in the program.

For the home-based program, records indicate that families typically received 74 percent of the scheduled weekly hour-long home visits. Average monthly family day attendance was variable, with families attending between 20 percent and 65 percent of the available sessions. The average attendance rate at family day hovered near 36 percent over an 18-month period. Among participants who terminated their involvement in EHS, the average family day attendance was even lower, at 21 percent. Attrition in the home-based program was very high, with 45 percent of families terminating prior to program completion and 25 percent of the families failing to complete at least one year of programming.

As described above, families enrolled in the center-based program demonstrated higher levels of compliance with program expectations and activities, whereas the home-based families showed less compliance and adherence to program schedules and activities. Thus, the doses appear to be very different across the two service delivery models. Of particular interest is the difference in attrition rates that show families enrolled in the home-based program often did not receive one full year of services--the minimum considered necessary for change (Olds et al., 1998).

Independent t-tests comparing scores from measures at time of entry and one-year post-test demonstrated that families involved in the program for at least one year showed gains on several measures. Families involved in both programs who completed at least one year showed gains on the Family Support Scale, indicating that their perceived levels of support increased. A similar result was found across the two program models on the family support subscale of the Family Environment Scale. Across both programs, no gains were found on the Maternal Self-Efficacy Scale. Families entered the programs scoring high on this scale and the high scores persisted. These results were common across both program models. Additionally, families enrolled in the center-based program for at least one year showed gains on some components of the Knowledge of Child Development Scale (knowledge of social and emotional development) and the Family Environment Scale (enhanced family functioning, personal growth, and improved family relationships).

Do the two delivery models provide families with opportunities to participate and engage?

Descriptive statistics were used to describe data from the Family Participation Scale, and the constant comparison technique was used to code anecdotal information provided by the staff regarding families' engagement with the program. Both programs offered opportunities for family involvement. Families enrolled in both programs did not report obvious barriers that prevented their participation in planned activities. However, the staff of both programs struggled with implementing the family involvement program component.

The Family Participation Scale indicated that families reported positive sentiments about their potential participation in the two programs. They reported that a high level of comfort with staff, convenient scheduling, diverse and enjoyable activities, comfortable opportunities to meet others, and the availability of child care supported their potential participation. For the center-based program, less than 1 percent of the families reported participating in family activities beyond the mandated monthly home visits. The one area of involvement reported by center-based families was volunteering in classrooms. Although this is helpful in forging partnerships among families and program staff and in providing parents additional time to be with their children, serving as a classroom volunteer does not offer the full range of opportunities embraced in the Head Start concept of family involvement. For home-based families, as documented above, results on the Home Visit Family Day Attendance Log showed that family attendance at family involvement activities was minimal.

Although both programs offered numerous opportunities for families to participate, staff and families reported a lack of engagement in family activities. The center-based families reported that a lack of flexibility in their work schedules prevented them from participating. Staff suggested that the mental health of the home-based families was a major contributor to their lack of attendance.

Do the two delivery models provide effective mechanisms for managing Early Head Start services?

Several instruments were used to detail if the programs were meeting Early Head Start Performance Standards and what barriers existed that prevented full adherence to these standards. Descriptive statistics were used to summarize the information from these instruments. According to the self-report of program coordinators on the Center Director Questionnaire, both programs were effective in meeting health and safety and parent communication standards. The Transition Checklist suggested both program models had adequate policies regarding transitioning children and families to other appropriate programming. The Assessment Checklist suggested that both programs had assessment policies in place, but that the available documentation did not provide a convincing link between the assessment information and planning in either program model. The Individualized Development Plan and developmental assessment tools used in the center-based program did not provide a basis for individualization or classroom planning. The assessment tools used in the home-based model were child-focused, but as described above, the program was family-focused. Thus, the available assessment information was unrelated to the implemented home-based programming.

Although challenges were evident in the administration of both program models, the home-based model appeared to present more challenges in ensuring effective program management. The center-based program had written policies about program philosophy and the delivery of services to children, whereas the home-based program lacked written policies in several areas, including proper cleaning and disinfecting, acceptable methods of guiding children's behavior, food preparation and handling, diapering and toileting techniques, and infant feeding techniques. Relative to barriers, staff of the center-based program reported having enough personnel to provide services and effective use of community resources to address the needs of families. The staff of the home-based program indicated that they had "somewhat less than enough" qualified staff to provide adequate education and social services and "far less than enough staff" to provide mental health and disability services. Additionally, staff reported that the home-based program did not effectively use community agencies to provide necessary services for enrolled children and families.

Discussion

This study of two Early Head Start service delivery models found a number of clear differences between the center-based and the home-based approaches. The most striking of these involve the content and intensity of the services delivered by the programs. The center-based program implemented a child-focused early childhood classroom model with little emphasis on family issues, whereas the home-based program focused largely on family and parenting issues and adult mental health. With regard to intensity, the children in the center-based program attended 30 to 40 hours of classroom-based programming per week and 100 percent of the families completed at least one year in the program. The home-based program engaged parents in an average of three 1-hour-long home visits per month. This is substantially fewer hours of programming, with the content being adult-focused, rather than child-focused. The home visitors reported that they were often not able to deliver the intended child development curriculum due to the pressing needs of the family, such as maternal depression, financial concerns, housing issues, and a myriad of other stressful situations often encountered by families living in poverty. On another indicator of intensity, length of participation, the two program models again differed substantially. As noted above, all center-based families completed one year in the program. However, attrition was very high among the home-based participants, with 45 percent of the families terminating prior to program completion and 25 percent failing to complete even one year. The differences found in this study regarding the core program attributes of content and intensity are far greater than one would expect, given that the Head Start Performance Standards direct all Early Head Start programs to meet common cognitive, social, and health child outcome objectives.

One program component on which the two programs were relatively similar was family involvement. Both the center-based and home-based programs offered family involvement opportunities, in addition to their basic program delivery modalities of staff home visits or child classroom attendance. Even though both programs offered numerous opportunities for families to be involved, participation in these activities was very low. The 1 percent of center-based families who engaged with the program did so by volunteering in their child's classroom. While this is valuable, it did not expose these families to the rich array of adult development activities that Head Start typically advocates. Due to welfare-reform requirements, these families were devoting significant hours per week to work or school. Thus, intense participation in family-oriented programming delivered during typical work hours did not fit their schedules. Arranging family involvement activities in the evening also posed scheduling difficulties. After the adults devoted long hours to work, and children spent long hours in center-based care, neither group was anxious to spend more time at the program in the evenings.

For participants in the home-based program, attendance at family involvement activities was minimal. The mental health issues that many of the home-based families experienced not only dominated the home visit but also impacted the home-based parents' ability to fully engage with the program and participate in expected activities. This result suggests that greater attention to adult mental health issues may be required for the home-based service delivery model to achieve its intended impact.

This study also examined program management issues within these different program models. A finding common to both the home-based and center-based programs was the failure to utilize child assessment data to support program planning. While both programs collected significant data on child development in a variety of domains, no evidence existed showing that this information provided a basis for planning for individuals, classrooms, or home visits. This result is congruent with Horton and Bowman's (2002) finding that, in early childhood programs, while the practice of assessment "has expanded significantly over the past 5 years, there is not a widespread understanding that assessment must be integrated with curriculum and used to inform its development in order to be fully meaningful and effective" (p. 21).

The program management results of this study also highlighted a series of problems in managing the home-based program, given the multiple demands of the families served and the crisis intervention required of program staff, based on the issues that the families faced. The findings suggest that the severity of the family functioning and adult mental health issues demonstrated by the participating families were beyond the level of training and expertise of most of the program administrators and the service delivery staff. Findings from the nurse home visiting literature echo the need to employ well-trained professionals to achieve optimal program implementation and outcomes (Korfmacher, O'Brien, Hiatt, & Olds, 1999; Olds, Kitzman, Cole, & Robinson, 1997).

Several other factors should be considered in future programming and research. The difficulty that these programs had implementing a successful family involvement component is a significant issue for the Early Head Start model. It is interesting to examine the demographics of the programs in this study. The home-based program enrolled families that were largely unemployed, whereas the center-based program had a mostly employed population. This finding could be related to welfare reform requirements. For example, the much higher attrition experienced by the home-based program could be related to welfare reform requirements mandating people to enter the workforce or job training. When home-based families found employment, they needed child care and also were no longer available for home visits during the day. Therefore, their children were moved to a center-based program or removed from Early Head Start, resulting in attrition from the home-based program. Likewise, center-based families (which tended to be a mostly employed population) had scheduling restrictions that made program participation unachievable.

A related consequence of welfare reform is that over time, the home-based population consists of more families that have significant mental health and other functional issues that prevent them from moving into the labor force. These same challenges make it difficult for them to benefit from the various components of the home-based program, especially the family involvement component. These findings obtained with this Early Head Start sample are consistent with those reported in the general home visiting literature. For example, Olds et al. (1997) found that major mental illnesses of some home visiting participants hampered program adherence and outcomes. Such risk factors as low literacy levels (Kitzman, Cole, Yoos, & Olds, 1997) and participants' lack of resources (Korfmacher, Kitzman, & Olds, 1998) also have been found to contribute to the multiple challenges of fully implementing successful home visiting programs.

It should be noted that this study involved two Early Head Start programs within their first few years of funding. It is clear from the findings, however, that the two different delivery models, while ostensibly focused on the same Early Head Start objectives, are based on different philosophies that result in significantly different experiences for participants. It seems reasonable to suppose that such variation in program content and intensity has the potential to result in significantly different child outcomes. These major program differences need to be examined more thoroughly in outcome studies of mature programs rather than with programs in their first few years of operation. Such studies would have the potential to inform policy decisions regarding the difficult issues that Early Head Start grantees must face and overcome to achieve program maturation and effectively deliver a comprehensive and high-quality program for children and parents facing significant economic and social challenges.

References

Arnett, J. (1989). Caregivers in day-care centers: Does training matter? Journal of Applied Developmental Psychology, 10, 541-552.

Dunst, C. J., Trivette, C. M., & Deal, A. G. (1988). Enabling and empowering families: Principles and guidelines for practice. Cambridge, MA: Brookline Books.

Gilliam, W. S., Ripple, C. H., Zigler, E. F., & Leiter, V. (2000). Evaluating child and family demonstration initiatives: Lessons from the comprehensive child development program. Early Childhood Research Quarterly, 15(1), 41-59.

Golas, J. C., Horm-Wingerd, D. M., Caruso, D. A., & Dickinson, L. (2001). The power of partnerships: Lessons learned from the C.H.I.L.D., Inc./University of Rhode Island Research Team Partnership. National Head Start Association Dialogue, 5(2), 149-159.

Harms, T., Cryer, D., & Clifford, R. (1990). Infant/toddler environment rating scale. New York: Teachers College Press.

Horton, C., & Bowman, B.T. (2002). Child assessment at the preprimary level: Expert opinion and state trends. Chicago: Herr Research Center/Erikson Institute.

Kitzman, H. J., Cole, R., Yoos, H. L., & Olds, D. (1997). Challenges experienced by home visitors: A qualitative study of program implementation. Journal of Community Psychology, 25(1), 95-109.

Korfmacher, J., Kitzman, H., & Olds, D. (1998). Intervention processes as predictors of outcomes in a preventative home-visitation program. Journal of Community Psychology, 26(1), 49-64.

Korfmacher, J., O'Brien, R., Hiatt, S., & Olds, D. (1999). Differences in program implementation between nurses and paraprofessionals providing home visits during pregnancy and infancy: A randomized trial. American Journal of Public Health, 89(12), 1847-1851.

Macphee, D. (1981). Knowledge of Child Development Inventory. Unpublished manuscript, University of North Carolina.

Moos, R. (1986). Family Environment Scale Manual (2nd ed.). Palo Alto, CA: Consulting Psychologist Press.

Olds, D., Henderson, C. R., Jr., Kitzman, H., Eckeurode, J., Cole, R., & Tatelbaum, R. (1998). The promise of home visitation: Results of two randomized trials. Journal of Community Psychology, 26(1), 5-21.

Olds, D., Kitzman, H., Cole, R., & Robinson, J. (1997). Theoretical foundations of a program of home visitation for pregnant women and parents of young children. Journal of Community Psychology, 25(1), 9-25.

Teti, D. M., & Gelfand, D. M. (1991). Behavioral competence among mothers and infants in the first year: The mediational role of maternal self-efficacy. Child Development, 62, 918-929.

U.S. Department of Health & Human Services. (2002). Building their futures: How Early Head Start programs are enhancing the lives of infants and toddlers in low income families [Summary report]. Washington, DC: Author.

Correspondence should be directed to Julianna Golas, Department of Human Development and Family Studies, 2 Lower College Road, University of Rhode Island, Kingston, RI 02881-0818; 401-874-2150.

Julianna C. Golas

University of Rhode Island

Diane Horm

University of Oklahoma-Tulsa

David A. Caruso

Antioch University New England 

Table 1
Early Head Start Instruments

Instrument Description                          Respondent   Frequency

Instruments to assess the content of services

Knowledge       An adaptation of Macphee's      Parent and   Annually
of Child        original 56-item survey         Staff
Development     measuring respondents'
(Macphee,       knowledge of children's
1981)           social, emotional, cognitive,
                and physical development. The
                adapted inventory consisted
                of 20 items, 4 questions from
                each developmental domain.
                The total score was
                determined by the number of
                correct items over the total
                number of items.

Family          Ninety true/false items         Parent       Annually
Environment     measuring the social and
Scale           environmental characteristics
(Moos, 1986)    of families. The scale is
                based upon a 3-dimensional
                conceptualization of
                families. Families are
                characterized as having
                either a healthy or unhealthy
                family relationship, personal
                growth, and system
                maintenance.

Family          A 21-item self-report measure   Parent       Annually
Support         designed to assess the degree
Scale           to which different sources of
(Dunst,         support have been helpful to
Trivette, &     families with young children.
Deal, 1988)     Ratings were made on a
                5-point Likert scale, ranging
                from not at all helpful" to
                extremely helpful."

Maternal        A 10-item self-report measure   Parent       Annually
Self-Efficacy   designed to assess the
Scale           behavioral competence of
(Teti & Gel-    mothers with infants in the
fand, 1991)     first year. Items included
                the mother's self-efficacy
                beliefs in relation to
                specific areas of infant
                care, such as performing
                daily routines, soothing a
                child, or engaging the baby.
                Ratings were made on a
                5-point Likert scale, ranging
                from "not good at all" to
                excellent.

Infant/         A 35-item rating scale used     Researcher   Quarterly
Toddler         to measure the quality of the
Rating Scale    infant/toddler caregiving
(Harms,         environment. The rating scale
Cryer, &        is divided into 6 subscales
Clifford,       (personal care routines,
1990            listening and talking, adult
                needs, learning activities,
                interaction items, and
                program structure). Each item
                is rated on a 7-point Likert
                scale, ranging from "high
                quality" to inadequate
                quality.

Arnett Scale    A 13-item rating scale used     Researcher   Quarterly
of Caregiving   to rate the warmth and
Behavior        attentiveness of the infant
(Arnett,1989)   toddler caregivers. Each item
                is rated on a 4-point Likert
                scale, ranging from "high" to
                "low."

Instruments to assess the dose and intensity of services

Center-based    Log to document child           EHS Staff    Daily
Attendance      attendance at the center-       and
Log             based program. The monthly      Researcher
                percentage of time a child
                attended the center was
                calculated by the number of
                days the child attended the
                center, divided by the total
                number of days the center was
                open.

Home Visit      Log to document the center-     EHS Staff    Daily
Attendance      based and home-based            and
Log             families' adherence to the      Researcher
                home visit schedule. A
                monthly percentage of home
                visits was determined by the
                number of completed visits,
                divided by the total number
                of scheduled visits.

Home-Visiting   Log to document the home-       EHS Staff    Daily
Family          based families' participation   and
Day             in Family Day. Monthly Family   Researcher
Attendance      Day participation was
Log             calculated as a percentage of
                the number of days a family
                attended Family Day, divided
                by the total number of
                scheduled Family Days.

Instruments to assess the opportunities for families to participate
and engage

Family          Created by the research team,   Parent       Annually
Participa-      the FPS is a multi-formatted,
tion Survey     self-report instrument for
(FPS)           parents to document their
                level of participation in
                the EHS Program. A checklist
                portion allowed parents to
                indicate the nature of their
                involvement, and a 5-point
                Likert scale, ranging from
                never to "always," allowed
                parents to rate their level
                of comfort while

                participating.

Community       EHS staff were required to      EHS Staff    Daily
Contact Log     document any communication      and
                they had with the EHS family    Researcher
                or regarding the EHS family.
                The Community Contact Log was
                designed and maintained by
                the researchers to tally the
                number of meaningful
                communications between staff
                and families, and other
                community agencies.

Instruments to assess program management

Center          A 29-item self-report           EHS Coor-    Annually
Director        checklist used to document      dinator
Question-       the center's adherence to
naire           Early Head Start performance
                standards. Subtopics included
                health and medication, parent
                communication, curriculum and
                practice, and program
                policies and philosophy.

Barriers to     A combination of open-ended     EHS Coor-    Annually
Providing       questions and 6-point Likert    dinator
Services        scale ratings, the Barriers
                survey had EHS coordinators
                document obstacles and
                barriers that may prevent
                them from delivering the
                program services to their
                full potential.

Assessment      A 24-item observational         Researcher   Annually
Checklist       checklist used to document
                if policies were available
                regarding the assessment of
                children and families and
                to document the adherence to
                those policies.

Transition      A 12-item observational         Researcher   Annually
Checklist       checklist used to document
                the availability of policies
                regarding transitioning
                children and families, and to
                document the adherence to
                those policies.

Home Visit      A 5-item observational tool     Researcher   Annually
Plan Check-     used to document the accuracy
list            and completeness the home
                visitor demonstrated when
                completing the family's Home
                Visit Plan. A 3-point Likert
                rating, ranging from "poor"
                to "high quality," was used.