Table of Contents | Previous | Next |
Implementation Levels
In addition to collecting detailed descriptive information about the research programs, we systematically assessed the programs' level of implementation. We defined the level of implementation as the extent to which the programs offered services meeting the requirements of selected key elements of the revised Head Start Program Performance Standards and the Early Head Start grant announcement. To assess the extent of program implementation in fall 1997, we developed implementation rating scales, checklists for organizing information needed to assign ratings (mainly information collected in site visits), and a consensus-based process for assigning implementation ratings to each research program. We considered programs that received ratings of 4 or 5 on a 5-point scale to be "fully implemented." (5)
A ratings panel consisting of four national evaluation team members and two outside experts used the checklists completed by site visitors to rate program implementation in the areas of child development and health services, family partnerships, community partnerships, staff development, and management systems, as well as overall program implementation. After independently rating each program, panel members discussed any differences in ratings and reached a consensus rating for each aspect of each program.(6) The ratings results are reported in Volume III of this report; the implementation checklists and rating scales are presented in the appendixes of Volume III.
The implementation ratings show that in fall 1997, when the research programs had been serving families for about one year:
- One-third of the research programs had reached "full
implementation." These programs had fully implemented
all or nearly all of the key program components by fall 1997, and
all of them had fully implemented the child development and health
services and the staff development requirements that we examined.
These "early implementers" were generally programs that
had previously served infants and toddlers as CCDPs, PCCs, or community
child care programs. Two were center-based, two were home-based,
and two were mixed-approach programs.
- Of the 11 programs that had not reached full implementation,
8 were rated as moderately implemented and 3 were judged to be at
a low level of implementation. Two of the latter three were
former CCDP programs that found it difficult to make the transition
to Early Head Start's explicit focus on child development.
- Nearly half of the research programs had reached
full implementation of child development and health services.
The programs were most likely to have fully implemented the requirements
for individualizing child development services, offering group socializations,
conducting developmental assessments, and involving parents in child
development services. They were least likely to have fully implemented
the requirements for ensuring child care quality (when center-based
care was not offered directly), providing the required frequency
of child development services (including frequency of home visits
in home-based programs and parent education in center-based programs),
and ensuring access to needed health services (including tracking
and following up on health services, and ensuring that children
received immunizations and well-child examinations). Center-based
and mixed-approach programs were most likely to have reached full
implementation in child development and health services by 1997.
- Slightly more than half of the research programs
had reached full implementation of family partnerships. The
programs were most likely to have fully implemented Individual Family
Partnership Agreements and the expected frequency of family development
services. They were least likely to have fully implemented requirements
for involving parents in policymaking and program operations and
for making a wide range of services available to families, directly
or by referral. Although a variety of services were usually offered,
some programs were not following up systematically with families
and service providers. Overall, home-based programs were most likely
to have fully implemented the family partnership component by 1997.
- Nearly half of the research programs had reached
full implementation of community partnerships. The programs
were most likely to have fully implemented collaborative relationships
with other community organizations and least likely to have established
transition planning procedures and have transition plans in place
for all children within six months of their third birthday.
- Nearly two-thirds of the research programs had fully
implemented staff development requirements. The programs
were most likely to have fully implemented the requirements for
staff supervision and training. Compared to similar agencies in
Early Head Start communities, eight of the programs offered higher
salaries, eight offered similar salaries, and one paid lower salaries.
Staff morale was high in eight programs and moderately high in nine.
- Slightly more than one-third of the research programs had reached full implementation of program management systems. Nearly all of the research programs had implemented the requirements for conducting community needs assessments, while fewer than half had yet fully implemented the requirements for establishing a Policy Council that meets regularly, developing written goals and implementation plans, and conducting annual program self-assessments.
Quality of Center-Based Child Care
The Head Start Bureau requires programs to provide child care directly or broker child care services in the community for all families who need it, and to take steps to ensure that all child care used by Early Head Start families meets the revised Head Start Program Performance Standards.(7) Preliminary data from observations (8) of center-based child care settings in which Early Head Start children received care at 14 and 24 months of age suggest that:
- The quality of care provided by Early Head Start
centers during their first two years of serving families was good.
On average, the Early Head Start centers planned teacher-child ratios
and group sizes that met the revised Head Start Program Performance
Standards, and the average score on the Infant/Toddler Environment
Rating Scale (ITERS) was 5.4 (in the good to excellent range). The
average quality of care observed in the Early Head Start centers
was above 4 (the middle of the minimal to good range) in all nine
research programs that provided center-based care. Across all programs,
the highest scores were in the personal care routines, interactions,
and program structure categories of the ITERS and the lowest scores
(still in the high end of the minimal to good range) in the learning
activities and furnishings categories.
- The quality of care in community center-based settings in which Early Head Start children were enrolled varied widely and was minimal to good, on average. On average, community centers caring for Early Head Start children received an ITERS score of 3.8 (in the minimal to good range). The scores ranged from 2.4 (inadequate to minimal) to 6.1 (good to excellent). The average ITERS scores were above 4 (above minimal) for all research programs that had begun assessing and/or monitoring the quality of child care that enrolled children received in community settings. As previously noted (p. 10), 6 of the 13 programs that did not provide center-based care to all children assessed and monitored the quality of community child care used by Early Head Start children.
5. The term "fully implemented" is a research term that we use to describe programs that substantially implemented the required program elements. (back)
6. Using the same rating scales, a member of the Head Start Bureau monitoring team independently rated program implementation based on information collected during monitoring visits completed a few months after the research site visits. The ratings assigned by the monitoring team member were very similar to those assigned by the rating panel. (back)
7. The Head Start Bureau was still clarifying these requirements at the time of the fall 1997 site visits. (back)
8. In Volume I we present child-staff ratios and group sizes reported by staff during site visits. The findings reported here (and in Volume III) are based on classroom observations. Observations were conducted in fall 1997 in 162 Early Head Start classrooms in 9 programs. The number of classrooms observed in each program ranged from 1 to 9. The 1997-1998 observations of community child care centers in which Early Head Start children were enrolled included 79 classrooms in 14 program sites where data were available. The number of centers observed at each site ranged from 1 to 9. Average scores of 5.0 and above on the 7-point ITERS scale are generally interpreted as good to excellent quality. Scores of 3.0 to 5.0 are considered minimal to good quality, and scores of 1.0 to 3.0 are considered inadequate quality. (back)
| Table of Contents | Previous | Next |