Description:
The program implementation checklist and rating scales
developed for the national Early Head Start Research and Evaluation Project (EHSRE) were designed to guide the collection
and organization of information related to 25 key elements
of the Head Start Program Performance Standards and
Early Head Start program guidelines. The checklist can
be used to guide the collection of information related
to the key program elements, and the rating scales can
be used to assess how fully the program has implemented
each key element, how fully key program areas are being
implemented, and how fully the program is being implemented
overall. The checklist contains 39 general criteria
with references to the performance standards, as well
as specific indicators for each general criterion. Completing
the checklist requires collecting information from staff,
parents, and program records. There are five ratings
scales, one each for early childhood development and
health services, family partnerships, staff development,
community partnerships, and management systems. Each
rating scale has multiple dimensions with ratings from
1 to 5. A rating of 4 indicates full implementation
and a rating of 5 indicates enhanced implementation.
Uses of Information: The
checklist and rating scales are intended to help Head
Start programs serving pregnant women and families with
infants and toddlers identify areas that need improvements
in order to continue to provide high-quality comprehensive
services that meet Head Start performance standards
and other regulations. They may be useful for organizing
information in preparation for Head Start Bureau monitoring
visits.
Reliability: Reliability
has not been established in the usual sense. However,
an informal assessment conducted by Head Start Bureau
monitoring staff concluded that the assessments of “full”
or “enhanced” implementation on these rating
scales were consistent with results of in-depth monitoring
conducted by the Bureau.
Validity: Content validity
was established by reviewing the specific criteria for
determining the rating on each dimension with representatives
of the Head Start Bureau and the Early Head Start National
Resource Center. A form of predictive validity was assessed
in the national Early Head Start Research and Evaluation Project (EHSRE), which found that programs that were rated as
fully implemented achieved a stronger pattern of impacts
on children and families.
Method of Scoring: The
Early Head Start evaluation team developed overall ratings
of each area and for the program overall by having multiple
team members rate each program independently, meet to
discuss any discrepancies in ratings, and agree on a
consensus rating for each dimension. The team also created
summary ratings of each area and for the program overall.
To be rated fully implemented overall, a program had
to receive a rating of 4 or 5 on most dimensions rated.
This process could be followed by program staff who
wanted to develop summary ratings.
Interpretability: The
results obtained from the ratings of program implementation
are readily interpretable by programs serving families
with infants and toddlers to show areas of programmatic
strengths and weaknesses. Because the scales are tied
to key dimensions of the performance standards, program
management and staff can see ways to focus program improvement
efforts.
Training Support: The
Early Head Start evaluation’s final implementation
report, Pathways to Quality (ACF 2002) describes the
use of the checklist and rating scales in the national
Early Head Start Research and Evaluation Project (EHSRE). Copies
of the rating are available in the report.
Adaptations/Special Instructions
for Individuals with Disabilities: Not applicable.
Report Preparation Support:
Not applicable.
References:
Administration for Children and Families. Pathways
to Quality and Full Implementation in Early Head Start
Programs. Washington, DC: U.S. Department of Health
and Human Services, 2003. |
