ACF OPRE: Survey of Early Head Start Programs

Results

Program Context

The Head Start programs that participated in the IM/IL training event were not randomly selected; however, this analysis reveals that their characteristics were generally similar to those of the Region III programs that did not participate (Table 1). Participating programs did differ from nonparticipating programs in the percentage of teachers with post-secondary degrees: on average, participating program had a higher percentage of teachers with that level of education. Additionally participating programs differed on some child characteristics: on average, participating programs had a significantly lower percentage of children who were non-Hispanic black or African American, and a significantly higher percentage of children who were non-Hispanic white, from a single-parent family, had a disability, or had an Individualized Education Plan (IEP). Overall, the programs that chose to participate in the IM/IL training event appear to have fewer minority children, but more children with a disability or an IEP and more teachers with a post secondary degree in early childhood education.

Training-of-Trainers Event in Spring 2006

The IM/IL training event in spring 2006 was both well attended and well received. Nearly 90 percent of programs sent four or five staff members; more than half (56 percent) sent five. The child development and education manager was the staff member most commonly sent to the training (72 percent of programs), followed by the health services manager (66 percent) and the family and community partnerships manager (58 percent). More than one-half of the programs (52 percent) sent the Head Start program director to the training, and more than one-quarter (28 percent) sent a teacher.

Participating programs gave the training a positive overall rating. On a scale of 1 (poor) to 5 (excellent), 71 percent rated the event as a 5. Respondents rated the event highly on its organization and the information that was presented (Table 2). For example, on a scale of 1 (strongly disagree) to 4 (strongly agree), 85 percent strongly agreed that the IM/IL goals were clearly explained; 82 percent strongly agreed that the workshop presented ideas for enhancements that addressed these goals; and 71 percent strongly agreed that the event provided new information and resources.

Programs rated the training somewhat lower on the practical aspects of implementing IM/IL in their own programs.8 For example, on a scale of 1 (strongly disagree) to 4 (strongly agree), only a third of programs strongly agreed that the training prepared them to implement IM/IL. Moreover, when asked about the allocation of time to the topics during the TOT, many programs reported that too little time was spent on engaging adults in IM/IL and planning their program’s implementation (37 and 40 percent, respectively; Table 3). Indeed, one-third of programs reported leaving the training event without a written action plan for implementing IM/IL (not shown).

**Table 1. Comparison of Program Characteristics: Head Start Programs in Region III That Did and Did Not Participate in the Spring 2006 *I Am Moving, I Am Learning* (*IM/IL*) Training-of-Trainers Event**
Program Characteristics	Participation in IM/IL Training Event
Program Characteristics	Yesa	Noa
Average Program Enrollmentc	432	447
Average Number of Centers per Program	11	10
Average Number of 3- and 4-Year-Old Children per Center	47	55
Average Number of Teachers per Center	3	3
Average Percentage of Teachers with a Postsecondary Degree in Early Childhood Education	89	83*
Program Auspice (Percentage)
Nonprofit	43	45
Community action agency	30	30
School system	23	18
Government agency	4	6
For profit	0	3
Type of Service Provided (Percentage)b
Full-day	83	83
Part-day	47	43
Center-based	93	91
Home-based	32	26
Combined Early Head Start/Head Start	26	21
Metropolitan Location (Percentage)
Metropolitan	57	71
Nonmetropolitan	43	29
Average Enrollment of Children with Child Characteristic (Percentage)
Non-Hispanic Black or African American	28	43**
Hispanic	11	13
Non-Hispanic White	60	43**
Live in Spanish-speaking homes	7	9
Live in single-parent homes	46	39*
Have health insurance	90	92
Have a disability	16	14*
Have an Individualized Education Plan	16	14*
Total Sample Size	53	121

Source: 2005-2006 Head Start Program Information Report.

Note: The participating programs only include the programs that participated in the TOT in spring 2006. The 17 programs that participated in the pilot effort in FY2005 are included in the “No” category (did not participate in IM/IL training event).

aIncludes only Head Start programs in Region III that provide direct services.

bPercentages not intended to add to 100.

cThe distributions for some of these continuous variables are skewed. These were examined as categorical and continuous variables, and the significant findings were generally similar in both of the analyses, with two variables emerging as significant as continuous variables: children living in single-parent homes and percentage of teachers with a postsecondary degree in early childhood education.

*P-value for the difference between these two groups is significant at the .05 level, two-tailed t-test.
**P-value for the difference between these two groups is significant at the .01 level, two-tailed t-test.

**Table 2. Ratings of Agreement with Statements About the Spring 2006 *IM/IL* Training-of-Trainers Event (Percentage of Programs)**
	Strongly Agree	Agree	Disagree	Strongly Disagree
The three IM/IL goals were clearly explained	85	12	0	2
The workshops presented ideas for program enhancements that addressed the goals of IM/IL	82	16	0	2
The instruction received at the training was adequate to train my own staff to implement IM/IL	50	46	2	2
Quality of the “take-home” materials (resource materials and handouts) was adequate to train my staff	49	49	0	2
The trainers explained how to adapt IM/IL to meet the needs of a program like ours	49	45	4	2
The ideas for program enhancements seemed like they would work in our program	49	49	0	2
The training prepared us to implement IM/IL	35	60	2	2
The training event provided new information and resources	71	26	0	2

**Table 3. Ratings of the Amount of Time Spent on Topics During the Spring 2006 *IM/IL* Training-of-Trainers Event (Percentage of Programs)**
	1 (Too Little Time)	2	3 (About the Right Time)	4	5 (Too Much Time)
Time for lecture and direct instruction	0	2	94	2	2
Time on how to engage adults in IM/IL	2	35	61	0	2
Time for asking questions	0	10	80	8	2
Time for practicing movement activities	6	8	78	6	2
Time for planning our implementation	13	27	57	2	0
Time for the topic of improving children’s nutrition	4	18	69	6	2

Programs’ perceptions of nutrition and physical activity as priority issues were substantially higher following the IM/IL training event (Table 4). Two separate items in the questionnaire asked IM/IL coordinators about their program’s views about the importance of the three IM/IL goals (MVPA, structured movement experiences, and healthy nutrition choices) “before the spring 2006 IM/IL training event” and “after the spring 2006 IM/IL training event.”9 Programs’ perceptions of the importance of the three IM/IL goals was significantly higher after the TOT.10 Table 4 presents results for all programs and shows these differences using categories that summarize the amount of change in the ratings. Interestingly, the largest changes in program ratings of the importance of IM/IL goals before and after training were for MVPA and structured movement. Fifty-two percent of programs ranked MVPA two or more points higher than they would have prior to the training event, 50 percent did so with structured movement experiences, and 19 percent did so with healthy nutrition choices. Conversely, MVPA, structured movement experiences, and healthy nutrition choices were rated as “very important” before and after the training event by 8, 6, and 25 percent of programs, respectively.

Table 4. Difference in the Programs’ Ratings of Perceived Importance of *IM/IL* Goals in Comparison with Other Head Start Services and Activities Before and After *IM/IL* Training-of-Trainers Event (n=48)
Difference in Perception of Importance of IM/IL Goal	Percentage of Programs
Moderate to Vigorous Physical Activity
No change (highest rating)	8
2 or more points	52
1 point	38
No change (less than highest rating)	2
Structured Movement Experiences
No change (highest rating)	6
2 or more points	50
1 point	40
No change (less than highest rating)	4
Healthy Nutrition Choices
No change (highest rating)	25
2 or more points	19
1 point	44
No change (less than highest rating)	13

Overall, programs rated the Spring 2006 TOT highly on its overall content and organization, and it appears that programs held the IM/IL goals as a greater priority following the spring 2006 TOT than prior to that event. However, some programs indicated that they would have liked a greater emphasis on the practical aspects of implementing IM/IL, such as engaging adults and planning their program’s implementation.

Implementation

M/IL Leaders. Programs assigned responsibility for leading IM/IL implementation to a variety of different staff members. Most commonly, IM/IL implementation was coordinated by the program director or assistant director (16 programs). The next most common staff positions were education coordinators or managers (9 programs), nutrition/health coordinators (8 programs), education specialists (5 programs), and early childhood/child development coordinators (4 programs). Eight programs assigned responsibility to other types of staff including family/parent/community coordinators, early childhood/child development specialists, social services coordinators, and training coordinators.

The individuals assigned to lead IM/IL were highly experienced and well-educated. The group had a median of 14 years of experience working with Head Start or other preschool programs, and 84 percent of leaders had obtained at least a bachelor’s degree.

Planning and Goals Selected. All but two of the programs that participated in the spring 2006 training reported implementing IM/IL activities.11 Most of these programs did some planning before trying to implement IM/IL activities. Nearly 80 percent reported conducting a needs assessment.12 Programs received a moderate level of input on IM/IL implementation plans from their stakeholders, including the health services advisory committee (68 percent of programs), policy council (53 percent), parent committee (45 percent), and governing board (20 percent). Roughly half (53 percent) of programs reported having a written implementation plan for IM/IL.

Three-quarters of the programs that implemented IM/IL activities involved all of their centers and classrooms. The 25 percent of programs (12 programs) that did not implement IM/IL activities program-wide selected centers and classrooms based on decisions by management staff (5 programs), volunteer participation by centers and classrooms (5 programs), and physical location (2 programs). Half of these programs (5 out of 12) implemented IM/IL in more than half of their classrooms.

In reporting on the areas that IM/IL enhancements were intended to address, more than 85 percent of programs that implemented IM/IL reported implementing activities that addressed the IM/IL goal to increase moderate to vigorous physical activity (MVPA). The same percentage reported implementing activities to improve the quality of structured movement, while a slightly smaller percentage (79 percent) reported implementing activities to improve healthy nutrition choices. In total, 63 percent of programs tried to address all three IM/IL goals.

Staff Training. Ninety percent of programs that implemented IM/IL activities (all but five) trained frontline staff on IM/IL. It is noteworthy that five programs are implementing IM/IL but did not report providing any training to staff. The most common approach to training, used by a third of programs was a pre-service training at the start of the program year and one or more in-service training sessions during the program year (Table 5).13 An additional 30 percent of programs conducted pre-service and in-service training and also had a targeted workshop led by a TA/content specialist or outside consultant. Twelve percent of programs provided only a pre-service workshop(s) and 12 percent provided only an in-service workshop(s), while 14 percent used either pre-service or in-service training and a targeted workshop. There were no cases in which programs distributed written materials or brought in an expert instead of conducting a pre-service or in-service training. Across all training models, staff received a median of three training sessions devoted to IM/IL.

As shown in Table 5, the hours of training provided increased in concert with the number of modes of delivery. Overall, reported training hours ranged from 1 to 24 hours, with a median of 6 hours.

**Table 5. Training Models Used by *IM/IL* Programs That Provided Training (n = 43)**
Type of Training Model	Percentage of Programs	Median Hours of Training
Pre-service training only	12	1
In-service training only	12	2
Pre-service and in-service training	33	6
Pre-service or in-service training and specialized workshop	14	7
Pre-service, in-service, and specialized workshop	30	12

As a followup to the training, one-third of programs received TA from the Region III TA system. One-quarter of programs received TA for IM/IL from some other consultant or provider, while 8 percent received TA from Region III and a consultant or other provider. Of programs that received TA from the Region III system, the most common staff member that provided this support was the TA specialist (81 percent). Despite the low level of TA received by programs, it is noteworthy that only 15 percent of programs cited needing more TA as a challenge to implementing IM/IL in their program.

In total, the majority of programs provided pre-service and in-service training, and the intensity of training increased with the number of modes of delivery. A relatively small percentage of programs utilized technical assistance from Region III or another source.

Child-Centered Enhancements. IM/IL is not a structured or curriculum-driven program. Rather, IM/IL allows programs to develop individualized approaches to promoting the IM/IL goals, selecting a mix of enhancements that best meets the needs of their program and the children they serve. A wide variety of child-centered IM/IL enhancements were implemented by programs. All but three programs (94 percent) implemented one or more enhancements that focused specifically on physical activity (either MVPA or structured movement; Table 6). A smaller percentage of programs (67 percent) reported implementing enhancements that were focused specifically on nutrition.

With regard to physical activity, the most commonly reported enhancement was introduction of the movement vocabulary—85 percent of programs reported using the equipment and/or vocabulary for teaching structured movement. More than three-quarters of programs reported introducing new equipment for indoor or outdoor play, and more than half (56 percent) reported enhancing or reconfiguring inside or outside space to facilitate play or other types of physical activity. Forty-two percent of programs reported implementing all of these physical-activity-focused enhancements.

**Table 6. *IM/IL* Enhancements Implemented with Children (n=48)**
Type of Enhancement by IM/IL Goals	Percentage of Programs
*Enhancements Focused on Physical Activity Goals*	94
Used equipment and/or vocabulary for teaching structured movement	85
Introduced new play equipment	77
Reconfigured or enhanced space to facilitate physical activity	56
All of the above	42
*Enhancements Focused on Nutrition Goals*	67
Changed policies or practices related to foods served to children	65
Established policy for foods brought in from home	31
All of the above	21
*Enhancements to Support Nutrition and/or Physical Activity Goals*	94
Purchased instructional materials or aids	88
Used “Choosy” in IM/IL activities	85
Used an existing physical activity/nutrition curricula	17
All of the above	15

Of programs that implemented nutrition-related IM/IL enhancements, the majority (65 percent) made changes in policies or practices related to the foods served to children at Head Start.14 Less than a third of programs (31 percent) established new policies about the food that children may bring in from home. Only 21 percent reported implementing both of these nutrition-related enhancements. Most programs that reported making changes in policies or practices related to foods served to children in Head Start modified the types of food being served; 52 percent of programs reported making such changes. Only 19 percent of programs reported making changes in the amounts of food served to children (not shown).

Most programs (94 percent) reported one or more enhancements that could be used to support either nutrition or physical activity goals. Most commonly, programs reported purchasing instructional materials or visual aids (88 percent) and using the animated “Choosy” character that was introduced at the IM/IL training event (85 percent). Eight programs (17 percent) reported using an existing curriculum to promote physical activity and healthy eating. Of these, four programs were using the “Color Me Healthy” curriculum.15

Overall, more programs reported implementing enhancements related to MVPA and structured movement than those related to nutrition. The most common activities involved using equipment, IM/IL vocabulary, and instructional materials for promoting physical activity. For nutrition, the most common activity carried out by programs involved modifying the foods served to children in Head Start. In general, most programs made use of the Choosy character introduced at the TOT.

Enhancements Focused on Parents, Staff, and Community. All but one program made an effort to reach parents as a part of IM/IL. The approach used most frequently for reaching parents was the distribution of written information in the form of flyers, pamphlets, or newsletters (85 percent), followed by workshops and events involving parents (71 percent) and discussion of nutrition and/or physical activity at parent-teacher conferences (63 percent). The majority of programs (65 percent) offered activities to parents that focus on altering eating and physical activity behaviors. These activities were not focused on the Head Start children, but were directed towards change in parental behaviors. Forty percent of programs established new policies regarding the types of food served at parent and staff meetings. Given that 40 percent of programs reported that the spring 2006 TOT should have spent more time engaging adults in IM/IL, it appears that programs were able to supplement knowledge that they gained from the TOT to develop enhancement activities that targeted parents.

Just over half of the programs (52 percent) reported offering activities that targeted the eating or physical activity behaviors of staff. Examples of staff-focused activities included offering discounts at local gyms, providing exercise classes or exercise equipment, having a nutritionist present a lecture or workshop, presenting information about diet and exercise in staff newsletters, and having staff set personal goals or develop walking plans. A few programs (6 percent) offered incentives to staff for meeting any goals related to IM/IL.

In addition to engaging parents and staff in the IM/IL enhancement, many programs partnered with community programs. About half of the programs (52 percent) reported having identified a community organization as a partner. More than 20 percent reported working with one organization, and close to 30 percent reported working with two or more organizations.16

Overall, 44 percent of programs indicated that they implemented IM/IL activities involving all three groups—parents, staff, and community partners.

Successes and Challenges

Programs generally had positive perceptions of their success in implementing IM/IL. Over half of the 48 programs that reported trying to implement IM/IL activities rated their program’s overall implementation of IM/IL as successful (Table 7).17 That is, on a scale of 1 (not at all successful) to 5 (extremely successful), they rated themselves as 4 or 5. Programs used this same scale to rate their success on implementing activities that targeted each of the three IM/IL goals. Two-thirds of programs rated their IM/IL enhancements that focused on healthy nutrition choices as successful. Sixty-three percent of programs rated enhancements that focused on MVPA as successful and 56 percent rated structured movement enhancements as successful.

Respondents indicated that staff enthusiasm for IM/IL was high. Seventy one percent of respondents rated the enthusiasm of their staff as a 4 or 5 on a scale of 1 (resistant) to 5 (enthusiastic) (Table 8). Results were comparable for each of the three IM/IL goals: MVPA (73 percent), structured movement (73 percent), and healthy nutrition choices (71 percent).

IM/IL coordinators were asked to identify factors that might have contributed to the success of IM/IL implementation as well as factors that posed challenges. The two factors that were most often cited as contributing to the success of IM/IL implementation were staff enthusiasm about the goals of IM/IL (77 percent) and the fact that the TOT training event “provided the programs with the necessary training to train staff” (75 percent). In addition, over half of respondents (54 percent) reported that having an enthusiastic and capable leader was an important factor, and 48 percent attributed success to the fact that before the TOT event their program was “already actively involved in efforts to increase children’s physical activity and improve their nutrition.” Parent enthusiasm, technical assistance, and community resources were less frequently identified as supports for successful implementation; each was cited by one-third or fewer of respondents.

**Table 7. Programs’ Level of Implementation Ratings: *I Am Moving, I Am Learning (*IM/IL*)* Goals (n=48)**
Perceived Success with Implementation	Percentage
IM/IL Overall
5 (extremely successful)	17
4	35
3	44
2	4
1 (not at all successful)	0
Moderate to Vigorous Physical Activity
5 (extremely successful)	19
4	42
3	31
2	8
1 (not at all successful)	0
Structured Movement Experiences
5 (extremely successful)	14
4	42
3	38
2	6
1 (not at all successful)	0
Healthy Nutrition Choices
5 (extremely successful)	21
4	46
3	25
2	6
1 (not at all successful)	2

When asked to identify the single, most important reason for implementation success, 34 percent of IM/IL coordinators cited staff enthusiasm and 32 percent cited the training event. The only other factors cited by more than two coordinators were the availability of community resources (either money or in-kind support, 11 percent or five programs) and having an enthusiastic and capable leader (9 percent or four programs).

The two most frequently reported challenges to implementation were (1) “management staff did not have enough time to devote to IM/IL” (cited by 59 percent of coordinators), and (2) “other areas in the program were a higher priority” (cited by 41 percent). One-third of coordinators reported that frontline staff did not have enough time to participate in the training, and 35 percent noted that they lacked funds to purchase materials they thought were needed to implement IM/IL. Overall, lack of managers’ time and competing program priorities were ranked as the most important challenges by 22 and 20 percent of coordinators, respectively. In both of the programs that did not try to implement IM/IL activities, the coordinators cited lack of management staff time as the major reason.

**Table 8. Staff Enthusiasm About *IM/IL* and *IM/IL* Goals (n=48)**
IM/IL Overall
	Percentage of Programs
5 (Enthusiastic)	25
4	46
3	23
2	6
1 (Resistant)	0
Moderate to Vigorous Physical Activity
5 (Enthusiastic)	25
4	48
3	21
2	6
1 (Resistant)	0
Structured Movement Experiences
5 (Enthusiastic)	21
4	52
3	19
2	8
1 (Resistant)	0
Healthy Nutrition Choices
5 (Enthusiastic)	21
4	50
3	27
2	2
1 (Resistant)	0

Measuring Potential Outcomes

Half the programs assessed children’s height and weight and calculated body mass index, a necessary first step to accurately determine whether children are at a healthy weight. Fewer programs (31 percent) reported recording time spent outdoors (which could serve as a surrogate measure for MVPA) or the quality of children’s movement experiences (25 percent). Only 4 percent (two programs) reported recording children’s food intake or selection. Overall, 38 percent of programs both calculated body mass index and recorded time spent outdoors, 25 percent both calculated body mass index and recorded quality of children’s movement experiences, and 6 percent recorded all three.

Perceptions of Obesity

The IM/IL coordinators did not perceive obesity to be as important a health problem for the children in their Head Start programs as they did for the children’s parents and the Head Start staff. Almost one-third of programs reported that obesity was “not a problem at all” or “a small problem” for the children, and only 18 percent reported it was “a large problem” or “a very large problem.” By contrast, over half the coordinators (56 percent) perceived that obesity was “a large problem” or “a very large” problem for the parents of Head Start children, and 36 percent perceived it was a “large problem” or “very large” problem for their program staff. Finally, when asked to rank the importance of three health conditions—asthma, oral health, and obesity—for children in their program, obesity was ranked as the most important condition by only 20 percent, which was less often than either oral health (46 percent) or asthma (34 percent).

Sustainability

The questionnaire assessed several factors that may be related to the sustainability of IM/IL enhancement activities into the future. Specifically, it assessed whether programs integrated IM/IL into their programs’ planning and practices, whether they have supports in place for promoting staff knowledge and skills, and whether frontline and management staff feel invested in these efforts.

On a positive note, following the spring 2006 TOT event, nearly every program tried to implement IM/IL, and only 7 percent (three programs) indicated that staff lost interest in IM/IL during the first full implementation year. This suggests that interest and enthusiasm for IM/IL were carried through that program year. Furthermore, the majority of programs identified an enthusiastic and capable leader (54 percent) and enthusiastic staff (77 percent) as factors that positively influenced their local implementation successes. Indeed, strong commitment by program leadership may be a key factor in sustaining IM/IL enhancements in the years to come.

Despite these initial efforts, several findings also raise questions about whether the implementation can be sustained. Only half of programs reported having created a written plan for IM/IL implementation, and 41 percent of programs indicated that other areas in their program were higher priority than IM/IL. With respect to staff knowledge and skills, nearly three-quarters of programs (72 percent) reported that more than half their frontline staff had participated in more than one training session. However, the total number of training hours in each program was a median of six hours per program.18 Furthermore, only one-third had received technical assistance for IM/IL from the Region III technical assistance system. Utilization of technical assistance may be an important support for new staff implementing IM/IL when there is staff turnover, which was identified as a challenge by 17 percent of programs. Finally, the survey results suggest that management staff’s participation may be limited by other factors: nearly 60 percent of programs report that management did not have enough time to devote to IM/IL, and 11 percent (five programs) reported that program managers did not have the skills to train staff on IM/IL.

Early Head Start and Home Visitors

The IM/IL training focused primarily on Head Start programs, but the questionnaire also assessed the implementation of IM/IL in Early Head Start programs or during home visits. Of the 15 programs19 with a combined Head Start/Early Head Start program (30 percent of all programs), 8 implemented IM/IL activities in their Early Head Start program. Respondents provided an open-ended description of the activities that their programs implemented: four described using the “Choosy” music with children, one provided “parent education on MVPA and better nutrition,” and one promoted “awareness” among program staff.

Of the 20 programs that offer home-based services (40 percent), 15 programs implemented IM/IL activities as a part of home visits.20 Respondents provided an open-ended description of the activities that they implemented, and six programs used the music activities with children, two carried out activities that involved parents, while the remaining programs provided general descriptions of activities for promoting the three IM/IL goals.

Among programs implementing IM/IL in Early Head Start, coordinators noted that most of the IM/IL activities they had learned about were for older children and that they lacked training in implementing IM/IL activities for infants and toddlers. Among programs that tried to implement IM/IL during Head Start home visits, challenges that were identified included (1) getting parents to continue activities afterward (three programs), (2) finding enough time during the visit (two programs), and (3) reinforcing IM/IL goals as part of the frequent (usually weekly) contact with children (one program).

Successes and Challenges Associated with High Implementation

Programs that rated their IM/IL implementation as being very successful were classified as “high implementing” programs.21 High implementing programs were significantly more likely than programs that were not high implementers to report that staff members were enthusiastic about IM/IL enhancement activities (Table 9). Ninety-two percent of high implementing programs had staff who were enthusiastic about IM/IL, overall, compared with only 48 percent of programs that were not high implementers. Similar differences in staff enthusiasm were observed for each of the IM/IL goals.

**Table 9. Staff Enthusiasm, Implementation Supports and Challenges, by Level of Implementation (Percentage)**
Characteristic of IM/IL Implementation	High Implementer a
	Yes (n=25)	No (n=23)
	Percentage of Programs
Staff Enthusiasm About IM/IL Goalsb
Enthusiastic about IM/IL enhancements overall	92	48***
Enthusiastic about IM/IL enhancements for increasing MVPA	92	52***
Enthusiastic about IM/IL enhancements for increasing structured movement experiences	88	57**
Enthusiastic about IM/IL enhancements for improving healthy nutrition choices	84	57**
Factors Supporting Implementing IM/IL
Had the community resources (either money or in-kind support) to help in implementation	36	22
The training event provided us with the necessary training to train our staff	88	61**
Good technical assistance	40	22
Had an enthusiastic and capable leader to implement IM/IL	68	39*
Staff members were enthusiastic about the goals of IM/IL	88	61**
Parents of children in the program were enthusiastic about the goals of IM/IL	44	17*
Obesity prevention was a priority of our program’s policy council, governing board, or health services advisory committee	40	30
Before the training event, were actively involved in efforts to increase children’s physical activity and improve their nutrition	48	48
Left training with a written action plan for implementation of IM/IL	79	52*
Challenges in Implementing IM/IL
The management staff did not have enough time to devote to IM/IL	44	74**
The frontline staff did not have enough time to participate in training	17	48**
Other areas in our program were a higher priority	35	48
We felt we needed materials to implement IM/IL, but our program did not have the funds to purchase them	35	35

Source: IM/IL Implementation Evaluation Stage 1 Questionnaire. Completed by IM/IL coordinators in spring 2007, approximately one year after program participation in the 2006 training-of-trainers event.

Note: Sample sizes ranged from 46 to 48, depending on the item-level missing values.

aRated as 4 or 5 on a 5-point scale with anchors at 1 (not at all successful) to 5 (extremely successful).

bRated as 4 or 5 on a five-point scale with anchors at 1 (resistant) and 5 (enthusiastic).

MVPA = moderate to vigorous physical activity.

*P-value for the difference between these two groups is significant at the .05 level, two-tailed t-test.
**P-value for the difference between these two groups is significant at the .01 level, two-tailed t-test.
***P-value for the difference between these two groups is significant at the .001 level, two-tailed t-test.

High implementing programs were also more likely to identify particular factors contributing to the success of their IM/IL implementation. For example, compared with programs that were not high implementers, a significantly greater percentage of high implementing programs credited (1) the IM/IL training event for providing the training necessary to train staff; (2) the enthusiasm of staff members about IM/IL goals; (3) an enthusiastic and capable leader; (4) the support of parents for the goals of IM/IL; or (5) the fact that they left the IM/IL training event with a written action plan for implementation. Among all programs that left the IM/IL training event with a written action plan, roughly the same percentage of high implementing and other programs (63% vs. 55%) had a written plan in place in Spring 2007.

Coordinators of high implementing programs were also less likely to identify certain factors as challenges to IM/IL implementation. For example, high implementing programs were less likely than programs that were not high implementers to report that management staff did not have enough time to devote to IM/IL or that frontline staff lacked time for training. There were no significant differences between high implementing programs and programs that were not high implementers in perceptions about other program areas being higher priority than IM/IL or about not being able to afford IM/IL materials.

Overall, high implementing programs had greater enthusiasm among staff and program leadership, were more likely than other programs to have left the TOT with a written plan, and were less likely to report that managers did not have time to devote to IM/IL.

Training and Enhancements Associated with High Implementation

Programs that perceived themselves as high implementing programs offered more types of training than programs that were not high implementers (Table 10) and provided significantly more training hours (not shown). One-quarter of the programs that were not high implementers offered only pre-service training and 20 percent offered only in-service training. None of the high implementing programs limited training to a single pre-service session and only 4 percent limited training to one or more in-service sessions. In contrast, 35 percent of high implementing programs provided pre-service training, in-service training, and a specialized workshop as compared to 15 percent of programs that were not high implementers. On average, staff in high implementing programs received nearly twice as much training as those in programs that were not high implementers (a mean of 9.7 hours versus a mean of 5.2 hours, p<0.01). There was not a statistically significant difference by implementation level in receipt of technical assistance from Region III TA staff (40 percent of high implementing programs versus 26 percent of programs that were not high implementers).

In general, few significant differences were noted in the types of child-centered enhancements implemented by high implementing programs and programs that were not high implementers (Table 11). However, two interesting differences were observed. High implementing programs were significantly more likely than programs that were not high implementers to carry out enhancements focused on structured movement. Almost all high implementing programs (96 percent) purchased equipment to support structured movement enhancements or used the movement vocabulary. In contrast, three-quarters of the programs that were not high implementers did neither of these things. A similar pattern was noted for use of the “Choosy” character: 96 percent of the high implementing programs used “Choosy,” compared with 73 percent of other programs.

**Table 10. Type of Training, by Level of Implementation**
Types of Training Offered*	High Implementera
	Yes(n=23)	No (n=20)
	Percentage of Programs
Pre-service training only	0	25
In-service training only	4	20
Pre-service or in-service training and specialized workshop	9	20
Pre-service and in-service training	43	20
Pre-service, in-service, and specialized workshop	43	15

Source: IM/IL Implementation Evaluation Stage 1 Questionnaire. Completed by IM/IL coordinators in spring 2007, approximately one year after program participation in the 2006 training-of-trainers event.

a “High implementers” rated their programs’ success implementing “IM/IL overall” as 4 or 5 on a five-point scale ranging from 1 (not at all successful) to 5 (extremely successful).

* P-value for the Fisher’s exact test assessing the overall association between the types of training offered and high implementation was significant at the .01 level. This analysis did not test pairwise comparisons across training types.

There was no difference between high implementing programs and programs that were not high implementers in the total number of child-centered enhancements implemented. On average, high implementing programs implemented a median of seven child-centered enhancements and programs that were not high implementers implemented a median of six child-centered enhancements (not shown).22

High implementing programs and programs that were not high implementers were equally likely to implement IM/IL enhancements designed to promote change in diet and physical activity behaviors of staff (60 and 43 percent of programs, respectively) and to involve community partners (48 percent and 57 percent, respectively). However, high implementing programs were significantly more likely to implement IM/IL enhancements that targeted diet and physical activity behaviors of parents (76 percent of high implementing programs versus 52 percent of other programs) and to establish policies for foods served at staff and parent meetings (48 percent versus 17 percent).

Overall, high implementing programs were more likely to implement enhancements focused on structured movement and use the Choosy character in their activities, but there was no overall difference in the number of enhancements carried out by high implementing Overall, high implementing programs were more likely to implement enhancements focused on structured movement and use the Choosy character in their activities, but there was no overall difference in the number of enhancements carried out by high implementing programs relative to programs that were not high implementers. With respect to staff training, high implementing programs were also more likely to provide staff with pre-service and in-service training on IM/IL. In fact, staff in high implementing programs received nearly twice as much training as those in programs that were not high implementers.

**Table 11. Types of *IM/IL* Enhancements Implemented, by Level of Implementation**
Type of Enhancement	High Implementera
	Yes (n=23)	No (n=20)

	Percentage of Programs
Child-Centered Enhancements Focused on Physical Activity
Purchased equipment and/or used vocabulary for teaching structured movement	96	74*
Purchased new play equipment	80	70
Reconfigured or enhanced space to facilitate physical activity	56	57
All of the above	50	45
Child-Centered Enhancements Focused on Nutrition
Changed policies or practices related to foods served to children	65	71
Established policy for foods brought in from home	48	30
Child-Centered Enhancements to Support Nutrition and/or Physical Activity
Purchased instructional materials or aids	96	78
Used “Choosy” in IM/IL activities	96	73*
Used an existing physical activity/nutrition curricula	16	17
All of the above	17	15
Enhancements Focused on Parents, Staff, and Community
Offered activities to alter diet and physical activity behaviors of staff	60	43
Offered activities to alter diet and physical activity behaviors of parents	76	52*
Established policy for foods served at staff/parent meetings	48	17*
Involved one or more community organizations as a partner	48	57

Source: IM/IL Implementation Evaluation Stage 1 Questionnaire. Completed by IM/IL coordinators in spring 2007, approximately one year after program participation in the 2006 training-of-trainers event.

a “High implementers” rated their programs’ success implementing “IM/IL overall” as 4 or 5 on a five-point scale ranging from 1 (not at all successful) to 5 (extremely successful).

* P-value for the Fisher’s exact test assessing the overall association between the types of training offered and high implementation was significant at the .01 level. This analysis did not test pairwise comparisons across training types.

8 As shown in Table 3, more than 95 percent of programs “agreed” or “strongly agreed” with each of these positively worded items about the IM/IL TOT event. (back)

9 Coordinators rated their perceptions in response to the following question: “Compared with all other services and activities your program provides in Head Start, how would you rank the importance of the following activities in your program before the spring 2006 IM/IL training event?” The next question asked them to rank the importance of the three IM/IL activities after attending the training. In both questions, respondents were asked to rate the importance of these focus areas using a 5-point scale that ranged from 1 (not important at all) to 5 (very important). (back)

10 Among programs that did not rate these goals at the highest level before the training (5 or very important), the rating of MVPA, structured movement, and health nutrition choices following the TOT was 1.6, 1.6, and 1.1 points higher respectively, (p<0.0001 for all three differences; data not shown). (back)

11 Reasons for not implementing IM/IL activities are discussed in the subsequent section on “Successes and Challenges.” (back)

12 These are programs that responded “yes” to the question, “Before selecting IM/IL activities to implement, did you review your current program activities and identify areas in which you were not implementing activities like the ones presented at the spring 2006 IM/IL training event?” (back)
13“Pre-service training” is training and education provided to teachers before the beginning of the program year, while “in-service” training is the training and education that teachers receive at times during the program year. (back)

14 Note that the Head Start performance standards include detailed standards for the foods provided to children. It is possible that some of the programs that did not focus on nutrition-related enhancements may have believed that their nutrition programs were already consistent with IM/IL goals. (back)

15 Based on the responses to the questionnaire, it was not possible to distinguish whether this curriculum was implemented as a part of IM/IL or whether it was already in place when IM/IL began. (back)

16 The questionnaire did not assess the types of community organizations with which programs partnered on IM/IL. This information will be collected during the next two phases of the evaluation. (back)

17 Note that in later sections of this report (“Successes and Challenges Associated with High Implementation” and “Training and Enhancements Associated with High Implementation”), we label these programs as “high implementers.” (back)

18 This figure (total hours of training) represents the amount of training that was offered to staff, but it does not capture the amount of training received by staff members. The time spent in training may have varied across staff. Moreover, it does not capture the intensity or quality of the training. (back)

19 One program in this sample reported having an Early Head Start program, but the PIR indicated that they did not have children enrolled in Early Head Start. This difference may be due to the fact that the PIR data are based on the 2005-2006 program year, while the questionnaire referred to the 2006-2007 program year. (back)

20 For seven programs, data from the questionnaire and the PIR are discordant: two programs reported that they deliver services through home visitors and the PIR data indicated that they did not, and two reported that they did not deliver services through home visitors and the PIR indicated that they did. This difference may be due to the fact that the PIR data are based on the 2005-2006 school year, while the questionnaire referred to the 2006-2007 school year. (back)

21 High implementing programs rated the success of their IM/IL implementation as 4 or 5 on a 5-point scale that ranged from not at all successful (1) to extremely successful (5). See Table 7. (back)

22 With the exception of physical activity enhancements, each of the enhancements in Table 11 was considered one enhancement. For physical activity enhancements, each of the items listed in Table 11 could be associated with up to two different enhancements: equipment or vocabulary for structured movement, indoor or outdoor play equipment, and enhanced indoor or outdoor space. (back)

Table of Contents | Previous | Next