Table of Contents | Previous | Next |
Appendix 4.6: Basis for Assuming That Non-Participants Experienced No Intervention Effects
Although a common assumption in experimental evaluations of social program impacts, the decision to consider nonparticipants from the Head Start group unaffected by the intervention merits discussion. The validity of this assumption begins with the very exacting definition of a “nonparticipant” that has been adopted for this study, i.e., if a child attended a Federal Head Start program for even a day during the 2002-2003 program year, she or he was considered a “participant.” This means that while the chances are slim that such a child (or her/his family) could be meaningfully influenced by so brief an experience, this definition allows that some Head Start impact could have occurred in this case. Only those children identified as never having attended Head Start were considered nonparticipants and made part of the “no-show” adjustment. In addition, the set of Head Start group members considered no-shows was further narrowed—thus strengthening the case for assuming zero impacts on nonparticipants even more—by looking at multiple sources of information for indications of participation as was described in Chapter 2 and in Appendix 2.2. Evidence of Head Start attendance from any of these sources took a child out of the “no-show” group and released him/her from the assumption of no program impact.
With such careful reduction of the non-participant group to only those children who never spent any time in any Federal Head Start program, only one means of being affected by the program remains open to those individuals—the possibility that just applying to Head Start and going through the random assignment and admission process influenced behavior and outcomes. Before this could create impacts on nonparticipants that would show up in the experimental comparison outcomes between the two groups created at random assignment, it would have to occur for children randomized into the group accepted into the program (but who never attend) and not to children randomized into the non-Head Start group. This follows from the fact that any symmetric impact of the application and randomization process on both groups will net out in the calculation of the experimental impacts, which focuses solely on the difference in outcomes between the Head Start and non-Head Start groups. The application and random assignment process was identical up to the point when program staff were told which children had been selected into the Head Start group; during this interval, when no one (not even study staff) knew who would be selected into the program and who would not be, it was impossible for different, non-canceling impacts to have occurred.
This leaves only the interval following notification of the program staff of admission of no-shows into the program as a time when differential impacts could have occurred. There were two possibilities during this period for the existence of the Head Start program to influence a nonparticipating child’s (or his/her family’s) outcomes:
-
Head Start program staff might do something to guide families to alternative non-Head Start services that those families would not have accessed otherwise, and those services influenced later child and family outcomes; and/or
-
The families themselves might choose different non-Head Start options, or alter the timing of when they used those options, simply because they knew Head Start was available to their child—and the use of those services then later alters child and family outcomes.
The potential for either of these influences to appreciably contribute to the overall gain caused by Head Start—the quantity the no-show adjustment attributes solely to actual program participants—is remote. Encouragement, information, and assistance in using non-Head Start services should rarely be a priority for Head Start programs seeking to meet Federal enrollment targets and that have already expressed the intention of (and been given “permission” by the random assignment process to) involving the child in Head Start itself. Moreover, there was some chance that similar children in the non-Head Start sample received this kind of aid as well,1 which would then result in a canceling out of its effects in the impact analysis. The window for parents to behave differently because Head Start admission had been granted but not pursued was presumably short, assuming that parents understand the offered “Head Start slot” would not be held open for their child indefinitely. Once that slot was presumed by the family to have been filled by some other child, the fact that it was once open to the family’s child should have no further effect on behavior.
The one real opportunity for Head Start to affect the longer run outcomes of no-shows arises in cases where notification of admission reached families well ahead of the time their children could actually begin their Head Start participation. Many grantees and delegate agencies focus new admissions on the start of the school year and also seek to inform families that their child has been admitted for the fall in the late spring or summer. The random assignment process was set up to accommodate this practice as much as possible, which meant that many families randomized into the Head Start group found out about this decision a number of weeks, or even months, before participation could begin. These families may have behaved differently in the interval between that point and when they presumed their “unclaimed” Head Start slot had been relinquished to another child (i.e., differently from families assigned to the non-Head Start group who were told they did not have option of joining the program). This behavior could then have led to different outcomes the following spring, making “no-shows” to some extent contributors to the total gain caused by the program.
The most likely type of altered family behavior to emerge during the “waiting period” between Head Start admission and the start of program services in the fall was a reduction in the pursuit of other types of child development assistance. Parents expecting to rely on Head Start to support their child’s development in fall 2002 presumably had less incentive to find alternative supports during the summer and to arrange in advance non-Head Start services for their child. As a consequence, their children’s outcomes could be set back relative to counterparts in the non-Head Start sample. This did not happen for families that quickly decided their children would not attend Head Start in the fall even though admitted (e.g., those who moved at the end of the previous school year), but this was the minority of all eventual no-shows. It is also possible that some families did more to push forward their children’s development while waiting for Head Start to begin in the fall, wanting to make sure the child was ready for the new experience or inspired by the theme of Head Start—of which they (temporarily) considered themselves a part—emphasizing intellectual stimulation and engagement of the child.
Overall, it is hard to gauge how much, and in what direction, the “anticipatory effects” of Head Start participation altered family behaviors over the summer for those families that ultimately did not participate. But, to assume there was an effect requires the following to be true:
-
Such families behaved differently;
-
Children ended up in different places cognitively and in terms of behavior and health care the following spring (when outcome data were colleted) as a result; and
-
This contribution to the overall measured gain attributable to Head Start appreciably affected the size of the measured average impact of the program among all children assigned to the Head Start group and, hence, the size of the average impact on participants inferred through the no-show adjustment.
However, because one cannot confidently rule this out completely, it is necessary to acknowledge that some small bias—either up or down—may be present in the no-show adjusted impact estimates for participants presented in Appendices 5.1, 6.1, 7.1, and 8.1.
1 Agreements with grantees and delegate agencies not to serve children and families assigned to the control group did not preclude referral for other services-or even direct provision of those services if no Federal Head Start funding was involved-for grantees/delegate agencies whose contacts with the community through its non-Head Start activities are extensive and often involve information and advice on service options. These activities, and their long-run consequences, are considered outside the Head Start intervention per se as something that would occur even if Federal funding for Head Start did not exist. (back)
| Table of Contents | Previous | Next |