Table of Contents | Previous | Next |
Research Design and Data Sources
The evaluation rigorously tests whether the core components of CEO’s program produce impacts on employment, recidivism, and other outcomes.
Research Design for the Evaluation
The impacts of CEO’s program are being assessed using a random assignment research design. Between January 2004 and October 2005, 977 ex-prisoners who were referred by their parole officer, reported to CEO, and met the study eligibility criteria,1 were randomly assigned to one of two groups:2
-
Neighborhood Work Project Group (the Program Group; N=568). Individuals who were assigned to this group were eligible to receive all of CEO’s program services, including the four-day Life Skills class, placement in a transitional job, job coaching, additional services (such as the fatherhood program), job development services, and all post-placement services.
-
Resource Room Group (the Control Group; N=409). The Resource Room program was designed to provide a basic level of service to individuals assigned to the control group and to provide a benchmark against which to compare CEO’s core program. Individuals assigned to this group participated in a revised version of the Life Skills class that lasted 1.5 days, rather than 4 days. Participants were then given access to a resource room that includes computers (with job search software), phones, voice mail, a printer, a fax machine, and other job search tools, including publications. When clients came into the Resource Room a staff person was available, if needed, to assist them with many aspects of job search, including use of the equipment, help developing a resume, and assistance with setting up a voice mail account so that potential employers could leave messages for participants.3
The research team is tracking both groups using a variety of data sources (see below). Any statistically significant differences that emerge between the groups over time — for example, differences in employment or incarceration rates — can be attributed to CEO’s core program with some confidence.4 These differences are referred to as “impacts” or “effects.”
It is important to note that the individuals who were targeted for the study — parolees who were referred to CEO by their parole officer — represent a subset of the parolees that CEO serves. For example, everyone who graduates from New York State’s Shock Incarceration (boot camp) program and returns to New York City is required to participate in CEO’s program. For contractual reasons, boot camp graduates and participants in some other special programs were not included in the study. CEO provides very similar services to all of these groups of parolees.
Data Sources Used in This Analysis
Data from several sources are used in the analysis presented in this paper:
-
Baseline data. Participants completed a short baseline information sheet when they entered the study. Some additional baseline data were also obtained from CEO’s internal database.
-
Criminal justice administrative data. These data were provided by the New York State (NYS) Division of Criminal Justice Services (DCJS), which compiles data from many city and state agencies. These data provide information on a range of outcomes, including arrests, parole violations, convictions, and incarceration in state prison, for each member of the study sample, both before and after study entry.5
-
Employment data. Data from the NYS Department of Labor show employment in jobs covered by the state unemployment insurance (UI) program (including CEO transitional jobs). These data cover the period from three years prior to study entry to one year after study entry for all sample members.
-
Program MIS data. These data, from CEO’s management information system (MIS), provide information on each individual’s participation in the CEO components for which he was eligible, including data on NWP work and on job coach and job developer appointments.
At this point, data from all of these sources are available for at least one year after study entry for all sample members. The study will eventually follow sample members for three years and, as discussed further below, will incorporate data from a number of other sources.
1 For both ethical and methodological reasons, individuals who had participated in the Neighborhood Work Project (NWP) program in the past year (“recycles”) were excluded from the study and assigned to the NWP program. Also, only individuals who signed an Informed Consent form were included in the study sample. (back)
2 The MDRC team worked with CEO and the New York State Division of Parole to design a random assignment process that ensured that the study did not decrease the number of people who received NWP services and that ensured that CEO had enough participants to fill its contractual obligations to NWP worksite sponsors. Random assignment was only conducted during weeks when the number of new enrollees exceeded the number of available NWP worksite slots. (back)
3 If a control group member worked diligently in the Resource Room for three months but was unable to find employment on his own, he was offered CEO’s job development services (but not a transitional job), but this very rarely happened. Of course, control group members may have sought assistance from other employment programs. A 15-month follow-up survey will gather information about program participation for both groups. (back)
4 Since the proportion of people randomly assigned to the program group varied from week to week, weights were used so that the impact results were not dominated by a disproportionate assignment to one group or the other in any given week. In particular, each program group member received a smaller weight in the impact calculation if they entered the study in a week when a higher proportion of individuals were assigned to the program group, and vice versa. The same was true for control group members. In addition, following standard practice in studies such as this, the estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. (back)
5 DCJS provided MDRC with three SPSS portable files matched to a sample request file using New York State Identification (NYSID) Numbers: an arrest, a parole, and a corrections file. Each file includes information about unsealed events only. The arrest file includes data from the DCJS rap sheet system, including arrests, convictions, and sentencing. The corrections file contains information furnished to DCJS from the New York State (NYS) Department of Corrections, including state prison admissions and releases (with a small number of records on jail stays). The parole file contains information furnished to DCJS from the NYS Division of Parole, including parole term start and end dates, abscond dates, and parole revocation information. The arrest and corrections files contain data reported to DCJS from January 1970 to present, and the parole file contains data reported to DCJS from January 1960 to present. DCJS takes no responsibility for the methods of statistical analysis or for the conclusions derived from the analysis. (back)
| Table of Contents | Previous | Next |