Table of Contents | Previous | Next |
Effects on Program Participation and Service Receipt
This section presents the effects of New York City’s Personal Roads to Individual Development and Employment (PRIDE) program on program participation and the receipt of support services. The analysis uses data from the ERA 12-Month Survey and from the automated tracking system of the city’s Human Resources Administration (HRA). Because the follow-up survey was limited to the single-parent sample, the findings reported here pertain only to that group.
- Survey respondents in the PRIDE group were substantially more likely than those in the control group to participate in work activities (primarily job search and work experience) but not in education or training.
Table 4 presents data on participation in employment-related activities, as reported on the 12-month survey. (Box 2 explains how to read the tables in the ERA evaluation.) A substantial proportion of the control group respondents reported participating in work activities. This pattern probably reflects the fact, discussed in the preceding section, that about one-third of control group members became nonexempt during the follow-up period and were required to participate in regular HRA work activities.
Nevertheless, PRIDE substantially increased participation in both job search activities and work experience –– two core program components. For example, 33 percent of respondents in the PRIDE group reported unpaid work/subsidized employment, compared with about 14 percent of respondents in the control group. Overall, including those who did not participate in this activity, the PRIDE group had almost three times as many weeks of participation in work experience as did the control group (7.7 weeks versus 2.8 weeks). Among survey respondents in the PRIDE group who participated in work experience, the average time spent in these positions was nearly six months (not shown).
There was a similarly large difference between the two groups in their participation in group job search/job club activities, which were provided by all the PRIDE vendors. More than 40 percent of the PRIDE group respondents reported participating in such activities –– more than double the figure for the control group.
The PRIDE program had no statistically significant effect on participation in education or training among the survey respondents. This is somewhat surprising, given that recipients who were assigned to the Work Based Education (WBE) track were expected to spend two days per week in education activities. However, it is notable that respondents in both groups were fairly likely to report participating in education and training; more than one-fourth of the control group and nearly one-third of the PRIDE group reported such participation, with adult basic education and English as a Second Language (ESL) courses being most prevalent. It may be that control group members, without work requirements, were more likely to enter education or training activities on their own, thereby offsetting the increase in such activities that is attributable to the WBE requirements.
- Respondents in the PRIDE group were more likely than those in the control group to receive help accessing benefits and other supportive services and help with job preparation.
| Outcome | PRIDE Group | Control Group | Difference (Impact) | P-Value | |
|---|---|---|---|---|---|
| Ever participated in any activitya (%) | 65.9 | 49.7 | 16.2 *** | 0.000 | |
| Participated in any employment-related activityb (%) | 58.7 | 40.1 | 18.5 *** | 0.000 | |
| Participated in a job search activity (%) | 51.3 | 36.2 | 15.1 *** | 0.000 | |
| Group job search/job club | 41.4 | 20.0 | 21.4 *** | 0.000 | |
| Individual job search | 34.2 | 26.9 | 7.3 ** | 0.028 | |
| Participated in an education/training activity (%) | 31.4 | 26.3 | 5.1 | 0.128 | |
| Adult basic education/GED classes | 14.6 | 11.7 | 2.9 | 0.236 | |
| English as a Second Language (ESL) classes | 10.4 | 9.1 | 1.3 | 0.545 | |
| College courses | 5.7 | 3.8 | 2.0 | 0.204 | |
| Vocational training | 8.4 | 7.4 | 1.0 | 0.601 | |
| Participated in unpaid work/subsidized employment (%) | 33.0 | 13.7 | 19.3 *** | 0.000 | |
| Average number of weeks participating in: | Job search activities | 10.3 | 4.8 | 5.5 *** | 0.000 |
| Education/training activities | 6.4 | 4.5 | 1.9 * | 0.071 | |
| Unpaid work/subsidized employment | 7.7 | 2.8 | 4.9 *** | 0.000 | |
| Sample size (total = 759) | 380 | 379 | |||
| SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. NOTES: See Appendix F. a "Any activity" includes employment-related activities, education/training activities, life skills, and other types of activities. b Employment-related activities include job search activities, unpaid jobs, and on-the-job training. |
Box 2How to Read the Tables in the ERA EvaluationMost tables in this report use a similar format, illustrated below, which shows a series of outcomes for the PRIDE group and the control group. For example, the table shows that 65.9 percent of PRIDE group members and 49.7 percent of the control group members have ever participated in any program activity. Because individuals were assigned randomly either to the PRIDE group or to the control group, the effects of the program can be estimated by the difference in outcomes between the two groups. The “Difference” column in the table shows the differences between the two research groups’ participation rates — that is, the program’s impacts on participation. For example, the impact on participation in any employment-related activity can be calculated by subtracting 58.7 percent from 40.1 percent, yielding an 18.5 percentage point impact. Differences marked with asterisks are “statistically significant,” meaning that it is quite unlikely that the differences arose by chance. The number of asterisks indicates whether the impact is statistically significant at the 1 percent, 5 percent, or 10 percent level (the lower the level, the less likely that the impact is due to chance). For example, as shown below, the PRIDE program had a statistically significant impact of 7.3 percentage points at the 5 percent level on participation in individual job searches. However, PRIDE had no statistically significant effect on participation in an education/training activity (there are no asterisks next to the difference of 5.1 percentage points). The p-values show the exact levels of significance.
|
|||||||||||||||||||||||||||||||||||||||||||||||
Table 5 presents data on the types of services that sample members received, as reported by respondents to the 12-month survey.
Access to the PRIDE program led to changes in the types of help received, and most of the changes are related to the increased emphasis on work. One of the largest effects on help received was in job preparation (see the middle of Table 5): 36.5 percent of respondents in the PRIDE group reported receiving help in this area, compared with 18.6 percent of respondents in the control group. Consistent with the goals of the program, more respondents in the PRIDE group reported that they had received help finding a job that took into account their health problems (though it is notable that only about one-fifth of the PRIDE group reported receiving such help).
The program also led to notable increases in help finding or paying for child care and transportation, both of which are work-related supports. Finally, the program led to an increase in assistance getting Medicaid and food stamps, consistent with families thinking about moving off welfare and making sure that they would continue to receive other benefits for which they were eligible. The next section explores whether this effect was driven by a reduction in welfare and an increase in work.
- Members of the PRIDE group were much more likely than those in the control group to be identified as noncompliant, and they were more likely to be sanctioned for failure to comply with program rules.
According to HRA’s tracking system, about 65 percent of the PRIDE group were considered out of compliance with HRA rules at some point within one year after random assignment; over a two-year period, the figure was nearly 76 percent. Most of these instances of noncompliance related to PRIDE’s requirements, so it is not surprising that the corresponding figures for the control group are much lower, about 16 percent and 24 percent, respectively.
A similar pattern is evident with regard to sanctioning. Within one year of random assignment, 20 percent of the PRIDE group and 3 percent of the control group had their grant reduced for noncompliance. Within two years, the figures were 32 percent for the PRIDE group and 8 percent for the control group. Further analysis (not shown) found that there were few differences in the characteristics of sample members who were sanctioned and those who were not.
| Outcome (%) | PRIDE Group | Control Group | Difference (Impact) | P-Value | |
|---|---|---|---|---|---|
| Received help with support services | 32.7 | 16.3 | 16.3 *** | 0.000 | |
| Finding or paying for child care | 22.1 | 12.9 | 9.2 *** | 0.001 | |
| Finding or paying for transportation | 21.4 | 8.9 | 12.5 *** | 0.000 | |
| Received help with basic needs | 57.0 | 51.3 | 5.6 | 0.125 | |
| Solving housing problems | 27.8 | 24.1 | 3.7 | 0.256 | |
| Getting access to medical treatment | 45.3 | 41.1 | 4.2 | 0.257 | |
| Handling a financial emergency | 17.8 | 16.2 | 1.6 | 0.549 | |
| Received help with public benefits | 58.7 | 51.7 | 7.1 * | 0.053 | |
| Getting Medicaid | 55.5 | 47.3 | 8.2 ** | 0.026 | |
| Getting food stamps | 56.0 | 45.4 | 10.6 *** | 0.004 | |
| Received help with job preparation | 36.5 | 18.6 | 17.8 *** | 0.000 | |
| Enrolling in job readiness or training | 27.9 | 12.9 | 14.9 *** | 0.000 | |
| Looking for a job | 28.3 | 13.3 | 15.1 *** | 0.000 | |
| Finding clothes, tools, or supplies for work | 7.8 | 6.1 | 1.7 | 0.372 | |
| Received help with retention and advancement | 12.7 | 7.1 | 5.6 *** | 0.010 | |
| Finding a better job while working | 4.3 | 3.1 | 1.2 | 0.389 | |
| Enrolling in life skills classes while working | 2.1 | 1.8 | 0.3 | 0.797 | |
| Getting a career assessment | 10.0 | 4.2 | 5.8 *** | 0.002 | |
| Dealing with problems on the job | 3.6 | 2.5 | 1.1 | 0.368 | |
| Received help with barriers to employment | 31.8 | 27.8 | 4.0 | 0.240 | |
| Addressing a personal problem that makes it hard to keep a job | 4.2 | 2.4 | 1.8 | 0.163 | |
| Addressing a health problem that makes it hard to find/keep a joba | 24.8 | 22.6 | 2.2 | 0.486 | |
| Finding a job that takes into account health problems | 19.4 | 10.1 | 9.3 *** | 0.000 | |
| Sample size (total = 759) | 380 | 379 | |||
| SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. NOTES: See Appendix F. a This measure includes other activities, such as life skills and child development classes. |
| Table of Contents | Previous | Next |