Table of Contents | Previous | Next |
Chapter 4. Philadelphia: Two Service Models for Welfare Recipients
Introduction
The Philadelphia site in the Hard-to-Employ demonstration is testing two service models designed to increase the employment and earnings of hard-to-employ welfare recipients. This chapter provides information on the policy relevance of the study, descriptions of the programs and the research design, data on the participants’ characteristics at enrollment, and early implementation findings based on program participation data and qualitative field research. The early implementation analysis indicates that intake for the study functioned as planned and that participants were correctly referred to the programs being studied. However, somewhat low enrollment and participation rates presented ongoing challenges to the study design.
Background and Policy Relevance
As welfare caseloads nationwide have declined, policymakers, program administrators, and researchers have increasingly focused attention on long-term and hard-to-employ recipients who have not made a stable transition from welfare to work. While many recipients of Temporary Assistance for Needy Families (TANF) receive welfare grants for a short period in a crisis situation or at a time of brief unemployment, a substantial proportion of the caseload is composed of hard-to-employ recipients, who often remain on TANF for longer periods. Many of these recipients face significant barriers to employment, such as physical health problems, mental health conditions, substance abuse, and limited employment and educational backgrounds.1
Until the 1990s, recipients with serious barriers to work were often exempt from requirements to participate in employment-related activities. During that decade, partly as a result of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996, many states began to extend work requirements to a broader share of the TANF population.2 TANF reauthorization, passed in January 2006, further strengthened the participation mandate, making it crucial that welfare agencies focus on working with hard-to-employ recipients.3 Welfare time limits and economic fluctuations — including the economic downturn from 2001 to 2003 — also increased the need to offer these recipients effective services to assist them in the transition from welfare to work.4
Over the past 30 years, many studies have provided insight into which programs are most effective in assisting recipients to move from welfare to work; however, fewer have targeted more disadvantaged recipients receiving welfare. An analysis of the results from 20 welfare-to-work programs targeted at the general welfare population concluded that the programs generally increased earnings about as much for the more disadvantaged groups (defined in this case as long-term welfare recipients with no high school diploma and no recent work history) as for the less disadvantaged groups. However, the more disadvantaged groups earned considerably less than the others. This outcome suggests that it may be necessary to target resources and develop specific programs to meet the needs of the most disadvantaged TANF recipients.5
The National Supported Work Demonstration, implemented in the 1970s, remains one of the most comprehensive evaluations to date of programs for recipients who are harder to employ. The program offered subsidized employment to long-term welfare recipients and showed particularly large impacts for the most disadvantaged participants within the sample (very long-term recipients and those without a high school diploma).6
As the welfare system evolved to strengthen the participation mandate and provide only temporary cash assistance, the subsidized employment model evolved as well. Facing time-limited welfare and an emphasis on meeting participation rates through employment-related services, administrators shortened the period of subsidized employment and increased the focus on the transition to permanent work. The modified model became known as the transitional employment model. Policymakers and practitioners have recently turned to this restructured model as a promising approach to assist hard-to-employ TANF recipients to leave the welfare rolls. However, further experimental research has not yet been conducted to assess the effectiveness of this model and to understand for which subgroups it is most effective.7
The transitional work model places participants almost immediately into subsidized work, on the assumption that barriers to employment will surface and be resolved through the working process. Another model often used with hard-to-employ TANF recipients is an intensive case management model, focusing on assessing and treating their barriers to employment “up front,” or before they go to work. However, this model has also not yet been rigorously tested.8
The Philadelphia Hard-to-Employ site tests both the transitional employment model and the model to treat barriers to employment up front for TANF recipients who have been identified as hard to employ — those who received TANF for at least a year and/or do not have a high school diploma.9 The evaluation compares each program group with a control group that is not required to participate in any program. It seeks to understand whether the programs improve recipients’ employment, income, earnings, and welfare receipt outcomes, as compared with recipients in the control group. The study will also examine which program model works best for particular subgroups of recipients.
Program Description
Faced with the challenge of how to serve hard-to-employ recipients on the TANF rolls, administrators are seeking to understand the effectiveness of different service models. The models that this study tests grew out of programs that Philadelphia was already implementing and that administrators felt showed promise in assisting more disadvantaged recipients to make the transition from welfare to permanent work.
Both programs in the evaluation are supported primarily by TANF funds, made available from the Pennsylvania Department of Public Welfare (DPW) through the Philadelphia Workforce Development Corporation, the Workforce Investment Act’s fiscal agent. These and other welfare employment and training funds are channeled through a memorandum of understanding between DPW and the Pennsylvania Department of Labor and Industry, which in turn is included in the Workforce Investment Act master agreements.
Transitional Work Corporation Program10
The Transitional Work Corporation (TWC) is administering the transitional employment program. TWC was formed in 1998 in a joint effort among the Commonwealth of Pennsylvania, the City of Philadelphia, the Pew Charitable Trusts, and Public/Private Ventures. It was founded to provide transitional employment to TANF recipients in Philadelphia who had received benefits for at least 24 months and were required to participate in work-related activities. It has since expanded its services to other groups and serves over 1,500 people a year. It is now one of the nation’s largest and most prominent providers of transitional employment to welfare recipients.11
The TWC model begins with a two-week orientation, consisting of intensive job-readiness activities. After the orientation, participants are placed in a transitional job, usually with a government or nonprofit agency, for which TWC pays the minimum wage ($5.15 per hour from the start of the study through December 2006, then $6.25 from January 2007 through June 2007) for up to six months. TWC identifies on-site work partners to provide additional guidance and act as on-the-job mentors during the transitional work period. Recipients are required to work 25 hours per week and to participate in 10 hours of professional development activities at TWC. These activities may include job search and job-readiness instruction, as well as preparation for a certificate of General Educational Development (GED) and other classes. During the transitional work period, TWC staff work with participants to find permanent, unsubsidized jobs. If recipients do not find a permanent job during the six-month transitional work period, staff continue to assist them to obtain unsubsidized employment. TWC also provides job retention services to participants for six to nine months after their placement in a permanent job. In addition, the program offers bonuses of up to $800 for recipients who retain their full-time jobs during the six months following their permanent employment start date. The services offered to participants in the Hard-to-Employ demonstration are the same as those offered to TANF recipients at TWC who are not part of the study.
Program staff at TWC are organized into small teams of four. Each team includes three career advisers, each of whom works with participants during one of the three phases of their trajectory at TWC (orientation, transitional work, and unsubsidized work), and a “sales person” in charge of helping participants find unsubsidized work. Participants work with the four staff members on their assigned team throughout their time at TWC. On the day they arrive, they are assigned to an orientation advisor for the two-week orientation period. After completing orientation, participants transfer to the transitional career advisor, who coordinates placement in a transitional job, as well as professional development activities. When participants are placed in transitional work, they also begin to work with the sales person, who helps place them in unsubsidized employment. Once participants are in an unsubsidized job, they transfer to the retention advisor, who helps coordinate services such as transportation, child care, and bonus payments. In addition to these staff, each participant works with facilitators who lead the orientation classes and the professional development activities.
Success Through Employment Preparation Program
The program focusing on preemployment strategies to remove barriers to work, the Success Through Employment Preparation (STEP) program, is run by Jewish Employment and Vocational Service (JEVS). JEVS is a nonprofit social service agency, founded in 1941, that provides a broad range of education, training, health, and rehabilitation programs in the Philadelphia area. The STEP program was derived from Philadelphia’s Maximizing Participation Project (MPP), a voluntary program for TANF recipients who are exempt from participating in work-related programs because they have a physical or mental disability or because they face multiple barriers to employment.12 It provides intensive case management and support to assess and treat drug and alcohol, behavioral health, and vocational barriers. JEVS is one of the county’s providers for the MPP program. JEVS designed the STEP program based on MPP, but targeted it for recipients who are not exempt from the participation requirement. STEP was developed specifically for this study and serves only study participants. It provides intensive services to help participants eliminate employment barriers and then helps them to find jobs.
In the STEP program, outreach staff first conduct home visits and address any barriers that might keep participants assigned to this group from coming into the office. Once the recipients are enrolled, the program begins with an extensive assessment period to identify participants’ barriers to employment. Specialized staff analyze the results of the assessments and then meet with the participant and her or his primary case manager to design a plan to address these barriers. Treatment can include various life-skills classes (including, for example, GED preparation, English as a Second Language classes, support groups, and professional development sessions) and counseling with behavioral health specialists, as well as ongoing case management meetings. If participants’ barriers are considered severe, staff may refer them to outside organizations for further assessment and treatment. After completing the life-skills courses, participants work with job coaches and job developers to find permanent employment. The timing of the employment search process depends on participants’ individual motivation levels and barriers to employment, but usually does not begin before they have completed the assessments and the team has designed treatment plans. To avoid overlap with the TWC model, participants in the STEP group cannot participate in subsidized employment.
STEP’s program staff are organized into small teams in charge of case management, as well as groups of clinical support specialists and employment services staff. The case management team consists of a case coordinator, who serves as the participant’s primary case manager, and her or his assistants. Participants begin meeting with their case coordinator from the first day they come in to STEP and stay with the same case coordinator throughout their time in the program. The case coordinators provide general case management and coordinate recipients’ interactions with the clinical support specialists and employment services staff. The clinical support specialists include behavioral health specialists, assessment counselors, and instructors. The employment services staff include job developers, who are in charge of helping participants find employment, and job coaches, who work with participants to help them retain jobs.
Research Design
This evaluation uses a random assignment design to determine whether the TWC and STEP program models are effective in assisting recipients to make the transition from welfare to work. The study is not a direct comparison of the two models; program participants’ outcomes will be compared with outcomes for participants in a control group who are not required to participate in any work-related activities. In addition, the study seeks to understand whether the models are generally more effective in assisting certain subgroups of recipients and which model best serves particular subgroups.
The target population for the study is TANF recipients who have received cash assistance for at least 12 months in their lifetime or who do not have a high school diploma. The study does not include “U” cases13 (two-parent cases, with some exceptions), recipients who are exempt from participation or have good cause not to participate, and recipients who are currently employed.
Recipients who met the study criteria were randomly assigned at the Philadelphia County Assistance Offices, or public benefits offices, into one of the two program groups or into the Voluntary Services group, which serves as the control group. The group to which they were assigned was selected entirely at random — it was not based on any background characteristics or assessments. Approximately 37.5 percent of the sample members were assigned to the TWC group; 37.5 percent to the STEP group; and 25.0 percent to the control group. Recipients placed in one of the program groups were referred from the public benefits offices to the appropriate program — TWC or STEP — and received the services described above. Voluntary Services recipients were given a list of community resources but were not required to participate in employment activities. They could choose to participate in any work or education-related activities, with the exception of TWC and STEP, but were not penalized for failing to meet the work requirement. Participants retain their group assignments for approximately three years; during that time, individuals in one group are not allowed to receive the services offered to the other groups.14
Because random assignment occurred in the welfare offices rather than at the point of entry into the programs, the study includes many people who did not receive the services they were referred to because they never showed up to the programs or quickly dropped out without ever participating substantially. The samples for both program groups include some of these cases, discussed in more detail later in the chapter. This may make the differences between the outcomes for the program groups and those for the control group smaller, because the program groups will include individuals who did not receive any services, similar to many control group members. However, this design provides a structure that also offers insight into the welfare system in which these programs operate.
MDRC will continue to track the members of all three groups for at least three years after random assignment. Several data sources will inform the evaluation:
-
Implementation research, including site visits and interviews with staff, will be used to understand how the services were implemented.
-
DPW’s and the programs’ databases will be used to examine the participation of program group recipients in TWC and STEP and the participation of control group recipients in activities recorded by public benefits offices staff (such as GED or job search programs).
-
Administrative records data will be used to measure participants’ receipt of welfare and food stamps, as well as their earnings and employment in jobs covered by unemployment insurance. MDRC has obtained wage data from the National Directory of New Hires. This is a national database maintained by the Office of Child Support Enforcement that can provide information on earnings from employment both within and outside Pennsylvania.
-
Surveys will track over time participants’ employment outcomes (including informal work outcomes that do not show up in the administrative data), health insurance outcomes, receipt of services (such as employment services, mental health services, or substance abuse counseling not provided through DPW), and other outcomes.
Because of the random assignment design, any significant differences that emerge between each of the program groups and the control group (the Voluntary Services group) will be attributable to the services provided by the programs. In other words, the Voluntary Services Group will provide a counterfactual against which the programs can be compared.
Random Assignment and the Sample Intake Process
Figure 4.1 provides an illustration of the sample intake process. Intake workers screened TANF applicants to ensure that they met the study criteria and sent them to a research aide stationed in the County Assistance Office.15 Career Development Unit workers, who are responsible for assigning recipients to employment and training activities, screened ongoing TANF recipients who were not participating in another activity. They then sent the recipients to a research aide for random assignment. Most recipients sent to the research aides entered the study, but recipients who showed that they already had specific education or employment-related plans were able to opt out of participating. Those who entered the study signed a data release form. The aide also collected their baseline information, including age, gender, race, education and employment history, family and living circumstances, and number of months of TANF receipt. Participants received a $10 gift card to compensate for the time they spent providing this information. The aide then randomly assigned recipients via a Web-based system or the telephone to one of the two program groups or to the control group.
 [D] |
Fewer recipients than expected were referred to MDRC’s research aides in the first few months of sample enrollment, resulting in intake rates that were lower than predicted. To increase the flow of recipients into the study, MDRC and DPW decided early on to expand the eligibility criteria. Initially, the study included only incoming TANF applicants (both new applicants and re-applicants). However, in November 2004, DPW and MDRC agreed to include ongoing TANF recipients who were not participating in any employment-related activities. This expansion may result in reaching a somewhat harder-to-employ population, as these recipients were already not in compliance with the work requirements. In addition to expanding the eligibility criteria, MDRC and DPW also worked with the County Assistance Offices to identify procedures to ensure that all potential participants were referred to the study.16
MDRC completed random assignment at the end of May 2006 with 1,944 participants, which neared the goal of 2,000 participants. With a sample of this size, the study has the potential to show impacts that have important policy relevance. For example, if the programs improve outcomes by even a small amount, the resulting changes over a larger population would be substantial.
Early Findings from the Assessment
This report examines baseline data on participants’ demographic characteristics, as well as participation data and qualitative field research data that allowed MDRC to provide an early implementation analysis of the programs. MDRC collected the following data:
-
Participant demographic data: MDRC’s research aides collected baseline demographic data for each participant at the time of random assignment. In this report, baseline and sample build-up data included all 1,944 sample members who entered the study from October 12, 2004 (when study enrollment began), through May 31, 2006 (when study enrollment ended).
-
Participant referral, enrollment, and participation data: Program referrals were entered into Pennsylvania’s Client Information System and transferred to the Automated Interface Management System (AIMS) database. This is the process by which recipients are referred to welfare-to-work contractors, and is used to track program activity information, including actual hours of participation, program rejections, and terminations. For this report, MDRC used information on program activities from the AIMS database for all three research groups.17 Program participation data were analyzed for 248 recipients who were randomly assigned between the start of random assignment and December 31, 2004. The analysis includes four and a half to six and a half months of follow-up with these recipients. Some data were also collected from STEP and TWC at a later date to conduct preliminary participation analyses for a larger proportion of the sample.
-
Qualitative program implementation data: MDRC staff visited the two program sites in May 2005 and interviewed case management staff to understand the structure of the programs and the activities in which recipients were participating. In addition, MDRC staff interviewed employees at the County Assistance Offices in order to clarify the intake process and the participation-monitoring procedures.
Characteristics of the Sample
Table 4.1 shows the baseline characteristics of the sample by research group. The characteristics across the three groups are very similar, which is expected because of the random assignment design. In addition, the participants’ characteristics indicate that the study is reaching a hard-to-employ population, as intended.
The average age of the participants is about 29. The large majority of the participants are female (not shown on the table). Just over 80 percent are black and about 14 percent are Hispanic.
Many of the participants have considerable barriers to employment, including low education levels, limited employment history, and responsibilities caring for children under 6. Compared with TANF recipients in other studies, the sample in this study have greater barriers to employment. Over half (56 percent) do not have a high school diploma or a GED.
| Characteristic | TWC Groupa | STEP Groupb | VS Groupc | Total | |
|---|---|---|---|---|---|
| Age (%) | 18 to 24 years | 34.0 | 35.2 | 33.3 | 34.3 |
| 25 to 30 years | 27.5 | 27.6 | 29.0 | 27.9 | |
| 31 to 40 years | 26.5 | 26.2 | 27.3 | 26.6 | |
| 41 years or older | 12.0 | 11.0 | 10.5 | 11.3 | |
| Average age (years) | 29.5 | 29.3 | 29.2 | 29.3 | |
| Race/ethnicity (%) | Black, non-Hispanic | 78.8 | 84.2 | 81.5 | 81.5 |
| Hispanicd | 17.1 | 11.8 | 13.5 | 14.2 | |
| White, non-Hispanic | 2.6 | 2.9 | 3.3 | 2.9 | |
| American Indian or Alaskan Native | 0.1 | 0.1 | 0.0 | 0.1 | |
| Asian or Pacific Islander | 0.3 | 0.0 | 0.2 | 0.2 | |
| Other | 1.1 | 1.0 | 1.5 | 1.1 | |
| Education (%) | High school diploma | 29.0 | 32.1 | 27.1 | 29.7 |
| GED certificate | 8.3 | 6.7 | 6.6 | 7.3 | |
| Technical/associate's/2-year college | 6.4 | 7.3 | 5.5 | 6.5 | |
| 4 years or more of college | 1.1 | 0.6 | 0.4 | 0.7 | |
| None of the above | 55.2 | 53.3 | 60.4 | 55.8 | |
| Highest grade completed (%) | 8th grade or lower | 4.6 | 3.5 | 4.6 | 4.2 |
| 9th grade | 10.3 | 9.2 | 10.2 | 9.9 | |
| 10th grade | 18.2 | 19.3 | 20.8 | 19.2 | |
| 11th grade | 29.4 | 28.9 | 31.5 | 29.7 | |
| 12th grade | 32.0 | 33.8 | 27.5 | 31.6 | |
| Beyond 12th grade | 5.5 | 5.3 | 5.4 | 5.4 | |
| High school diploma or GED certificate (%)* | Yes | 44.8 | 46.7 | 39.6 | 44.2 |
| No | 55.2 | 53.3 | 60.4 | 55.8 | |
| Marital status (%) | Unmarried, not living with a partner | 91.3 | 89.5 | 90.2 | 90.3 |
| Married, living with spouse | 1.7 | 1.5 | 2.9 | 1.9 | |
| Married, separated | 5.8 | 6.0 | 4.4 | 5.5 | |
| Unmarried, living with a partner | 1.2 | 3.0 | 2.5 | 2.2 | |
| Number of children under age 18 (%) | None | 4.0 | 2.6 | 3.3 | 3.3 |
| 1 child | 32.9 | 31.3 | 28.5 | 31.2 | |
| 2 children | 28.6 | 30.6 | 31.8 | 30.1 | |
| 3 children | 18.4 | 18.2 | 20.7 | 18.9 | |
| 4 children or more | 16.1 | 17.2 | 15.6 | 16.4 | |
| Average number of children | 2.2 | 2.3 | 2.3 | 2.2 | |
| Children under age 6 (%) | Yes | 63.7 | 65.2 | 66.1 | 64.9 |
| No | 36.3 | 34.8 | 33.9 | 35.1 | |
| Limited English (%) | Yes | 5.3 | 4.4 | 5.0 | 4.9 |
| No | 94.7 | 95.6 | 95.0 | 95.1 | |
| Public housing (%) | Yes | 28.5 | 33.5 | 32.0 | 31.3 |
| No | 71.5 | 66.5 | 68.0 | 68.7 | |
| Housing status (%) | Rents house or apartment | 60.0 | 62.6 | 63.4 | 61.8 |
| Owns house or apartment | 6.1 | 6.1 | 5.6 | 6.0 | |
| Lives with friends or relatives | 30.1 | 27.6 | 27.2 | 28.4 | |
| Has transitional/ emergency/temporary housing |
2.6 | 2.6 | 3.5 | 2.8 | |
| Is homeless, living on the street | 0.1 | 0.0 | 0.0 | 0.1 | |
| Other | 1.1 | 1.1 | 0.2 | 0.9 | |
| TANF receipt (%) | Never received TANF | 6.7 | 5.5 | 5.3 | 5.9 |
| Less than 1 year | 9.0 | 10.3 | 7.8 | 9.2 | |
| 1-2 years | 16.3 | 14.5 | 15.6 | 15.4 | |
| 2-5 years | 43.0 | 44.3 | 44.1 | 43.8 | |
| 5 years or more | 25.0 | 25.4 | 27.1 | 25.7 | |
| Average months of TANF receipt | 39.9 | 40.4 | 40.7 | 40.3 | |
| Always lived in United States (%)*** | Yes | 90.6 | 94.7 | 94.7 | 93.2 |
| No | 9.4 | 5.3 | 5.3 | 6.8 | |
| If not always, average years lived in United States | 9.5 | 8.7 | 10.6 | 9.5 | |
| Ever employed (%) | Yes | 91.7 | 92.6 | 90.3 | 91.7 |
| No | 7.7 | 6.6 | 8.4 | 7.5 | |
| Time since last employment (%) | 6 months or less | 37.6 | 38.8 | 41.5 | 39.0 |
| 7-12 months | 14.8 | 13.5 | 13.8 | 14.0 | |
| 13-24 months | 16.3 | 18.8 | 14.4 | 16.7 | |
| More than 24 months | 21.3 | 20.1 | 18.7 | 20.2 | |
| Missing | 10.1 | 8.8 | 11.7 | 10.0 | |
| Average months since last employment | 18.1 | 18.3 | 16.0 | 17.6 | |
| Ever worked 6 or more months for one employer (%) | Yes | 69.9 | 70.1 | 66.5 | 69.1 |
| No | 28.8 | 28.3 | 31.4 | 29.3 | |
| Months employed in past 3 years | None (Did not work) | 19.7 | 19.3 | 20.1 | 19.7 |
| Less than 6 months | 24.2 | 19.3 | 22.0 | 21.8 | |
| 7-12 months | 24.6 | 24.1 | 21.1 | 23.6 | |
| 13-24 months | 15.7 | 20.1 | 20.7 | 18.6 | |
| More than 24 months | 14.5 | 16.0 | 13.8 | 14.9 | |
| Month of random assignment | October 2004 | 1.4 | 1.1 | 1.6 | 1.3 |
| November 2004 | 4.8 | 4.0 | 4.5 | 4.4 | |
| December 2004 | 6.8 | 7.2 | 7.0 | 7.0 | |
| January 2005 | 7.0 | 7.3 | 7.2 | 7.2 | |
| February 2005 | 9.3 | 8.6 | 8.6 | 8.8 | |
| March 2005 | 7.9 | 9.1 | 8.8 | 8.6 | |
| April 2005 | 4.4 | 4.0 | 4.1 | 4.2 | |
| May 2005 | 5.3 | 5.9 | 5.5 | 5.6 | |
| June 2005 | 4.9 | 4.8 | 4.7 | 4.8 | |
| July 2005 | 1.6 | 1.5 | 1.6 | 1.6 | |
| August 2005 | 5.3 | 5.2 | 5.1 | 5.2 | |
| September 2005 | 4.0 | 4.1 | 4.1 | 4.1 | |
| October 2005 | 3.3 | 3.6 | 3.7 | 3.5 | |
| November 2005 | 2.5 | 2.2 | 2.5 | 2.4 | |
| December 2005 | 2.0 | 2.3 | 2.1 | 2.2 | |
| January 2006 | 2.6 | 2.1 | 1.8 | 2.2 | |
| February 2006 | 6.4 | 6.6 | 7.0 | 6.6 | |
| March 2006 | 8.9 | 9.0 | 8.6 | 8.8 | |
| Sample size | 732 | 725 | 487 | 1,944 | |
| SOURCE: Philadelphia Baseline Information Form. NOTES: In order to assess differences in characteristics across research groups, chi-square tests were used for categorical variables, and analysis of variance (ANOVA) tests were used for continuous variables. Levels for statistically significant differences between program and control groups are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. a TWC: Transitional Work Corporation. b STEP: Success Through Employment Preparation. c VS: Voluntary Services. d Sample members are coded as Hispanic if they answered “yes” to that race/ethnicity category. |
In comparison, a study of welfare recipients in three cities in Texas showed that approximately 45 to 55 percent did not have a high school diploma or a GED, and a study of recipients identified as hard to employ in Hennepin County (Minneapolis), Minnesota, showed that about 46 percent did not have a high school diploma or a GED.18
Approximately 92 percent of the Philadelphia sample participants have been previously employed; however, about two-thirds have worked a year or less in the past three years. This indicates a more limited work history, compared with recipients in the Texas and Minnesota studies. In Texas, 51 to 58 percent of recipients had worked a year or less in the past three years and, in Minnesota, 57 percent of hard-to-employ recipients had worked a year or less during that time.
Approximately two-thirds of the Philadelphia participants have a child under 6, and the average number of children under 18 is 2.2. A higher proportion — 35 percent — of the Philadelphia participants have three or more children, compared with participants in the Texas study (26 to 32 percent) or in the Minnesota study (32 percent).
In addition, according to data from the Philadelphia County Assistance Offices, about 70 percent of the sample members have received TANF for more than two years, and the average number of months of TANF receipt is about 40. National law stipulates that federal funding cannot be used toward recipients’ TANF grants after they have received 60 months of TANF, except under certain circumstances. Many participants in the demonstration are nearing this limit, and over a quarter are already receiving Extended TANF, Pennsylvania’s state-funded assistance for recipients who have received TANF for over 60 months.
Program Enrollment and Participation
Early data indicate that the referral process functioned correctly, and that almost all recipients were referred properly. Usually the referral took place very soon after the recipient was randomly assigned, and the programs received timely notice of the referrals.19 However, there were some early concerns regarding enrollment and participation levels at both programs. The percentage of recipients who enrolled in TWC’s program was somewhat low, although this was anticipated, and discussion with TWC confirmed that the enrollment rate was not unusual. In addition, there was concern that recipients at STEP were not participating for a high number of hours and may not have received services beyond assessment. The program made some changes to address this issue, and subsequent analysis suggests that participation may have increased.
Program Enrollment
When a recipient was assigned into one of the two programs, County Assistance Office staff entered the appropriate program code into the state’s database and the provider received notice of the referral the following day through the interface system. Early data indicated that this process worked correctly for most participants; as shown in Table 4.2, almost all participants were referred to the program to which they were assigned.
Once a provider received a referral from the County Assistance Office, program staff could choose whether or not to officially enroll the recipient in the program. Each program determined its own criteria for enrollment. TWC and STEP differed in their enrollment processes and in how they determined which recipients to enroll. Table 4.2 indicates the proportion of participants who enrolled in both programs.
STEP enrolled recipients after the program’s outreach staff completed the initial home visit and the recipient came into the STEP office to meet with her or his case coordinator. Most (80.9 percent) of the early recipients referred to STEP completed this process and enrolled in the program. Those who did not enroll included recipients who could not be located by the outreach workers and those who were located but never came in to the office. Participants’ early attendance and participation, after the first visit, did not affect the enrollment process.
| Outcome | TWCa | STEPb | Voluntary Services |
|---|---|---|---|
| Referred to program (%) | 92.6 | 95.5 | NA |
| Enrolled in program (%) | 55.8 | 80.9 | NA |
| Ever participatedc (%) | 55.8 | 80.9 | 28.1 |
| Average total hours of participationd | 172 | 24 | 168 |
| Sample sizee | 95 | 89 | 64 |
| SOURCE: MDRC calculations from Pennsylvania's Automated Interface Management System (AIMS) database. NOTES: The table reflects referrals, enrollment, and participation through May 2005. a TWC: Transitional Work Corporation. b STEP: Success Through Employment Preparation. c TWC does not enter into the AIMS database the hours that customers participated in orientation activities, because it is likely that some customers participated in these activities but were never officially enrolled in the program and do not show any hours. d The sample for the average total hours of participation includes only those who ever participated (53 in the TWC group, 72 in the STEP group, and 18 in the Voluntary Services group). For TWC, if a customer completed the orientation and enrolled in the program, 30 hours were added to the hours of participation. (Ten customers enrolled in TWC but did not have any other activity hours.) This adjustment was not necessary for STEP customers. STEP’s internal Management Information System showed a slightly higher number of hours than the AIMS database for some clients. e The sample in this table is limited to recipients who were randomly assigned between October 12, 2004 (when study enrollment began), and December 31, 2004, to allow between four and a half months and six and a half months of follow-up. |
During most of the study period, TWC enrolled recipients only after they had completed the program’s two-week orientation process discussed above. The proportion that never enrolled at TWC thus included recipients who never attended the program at all, as well as those who showed up but did not complete the full orientation. Therefore, early data show that enrollment at TWC was somewhat low (55.8 percent). Follow-up with participants who did not show up or had poor initial attendance varied among TWC staff, but TWC’s general philosophy was that participants who did not consistently attend orientation may not have been ready for the program. TWC staff confirmed that, according to TWC’s program model, they traditionally did not conduct extensive outreach with recipients and that the relatively low enrollment rate was not unusual. MDRC did not encourage TWC to significantly modify its program model to attempt to enroll more recipients.
Nonetheless, the relatively low enrollment rate is of concern for the study, because if the program has impacts for the recipients who do participate, these effects may be diluted when evaluating the results of the full TWC sample. MDRC hoped the issue would be addressed through Philadelphia’s presanction review process, in which third-party providers attempt to locate recipients who are not participating and bring them into compliance. However, it is unclear whether this system provided a strong mandate to participate during the research period for this report.
In summer 2005, DPW issued a directive to limit multiple referrals to the third-party sanction reviewers in order to simplify the process of implementing sanctions in cases of repeated noncompliance. In addition, Pennsylvania issued memos to staff in June, October, and December 2005, describing a universal engagement process and reiterating the new sanction procedures, in which offices would receive lists of unengaged recipients and would be required to follow up with them. However, these processes were not in place when the research for this report was conducted.
Program Participation
MDRC also measured the number of hours that recipients participated in the programs. Table 4.2 shows the percentage of early TWC, STEP, and Voluntary Services participants who ever participated and the average number of hours of group members’ participation during the four and a half to six and a half months of follow-up.
Over the course of the follow-up period for this report, TWC recipients averaged a relatively high number of hours of participation — 96 hours (not shown in table). (This calculation includes group members who did not show any hours of participation. Among those who showed any hours of participation, the average number of hours was 172. There may also be some recipients who participated in some orientation activities but who do not show any participation hours, because TWC does not record those hours in the AIMS database unless the participant completes orientation.) Most of the recorded hours were probably in transitional or unsubsidized jobs, as case management staff explained that many recipients did not participate as regularly in professional development activities as they did in employment.
Early data indicated that recipients in the STEP program participated for a relatively small number of hours — 19 hours — although program staff later took steps to increase participation. (Among those who showed any participation, the average was 24 hours.) Based on field research at STEP, MDRC was concerned that many STEP recipients were not receiving significant services beyond initial outreach and assessment and that some recipients were not even participating in assessment activities. Interviews with case management staff at STEP revealed that, despite ongoing outreach efforts, some recipients did not come in for the assessments and others took several months to complete them. Further, even for those who did complete the assessments, there was sometimes a lengthy lag time before staff analyzed the results, because few staff were allocated to this task. Because the program design required that the assessment phase be completed before recipients could participate in most other activities, many recipients remained in a waiting period, during which most of their treatment consisted of life-skills classes for a few hours per week and meetings with case management staff. As a result of the low participation hours, the differences in outcomes for the STEP group and the control group may be smaller than if more participants had received services beyond assessment. It also suggests that it will be important to try to understand the effects of receiving the full set of STEP services (as opposed to the average effects for everyone assigned to the STEP group or even everyone who showed any hours of participation) in order to draw implications about the potential effect of the program without the lag between assessment and other program services.
To address these concerns, STEP managers reported that they implemented strategies to increase participation, including streamlining the assessment process and providing immediate engagement activities that recipients could participate in concurrently with the assessments.
Although recipients in the Voluntary Services group (the control group) were not required to participate in any programs, a substantial proportion reported to the County Assistance Offices that they participated in activities during the follow-up period.20 The majority of this participation was in vocational-type education programs, GED or high school classes, and basic education activities, although a few were engaged in structured job search activities.
Updated Program Enrollment and Participation Data
MDRC was also able to conduct preliminary analyses of enrollment and participation, using data from TWC and STEP, for a larger proportion of the research sample.21 The percentage of the sample that was referred to each program remains high, although it is slightly lower for both programs, compared with the earlier data. The percentage that enrolled also remains similar, but increases slightly for the TWC group.
At TWC, the proportion that was ever placed in a transitional job is similar to the earlier data on the proportion of the sample that had ever participated in the program, about 60 percent. The TWC data also show that, among those who enrolled in the program, almost all were placed in a transitional job. This indicates that most enrolled participants received at least some treatment.
More recent data from STEP on the proportion of the sample that ever participated also correlates with the earlier data, about 80 percent of the sample. The new data also indicate that the number of hours of sample members’ participation increased, although it is unclear whether this reflects an increase in the number of hours recipients spend in activities or improvement in recording practices.
Conclusions
The Philadelphia site of the Hard-to-Employ evaluation tests two service models designed to offer employment-related services to hard-to-employ welfare recipients. The TWC program provides up to six months of transitional employment, combined with case management services and professional development activities. The STEP model provides intensive services to assess and treat barriers to employment, followed by job search services. The evaluation is an important test of two programs that may potentially assist hard-to-employ welfare recipients to find employment. It will compare each model with a control group of recipients who are not required to participate in any activity.
The early implementation analysis shows that the random assignment process worked correctly and that sample members show some participation in both programs. However, somewhat low enrollment and participation rates presented ongoing challenges to the programs, as well as to state and local welfare administrators. The TWC group has a relatively low enrollment level, because participants were enrolled only after they completed a two-week orientation. TWC staff confirmed that this is standard for its program model and that the enrollment process for the evaluation functioned no differently than it did for nonsample members. The STEP group, on the other hand, shows somewhat low participation rates, even among those enrolled, because many participants remained in the assessment phase for lengthy periods without receiving other services. STEP administrators reported that they implemented changes to increase participation, including offering immediate engagement activities during the assessment period. Preliminary analysis of participation data confirms that participation did increase. Further research in subsequent reports will indicate the impacts of the programs on participants’ welfare, employment, income, and earnings outcomes.1 For example, one study synthesized results from a survey that was administered to welfare recipients in six states in 2002. It found that 40 percent of recipients lacked a high school diploma or GED, 21 percent had a physical health limitation, 30 percent met the diagnostic criteria for major depression or were experiencing severe psychological stress, and 29 percent had a child with health problems (Hauan and Douglas, 2004). (back)
2 Bloom and Butler (2007). (back)
3 TANF reauthorization strengthened the participation mandate in several ways. It adjusted the caseload reduction credit — by which states can reduce their minimum required participation rate if they reduce their caseload — so that the baseline year against which the current caseload is compared is 2005, rather than 1995. The bill also required states to count toward the participation rate families receiving TANF through separate state programs — programs that receive no federal TANF funding but do receive state funding that counts toward the state’s Maintenance of Effort requirement. In addition, the bill called on the U.S. Department of Health and Human Services to disseminate more explicit regulations on countable activities and required states to implement stricter internal controls to verify reporting procedures. (back)
4 According to the National Bureau of Economic Research, the economy went into recession beginning in March 2001. Employment declines lasted through August 2003. (back)
5 Michalopoulos and Schwartz (2000). (back)
6 The Board of Directors, MDRC (1980). The National Supported Work Demonstration showed different results for different subgroups: For example, it showed significant results for welfare recipients but not for ex-offenders. (back)
7 However, the nonexperimental research into transitional work is promising. For example, a study of six transitional work programs found that rates of placement in permanent, unsubsidized employment for participants who completed the programs ranged from 81 to 94 percent (Kirby et al., 2002). See also Pavetti and Strong (2001). (back)
8 MDRC’s Employment Retention and Advancement Project has one site — Minneapolis — that tests an intensive case management strategy to treat barriers to employment before recipients go to work, although participants in this program may also be placed into transitional employment. Early results of this test are published in LeBlanc, Miller, Martinson, and Azurdia (2007). (back)
9 The transitional employment model being studied in Philadelphia is similar to the model being tested in the New York site for this project; however, the New York program is targeted at ex-offenders, rather than TANF recipients. (back)
10 This section describes the Transitional Work Corporation program as it operated during most of the study period; there may have been changes since then. (back)
11 A 2004 report examining the TWC program showed increased employment and earnings outcomes and decreased TANF receipt outcomes for TWC participants. However, the study did not use a random assignment design to compare the outcomes with those of similar individuals who did not receive TWC services (VanNoy and Perez-Johnson, 2004). (back)
12 The Maximizing Participation Project becomes mandatory for these recipients if they are receiving TANF after they have reached the 60-month time limit. (back)
13 A family meets the criteria for “U,” or the unemployed parent category, if: it is a two-parent household with at least one common child; at least one parent is able to work; and both parents are unemployed, or at least one parent has work in which the net earned income of the TANF budget group (after allowable deductions) is less than the family size allowance for the budget group, or at least one parent has “on the job training” in a project approved or recommended by the Job Service of the Road to Economic Self-Sufficiency through Employment and Training (Pennsylvania’s TANF program). (back)
14 If TWC or STEP decides to permanently terminate a recipient, she or he is still mandated to participate, and it is possible that she or he would participate in a program that Voluntary Services recipients can participate in. (back)
15 The research aides were MDRC employees. Three of the four research aides initially hired were former Pennsylvania TANF recipients. (back)
16 Sample intake was also lower than expected because random assignment ended early in one office. Random assignment initially took place in three of the Philadelphia County Assistance Offices. The study design called for random assignment to continue in all three offices until the sample goal was met. However, one office ended random assignment in June 2005 to take part in a countywide initiative to provide case management services at nonprofit providers, rather than at the public benefits offices. This left only two offices. One of these offices closed down seven months later, in January 2006, although an additional office was identified to begin random assignment at that time. Intake at this added office proceeded rapidly and helped to make up for the slower flow of intake earlier in the random assignment process. (back)
17 Although program participation data were available from the programs’ MIS systems, these data were not included in the main participation analysis for this assessment in order to maximize comparability of measures across all three research groups. The data primarily used in this report reflects the information that DPW receives from the programs about recipient participation; it may not be exactly comparable with each program’s internal tracking system. In addition, the 15-month survey, which began in early 2006, asked the recipients directly about their participation and may capture information about program participation that is not captured in the data presented in this report. (back)
18 Martinson and Hendra (2006); LeBlanc, Miller, Martinson, and Azurdia (2007). The Minnesota study includes recipients who had been assigned to TANF employment services for 12 months or longer, were unemployed at the time of random assignment, and had not worked in the preceding three months. (back)
19 The referral process was designed to prevent staff from randomly assigning recipients who were ineligible to participate in the study. Nevertheless, a small proportion of recipients were randomly assigned but never referred to the programs, because Career Development Unit staff later determined that they were not required to participate in work activities for reasons such as medical exemptions or responsibilities caring for children under age 1. However, this did not happen often enough to be of major concern. (back)
20 Voluntary Services recipients could report participation in order to receive supportive services such as child care and transportation. (back)
21 Data were analyzed for all participants in the STEP and TWC groups randomly assigned through December 2005. The data include at least three months of follow-up for STEP participants and at least six months of follow-up for TWC participants. (back)
| Table of Contents | Previous | Next |