Table of Contents | Previous | Next |
Evaluation Results
ACT’s Implementation and Operation
TDHS evaluation staff documented the implementation of the Texas ACT waiver demonstration and its operation over time. To inform the impact findings, they also examined how well staff and clients understood various ACT waiver policies. The evaluators conducted several rounds of site visits to local TDHS pilot offices, where they interviewed and observed staff interacting with clients. They also surveyed staff and clients and reviewed historical documents, policy documents, and administrative data relevant to the demonstration. Finally, through a contract with the University of North Texas, they surveyed experimental and control group members to ascertain how well each group understood state and federal time limits.
Key findings from this analysis are presented here, along with an analysis of clients’ overall understanding of time limits. More detailed results can be found in the full reports on the process evaluation, Achieving Change for Texans: Final Process Evaluation Report and Understanding of Time Limits: Supplement to Achieving Change for Texans Process Evaluation Report.
As displayed in Figure 1, the Time Limits pilot operated in Bexar County, which includes the San Antonio metropolitan area. The RER Choices pilot operated in Beaumont, Odessa, Corpus Christi (Dillon office) and El Paso (Clint office). RER Non-Choices operated in four TDHS offices in smaller communities and rural areas throughout Texas (Hondo, Huntsville, Lockhart and Luling).
 [D] |
| Source: Act Process Evaluation |
- An automated system assigned and tracked participation in this demonstration properly. This system provided consistency during a period when eligibility workers did not fully understand the complex rules of the ACT demonstration.
An automated system was designed to aid the implementation of the ACT pilots. The automated system ensured that ACT policies were applied appropriately, both in the pilot offices and in the remainder of the state. This system randomly assigned TANF clients in ACT pilot offices to the experimental or control groups and used various codes (known as ‘pilot indicators’) to apply the proper combination of policies in pilot offices. These functions included assigning clients’ time-limit tiers, tracking how many months of TANF benefits were used by clients toward time limits, assigning appropriate penalties for non-compliance with the PRA, and removing persons from the experiment when changes in their life circumstances contaminated their treatments in some way.
The automated system developed for the ACT demonstration worked properly throughout the demonstration period. With policies as complex as those in some of the waiver sites and staff turnover in some of the waiver sites and staff turnover in some local pilot offices, staff often relied upon the information in the automated system to inform them of appropriate policies to apply and to guide their interactions with clients.
- Exemption from workforce services was the main reason that clocks did not ‘tick’ in the two experiments with state time limits.
TDHS evaluators analyzed administrative data in September 1999 and September 2001 to determine how quickly experimental group members were advancing toward their state time limits. In the two experiments that included time limits (TL and RER Choices), researchers found that the majority of state time limits clocks in both pilots did not ‘tick’ in September 1999 or September 2001, although the share of non-ticking clocks declined between the two periods (Table 3). This means that fewer than half of persons who were subject to state time limit policies actually advanced toward that time limit.
Over half of those whose clocks were not ticking were exempt from participating in Choices, which is the trigger for the application of Texas time limits. The share of clients who had not been notified to begin participating in Choices or with a closed Choices case — other reasons that clocks did not tick — declined between the two periods.
Except in Clint, most exempt clients were caring for a young child. Exemptions for this reason decreased somewhat between the two periods because of 1999 policy changes that restricted eligibility for this exemption. Incapacitation was the second most frequently used exemption. Use of this exemption increased between the two periods, most likely because some persons who had previously used the ‘age of child’ exemption were also incapacitated in some way. In Clint, the most common exemption was for being too remote due to its location on the outskirts of El Paso.
| TL | TL | RER Choices | RER Choices | |
|---|---|---|---|---|
| Date of time limit clock analysis | 9/1999 | 9/2001 | 9/1999 | 9/2001 |
| Share of time limit clocks not ticking | 79% | 56% | 75% | 57% |
| Reasons clock did not tick | ||||
| Exempt from Choices | 54% | 65% | 62% | 62% |
| Not notified of Choices slot | 17% | 6% | 11% | 5% |
| Closed Choices case | 17% | 1% | 11% | 5% |
| First year after Tier 3 notice | 6% | 14% | 8% | 10% |
| Good cause | 2% | 4% | 2% | 5% |
| Other | 5% | 11% | 7% | 12% |
| Type of Choices exemption | ||||
| Caring for young child | 67% | 26% | 62% | 33% |
| Incapacitated | 12% | 33% | 13% | 28% |
| Other | 21% | 41% | 25% | 39% |
| Source: ACT Process Evaluation |
- Staff initially had difficulty understanding and explaining ACT time limit provisions to clients, but their performance in this area improved over time.
TDHS evaluators observed the operations in local ACT demonstration offices three different times during the experiment, interviewing staff in these offices and talking to clients after they had met with TDHS eligibility workers. They also administered several staff surveys about key features of the evaluation.
During the first site visits in the fall of 1997, eligibility staff reported some confusion regarding time limits policies and tended to rely on the automated system through a series of computer-generated letters to provide time limits information to clients. Workers in the RER Choices sites relied on the automated system to a greater extent than did workers in the TL sites. In both the TL and RER Choices sites, some eligibility workers erroneously informed control group clients about state time limit policies. Local office staff members were concerned about giving incorrect information to experimental and control group members and were pessimistic about the potential for the experimental interventions to affect client behavior. In RER Choices sites, staff also cited the difficulty of accurately implementing different policies for experimental and control group members in the same office.
By the second local visits in calendar years 1999 and 2000, workers did not discuss state time limits with control group members (to whom these policies did not apply) during eligibility interviews. However, an Internet survey of eligibility workers found that most eligibility workers still did not understand the pilot indicators or the policies associated with the control group.
- Clients understood that there were time limits but few knew what would happen to their benefits (TANF, food stamps, Medicaid) when they reached theirs.
In the first two rounds of interviews, experimental group clients in both the TL and RER Choices pilot sites knew they faced a time limit. In the TL sites but not the RER Choices sites, a large number of control group members (to whom state time limits did not apply) also thought that their TANF benefits were time-limited. Across all sites, most experimental clients did not know what would happen to their benefits (TANF, food stamps, Medicaid) when they reached their time limit. Almost half of interviewed eligibility workers felt that experimental group clients did not understand the state time limits message “at all.”
During the third phase of the process evaluation, a special analysis was conducted in the TL and RER Choices sites to learn why clients had such a poor understanding of time limits. Evaluators interviewed staff and clients in TDHS offices and mailed a survey to experimental and control group clients in both pilots. They found that:
-
The general message, “TANF benefits are time limited,” was presented effectively enough to make most TANF clients aware of time limits. However, some clients confused time limits with other time periods related to their case (such as the recertification period). Few clients correctly understood the implications of time limits — especially federal time limits — and few could correctly estimate the amount of time left on their state and federal clocks.
-
Eligibility workers typically did not spend their limited interview time trying to explain complex time limits policies to clients.
-
Clients typically did not read or recall the information about their time limits that was printed on their TANF eligibility or recertification notices. When an eligibility worker told a client about time limits, the client often did not seem to remember what she was told minutes later.
-
Responses of experimental groups on the client mail survey were similar across TL and RER Choices sites. The most frequent response to survey questions about time limits for both experimental and control group members was, “I don’t know.” But experimental group clients were somewhat more likely to know that they had a time limit and to understand the implications of state and federal time limits better than control group clients.
- Persons who had reached their state time limits understood the implications of state time limits better than persons still on TANF but were less aware of the meaning of federal time limits.
- Workers explained personal responsibility agreement provisions accurately, but clients still did not understand the intent of the PRA or the consequences of not complying with its requirements.
RER Choices workers were more familiar with personal responsibility agreement policies than time limit policies. They consistently took time to explain PRA policies and verify compliance with experimental group members, making interviews with experimental group members substantially longer than interviews with control group members. Most interviewed eligibility workers believed that the PRA conditions affected client behavior. However, interviews with clients immediately after their eligibility appointments revealed that they did not understand the purpose of the personal responsibility agreement or the consequences of not meeting PRA requirements.
Similar findings occurred in RER non-Choices sites. Because of smaller caseloads, less turnover among eligibility workers and simpler ACT policy provisions (i.e., no state time limits), clients typically received very clear messages about the PRA. Even so, they had difficulty understanding its intent or the consequences of not meeting its provisions.
Impacts of the ACT Demonstration
Ray Marshall Center evaluators analyzed administrative data from fourteen programs administered by seven Texas agencies to measure program participation, demographic characteristics, and outcomes at the individual and case levels across programs and over time. Net impacts (differences in outcomes for experimental and control group members) were computed for many public assistance, economic, and family and child measures. A list of these measures and the overall impacts of ACT waiver policies on each of them are shown in Table 4 for all three pilots.
Impacts on certain subgroups within the larger experimental and control groups were also studied to determine if ACT’s effects varied for different kinds of people, in different time periods, or under different policy contexts. To test for such differential impacts, analyses of twelve key variables were added for the following subgroups:
-
Four-year outcomes - To test whether the demonstration had different impacts for persons with long histories with the welfare system, four-year outcomes were computed separately for short-term and long-term TANF recipients assigned within the first six months of each experiment.5
-
Before/after 1999 policy changes – To judge whether the demonstration had different effects under a later Texas TANF policy environment, one-year outcomes for were compared for families assigned before and after several key policy changes implemented between October 1999 and March 2000 (1999 policy changes). These included the imposition of federal time limits, an expansion of the earned income disregard, and a tightening of the ‘age of child’ exemption.
-
Tier group analysis – To test whether the overall impacts varied for those with varying prior education and work experience and with different time limits, separate impacts for members of Tiers 1, 2, and 3 were computed.6
Complete results from the impact analysis are contained in a companion publication to this summary, Achieving Change for Texans Final Impact Report. Significant differences in outcomes between experimental and control groups in each pilot are discussed below.
| Time Limits |
RER | RER Non- Choices |
|
|---|---|---|---|
| Welfare Dynamics | |||
| Percent of time spent on TANF by caretaker | -0.6% | 1.5% | 1.4% |
| Percent of time
spent by caretaker in payee-only status |
0.3% | -2.8% | -1.1% |
| Percent of time spent on TANF by any child | -0.3% | -1.1% | |
| Percent of time
spent on TANF by any family member |
-0.4% | -1.1% | |
| Average monthly TANF benefit | -$1 | -$2 | |
| Percent of months
in child support penalty status |
2.9% | 1.9% | |
| Percent of months in Choices penalty status | -0.4% | 1.9% | n.a. |
| Average length of penalties in months | 0.9 | ||
| Percent of time on Medicaid by caretaker | 0.2% | 2.6% | 1.9% |
| Percent of time on Medicaid by any child | 0.3% | ||
| Percent of time on Food Stamps | -0.6% | -3.2% | |
| Percent of time on transitional Medicaid | 0.8% | 1.2% | |
| Percent of time on transitional Child Care | 0.1% | 0.3% | 0.1% |
| Self-Sufficiency Outcomes | |||
| Percent of
quarters in which caretaker had wages of any amount |
0.7% | 0.8% | |
| Average quarterly caretaker wages | |||
| Average quarterly family wages earned | $27 | ||
| Percent of
quarters in which caretaker wages exceeded 155% of poverty |
-0.1% | ||
| Percent of
quarters in which family earnings exceeded 155% of poverty |
|||
| Average quarterly
family earnings plus child support collections retained |
$30 | ||
| Percent of
quarters in which earnings plus child support greater than poverty |
|||
| Workforce Development Participation | |||
| Percent of
months participating in Choices program |
-0.4% | n.a. | |
| Ever participated in Choices | |||
| Average hours of Choices participation per month | n.a. | ||
| Percent ever participating in JTPA, WIA, or WtW | |||
| Percent ever
participating in post-secondary education |
|||
| Percent ever receiving post-secondary degree | -0.2% | ||
| Family and Child Indicators | |||
| Percent of
families with an OAG child support case open |
|||
| Percent of
families with an OAG case open experiencing new paternity |
|||
| establishment(s) monthly | |||
| Average number of children for whom paternity established | |||
| Percent of months in which any child support was collected | 0.5% | 1.0% | |
| Average monthly child support collections | $2 | -$5 | |
| Percent of
cases using subsidized child care (SCC) monthly |
0.4% | 0.5% | |
| Average number
of children using SCC monthly, of families receiving SCC |
0.0 | 0.1 | -0.3 |
| Subsidy per child-month using SCC | |||
| Percent of
pre-school children with any immunizations reported in ImmTrac† |
-1.8% | ||
| Percent of
pre-school children fully immunized as reported in ImmTrac† |
|||
| School attendance rate† | |||
| School mobility† | -15% | ||
| School dropout rate† | |||
| TAAS reading: percent passed† | |||
| TAAS mathematics: percent passed† | |||
| Rate of foster care placement | |||
| Rate of substantiated
reports of abuse or neglect per month |
|||
| NOTE: Only statistically significant parameters (p<.
01) are listed. †For confidentiality reasons, described fully in the Appendix, certain effects listed are unadjusted or repeated measures effects, rather than adjusted net effects. Source: ACT Impact Evaluation |
Impacts of Texas Time Limits
The time limits pilot measured the impact of state time limits and additional transitional benefits for persons reaching time limits while holding all other policies constant. In the TL pilot, outcomes for the 29,795 cases assigned to experimental and control groups from June 1996 through September 2000 were tracked through September 2001.
Over half of the cases included in this analysis entered the experiment during its first six months. The sizes and characteristics of the experimental and control groups were nearly identical when they entered the pilot. As shown in Figure 2, the overall TANF caseload declined steadily throughout most of the evaluation period.
[D] |
Evaluators first measured the impact of state time limits on a number of public assistance outcomes. TANF and Medicaid enrollments were measured separately for adult TANF recipients and their children because state time limits remove only the adult from the TANF case while increasing adults’ access to transitional Medicaid benefits.
Impacts of state time limits on food stamps, financial penalties for not meeting personal responsibility agreement requirements, and use of transitional Medicaid and child care were also computed. Researchers found that:
- Texas time limits caused small reductions in TANF receipt, slight increases in enrollment in Medicaid and transitional benefits but no change in use of food stamps.
Under Texas policy, the removal of an adult from TANF due to reaching the state time limit creates a ‘child-only’ (or payee) TANF case and a reduction in the average monthly TANF benefits available to the affected family. Adult use of transitional Medicaid would also be expected to increase. Children’s use of TANF and Medicaid were expected to be unaffected by the imposition of time limits for their parents.
Impact results showed that TANF usage actually declined slightly for both caretakers and children affected by time limits but the amount of time spent in ‘payee-only’ status did increase. Use of TANF by any family member declined by 0.6 percentage points, or 1.5 days per year. These reductions tended to be more pronounced for Tier 2 caretakers, who had some high school education and limited work experience. Average TANF benefits for families subject to time limits decreased by only $1 per month.
Medicaid enrollment increased for both adults and children, but the sizes of these effects were very small. Use of transitional Medicaid increased by three days per year, the largest increase of all types of Medicaid measured. Adult short-term recipients subject to time limits spent significantly less time on Medicaid than did control group members.
Time limits also increased use of both transitional Medicaid and transitional subsidized child care, but had no impact on the use of food stamps. Thus, on the whole, the Texas time limits experiment had small and inconsistent impacts on Medicaid and Food Stamp program participation for TANF families.
- Persons facing state time limits received PRA penalties less often than other caretakers but differences were very small. For both groups, about two months of every year on TANF were spent in penalty status.
Overall, persons subject to time limits spent one less day in penalty status than other caretakers for each year that they received TANF. This difference was driven by lower rates of financial PRA penalties for Choices and child support among Tier 1 caretakers, a group with at least a high school education and the most work experience of all groups. Most penalties, averaging about four months for each occurrence, were imposed for failure to comply with PRA provisions related to Choices participation, Texas Health Steps and school attendance.7
- The impacts of time limits on public assistance use differed by tier.
The small impacts of time limits on various public assistance measures sometimes masked differences in impacts for persons of different backgrounds. In some cases, time limits affected members of these subgroups in opposite ways. For example, Tier 1 families subject to time limits received Choices penalties less often than families not subject to time limits. As shown in Table 5, however, state time limits had no effect on these penalty rates for Tier 2 families while Tier 3 families subject to time limits were penalized more often. Differing impacts among subgroups were common, particularly by tier. This suggests that state time limits affect persons with different educational and work history backgrounds in quite different ways.
- Time limits caused a very small increase in caretaker employment but had no effect on overall quarterly earnings. Time limits affected Choices participation differently for short and long-term recipients and by tier.
Ray Marshall Center evaluators also measured the impacts of time limits on a number of labor market and related outcomes. These included quarterly employment rates and earnings for caretakers and all family members on the TANF case, as well as total family earnings plus child support. Impacts of time limits on participation in various programs designed to increase employment skills — including Choices, Job Training Partnership Act (JTPA) and the Workforce Investment Act (WIA), and post-secondary education were also measured.
| Time Limits | RER | RER Non-Choices | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Tier1 | Tier2 | Tier3 | Tier1 | Tier2 | Tier3 | Tier1 | Tier2 | Tier3 | |
| Welfare Dynamics | |||||||||
| Percent of time spent on TANF by any family member | -0.9% | -2.2% | -1.7% | 2.5% | |||||
| Average monthly TANF benefit | -$3.62 | -$3.35 | -$1.57 | ||||||
| Percent of months in child support penalty status | -0.14% | -0.16% | 0.57% | 1.64% | 1.66% | 0.48% | 0.57% | 1.18% | |
| Percent of months in Choices penalty status | -0.43% | 0.22% | 0.72% | 1.68% | n.a. | n.a. | n.a. | ||
| Percent of time on Medicaid by caretaker | 2.1% | 1.3% | 4.4% | 4.0% | 1.8% | ||||
| Percent of time on Medicaid by any child | 0.7% | -1.2% | -0.8% | ||||||
| Percent of time on Food Stamps | -0.7% | -1.1% | -1.7% | -3.7% | -3.9% | ||||
| Self-Sufficiency Outcomes | |||||||||
| Percent of quarters in which caretaker had wages of any amount | 1.5% | 4.4% | |||||||
| Average quarterly caretaker wages | $163 | ||||||||
| Workforce Development Participation | |||||||||
| Percent of months participating in Choices program | -0.2% | 0.2% | -1.0% | 0.4% | n.a. | n.a. | n.a. | ||
| Family and Child Indicators | |||||||||
| Percent of months in which any child support was collected | 0.8% | 1.0% | -0.3% | 1.0% | 1.2% | 1.9% | -4.5% | ||
| Percent of cases using subsidized child care (SCC) monthly | 1.2% | 0.8% | -0.7% | 0.6% | |||||
| NOTE: Experimental effects by tier are only
listed for those cases in which 1) the overall experimental effect
was found to vary significantly by tier, and 2) the effect within
a given tier is significantly different from zero. Source: ACT Impact Evaluation |
Overall employment gains were quite small, an increase equivalent to 2.5 days per year. Employment gains occurred primarily among short-term recipients, (i.e., persons with less than 30 months of prior TANF receipt at the point of random assignment), as well as among Tier 2 caretakers. Even so, caretakers’ quarterly earnings did not increase as a result of state time limits. Very few families generated enough income from sources measured here to become independent of public assistance.
Overall rates of participation in Choices, other short-term workforce development programs, and post-secondary education were unaffected by Texas time limits. Those subject to time limits were significantly less likely to achieve a post-secondary degree.
The overall insignificant effects on Choices participation masked interesting differences for some subgroups, with time limits causing both short-term recipients and those in Tier 1 to reduce their Choices participation, relative to controls.
- Time limits slightly increased the number of months in which child support was collected but produced no differences in other family or child outcomes.
Finally, family and child outcomes were measured to determine if the demonstration affected them. Several child support measures were analyzed, including rates of paternity establishment and opening of new child support cases, the two measures most closely associated with PRA requirements to cooperate with child support enforcement. Additional child support measures included the percent of months in which child support was collected and the amount of child support collected each month. Among other family and child outcomes tested were use of subsidized child care, immunization rates for young children, school attendance and TAAS performance, rates of foster care placements and incidence of substantiated child abuse or neglect.
Child support collections occurred more often for families subject to time limits, but as with other impacts from this experiment, the overall impacts, about two days per year, were very small. These impacts varied by tier, with Tier 1 and Tier 2 families receiving child support more often and Tier 3 less often than their control groups. These findings suggest that time limits induced noncustodial parents with existing child support orders to pay child support more regularly but did not increase the number of paternity establishments or open child support cases.
Impacts of the Combination of Texas Time Limits and RER
The RER Choices pilot compared the combination of state time limits, expanded TANF eligibility rules and the personal responsibility agreement to the AFDC rules in effect prior to the ACT demonstration. From June 1996 through September 2000, 13,373 cases were assigned to participate in RER Choices as either experimental or control group members.8 Over 35 percent of these cases were already receiving TANF at the beginning of the experiment. As in the TL pilot (see Figure 2 above), the size of the TANF caseload declined in the first few years of the demonstration.
- Expanded TANF eligibility rules allowed more working but very poor two-parent families onto TANF in the site with the highest share of Hispanic families.
In the ACT demonstration, full tracking of families assigned to experimental and control groups only began after their TANF applications were approved. So the impact of expanded eligibility rules in the RER pilots (known as ‘entry effects’) could be assessed only for those whose applications were approved. In three of the four RER Choices sites, the sizes and characteristics of the experimental and control groups were essentially identical. However, in El Paso’s Clint office, a heavily Hispanic area, more two-parent families entered the experimental group. Most differences in the number and characteristics of the experimental group members in Clint were due to the elimination of the 100-hour and work history rules for two-parent TANF families. ACT’s disregard of children’s earnings also accounted for a small portion of the entry effects in Clint. Because of these and other differences in the Clint office, other results for that location are reported separately and not included in this summary.
The following results measure the combined impacts of time limits and the PRA in all of the RER Choices sites except Clint. Application of these policies produced the following impacts:
- The combination of time limits and the personal responsibility agreement caused very small reductions in TANF use for children, while adults’ TANF use increased due to penalty rules. Adult Medicaid enrollment also increased but there were no impacts on children’s Medicaid.
ACT rules allowed caretakers to remain on the TANF rolls even when they were financially penalized for not meeting PRA requirements. This varied from earlier AFDC rules that removed sanctioned caretakers from the AFDC grant when they did not comply with Choices or child support requirements. This rule change was largely responsible for the increase in TANF receipt among adult caretakers. However, the average monthly TANF grant was slightly lower for experimental group families because more financial penalties were imposed under ACT rules than sanctions under the old AFDC rules. The impacts of the RER experiment on TANF receipt varied by tier, with Tier 1 and 2 experimental group families receiving TANF less often than their control group counterparts.
Medicaid usage was higher for adults in this pilot because penalized caretakers retained their Medicaid eligibility by remaining on TANF. Among subgroups, these effects were particularly strong for long-term recipients and Tier 3 caretakers. There were no overall differences in Medicaid receipt among children in the two groups and small differences by tier. Persons subject to RER Choices provisions were less likely to receive food stamps than control group families but the impacts were very small. Caretakers subject to RER Choices provisions used both transitional Medicaid and transitional child care 20 percent more of the time than did those in the control group.
- The personal responsibility agreement rules resulted in large increases in financial penalties, compared to earlier AFDC sanctions.
Over 29 percent of families subject to the PRA in the RER Choices sites received financial penalties, compared to only five percent of families in the control group sanctioned under the old rules. Financial penalties for failure to comply with PRA provisions were in effect 16 percent of the time that caretakers in the experimental group were on TANF following random assignment. Most of these penalties were imposed for not complying with PRA provisions related to Choices, Texas Health Steps program, child support, and school attendance. By comparison, control group members spent less than six percent of the time being sanctioned for failure to cooperate with the child support or Choices provisions in effect under the AFDC rules. The average length of time caretakers remained in penalized status increased by almost a month when families were subject to the PRA. Long-term recipients subject to RER Choices provisions experienced larger increases in penalties, relative to controls, than did short-term recipients for both child support and Choices penalties. Impacts of these penalties were also stronger for Tier 2 and 3 caretakers than for Tier 1 caretakers, as illustrated in Figure 3 for child support penalties.
[D] |
- The combination of state time limits and the PRA produced small gains in caretaker employment and total family earnings, but had no impact on caretaker earnings.
While persons subject to RER Choices policies worked in more calendar quarters after entering the demonstration, their earnings were not any different than those of control group members. The increase in family earnings occurred because other family members (typically teenage children) earned more than those in families not subject to these provisions. These small impacts were driven primarily by four-year outcomes for short-term TANF recipients, with no differences in impacts measured for any of the other subgroups.
Caretakers subject to RER and time limit provisions spent less time enrolled in the Choices program and were more likely to receive financial penalties for failure to participate in Choices. A strong increase in the rate of Choices participation occurred for experimental group caretakers following the 1999 policy changes. Tier 1 caretakers subject to RER and time limit provisions spent less time in Choices than did control group members, while those in Tier 3 did the opposite. No differences were observed between experimental and control group members’ participation in other workforce development services.
- The combination of time limits and the personal responsibility agreement caused small increases in child support collections but few impacts on other family and child measures. This lack of impacts occurred even though many penalties were imposed for not complying with PRA provisions related to these outcomes.
Overall increases were observed both for the number of months of child support collections and the amount collected but not for paternity establishments or the number of new child support cases that were opened. As shown in Figure 4, impacts on the number of months in which child support was collected varied considerably by subgroup. These differences may have been associated with the penalties imposed for failure to cooperate with child support collections and/or the length of time needed to complete the process to collect child support.
Increases in the use of subsidized child care were partially attributable to higher usage of transitional child care by experimental group members. Greater use of subsidized child care was also observed for long-term recipients, caretakers assigned following the 1999 policy changes, and Tier 3 caretakers. Because fewer experimental group families were exempt from Choices after the 1999 policy changes, a greater share of these families used child care to participate in Choices than was true for control group families.
No impacts were observed for rates of school attendance or immunization, even though a number of penalties were imposed for failure to comply with PRA provisions in these areas. Very few children needed foster care or child protective services during the observed time period, and the combined time limit and PRA policies did not have any impact on these measures.
[D] |
Reaching Texas Time Limits
Ray Marshall Center evaluators tracked persons who were forced to leave TANF due to time limits for a year after exit using administrative data. By the end of September 2001, 816 caretakers in the TL pilot and 313 in the RER Choices pilot were removed from TANF because of reaching their Texas time limits. Ninety percent of this group had twelve month (Tier 1) time limits while most of the others had been assigned to Tier 2 (24-month time limits). As shown in Table 6, nearly all affected Tier 1 and 2 families received at least some Medicaid and food stamps after exiting due to time limits, and the children in about 90 percent of these families continued to receive at least some TANF benefits. More than 70 percent were employed at some time in the year after their exit, but their earnings were low. One year following exit, 25-33 percent of children in these families were still receiving TANF, and around 80 percent were enrolled in Medicaid. Four in ten families still received food stamps and approximately half of time-limited caretakers were employed.
The Center for Social Work Research conducted interviews with 39 welfare recipients in Bexar and McLennan/Bell Counties who had left TANF after reaching state time limits, as reported in Texans Who Timed Out of Welfare: The Year After. These researchers found that less than half of the respondents in each site were employed at the time of the first interview. Respondents from Bexar County were employed primarily in education or the service sector, earning wages between six and eight dollars an hour. In addition to service sector jobs, persons in McLennan/Bell Counties were working as mechanics and at jobs in the criminal justice system. Their wages ranged from $5.50 to $9.50 an hour. Most held part-time jobs. Persons not employed cited health problems, difficulties with transportation, and access to child care as barriers to employment.
Interviewed persons who had reached time limits thought that time limits were a good idea. However, they felt that they needed more time to prepare for leaving welfare (most persons interviewed had a 12-month time limit). They also felt that caseworkers needed better skills to recognize and deal with clients’ multiple employment barriers.
After losing TANF due to time limits, former recipients reported that they survived through a mix of irregular employment (including employment in the informal sector), public assistance, private or local services, and help from family and friends. They bought groceries in bulk, got help from family and friends with a month’s rent or a bag of groceries, and received periodic help from community agencies with food, clothing (particularly school clothing), school supplies, and utility payments. Most used the same mix of survival strategies that they had used while receiving TANF.
| Time Limits | RER | ||
|---|---|---|---|
| Tier 1 | Tier 2 | Tier 1 | |
| Number of Caretakers Reaching Time Limit | |||
| Number of caretakers reaching time limit | 739 | 55 | 286 |
| Number of caretakers reaching time limit with at least 12 month followup† | 428 | 26 | 156 |
| Children remaining on TANF | |||
| Percent of cases in which at least one child received any TANF | 88.1% | 92.3% | 87.8% |
| Average number of months at least one child was on TANF | 5.4 | 6.0 | 5.8 |
| Percent of cases in which any child received TANF in 12th month after exit | 24.2% | 24.0% | 32.9% |
| Medicaid receipt | |||
| Percent of cases in which caretaker received any Medicaid | 93.5% | 92.3% | 94.2% |
| Average number of months in which caretaker received any Medicaid | 9.3 | 8.9 | 9.1 |
| Percent of cases in which caretaker received Medicaid in 12th month after exit | 64.5% | 65.4% | 62.8% |
| Percent of cases in which caretaker received any transitional Medicaid | 88.1% | 88.5% | 89.1% |
| Average number of months in which caretaker received transitional Medicaid | 8.5 | 8.4 | 8.0 |
| Percent of cases in which caretaker received transitional Medicaid in 12th month after exit | 59.3% | 61.5% | 51.3% |
| Percent of cases in which any child received Medicaid | 98.1% | 96.2% | 98.1% |
| Average number of months in which any child received Medicaid | 10.4 | 10.3 | 10.5 |
| Percent of cases in which any child received Medicaid in 12th month after exit | 78.7% | 80.0% | 81.6% |
| Food Stamps receipt | |||
| Percent of cases in which household received any Food Stamps | 93.2% | 96.2% | 94.2% |
| Average number of months in which household received Food Stamps | 6.6 | 6.9 | 6.9 |
| Percent of cases in which household received Food Stamps in 12th month after exit | 41.4% | 46.2% | 45.5% |
| Employment and Earnings | |||
| Percent of cases in which caretaker was employed at all in 12 months | 72.4% | 76.9% | 67.9% |
| Average number of months in which caretaker was employed | 6.1 | 6.5 | 5.8 |
| Total wages earned in follow up period (among those who earned wages) | $6,548 | $5,169 | $5,250 |
| Percent of cases in which caretaker employed in 12th month after exit | 53.0% | 50.0% | 50.0% |
| Percent of cases in which caretaker was employed for all 12 months after exit | 25.5% | 19.2% | 26.3% |
| Total wages earned in follow up period (among those employed all 12 months) | $11,358 | $9,196 | $8,207 |
| Child care | |||
| Percent of cases in which caretaker received any subsidized child care | 29.0% | 19.2% | 22.4% |
| Average number of months in which caretaker received any subsidized child care | 2.1 | 1.3 | 1.4 |
| Percent of cases in which caretaker received subsidized child care in 12th month after exit | 14.7% | 3.8% | 8.3% |
| Percent of cases in which caretaker received any transitional child care | 19.9% | 11.5% | 14.1% |
| Average number of months in which caretaker received transitional child care | 1.5 | 0.4 | 0.9 |
| Percent of cases in which caretaker received transitional subsidized child care in 12th | 11.7% | 3.8% | 6.4% |
| Child support | |||
| Percent of cases in which any child support was collected | 36.9% | 46.2% | 35.3% |
| Average number of months in which caretaker received child support | 2.5 | 2.6 | 2.2 |
| Total amount of child support received during follow-up period (among those who | $1,093 | $766 | $865 |
| Percent of cases in which any child support was collected in 12th month after exit | 24.1% | 30.8% | 20.5% |
† Except for the first row, the population of this table is limited to caretakers who were forced to exit with at least twelve months available for follow up. Only groups with at least 20 caretakers meeting these criteria are shown. All statistics except those in the first row are based on a twelve month follow-up period. Source: ACT Impact Evaluation |
Impacts of RER
The RER Non-Choices experiment measured the impact of RER provisions (expanded eligibility rules plus the PRA) in the absence of time limits.9 This pilot was implemented in January 1997. By the end of September 2000, 1,684 cases were assigned as experimental or control group members. The expanded TANF eligibility rules in these sites did not affect the composition of the experimental group. Over 60 percent of families participating in RER Non-Choices were already receiving TANF at the beginning of the experiment. As in the other pilots, the overall size of the TANF caseload declined during this period (see, for example, Figure 2).
After 57 months of operation, the RER experiment across four rural offices in counties without Choices programs produced the following impacts:
- The personal responsibility agreement produced no overall impacts on the use of children’s TANF or Medicaid but increased caretakers’ time on TANF and Medicaid due to PRA rules. Use of food stamps declined for families subject to PRA provisions.
No differences between experimental and control groups were found for children’s overall TANF usage or the monthly amount of the TANF grant. Caretakers subject to the PRA spent more overall time on TANF but less time in ‘payee-only’ status, due to the rules governing penalties and sanctions. The PRA slightly increased the percent of time on TANF for Tier 2 families. A descriptive analysis showed that the active TANF caseloads in these sites became increasingly comprised of long-term welfare recipients as the study progressed.
While caretakers in families subject to RER Non-Choices provisions were more likely to enroll in Medicaid, the experiment had no overall impact on children’s Medicaid receipt. These same families used food stamps less often. Subgroup analyses qualified these effects to some extent. No effects of the PRA were found on transitional Medicaid usage, while the increase in transitional child care usage was so small as to be practically insignificant.
- Families subject to the personal responsibility agreement spent more time in penalty status, as expected.
Experimental group members spent eleven percent of their time on TANF after random assignment in penalty status, most often due to non-compliance with Texas Health Steps. The average penalty lasted 4.6 months in duration. Control group members were subject to sanctions instead of penalties, and spent about one percent of the time following random assignment in sanctioned status for failure to cooperate with child support enforcement. Subgroup analyses revealed that the impact of penalties for failure to cooperate with child support enforcement was greatest before the 1999 policy changes, among short-term recipients, and among caretakers in Tiers 1 and 3.
- The personal responsibility agreement produced no overall impact on employment or earnings.
Overall, RER Non-Choices provisions did not impact any of the employment or earnings measures. These findings were qualified, however, by the tier level of the caretaker. For reasons that are not clear, Tier 3 caretakers, who are the most disadvantaged in terms of education and employment history, experienced substantial increases in employment and earnings when subject to the PRA.
Because of the somewhat more rural locations of these sites, most offered no Choices program. Participation in other available workforce development programs was low, with no difference measured in overall rates of participation between the groups.
- No impacts were found for school attendance rates, most of the child support measures, or immunization rates, despite experimental group members being subject to PRA penalties in these areas.
Use of subsidized child care increased and the average amount of child support collections actually decreased. No other family and child outcomes subject to PRA were affected by those rules, including immunization rates, school-related measures and use of child protective services.
One-Time Recipients
The TANF One-Time program was implemented in Hidalgo and Cameron counties in the Lower Rio Grande Valley in the fall of 1997, was extended to the rest of that TDHS region in April 1998, and became a statewide program in August 1998. Unlike the other ACT components, this provision was not evaluated as a randomized experiment. TDHS used administrative data to describe its implementation, and CSWR conducted detailed interviews with families who received one-time payments to better understand the uses of this payment and their perceptions of the program.10
A total of 4,715 one-time payments were issued through May 2001. About 85 percent of TANF One-Time recipients had not received TANF in the two years prior to receiving their first one-time payment. About 45 percent had not received food stamps in the six months before they received one-time payments for the first time. These findings were consistent with the policy that the TANF One-Time program was only available to families who met ‘crisis’ criteria.
CSWR researchers tried to contact 182 households shortly after they received the one-time payment and conducted in-depth interviews with 55 of these households several times in the months following receipt of the one-time payment. Complete results from these interviews are reported in Texans Who Receive a One-Time Benefit: The Year After. These families’ experiences may not be representative of the entire state but suggest common uses of one-time payments. Researchers found that:
- Families typically needed one-time payments due to losing their employment and used these payments to cover a one-time emergency expense or to purchase or repair a vehicle.
Over 38 percent of interviewed families had been laid off, fired or were receiving no income from their employment. Another 26 percent worked in seasonal jobs. Smaller shares of persons interviewed reported health, transportation or child care difficulties, as well as a variety of other reasons for not working. Over half of the interviewed families used the $1,000 payment to pay overdue rent or mortgage or other overdue bills, while another fourth bought or repaired a car. In almost all cases, these families were living at or below the poverty level and payment of ongoing bills was somewhat precarious on a month-to-month basis.
- One-time payments produced few changes in household budgeting, employment, access to education and training, or access to child care.
Being able to budget presumes the availability of money to manage. Most recipients of the one-time payment used the money in one of two ways to alleviate pressing bills or as an investment in transportation. For most of the recipients, the money was already allocated before it was received. Thus, few funds were available for other purposes.
Although the one-time payment for transportation-related expenses enabled many study participants to begin a more extensive job search, employment remained fairly low among the study participants. Persons in seasonal employment and those using the payment to improve their available transportation reported the biggest employment gains.
- One-time payments were not used to support persons outside the household.
One-time recipients often received additional help from persons outside their household and none mentioned that they aided relatives or friends outside of their own household with the payment. Forty percent of persons interviewed reported receiving income support from ‘other’ sources (most commonly family and friends). On average, households reported receiving $275 per month from other sources, as well as many types of non-monetary supports.
All in all, the effect of the one-time payment on households depended on many factors, including the number of other supportive resources that families could piece together. In many instances, a heavy debt load that quickly absorbed the $1,000 with no long-term lasting effects already hindered households. The households gaining the most benefit from the one-time payment were those for whom unemployment was temporary. In those cases, the one-time payment filled an immediate need that allowed for a continuation of a present (and generally modest) standard of living.
5Short-term
recipients had less than 30, and long-term recipients 30 or more months
of prior TANF receipt when they entered the demonstration.(back)
6Persons assigned to Tier 1 had
the most prior education and work experience and a 12-month time limit.
Tier 2 caretakers had a 24-month time limit. Those in Tier 3, who had
the least education and work experience, had a 36-month limit.(back)
7Texas Health Steps requires
health screening and regular check-ups for young children. It replaced
the Early and Periodic Screening, Diagnosis and Treatment (EPSDT) program
in May 1996.(back)
8El Paso’s Clint office
accounted for 2,888 of these cases.(back)
9Texas time limits do not apply
in sites without a Choices program because state time limits are conditional
on the offer of a Choices slot.(back)
10This program is described
more fully in Schexnayder, Lein et al. (January 2002).(back)
| Table of Contents | Previous | Next |