Assets for Independence Act Evaluation:
Design Phase Final Report
August 9, 2000
5. |
Nonexperimental Impact Analysis |
||
| 5.1 | Purpose | ||
| 5.2 | Data Collection Plan | ||
| 5.3 | Data Analysis Plan | ||
| 5.4 | Cost Estimate | ||
Either instead of or in addition to an experimental design, another
approach to estimating the effects of AFIA programs on participants
is to undertake nonexperimental impact analysis.
5.1 Purpose
Under the nonexperimental approach, instead of using a randomly assigned control group to represent the policy counterfactual, one uses available data on nonparticipants within the general population. Comparable data would then be collected on program participants. Multivariate statistical techniques would be employed to account for observable differences between participants and nonparticipants on individual background characteristics and other contextual factors, such as local economic conditions.
Nonexperimental analysis requires that one has adequate data to parcel out program effects from non-program "external" effects on savings and asset outcomes. If one is unable to control adequately for the external factors, the resulting impact estimates could falsely attribute to the program the effects of underlying demographic or socioeconomic differences between participants and nonparticipants. This is especially problematic in programs such as IDAs, where one expects that participants have greater motivation and initiative than nonparticipants. Such personal traits are typically unmeasured in available data; without any control mechanism, one tends to overstate the program's effects.
With these limitations in mind, it is nonetheless worth considering the merits of nonexperimental approaches. To be feasible, this strategy requires a database that would enable one to measure the savings and asset patterns among households who participate in an AFIA-funded program and also among those who would qualify for, but are not participating in, such a program. For the program participants, as noted above, comparable data would need to be collected through a separate primary data collection effort, to the extent that participants would be found in very small numbers in any national database.
Such a database would need to meet the following criteria:
- It would contain national data with oversampling of the low-income population, to provide sufficient numbers of AFIA-eligible households.
- It would provide detail on income, savings, assets, and liabilities, both to identify the AFIA-eligible households and to track outcomes on savings and asset accumulation.
- It would follow households longitudinally (i.e., over multiyear intervals), to enable one to profile the year-to-year changes in household savings and asset-holdings.
The one dataset that appears to meet these requirements is the Survey of Income and Program Participation (SIPP), which is administered by the U.S. Bureau of the Census. This survey is a national, multi-panel longitudinal survey of adults, measuring their economic and demographic characteristics over a period of four years. Panel members are interviewed once every four months over the four-year life of the panel. At each of these waves, the interview includes several "topical modules." Once a year, panel members are asked to complete a topical module on Assets and Liabilities. A copy of this module can be found in Appendix E. The features of SIPP that make it well-suited for such analysis are as follows:
- The survey is a series of national "panels" or household samples. The members of each panel are interviewed in successive "waves" every four months. The most recent panel, the 1996 panel, was introduced in April 1996 and will be interviewed over 12 waves, encompassing 4 years. The twelfth and final wave is about to begin in December 1999.
- Each panel is a stratified sample of the U.S. civilian noninstitutional population, with oversampling of low-income households. The 1996 panel consists of 36,700 households.
- Detailed financial information is collected for each household. The "core module" of questions administered to each panel at each wave includes items on income sources and amounts, labor force status, living arrangements, and participation in income support programs. Such basic information is recorded for each of the last four months. Additionally, asset information is asked as of the last day of the four-month reference period. The latter items include checking account balances, value of U.S. savings bonds, amounts in individual retirement accounts (IRAs), and outstanding debts and obligations, including unpaid bank loans and credit card bills.
- At each wave, the core questions are supplemented
by several "topical modules" that address particular household
circumstances. One of the topical modules pertains to "Assets
and Liabilities." It is administered every year (i.e., every
third wave for each panel).[15] The items
include savings accounts, stocks, mutual funds, bonds, Keogh and
IRA accounts, and unsecured liabilities (e.g., loans, credit cards,
medical bills).
5.2 Data Collection Plan
This section discusses the data collection plan for the nonexperimental approach using SIPP and the survey of AFIA grantees and subgrantees.
5.2.1 Nonexperimental approach using SIPP
The 1996 SIPP panel is the most promising data source for this analysis, compared to either previous or upcoming panels, for the following reasons:
- It is a larger panel than others recently enrolled and is larger than the upcoming 2000 panel. The 1996 panel, introduced in April 1996, consists of 36,700 households. (The 1993 panel consisted of 21,800 households; the 2000 panel includes only 11,500 households.)
- It incorporates an oversampling of the low-income population. Of the 36,700 households in the 1996 panel, 9,900 have incomes below 150 percent of the federal poverty level at sample entry; 6,000 are below 100 percent of the poverty level. (These counts are 8 to 10 percent higher than those resulting from a design without such oversampling.)
- It has been followed for a longer period than the 32-month follow-up period of previous panels, which were interviewed every four months in eight successive "waves." The 1996 panel is now in the midst of its twelfth and final wave, thus encompassing four years of follow-up data.
- The Assets and Liabilities topical module has been administered to this panel every year, at Waves 3, 6, 9, and 12. These waves were initiated in December of 1996, 1997, 1998, and 1999, respectively. The data thus provide a detailed, annual, longitudinal record of household saving and asset accumulation behavior.
- It has been administered predominantly through computer-assisted telephone interviewing (CATI), with once-annual computer-assisted personal interviewing (CAPI). Any corresponding data collection undertaken among AFIA program participants for this evaluation would also be CATI-based (and perhaps exclusively CATI).
- Even with the three-year lag in the release of public use files, the data from this panel will be much more timely for this analysis. Wave 3 topical module data from the 1996 panel will be soon released. One can expect data through Wave 6 in early 2001, through Wave 9 in early 2002, and Wave 12 in early 2003.
- One avoids the need to separate from this sample those who have participated in AFIA-funded programs, as the data cover a period that predates the startup of such programs.
The pre-AFIA timing of the 1996 sample has advantages, as noted in the last two bullets. It does mean, however, that the estimation of impacts must make use of economic indicators, such as local unemployment rates, to control for the effects of shifting labor market conditions or other external intertemporal factors. To the extent that such explanatory variables do not fully capture such "exogenous" effects on relevant outcomes, one may falsely attribute these effects to the AFIA programs.
The 1996 SIPP panel would thus be the data source for AFIA nonparticipants. The comparison group to be used for the nonexperimental analysis would be a subset of the 1996 panel. Specifically, we would identify those panel members whose employment status and financial circumstances at panel entry (Wave One) would have made them AFIA-eligible (i.e., if AFIA programs had existed in 1996). Most importantly, they would have needed to be employed (or recently employed) in 1996 with household income no greater than the prevailing earned income guidelines for the Earned Income Tax Credit (EITC).[16]
For the AFIA program participants, as candidates for the nonexperimental treatment group, the data collection would involve the following steps:
- Based on the annual progress reports submitted by the FY 1999 and FY 2000 AFIA grantees in early 2001 (for the calendar period 2000), we will estimate the number of program participants enrolled by each grantee (and by subgrantee, if applicable) during the 12-month period April 2000 - March 2001. This total number will constitute the "sampling universe." (For the FY 1999 grantees, the expected number of accounts to be established collectively, within their project periods, is about 9,700, based on the information presented in Appendix A of this report. We do not know, however, how many participants might be enrolled during April 2000 - March 2001 by these programs and by those in the FY 2000 cohort.)
- We will then implement a multi-stage cluster sampling design that maintains the sampling principle of "probability proportional to size" (PPS). The objective of this design will be to identify a treatment sample (of yet-to-be-specified size) whereby each IDA participant in the sampling universe stands an equal chance of selection. Because participants occur in clusters (by grantee and then subgrantee), the sampling will occur in stages. At the first stage, a number of grantees will be selected on a PPS basis. At the second stage, applicable only where subgrantees operate the program, a number of subgrantees will then be selected, also on a PPS basis. At the third and final stage, participants will themselves be randomly selected from among the selected grantees and subgrantees. Only at the final sampling stage will it be necessary to obtain lists of the program participants from each selected grantee and subgrantee. We expect that the use of MIS IDA (or another equivalent information system) by grantees and subgrantees will facilitate this process.
- The participants selected into the nonexperimental treatment group will then form a survey sample for a multi-wave telephone interviewing process patterned after SIPP. That is, we will administer the SIPP topical module on Assets and Liabilities to each participant, at follow-up intervals of twelve months, with the first wave to occur during April 2001 - March 2002. The second, third, and fourth waves would commence in April of 2002, 2003, and 2004, respectively. Given the geographic dispersion of the sample, we expect that all interviewing would be conducted by telephone, using CATI techniques. See Appendix E for a copy of this module.
- Contemporaneous local economic data would be linked with each nonexperimental case, in both the treatment and comparison groups, to be used as additional explanatory variables in the impact analysis, as explained further below.
This data collection strategy would thus yield a body of data that would combine Census-collected SIPP information during the period 1996-2000 for AFIA-eligible program nonparticipants with comparable evaluator-collected information during the period 2000-2004 for AFIA program participants.
5.2.2 Survey of AFIA grantees and subgrantees
Information on the non-IDA program services offered by AFIA grantees and subgrantees will be collected through a survey of all grantees and subgrantees in the FY 1999 and FY 2000 cohorts and will be conducted annually for four years.
The survey will supplement findings from the site visits, which will necessarily be restricted to a small group of grantees. Our experience is that short program surveys can be very effective to capture straightforward descriptive information about program features. Having this information available for the universe of grantees can be useful in several ways. First, it can identify the degree to which IDA programs are implemented consistently with what was intended (as articulated in grantees' applications). Second, it can place the program features identified in the process analysis in a broader context. (For example, how common is a particular program feature observed during visits to several sites?) Third, it can identify any differences in cohorts over time. It may be expected that, as each successive cohort's experiences become known, IDA programs "mature" over time. That is, newer sites will take into account their predecessors' successes and challenges. Cohort differences may also occur if certain AFIA requirements, over time, tend to encourage or discourage certain types of program models.
The program survey will collect the following types of information:
- funding levels (federal, state, and local);
- eligibility requirements;
- program requirements (minimum deposits, counseling requirements, etc.);
- number of account holders currently anticipated;
- length of time the program has been operational;
- financial institutions involved in the program;
- terms of the savings accounts; and
- support services offered to IDA participants.
5.3 Data Analysis Plan
A key challenge for the nonexperimental analysis is the presence of selection bias. Specifically, IDA participants are likely to differ importantly in their savings and investment decision-making from nonparticipants who share the same observable demographic characteristics. The outcomes measured for nonparticipants, even after adjusting for participant-nonparticipant demographic differences, may not reliably represent the outcomes that participants would have experienced in the absence of the IDA program.
The issue of selection bias is especially problematic in this context because IDA participants, as voluntary program entrants, are a self-selected group of individuals. They will tend to be more highly motivated than demographically comparable members of the nonparticipant population. This calls for an empirical strategy to take account of motivational factors, which are of course not directly observable. Without such a strategy, one runs the risk of falsely attributing to the program the effects of these motivational traits.
Our proposal for this analysis makes use of "propensity score" methods first developed to evaluate nonexperimentally the effects of differing forms of medical treatment.[17]The basic logic of propensity scoring is that, in either a medical or nonmedical context, one can obtain improved estimates of a treatment effect by first dividing the research sample into subgroups based on each case's estimated likelihood or "propensity" of having been observed in the data as a member of the treatment group versus the comparison group. The propensity scores serve as a composite indicator of multiple case-specific characteristics (commonly referred to as the "covariates"). The objective of subclassifying the sample, using the propensity score as the criterion, is to better match the treatment cases with members of the comparison group whose experience as nonparticipants forms the counterfactual against which to measure program effects.
This nonexperimental approach requires a first-stage analysis of the treatment-comparison status of sample cases, to estimate the propensity scores and ordinally rank both treatment and comparison cases. Based on this rank ordering, one then divides the sample into subgroups, each comprised of those treatment cases and comparison cases whose propensity scores fall within a specified interval. In the second-stage analysis, one then estimates treatment effects by subgroup and then computes the treatment effect for the entire sample as a weighted average of the subgroup estimates.
The challenge of applying the propensity score approach in this evaluation is that AFIA-funded programs have not existed long enough, or in enough communities across the country, to enable one to estimate propensity scores specific to AFIA program participation itself. To address this, we propose using participation in the federal earned income tax credit (EITC) program as a proxy for AFIA program participation.
The federal EITC is a logical choice as the basis for estimating the likelihood of participation in AFIA programs, for the following reasons:
- The income threshold for AFIA participation is itself the EITC income eligibility level
- Both the EITC provision and AFIA programs require that participants be employed.
- Previous EITC studies show the participation rate among eligibles as existing in an intermediate rangeCapproximately 80 percent nationallyCthat would make it a meaningful basis for disaggregating low-income individuals into subclasses that reflect motivational characteristics.[18]
The specific steps in the nonexperimental impact analysis will be as follows:
- Identify the members of the 1996 SIPP panel whose baseline characteristics (at panel entry) would have met the income and asset eligibility criteria for AFIA participation, i.e., with income below the EITC level, and with assets below $10,000.
- Use multivariate regression techniques to estimate the probability of EITC participation among AFIA eligibles in the 1996 SIPP panel.[19] Use these first-stage regression results to assign to each AFIA-eligible member of the SIPP panel (the comparison group) and each member of the selected sample of 1,600 AFIA participants (the treatment group) a propensity score that represents the estimated probability of their participation in EITC.
- Establish quintile values for the propensity scores, based on the distribution of values derived above for those in the treatment groupCthus creating five subgroups, each consisting of 320 treatment cases and the corresponding comparison cases from SIPP whose propensity scores fall within the same range. (Unlike the treatment cases, the comparison cases will thus not be distributed across the five subclasses in equal numbers. Given the size of the SIPP panel, however, with nearly 10,000 sample members having incomes below 150 percent of the poverty level, it is expected that the number of comparison cases in each subclass will be more than ample to support the proposed analysis.)
- Check to ensure that, within each subclass, both the treatment and comparison cases exhibit substantial within-group variation and between-group overlap in basic demographic characteristics. (If such balance does not exist, it may be necessary to reformulate the scoring approach until this condition is met.)
- For each outcome measure of interest, use multivariate regression techniques to estimate the treatment effect within each subclass. This second-stage modeling will: (a) adjust for the within-subclass variation in explanatory characteristics, and (b) adjust for differences in time-varying conditions (such as unemployment rates) between the 1996-2000 observation period for comparison cases and the 2000-2004 observation period for treatment cases.[20] For each outcome measure of interest, a "random effects" (or "random coefficients") model will be specified. Under such an approach, each case is assumed to have a unique set of coefficient values and a unique intercept value in the regression equation.[21]
- For each outcome variable, construct the overall treatment effect as the simple average of the subclass-specific estimates. (The use of an arithmetic average is enabled by having constructed the subclasses as quintiles, with each subclass containing an equal number of treatment-group members.)
It is important to note that the second stage of regression estimation will focus on respondent-level changes in outcomes, such as changes in ownership of the following types of assets:
- interest-earning assets at financial institutions (including passbook savings accounts, money market deposit accounts, certificates of deposit, interest-earning checking accounts, and—for IDA participants—individual development accounts);
- other interest-earning assets (including money market funds, U.S. government securities, and municipal and corporate bonds);
- stocks and mutual fund shares;
- equity in one's own home;
- equity in motor vehicles;
- equity in one's own business or profession; and
- IRA or Keogh accounts.
These are the asset types already measured
for comparison cases through the SIPP topical module on assets and
liabilities, and to be measured for treatment cases through the proposed
survey of 1,600 IDA participants.[22]
5.4 Cost Estimate
This section provides the estimated costs associated with conducting the nonexperimental impact analysis as a component of the AFIA evaluation and the survey of AFIA grantees and subgrantees. This analysis will provide an estimate of the impacts of AFIA-funded programs by comparing the pattern of savings and asset accumulation among AFIA program participants with the corresponding pattern of outcomes among AFIA-eligible nonparticipants during the period preceding the first awards of AFIA grants in late 1999.
The cost estimates, as shown in Exhibit 5-1, are based on the following assumptions:
- National household data on AFIA nonparticipants will be obtained through the public use data files for the 1996 SIPP panel. A sample of AFIA-eligible households will be constructed from the 1996 panel based on information collected through the Wave 1 core module and the Assets and Liabilities topical module administered at Wave 3 (conducted in December 1996-March 1997 and about to be released by the Census Bureau). This information will be used to identify all members of the 1996 panel who would have met the income and asset eligibility criteria for AFIA participation, had the AFIA legislation been in effect at that time.
- For this identified sample of AFIA-eligible nonparticipants, the "comparison group," we will use information collected in the core module and in the Asset and Liabilities topical module at Wave 3 (December 1996-March 1997), Wave 6 (December 1997-March 1998), Wave 9 (December 1998-March 1999), and Wave 12 (December 1999-March 2000). This will enable us to construct a 48-month record of their savings and asset ownership. Wave 3 data are about to be released by the Census Bureau. Data for Waves 6, 9, and 12 are to become available in early 2001, 2002, and 2003, respectively.
- A nonexperimental treatment group will be identified, consisting of individuals beginning their participation in AFIA programs during the 12-month period April 2000-March 2001(i.e., making their first IDA deposits into AFIA-matched accounts during this calendar period). A nationally representative sample of 1,600 such individuals will be identified from among selected AFIA program sites, where the selected sites are chosen on a probability-proportional-to-size (PPS) basis from among those in the first and second funded cohorts (FY 1999 and FY 2000), using a multi-stage cluster design. Only the experimental site will be excluded from selection, to avoid an undue respondent burden on program recipients at that site. The sample design will be addressed in the initial site selection memorandum (December 2000 draft and January 2001 revised).
- The nonexperimental treatment group will be interviewed at four follow-up intervals, occurring for each sample member at months 12, 24, 36 and 48 after the start of their AFIA participation. As shown in Exhibit 1-1, the first-round follow-up interviewing will take place during April 2001-March 2002; the second-round, third-round, and fourth-round follow-up interviewing will take place during each successive April-March period. The follow-up interviews will be conducted by telephone, using computer-assisted telephone interviewing (CATI) techniques. The survey instrument will consist of questions from the SIPP core module and the Assets and Liabilities topical module. The assumed interview length is 40 minutes. The response rates at each wave (computed as a percentage of the initial sample of 1,600) are assumed to be 60 percent at month 12, 50 percent at month 24, 40 percent at month 36, and 30 percent at month 48. This implies that 480 sample members will be interviewed through the final wave (month 48).
- To achieve these assumed response rates, sample tracking activities will be conducted. These activities will include several tracking letters sent to the survey sample prior to each interviewing wave. For the first-round follow-up interviews, these letters will be sent at months 5 and 10. Respondents will be offered a $10 incentive payment for providing the updated locating information requested at months 10, 22, 34, and 46. At each interviewing round, the survey respondents will then receive a $35 incentive payment.
- For both the comparison group and the nonexperimental treatment group, data will be compiled from existing governmental data sources on local economic conditions (such as the unemployment rate) during the time periods corresponding to the survey observations. This information will be used to construct additional explanatory variables for the impact analysis.
- Multivariate regression techniques will be used to model the savings and asset behavior of the combined treatment-comparison sample and to estimate the effects attributable to AFIA participation, as described in Section 5.3.
- The findings of the analysis of first-year effects will be presented in the September 2002 Interim Report. The findings with respect to the second-year, third-year, and fourth-year effects will be presented in the September 2003 Interim Report, the Spetember 2004 Interim Report, and the September 2004 Final Report, respectively.
| Item |
Rate
|
Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Units | Cost | Units | Cost | Units | Cost | Units | Cost | Units | Cost | Units | Cost | ||||||
| Staff Labor | |||||||||||||||||
| Mills, Greg | $45.86 |
444
|
$20,362
|
234
|
$10,731
|
250
|
$11,465
|
250
|
$11,465
|
136
|
$6,237
|
1314
|
$60,260
|
||||
| Hamilton, Chris | $76.92 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| DeMarco, Donna | $27.18 |
356
|
$9,767
|
320
|
$8,698
|
320
|
$8,698
|
320
|
$8,698
|
80
|
$2,147
|
1396
|
$37,943
|
||||
| Survey Director | $37.02 |
227
|
$8,404
|
408
|
$15,104
|
366
|
$13,549
|
366
|
$13,549
|
181
|
$6,701
|
1548
|
$57,307
|
||||
| Lam, Ken | $23.56 |
360
|
$8,482
|
280
|
$6,597
|
240
|
$5,654
|
24-
|
$5,654
|
200
|
$4,712
|
1320
|
$31,099
|
||||
| Battaglia, Mike | $62.37 |
40
|
$2,495
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
40
|
$2,495
|
||||
| Research Assistant | $15.87 |
572
|
$9,078
|
440
|
$6,983
|
440
|
$6,983
|
440
|
$6,983
|
80
|
$1,270
|
1972
|
$31,296
|
||||
| Secretary | $18.27 |
168
|
$3,069
|
112
|
$2,046
|
104
|
$1,900
|
104
|
$1,900
|
40
|
$731
|
528
|
$9,647
|
||||
| Ciurea, Michelle | $28.88 |
128
|
$3,697
|
70
|
$2,022
|
62
|
$1,791
|
62
|
$1,791
|
0
|
$0
|
322
|
$9,299
|
||||
| Person10 | $100.00 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Raise Pool |
4%
|
$2,610
|
$4,258
|
$6,248
|
|
$8,500
|
|
$4,728
|
$26,345
|
||||||||
| Subtotal Staff Labor | 2295 |
$67,872
|
1864 |
$56,438
|
1782 |
$56,288
|
1782
|
$58,539
|
717
|
$26,553
|
8440 |
$265,690
|
|||||
| Fringe |
42%
|
$28,506
|
$23,704
|
$23,641
|
|
$24,587
|
|
$11,152
|
$111,590
|
||||||||
| Overhead |
48%
|
$46,262
|
$38,468
|
$38,366
|
|
$39,900
|
|
$18,098
|
$181,095
|
||||||||
| Total Staff Labor |
2295
|
$142,640
|
1864
|
$118,611
|
1782
|
$118,295
|
1782
|
$123,026
|
717
|
$55,803
|
8440
|
$558,375
|
|||||
| Other Direct Costs | |||||||||||||||||
| Travel | |||||||||||||||||
| Airfare BOS/DC | $587 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Per Diem - Lodging DC | $118 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Per Diem - M&IE DC | $46 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Airfare BOS/KC | $1,190 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Per Diem - Lodging KC | $85 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Per Diem - M&IE KC | $38 |
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Ground Transportation | $70 |
08
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
0
|
$0
|
||||
| Subtotal Travel |
|
$0
|
|
$0
|
|
$0
|
|
$0
|
|
$0
|
|
$0
|
|||||
| Telephone (/hour) | $12 | 690 |
$8,280
|
570 |
$6,840
|
570 |
$6,840
|
570
|
$6,840
|
300
|
$3,600
|
2700 |
$32,400
|
||||
| Duplicating (/page) | $0.07 | 22700 |
$1,589
|
17,700 |
$1,239
|
17,700 |
$1,239
|
17700
|
$1,239
|
9000
|
$630
|
84800 |
$5,936
|
||||
| Equipment/PCs (/hour) | $3.36 | 2295 |
$7,711
|
1864 |
$6,263
|
1782 |
$5,988
|
1782
|
$5,988
|
717
|
$2,409
|
8440 |
$28,358
|
||||
| Postage/Delivery | $11 | 130 |
$1,430
|
125 |
$1,375
|
125 |
$1,375
|
125
|
$1,375
|
15
|
$165
|
520 |
$5,720
|
||||
| Inflation Adjustment |
3%
|
$570
|
$957
|
$1,432
|
|
$1,938
|
|
$1,084
|
$5,981
|
||||||||
| Total Other Direct Costs |
$19,581
|
$16,674
|
$16,873
|
|
$17,380
|
|
$7,888
|
$78,395
|
|||||||||
| Survey Research Group |
$141,154
|
$241,280
|
$217,998
|
|
$194,969
|
|
$76,144
|
$875,544
|
|||||||||
| G&A |
19%
|
$58,401
|
$71,547
|
$67,101
|
|
$63,721
|
|
$26,569
|
$287,340
|
||||||||
| Total Cost |
$365,775
|
$448,112
|
$420,267
|
|
$399,096
|
|
$166,404
|
$1,799,654
|
Fixed Fee |
6%
|
$21,947
|
$26,887
|
$25,216
|
|
$23,946
|
|
$9,984
|
$107,979
|
Total Worth |
$387,722
|
$474,999
|
$445,483
|
|
$423,042
|
|
$176,388
|
$1,907,633
|
Also now included in the estimated cost of the nonexperimental impact analysis is the cost of conducting the survey of grantees and subgrantees in the FY 1999 and FY 2000 cohorts. The aim of this survey is to obtain information about non-IDA benefits and services offered by each grantee or subgrantee. This will enable the impact analysis to examine whether the non-IDA "service configuration" is a significant factor in explaining the experience of AFIA program participants. The cost assumptions for this survey are as follows:
- The number of grantees in these two cohorts combined will be 80, with an average of three subgrantees per grantee. State-level grantees will be excluded.
- The survey will consist of a brief two-page questionnaire, to be mailed out annually to each grantee. Grantees will be responsible for assembling the responses from all subgrantees.
- In each year, an assumed 50 percent of grantees will respond to the survey without any additional effort from the evaluation contractor. The remaining 50 percent will be called by telephone, to remind them of the importance of completing the survey and to offer assistance in completing it. One-half of those called (i.e., 25 percent of the total) are assumed to respond to the telephone reminder. The remaining grantees and subgrantees will require a telephone call to actually obtain their responses through a telephone interview.
The survey will be conducted annually for four years, from 2001 through
2004.
Notes
[15] For the 1996 panel, the "Assets and Liabilities" topical module was administered during Waves 3, 6, 9, and 12, which occurred in December of 1996, 1997, 1998, and 1999, respectively. For the 2000 and 2001 panels each, this topical module will be administered during Waves 3 and 6. These waves will occur in October of 2000 and 2001 for the 2000 panel, and in October of 2001 and 2002 for the 2001 panel. [Return to Text]
[16] In 1996, the EITC limits on earned income (i.e., the income level at which the EITC phases down to zero), were as follows: $9,500 for a household without a child, $25,078 for a household with one child, and $28,495 for a household with two children. See U.S. House of Representatives, Committee on Ways and Means, 1996 Green Book: Background Material and Data on Programs within the Jurisdiction of the Committee on Ways and Means, November 4, 1996, p. 805. [Return to Text]
[17] See, for instance, Donald Rubin, "Estimating Causal Effects from Large Data Sets Using Propensity Scores," Annals of Internal Medicine, Part 2, October 15, 1997, 127: 757-763. [Return to Text]
[18] See John Karl Scholz, "The Earned Income Tax Credit: Participation, Compliance, and Anti-Poverty Effectiveness," National Tax Journal, March 1994, pp. 59-81. Scholz estimated the national EITC participation as between 80 and 86 percent for 1990. If this rate were closer to 100 percent, it would be difficult to identify the differences between participants and nonparticipants in observable characteristics. [Return to Text]
[19] In the SIPP Tax Module, administered to the 1996 panel in Waves 4, 7, and 10, respondents were asked "Did you claim an earned income credit on your Federal income tax return?" If so, they were then asked, "What was the amount of earned income credit claimed?" [Return to Text]
[20] The extent to which one can control for time-varying factors will depend on the availability of SIPP information regarding the locality of residence for comparison cases. This may require special arrangements with the Census Bureau to obtain respondent-level information beyond that normally contained in public-use data files. [Return to Text]
[21] For any give outcome, the estimating equation used across the five subclasses will have a consistent functional form and will include a consistent set of explanatory variables. From one outcome measure to another, however, there may be differences in the model specification. For instance, limited dependent variables (e.g., those whose values range between zero and one) will be treated differently than those continuously measured. [Return to Text]
[22] See, for instance, the following Census publication based on SIPP data from the 1991 and 1992 panels: T.J Eller and Wallace Fraser, Asset Ownership of Households: 1993, U.S. Bureau of the Census, Current Population Reports, P70-47, U.S. Government Printing Office, Washington, DC, 1995. [Return to Text]
