Table of Contents | Previous | Next |
Methods
Study Population
Recruitment of women into the three trials is described elsewhere (Olds, Henderson, Tatelbaum, et al., 1986; Kitzman, Olds, 1997; Olds, Robinson, O’Brien, et al., 2002). The study populations in the three sites were all primiparous pregnant women. Most were in low-income households, and most were unmarried. But the composition of the population in the three sites differed in some respects. The data in Table 1 suggest that the three samples differed with respect to level of risk for negative pregnancy outcomes, inadequate parenting and compromised child health and development, and other negative life-course outcomes. In particular, the Elmira group had fewer and lower levels of risk factors than did the other two samples, and the Memphis sample was at greatest risk.
| Characteristic of Mother | Elmira, New York | Memphis, Tennessee | Denver, Colorado |
|---|---|---|---|
| Under 19 years | 47% | 65% | 58% |
| Unmarried | 62 | 97 | 87 |
| Low income | 61* | 85* | 95* |
| Minority group | 11 | 92 | 65 |
| * Low income in Elmira was defined as Hollingshead class IV or V, semi-skilled and unskilled laborers. Low income in Memphis was defined as <100% of the federal poverty level. Low-income in Denver was defined as qualifying for Medicaid (<133% of poverty). |
Government Expenditures
Government expenditures we measured included those that are usually considered part of the social welfare system, e.g., Aid to Families with Dependent Children (AFDC), now transformed to Temporary Assistance for Needy Families (TANF), Food Stamps, low-income energy assistance, child care subsidies, Medicaid and State Children’s Health Plan programs. The aspects of government expenditures we examined in the study went beyond welfare. As noted above, we examined major government expenditures that theoretically could be affected by this program, such expenditures made for children who repeat a grade in school or those required to prosecute crimes committed by participating mothers.
We were not able to measure costs for all programs in each site for several reasons. First, some programs did not exist in all three sites during the period analyzed. In Elmira, the period included in this analysis was 1978-1994. Subsidized child care did not exist during the period in which the Elmira children would have been able to use it. Also, the State Children’s Health Plan did not exist in any state until 1998.
The second reason for differences in the government programs included in this analysis has to do with data availability. Surveys of study families have been conducted periodically since the study child’s birth and are an important source of data on government expenditures. Because the cost analysis is a relatively recent addition to the numerous studies conducted on these samples, there is some variability in government program-related questions asked at different periods for the three samples. The government program information gathered for the Elmira group was somewhat limited compared with that gathered for the other two sites, largely because it was collected prior to the initiation of these cost analyses.
The primary source of information on government programs, other than surveys of the families, is administrative data maintained by government programs. Where possible, we used administrative data for our estimates of government expenditures, but there are some gaps in administrative data that are the result of the length of time between use of a program by a study family and the time data were collected for the current analysis. The databases maintained by states for beneficiaries of government programs are quite large, and, as a result, they are regularly purged of data for which the state no longer has a use. This practice most affected the Elmira analysis in that, with the exception of arrest and child welfare data, none of New York state’s administrative data were available for study. The Elmira analysis therefore relied principally on survey data. For Memphis and Denver, we were able to obtain much administrative data and used them in our analyses.
Another reason for differences in the programs whose expenditures we measured had to do with the age of the child during the study period. In Elmira, because the study period ended at the child’s 15(th) year, we were able to include some school-related variables: special education and grade repetition. Children in the other two sites were too young to attend school during the study period.
There were some government programs whose use might be expected to be affected by the nurse visiting program, but which we decided not to measure. When a government program’s expenditures could conceivably be affected by nurse visitation, we included it unless local conditions suggested that there could not be a program effect. An example of this is subsidized housing. In both Memphis and Denver, the waiting list for subsidized housing was approximately 5 years long. When, as in this case, we found the supply of government services to be so inelastic that it was unlikely that it could be affected by the nurse visitation program, we did not measure it.
The programs included for all three sites were:
- AFDC/TANF
- Food Stamps
- Medicaid
- Child Protective Services
- Foster care (Elmira and Memphis) and Out-of-home placement and family preservation in Denver (part of its foster care system)
The programs included in two of the three sites were:
- Supplemental Security Income (SSI) (Elmira and Denver)
- Subsidized child care (Memphis and Denver)
- Job training (Memphis and Denver)
The programs included in only one site were:
- Repeated grades (Elmira)
- Special education (Elmira)
- Crime Investigation and court costs (Elmira)
- Child Health Plan (Denver)
- Head Start and Early Head Start (Denver)
- Low-income Energy Assistance Program (Denver)
Taxes Paid
An important component of this net cost analysis is the contribution study subjects made to social welfare in the form of taxes paid. We have therefore included taxes, primarily income and social security taxes, paid by working study subjects in our calculations.
Analytic Methods
Methods of Measuring Visitation Program Costs. We calculated visitation program costs for Memphis and Denver from accounting documents that reported actual spending on the nurse home visitation program, which was delivered until the study child’s 2(nd) birthday in all sites. We excluded research-related costs in order to isolate the costs that would be incurred by those wishing to replicate the visitation program. For Elmira, we used visitation program costs calculated for the 1993 cost study (Olds, Henderson, Phelps, et al., 1993). Visitation program costs were apportioned to the months in which they were incurred for Denver and Memphis. For Elmira, they were apportioned equally to the first 24 months of the study child’s life. This process created visitation program cost streams for each subject for the entire study period for each site. All program costs included actual overhead costs (rent, utilities, administrative expenses, etc.). This differs from the approach we took to accounting for government costs, described below, which did not monetize overhead expenses. This is a conservative approach.
Methods of Measuring Government Program Costs. Detailed methods used for each program in each site as well as sources of all data and program values will be described in an Appendix, which is forthcoming. The following paragraphs describe our general approach to using both administrative and survey data and summarize our methods.
Administrative Data. When we were able to obtain administrative utilization data from government agencies, we used them, rather than survey data, in estimating costs for each family, because we believed that government program data were likely to be more accurate than information provided by study participants who were asked to recall program use over a number of years. Sometimes we received administrative utilization data only; sometimes we received both utilization and cost data. In Memphis, for example, we received administrative files from AFDC and Food Stamps for the families in the study. These showed every month in which the family received AFDC grants and food stamps as well as the grant amounts. Our examination of the cost data revealed that they often were not credible—the most common problem being that the amounts shown greatly exceeded the maximum grant amount for the number of people in the family. We therefore assumed that the family received the maximum grant amount allowable based on the number of persons in the family for each month in which they were enrolled. In Denver, we were provided only utilization data for AFDC, so we used the same approach: assuming the maximum grant amount for all families, based on family size. This is a conservative approach, in that we would expect intervention-group families to have smaller grant amounts due to their earlier labor force participation and higher incomes. We therefore could have overestimated these costs for the intervention group.
From the Food Stamps program, we received data for units other than the study families. Food stamp amounts are based on the number of people who eat together rather than family size, so we adjusted the coupon amounts to reflect only the number of people in the study family.
For several programs, we received administrative data for enrollment rather than utilization. In those cases, we imputed costs. This was the case for Medicaid in both Memphis and Denver, so we estimated costs based on the category of eligibility for each family and the applicable capitation rates provided by both states (most families in Denver and all families in Memphis were in managed care plans). In New York, where Medicaid recipients were not in managed care plans, we used average Medicaid expenditures for enrollees according to age to estimate costs.
For cases in which we received administrative data on incidents of use, we assigned average costs per case. For instance, for child protective services in Elmira, we applied the New York state average cost of investigation per case of child maltreatment to the administrative data on number of investigated reports for the study families.
In Elmira, the only site for which we had administrative arrest data, we had only number of maternal arrests and general type of crime from New York State. We imputed costs based on the type of crime for which the mother was arrested, according to state records and the number of days she reported spending in jail in response to the 15-year survey. We used arrest and jail costs estimated for Washington State by Aos et al. (2001) to estimate costs of crime.
Survey Data. For a number of government programs, we did not have administrative data; we had only survey data collected at varying intervals and describing varying periods of time. For nearly all government programs used by the Elmira families, we had only survey data (the only exceptions were child protective services, foster care, and criminal justice system services). In Memphis, use of subsidized childcare and job training were reported in response to follow-up surveys. In Denver, receipt of SSI, subsidized childcare, and low-income energy assistance, as well as enrollment in the Child Health Plan and Head Start/Early Head Start were reported in response to surveys. The surveys typically asked the respondent questions about whether they used a program since the prior interview date or the birth of the child and, if so, for how long. Since the period between interviews could be lengthy, and dates of use often were not provided, it was necessary in numerous cases to make assumptions about when the families used the program. For childcare, it was logical to assume that use occurred at the beginning of the period covered by the survey, when children were younger, since that is when childcare is needed most. Moreover, it is the most conservative assumption possible because of the declining value of money over the study period. For other programs, when we did not know the time period during which the program was used, we usually assigned it to the beginning of the interview period. Exceptions included repeated grades in Elmira, which could not have occurred until the child was old enough to attend school. Assuming that utilization occurred at the beginning of the period covered by the interview was inevitably the most conservative assumption possible because of the discounting and inflation adjustment process we used, which insured that dollars spent at earlier time periods were more valuable than those spent later.
Survey data on use of government programs often were limited to frequency and duration of utilization and did not include actual government expenditures. For those programs for which we had utilization data only, we estimated government expenditures by multiplying utilization by average program cost, which was obtained from the study state for the period of interest. All costs were assigned to the month in which they occurred or were assumed to occur. In this way we created monthly government expenditure cost streams for each subject.
Taxes Paid. Survey respondents were not asked for tax data, but they were asked about employment, including job title and the period during which they worked at each job and wages. From this, we were able to estimate income for each study subject for Elmira and Denver. In Memphis, we had access to administrative income data maintained by the state’s unemployment insurance program. To estimate federal income tax, Internal Revenue Service tax rates for each year were applied to reported income for that year. If the subject’s income qualified for the Earned Income Tax Credit, the credit was calculated and subtracted from estimated taxes. We also estimated Medicare tax and Social Security taxes (a fixed rate is assessed for both) based on each family’s reported income. In the case of Memphis, the only site for which administrative income data were available, we compared income estimated based on survey responses with that from the Tennessee Department of Labor and found that survey data produced, on average, higher income estimates than Labor Department data. The difference was not significant, however. It is possible that the survey captures some unreported income, which would account for the higher self-reported estimates. For our purposes, the income of interest is reported income, since that is what taxes are based on. We therefore used the Department of Labor’s figures. The Department of Labor data included only income reported by Tennessee companies. It therefore does not include income information for families that moved out of state. About 10% of the Tennessee families lived out of state for some part of the study period. More nurse-visited families moved out of state than did comparison group families. As a result, it is possible that taxes paid by nurse-visited families are understated relative to comparison families. Cost (in this case, revenue, or negative costs) streams were also created for taxes paid.
Inflation Adjustment and Discounting. The three nurse visitation programs were conducted at different times: Elmira in the late 1970s and early 1980s; Memphis in the early 1990s; and Denver in the mid-1990s. Because of this and since the costs of the interventions and their associated outcomes occurred at different times, and because it was necessary to develop a summary net cost for each intervention, we adjusted the cost streams described in the preceding paragraphs for both inflation and the time value of money. To adjust for inflation, we used the Consumer Price Index for all items for the Denver-Boulder-Greeley area (for Denver), for the southern United States (for Memphis), and for the northeastern United States (for Elmira). We then discounted these costs to adjust for the time value of money to arrive at their present value, which allows us to make more accurate comparisons among expenditures that occur at different times. The discount rate we used was 3 percent, as recommended in Gold et al. (1996). All values are expressed in 2001 dollars.
Estimating Net Costs. Once all cost streams were adjusted for inflation and discounted, it was straightforward to calculate total visitation program costs, total government costs and total taxes paid. All amounts are summed separately for the comparison group and the intervention group. The sum of all tax revenues is subtracted from the sum of all government program costs to arrive at net government cost for each group. If the comparison group’s net government expenditures exceed those for the intervention group, the difference is compared with the sum of all visitation program costs to determine the ratio between them. This expresses the degree to which the visitation program’s costs have been recouped through savings in government expenditures.
Statistical Analysis
The cost data presented herein, like much economic data, are not approximately normal. After examining histograms of raw data and log-transformed data, we chose to employ the non-parametric Wilcoxon rank sum to test the hypothesis of equivalence between the control and visited groups. This test, which uses ranks of the data rather than the data itself, is not unduly influenced by extreme values. For the Elmira data, in addition to testing the whole sample, we performed the test separately for the low SES group and for the high SES group.
| Table of Contents | Previous | Next |