LIHEAP Research: Data Technical Notes on Estimates of Recipiency Targeting Indexes (Attachment B) FY 2008

Technical Notes on Estimates of LIHEAP Heating Assistance Recipiency Targeting Indexes for Households Receiving LIHEAP Heating Assistance in Federal Fiscal Year (FFY) 2008

Division of Energy Assistance, U.S. Office of Community Services

                                                                    February 17, 2012

The Office of Community Services (OCS) within the Administration for Children and Families (ACF) of the U.S. Department of Health and Human Services (HHS) administers at the Federal level the Low Income Home Energy Assistance Program (LIHEAP).  APPRISE, under contract to OCS’ Division of Energy Assistance, has prepared State-level targeting performance data that indicate the extent to which States targeted heating assistance to low income, vulnerable households in FFY 2008, i.e., elderly or young child households.

The targeting performance measure relies on low income household estimates and LIHEAP recipient household counts for FFY 2008.  The Technical Notes provide the context for understanding the data sources, calculations, interpretation, and caveats for the data tables which are included in Attachment A LIHEAP Information Memorandum-2012-03, dated February 17, 2012.  The Technical Notes cover the following topics:

• Low Income Household Estimates;

• Heating Assisted Household Population Counts; and

• Recipiency Targeting Indexes.

                                              Low Income Household Estimates

Low income household estimates by States are needed in the calculation of State LIHEAP recipiency targeting performance.  The topics of low income households, methodology, data source, population estimates, confidence interval, and caveats are described below.

Low Income Households

Section 2605(b)(2) of the LIHEAP statute allows grantees to serve households that have incomes at or below the Federal LIHEAP maximum income standard, i.e., the greater of 150 percent of the poverty level or 60 percent of a State’s median income.  States may set their LIHEAP maximum income standards equal to or lower than the Federal LIHEAP maximum income standard as long as their income standards are not set below 110 percent of the Federal poverty level, i.e., the HHS Poverty Guidelines.

States’ FFY 2008 LIHEAP maximum income standards for heating assistance (including any special cutoff rules) were obtained from OCS’ LIHEAP Grantee Survey for FFY 2008.  Those standards are expressed as a percent of the 2007 HHS Poverty Guidelines, and are included in the LIHEAP Report to Congress for FFY 2008.

Methodology

The following variables were used in classifying low income households for each State, using a public-used microdata file that includes records on individual housing units:

• Household population, household size, household composition, and household income;

• The 2007 HHS Poverty Guidelines that were published on January 24, 2007, in the Federal Register  (72 FR 3147-3148); and

• The State median income estimates for FFY 2008 that were published in the Federal Register on March 28, 2007 (72 FR 3147-3148).

The 2007 HHS Poverty Guidelines and the State median income estimates are used as they were in effect at the beginning of FFY 2008 (October 1, 2007).

Data Source

The calculation of State-level recipiency targeting indexes relies, in part, on data from the Census Bureau’s American Community Surveys (ACS) for 2006, 2007 and 2008.  ACS provided statistical estimates on the number of a State’s LIHEAP income eligible populations and of vulnerable households within that population. The ACS is an ongoing nationwide household sample survey that replaces the Decennial Census long-form questionnaire.  It provides a moving statistical picture of population and housing information every year instead of once every 10 years.  Its annual sample size is approximately three million households.

The Census Bureau produces ACS public-use microdata files that include records on individual housing units and the people living in those housing units or a group quarters.  Each record includes information associated with a specific housing unit or individual except for names, addresses, or other identifying information.

Population Estimates of LIHEAP Income Eligible Households

The estimated number (or population estimate) of LIHEAP income eligible households for each State for FFY 2008 is based on household sample data, using three-year State estimates from the 2006, 2007, and 2008 ACS.  Detailed ACS information is described in the American Community Survey’s Design and Methodology paper (April 2009).

Sample data not only describe the particular set of households in the sample, but are primarily used to estimate or weight the data that would have been obtained if a complete census count of the variable of interest was available.  All data that are based on samples, such as the ACS, are subject to the two broad types of errors that are described below.

• Sampling Error.  Sampling error refers to the chance error that results in estimating data, such as household income, from a sample rather than a known population count.  Such an error arises from the selection of persons and housing units in the sample household surveys.  For example, if the sample includes disproportionably less low income households than in the general population, the low income sample estimates could be undercounted.  The magnitude of sampling error is relatively small if a large number of households are sampled, but relatively large if a small number of households are sampled.

Weighted sample data from the ACS are quite good relative to most other sources of comparable information. The data are unique in that their reliability and validity continues to be studied.  Consequently, the data user can be well informed about the quality of the data and the potential limitations of their use.

• Nonsampling Error.  Nonsampling error, which affects both sample and complete count data, is the result of errors that may occur during the ACS data collection and processing phases.  Nonsampling errors include undercounting of persons and housing units, respondent errors such as the underreporting of income or misclassification of public assistance benefits, and errors that may occur during the clerical handling and electronic processing of questionnaires.  The Census Bureau tries to minimize nonsampling errors by using trained interviewers and by carefully reviewing the survey’s sampling methods, data processing techniques, and questionnaire design.

Confidence Interval

A confidence interval is the range wherein the true value of a population estimate (or point estimate) falls within a certain level of confidence due to sampling error or variability).  A confidence interval is expressed or displayed as the range in which the true population value falls.  The range includes a lower and upper limit with the population estimate being at the midpoint of the range.  The wider the confidence interval the less precise is the estimate.  Confidence intervals which overlap each other generally indicate that the difference between two estimates are not statistically significant, i.e., whether a statistical estimate is likely to have occurred by chance

The confidence evidence is expressed as a percentage, usually 90 or 95 percent.  For example, the recipiency targeting indexes use a 95 percent confidence interval.  This means that there is a 95 percent chance that the actual targeting index, i.e., the population estimate falls within the estimated lower and upper limits of the confidence interval (CI), as shown below:

The width of the confidence interval will vary depending on a number of factors. In general, confidence intervals for smaller States are wider than for larger States, though there are certain exceptions.  In addition, confidence intervals are generally wider for State maximum income than for the Federal maximum income. In both cases, the confidence intervals are wider because the estimate is based on a smaller sample size. The reliability and preciseness of estimates decrease as the width of the confidence interval increases

                                                           Lower     Population    Upper

                                                            Limit      Estimate       Limit

ACS sampling variability is due to the ACS collecting data based on a sample, not a population count.  The recipiency targeting indexes include the estimate (also known as the point estimate) as well as the upper and lower limits around the estimate using a 95 percent statistical confidence interval.  Such an interval indicates that we are 95 percent confident that the actual recipiency targeting index for a particular State falls within the lower and upper limits surrounding the estimate.  Confidence intervals which overlap between or among States indicate that the difference in the targeting index estimates for those States are not statistically significant.

Caveats

The following qualifications apply in interpreting estimates of the number of low income households:

• Household income.  The LIHEAP statute does not define household income.  ACS defines household income as the total money income received in a calendar year by all household members 15 years old and over.  State LIHEAP grantees may choose to use net household income (excluding certain household costs or expenses) or gross (total) household income in establishing their States’ LIHEAP maximum income standard.  However, OCS requires States to report on the LIHEAP Household Report the number of assisted households by poverty levels, using gross household income.

• Households.  A small variation in the counts of low income households, particularly for those households having renters or boarders, can result due to differences in how households are defined by the Census Bureau and by the LIHEAP statute.  The Census Bureau’s definition of a household focuses on all persons living in a housing unit.  The LIHEAP statute’s definition of household focuses on “any individual or group of individuals who are living together as one economic unit for whom residential energy is customarily purchased in common or who make undesignated payments for energy in the form of rent.”

• Income Eligibility.  State procedures for determining income eligibility may include annualizing one or more months of household income to test against the State's LIHEAP maximum income eligibility standard.  Therefore, some households may be income eligible for LIHEAP even though their actual annual incomes exceed the Federal or State LIHEAP maximum income standard.

As noted above, State procedures for determining LIHEAP income eligibility may also rely on using household net income in establishing LIHEAP income eligibility.  Therefore some households may be income-eligible for LIHEAP even though their gross incomes exceed the Federal or State LIHEAP maximum income standard.

• LIHEAP income standards.  States may set their LIHEAP maximum income standards lower than the Federal LIHEAP maximum income standard as long as their income standards are not set below 110 percent of the Federal poverty level.  Consequently, the number of income eligible households under a State’s LIHEAP maximum income standard can be equal to or lower than under the Federal LIHEAP maximum income standard.  If a State’s LIHEAP maximum income standard is less than the Federal LIHEAP maximum income standard, then its number of income eligible households becomes smaller, resulting in a larger sampling error.

Heating Assisted Household Population Counts

The calculation of State-level recipiency targeting indexes also relies, in part, on data from OCS’ LIHEAP Household Report for FFY 2008.  The Report provided state-reported numbers of heating assistance households, including those households that had at least one member who was elderly or a young child.  These counts are included in HHS’ annual LIHEAP Household Report to Congress for FFY 2008.  States are required to report complete household counts of the population that received LIHEAP assistance.  The topics of vulnerable households, heating assistance, and caveats are described below.

Vulnerable Households

The LIHEAP statute defines vulnerable households as those households having at least one member that is a young child, an individual with disabilities, or a frail older individual.  The LIHEAP statute specifies that vulnerable households and high-home energy burden households are especially made aware of LIHEAP assistance.  OCS requires State LIHEAP grantees to report annually on the number of assisted households with at least one member 60 years or older, disabled, or a child 5 years or under.

Heating Assistance

The state targeting indexes are based, in part, on those households that received benefits for heating assistance, which is the largest component of LIHEAP assistance.  The data for each State include the number of households receiving heating assistance and, of those households, the number of heating assistance households having at least one person 60 years or older or having at least one person 5 years or younger, as reported to OCS in each State’s LIHEAP Household Report.  Beginning with household data for FY 2011, state targeting indexes will be based on the count of households that receive any type of LIHEAP assistance, as reported by the States.  Such counts will provide for more robust performance data by expanding State targeting performance to include all types of LIHEAP assistance provided by each State.

Caveats

The following qualifications apply in using the state targeting indexes that rely, in part, on state-reported numbers of LIHEAP heating assistance households with vulnerable members:

• States’ targeting performances do not take into account that a State may have a higher targeting index score for other types of LIHEAP assistance such as cooling assistance.

• As noted above, some States’ targeting indexes may be larger due to differences in how States define and calculate household income.  For example, States that determine eligibility based on net household income, rather than gross income, may have higher targeting indexes compared to other States that used gross household income to determine LIHEAP income eligibility.  Therefore, more vulnerable households may be served within the State income maximum than would have been included using gross income.  Even among States using net income, there may be differences in recipiency targeting indexes due to differences in how each State defines net income, i.e., the range of income exclusions that a State permits.

• Counts of LIHEAP recipient households do not take into account if a State has other types of energy assistance programs that deal directly with vulnerable households.  This would decrease the possible number of LIHEAP assisted vulnerable households to the extent that other programs serve such households.  For example, Ohio provided heating assistance in FFY 2008 to low income households with young children primarily through its Temporary Assistance for Needy Families program.

• As “disability” is not defined, targeting indexes have not been computed for this group of households due to the lack of uniformity in States’ definitions.

• The counts rely on the accuracy of state-reported data on households receiving LIHEAP heating assistance that are included in the Department’s annual LIHEAP Report to Congress.

State Targeting Indexes

Performance goals must be measurable in order to determine if the goals are being achieved.  OCS has developed a set of performance measures (i.e., targeting indexes) that allow for quantitative analyses of various aspects of program targeting performance.  The calculation of state targeting indexes and caveats are described below.

Targeting Indexes

The recipiency targeting index measures the extent to which a specific group of households receives program benefits, e.g., LIHEAP assistance.  This index is computed for a specific group of households by dividing the percent of LIHEAP recipient households that are members of the target group by the percent of all income eligible households that are members of the target group.  For example, if 25 percent of LIHEAP recipient households are elderly households and 20 percent of all income eligible households are elderly households, then the recipiency targeting index for elderly recipient households is 125 (100 times 25 divided by 20).

A recipiency targeting index greater than or lower than 100 indicates that the target group’s incidence in the LIHEAP recipient population is higher or lower than that group’s incidence in the income eligible population, respectively.

Caveats

The following qualifications apply in using the state targeting indexes:

• Each confidence interval is represented as a line with the population estimate at the midpoint of the interval. Confidence intervals which overlap each other between States indicate that the difference in the targeting index estimates between those States is not statistically significant.  Caution should be exercised in comparing state targeting indexes which are not statistically significant.

• The state targeting indexes are limited to LIHEAP heating assistance at least through FY 2010 data.

• State targeting indexes do not take into account that States may be targeting another LIHEAP group, i.e., those households with the highest home energy costs and lowest household incomes, as specified in the LIHEAP statute. State-specific data on such households and the extent to which such household include vulnerable members is not available at the national level.

• The targeting performance data results indicate that a higher percent of States target young child than elderly households.  A review of the literature on the targeting of households with elderly members indicates that other federal social programs also have limited success in serving eligible elderly households, especially in comparison to households with young children.  The research appears to show that program participation barriers are most significant when elderly households have not made previous use of public assistance programs.