Implementing LIHEAP Outcome Performance Measures
THIS CONTAINS INFORMATION ISSUED BY THE U.S. ADMINISTRATION FOR CHILDREN AND FAMILIES IN LIHEAP ACTION TRANSMITTAL NO. LIHEAP- AT-2010-4, DATED 3/17/10
TO: LOW INCOME HOME ENERGY ASSISTANCE PROGRAM (LIHEAP) STATE GRANTEES
SUBJECT: Implementing LIHEAP Outcome Performance Measures RELATED Low Income Home Energy Assistance Act, as amended
REFERENCES: (Title XXVI of Public Law 97-35, the Omnibus Budget Reconciliation Act of 1981, as amended); 45 Code of Federal Regulations, Part 96; Final Rule amending HHS block grant regulations (64 Federal Register, 55843, October 15, 1999); Government Performance Results Act of 1993 (Public Law 103- 62); Office of Management and Budget's Program Assessment Rating Tool.
PURPOSE: (1) To provide background information on the LIHEAP Performance Measures Work Group activities and the Proposal for LIHEAP Performance Measures Draft Report. (2) To request participation and collaboration from State grantees in forming an informal, long-term work group for LIHEAP performance measures implementation.
BACKGROUND: LIHEAP 2003 Program Assessment
Beginning in 2002, the Director of the U.S. Office of Management and Budget (OMB) administered the Program Assessment Rating Tool (PART) for every Federal program as a method for implementing the program performance requirements imposed by the Government Performance Results Act of 1993 (GPRA).
In 2003, LIHEAP received its first Federal PART assessment and the score of "results not demonstrated" (see Link A below). For long-term performance measurement, LIHEAP received zero points on the following questions (see Link B below):
• Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?
• Does the program have ambitious targets and timeframes for its long-term measures?
• Does the program have baselines and ambitious targets and timeframes for its annual measures?
• Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?
• Has the program taken meaningful steps to correct its strategic planning deficiencies?
As a result of LIHEAP's initial PART rating of "results not demonstrated," OMB required the establishment of a LIHEAP performance improvement plan (PIP) that covered three aspects of the LIHEAP program. OCS updates this plan on a semi- annual basis with milestones for reaching improvement in several areas, including the implementation of long-term measures.
The heating assistance recipiency targeting indexes have generated useful information for analyzing to what extent LIHEAP provides heating assistance to elderly households and young child households. However, the indexes provide only proxies for program performance. They do not demonstrate to what extent LIHEAP affects recipients' energy security or long-term health and safety. To more accurately capture and evaluate performance and impact, LIHEAP needs to track outcome performance measures. Rather than develop these measures alone, OCS decided to collaborate with State LIHEAP Directors.
LIHEAP Performance Measures Work Group
In June 2008, OCS formed the LIHEAP Performance Measures Work Group (PMWG) as a way to collaborate with State LIHEAP Directors to create the appropriate LIHEAP outcome performance measures. The PMWG consisted of the following members: Jim Cain (Wisconsin); John Harvanko (Minnesota); Heather Jones (Missouri); Diane Telencio (New Jersey); and Melissa Torgerson (Oregon).
Through various meetings and discussions over the phone and in person from June 2008 to November 2009, the PMWG developed a draft proposal for a LIHEAP performance measurement system (see Attachment A below). In developing the proposal, the PMWG built upon feedback from a meeting facilitated by the National Association of Energy Assistance Directors (NEADA) where State LIHEAP Directors met in September 2009 to discuss performance measures. The PMWG also presented its proposal to State LIHEAP Directors at the winter NEADA meeting in February 2010. Attendees at this meeting supported the proposal's implementation.
Proposal for LIHEAP Performance Measures
The PMWG offers the proposal for adoption by States and considers it a living document. The proposal provides several recommendations regarding LIHEAP outcome performance measures. These recommendations consist of the following:
• Measuring the impacts of the three major types of services delivered by State grantees, including energy assistance, energy services, and client services;
• Developing a tiered set of performance measures which increase in complexity from Tier 1 to Tier 4, as follows:
o Tier 1 containing output measures that every State is already collecting;
o Tier 2 containing outcome measures that most States can or are collecting but not necessarily reporting;
o Tier 3 containing the most complex outcome measures that few States are presently collecting;
o Tier 4 containing the broadest outcome measures that provide a national picture of how well LIHEAP provides recipients with continuous, safe, and affordable energy service; and
• Executing this tiered approach to data collection and reporting in which States will gradually implement the measures in Tiers 1-3— relying upon technical assistance facilitated by OCS—and OCS will have responsibility for collecting and reporting on the Tier 4 measures.
CONTENT: Performance Measures Implementation Work Group
Program performance and assessment information play an increasingly important role in Federally funded programs. OMB reviews this information when developing the President's annual budget. Numerous policies and regulations, such as the annual Congressional Budget Justification, mandate this type of information in order to comply with annual budget and program effectiveness reporting requirements. Although new guidance on program performance is still forthcoming, State grantees should continue to act proactively to avoid Congressionally-imposed performance measures associated with future reauthorization legislation. Robust performance measures will also aide States in communicating performance achievements and obstacles to stakeholders such as State legislatures, Congress, and the public.
To build on the success of the PMWG, OCS is soliciting meaningful stakeholder consultation in developing an implementation plan for the LIHEAP outcome performance measures outlined in the PMWG proposal.
»Action Item: OCS requests volunteers from State grantee offices to participate in a small, informal work group. Ideally, the work group will consist of a balanced mix of States, both geographically and in size, as well as in level of experience and capacity in working with data, performance measures, and IT reporting systems. Work group members may be either State LIHEAP Directors or someone on their staff.
As developing and executing a performance measures implementation plan requires long- term work and commitment, the work group will have a systematically staggered rotation of members, soliciting new membership every 12 to 15 months on an ongoing basis. This will ensure that at any given time there will be a combination of new and existing work group members. OCS encourages every State grantee to consider participating in the work group as it progresses. This will ensure that the work group and its deliverables represent and address the diverse needs and capacities of grantees.
The work group meetings will occur via teleconference and, when feasible, face-to- face meetings. It is anticipated that meetings will be scheduled at least on a monthly basis for the first year. It is expected that work group members will spend some time in between meetings reviewing relevant documents, researching issues, and drafting work products. All work group members will receive recognition for their involvement.
The work group has the following objectives:
• Develop a performance measures implementation plan and timeline that addresses, at minimum: data collection, management, and analysis; instrument design and selection; feasibility and fidelity (e. g., data integrity and uniformity); and ethical concerns;
• Determine the training and technical assistance needs and obstacles of grantees through a needs assessment; and
• Provide on-going assistance to OCS in identifying and mitigating implementation issues and in improving the implementation process.
State grantees should contact Chad Sawyer by email for further information and/or to join the performance measures implementation work group. We request that interested staff indicate their willingness to join the work group by Friday, April 9, 2010.
INQUIRIES TO: Chad D. Sawyer, Program Specialist
Division of Energy Assistance Office of Community Services, ACF, HHS
370 L'Enfant Promenade, S.W.
Washington, D.C. 20447
Telephone: (202) 401-4870
ATTACHMENTS: A. Proposal for LIHEAP Performance Measures Draft Report
LINKS: A. PART Program Assessment Summary
Yolanda J. Butler, Ph.D.
Office of Community Services
Administration for Children and Families
U.S. Department of Health and Human Services
370 L'Enfant Promenade, S.W. Washington, DC 20447