CSBG IM #144 State and Federal Accountability Measures and Data Collection Modernization

U.S. Department of Health and Human Services
Administration for Children and Families
Office of Community Services
Division of State Assistance
370 L'Enfant Promenade, S.W.
Washington, D.C. 20447

COMMUNITY SERVICES BLOCK GRANT PROGRAM
Information Memorandum

Transmittal No. 144

Date: October 2, 2015

TO: State Community Services Block Grant (CSBG) Administrators, U. S. Territory CSBG Administrators, Eligible Entities, and State Community Action Associations

SUBJECT: State and Federal Accountability Measures and Data Collection Modernization

RELATED Community Services Block Grant Act 42 U.S.C. § 9901 et seq., hereafter
REFERENCES: referred to as “the CSBG Act.”

In collaboration with the Community Services Block Grant (CSBG) Network, the Office of Community Services (OCS) developed State and Federal Accountability Measures to track organizational performance by State CSBG Lead Agencies and OCS.  These measures are part of an enhanced framework for accountability and performance management across the CSBG Network.

OCS will require States, no later than FY 2016, to collect data and prepare to analyze and report on State CSBG Accountability Measures through their online State plan and annual reports. Concurrently, OCS will collect data and prepare to analyze and report on Federal CSBG Accountability Measures in FY 2016.

Performance Management for CSBG

Budget constraints, high poverty levels, changing demographics, and income inequality demand that the CSBG Network remains vigilant in our shared mission of creating opportunity and security for all Americans.  We must look at all levels of the CSBG Network — local, State, and Federal — to assess and increase CSBG impacts.  The CSBG Network is far-reaching and nationwide.  Together, we have the potential to achieve even greater results, in every community, by improving our accountability to one another, our customers, and our communities.

In an effort to help the CSBG Network increase accountability and achieve results, OCS launched several initiatives in 2012.  One focused on establishing organizational standards for eligible entities.  Under this effort, CSBG Network leaders developed and recommended a set of organizational standards to strengthen the capacity of the more than 1,000 eligible entities providing services across the country.  In FY 2016, States are implementing organizational standards across the Network. (See Information Memorandum 138: “State Establishment of Organizational Standards for Eligible Entities.”)

A second performance management initiative, currently underway, is focused on enhancing the CSBG Network’s performance and outcomes measurement system for local eligible entities — identified in the CSBG Act as Results Oriented Management and Accountability System (ROMA).  Finally, a third initiative focused on creating State and Federal-level accountability measures to track, measure, and improve organizational performance by State CSBG Lead Agencies and OCS.

These three efforts are complementary and integrated; together they comprise a network-wide accountability and performance management system for CSBG.  They will ensure eligible entities, States, and OCS operate within Federal law and regulation, and will build accountability and continuous management improvement into all three levels of the Network (local, State and Federal).  As shown in Appendix 1, Measuring the Success of Community Action and CSBG, these efforts will help us answer the questions: How well did the Network perform? and What difference did it make?  Ultimately, using these new and enhanced tools and information, the CSBG Network will make better program decisions and generate stronger results for low-income families and communities.

State and Federal Accountability Measures - Background

OCS developed the State and Federal accountability measures, with guidance and assistance from the Urban Institute and in consultation with the CSBG Network, in two phases.  The initial development phase, starting in 2013, included multiple listening sessions (conducted online and in-person), three CSBG Performance Management Task Force meetings with representatives from all three levels of the Network, and two expert meetings.  At the end of this process, OCS published a Dear Colleague Letter on February 28, 2014 to solicit comments on an initial list of proposed State and Federal accountability measures.  In response, the Network — including States, State associations, national organizations, and eligible entities — submitted thirty eight sets of comments to OCS.

In the second phase, OCS worked with a small working group of States and eligible entities to  rework the State accountability measures in response to the comments OCS received on the initial version of the State measures.  On January 28, 2015, OCS published the revised State and Federal accountability measures in draft Information Memorandum (IM), State and Federal Accountability Measures and Data Collection Modernization, and requested a second round of feedback.

Concurrently, OCS published a draft revised CSBG Model State Plan and sought input from the CSBG Network through a 60-day Paperwork Reduction Act (PRA) comment period that ran from January 26 to March 27.  (See Dear Colleague Letter: CSBG Model State Plan Revision: Open Comment Period, January 29, 2015.)  The draft revised CSBG Model State Plan included information on State accountability measures using the version of the measures published in the January 28 draft IM.

In response to the draft IM and the PRA 60-day comment period on the Model State Plan, OCS received dozens of extensive comments from a broad group of States, eligible entities, State associations, and national CSBG partners.  (See Dear Colleague Letter: CSBG Model State Plan Revision: OMB Clearance and 30-Day Comment Period.)  OCS carefully considered all these comments and consulted further with National and State partners before finalizing the State and Federal measures in this guidance.

State and Federal Accountability Measures

The final State and Federal accountability measures are designed to create transparency and accountability for performance at the State and Federal levels, and to help OCS and the States identify successful practices and areas for improvement.

The State accountability measures capture performance data about the critical activities and functions performed at the State level.  They indicate how efficiently and effectively a State implements the activities described in their State plan, and what impact the State’s efforts have on the performance of local eligible entities.

The State accountability measures address efficiency and effectiveness characteristics such as timeliness, accuracy, standards, and stakeholder satisfaction in the critical activities and functions listed below:

• Development of the State plan
• Implementation of the State plan, including:
  • Distribution of funds
  • Use of remainder/discretionary funds
  • Grantee monitoring and corrective action
  • Data collection, analysis, and reporting
  • Organizational standards for eligible entities
  • State linkages and communication

See Appendix 2 for the State accountability measures.

The Federal accountability measures are tied to the critical roles and responsibilities of OCS, and, where applicable, align with the State measures.  The Federal accountability measures indicate OCS’s effectiveness and efficiency as well as OCS’s impact on improving the performance of State Lead CSBG Offices.

Like the State measures, the Federal measures address such efficiency and effectiveness characteristics as timeliness, accuracy, standards, and stakeholder satisfaction in the following critical activities:

• State plan review and acceptance
• Distribution of funds
• Grant monitoring and corrective action
• Data collection, analysis, and reporting
• Organizational standards
• Training and technical assistance
• Communications

See Appendix 3 for the Federal accountability measures.

These State and Federal accountability measures are implemented within current Federal and State administrative authorities.  The CSBG Act requires States to report on performance, according to the annual reporting provision in Section 678E (42 U.S.C. § 9917), and allows OCS to request additional information through the State plan, as described in Section 676(b) (42 U.S.C. § 9908(b)).

State Accountability Measures - Data Collection

State accountability measures data will be collected using three mechanisms: 1) the CSBG Model State Plan, 2) the State CSBG Annual Report, and 3) a nationally administered survey.   Generally, States will not need to collect accountability measures data outside of the State plan and annual report.  Because OCS will manage the national survey, there will be no survey-related costs or burden for the States.  States will collect data on the majority of measures on an annual basis, and, at some point in the future, more frequently on a very small number of measures, as noted in Appendix 2.

States will collect accountability data in a seamless, integrated fashion, through regular planning and reporting processes and through the national survey.  For example, States will enter information in the State plan about planned performance in critical activity areas (e.g., development of the State plan, use of funds, grant monitoring, and training and technical assistance).  In the annual report, States will enter information on the actual performance in these same areas, and strategies for improving performance as appropriate and necessary.  Finally, the States will receive feedback on their performance in these activity areas from the national survey.

Revision of State Plan and Annual Report Forms, Including OMB/PRA Clearance

Over the past year, and with extensive input from the CSBG Network, OCS revised the Model State Plan for the FY 2016 application cycle (for applications due September 1, 2015) to streamline and automate content and to incorporate items related to State accountability measures.  Similarly, OCS, through a cooperative agreement with our national partner the National Association for State Community Services Programs (NASCSP), plans to revise the annual report forms in the coming year to include State accountability measures data, among other changes.

OCS is automating these revised forms through ACF’s On-Line Data Collection (OLDC) system.  The OLDC Model State Plan and annual report forms will include definitions and instructions and will apply data logic and validity checks to assure that data are reported accurately and consistently.  Automation provides new opportunities for integration of data sources and for using data to make program and resource decisions by comparing results over time.  While States may need additional time to complete the new automated Model State Plan and annual reports in the first year, they will save time significantly in subsequent years and benefit from the host of new automated performance management tools and resources.

As part of the revision process, OCS obtained approval for the revised and automated Model State Plan from OMB, as required under the Paperwork Reduction Act of 1995 (PRA), and will also seek PRA approval for the revised annual report form.  The PRA requires Federal agencies and OMB to ensure that information collected from the public minimizes burden and maximizes practical utility.  The OMB/PRA review and approval process includes a 60-day and a 30-day public comment period for each submission.  For more information about the OMB/PRA clearance process, please see the Frequently Asked Questions on the U.S. Department of Health and Human Service’s website.

During the PRA approval process for the Model State Plan, OCS collaborated closely with the CSBG Network.  The Network’s robust, thoughtful, and helpful participation in the PRA process directly contributed to the effectiveness and smooth implementation of the new Model State Plan.  OCS looks forward to the continued engagement of the field in the development of the annual report.

The American Customer Satisfaction Index (ACSI)

In 2012, OCS used the American Customer Satisfaction Index (ACSI) as the methodology for surveying States’ perceptions of OCS performance.  OCS plans to use this same methodology for the nationally administered survey of CSBG eligible entities that will collect information on State and Federal accountability measures.

The ACSI provides an independent, cost-effective, highly valid and reliable measure of satisfaction.  The ACSI methodology is the “gold standard,” and allows for the collection of consistent, uniform information.  It will provide OCS and the States with actionable insights to assure strong working relationships at all levels of the CSBG network and, ultimately, boost program results.

OCS engaged members of the CSBG Network in both the development of the survey and in discussions about how the results will be used to improve State and OCS performance.  OCS is preparing to conduct the first survey of CSBG eligible entities in October, 2015.  After the survey is completed, each State will receive a report detailing the results of the survey with recommendations for where to focus follow-up actions.  OCS will not distribute data comparing States.  For more information about OCS’ use of the ACSI see Appendix 4.

Data Reporting and Analysis

Once States have collected data on accountability measures through the Model State Plans, annual reports, and nationally administered survey, States will 1) analyze the data, 2) identify performance strengths and weaknesses, 3) make performance management decisions (to improve the effectiveness and efficiency of their CSBG operations), 4) report on these efforts to OCS and their eligible entities, and 5) use the data as part of their ongoing strategic planning.

States and OCS can use State accountability measures data to identify areas for improvement and determine if program changes are appropriate.  For example, a State’s accountability measures might indicate the State is not meeting the timeframes for disseminating monitoring reports or distributing funds.  The State could then take actions to improve performance in those areas.  By collecting consistent data over time, OCS and States will illuminate performance issues and encourage continuous improvement.  In the short-term, States can aim for performance improvement over the previous year.  In the longer term, consistent performance data across the entire CSBG Network can provide network-wide information about performance and best practices.

To assist with data analysis, OCS will provide each State with State-specific feedback on accountability measures after States have completed their annual reports.  OCS expects States to communicate their performance data to their eligible entities, and to use the data as part of their ongoing strategic planning. We encourage States to do additional analysis to supplement the State-specific feedback.  Under the new CSBG cooperative agreement mentioned earlier, NASCSP will help create web-based tools that in future years will make it easy for States and other CSBG Network partners to produce reports that compare the State’s performance over time with national averages and with selected groups of similar States.  OCS will engage States in discussions of data configurations and reporting that will guide decisions about State program performance.

OCS will use the State accountability data and analysis in its oversight of and guidance to States to encourage States to improve their performance.  OCS will communicate progress on State accountability measures to CSBG stakeholders through the OCS website and other means, as appropriate.

Federal Accountability Measures - Data Collection and Analysis

For the Federal accountability measures, OCS will collect data on critical Federal activities from 1) a nationally administered survey, 2) automated State plans and reports, and 3) other Federal grants systems.  OCS will begin to collect data on the Federal measures in FY 2016.

For example, in order to collect data for OCS performance in the State plan review process, OCS will use the Federal OLDC system to track timeframes for OCS review and acceptance of State plans.  In addition, OCS will gather data on OCS’s State plan review performance through the nationally administered survey.

As with the State measures, OCS will use the ACSI methodology as the nationally administered survey to collect information from States on Federal accountability measures.  OCS plans to conduct the first national survey to States about Federal performance in the Fall of 2015.

OCS is committed to a system that focuses on improved Federal accountability as a part of the national performance management framework.  Like the States, OCS will analyze and use the data on the Federal accountability measures to improve our performance.  We will communicate progress on the measures to the States and other stakeholders through the OCS website and other methods.

Conclusion

We look forward to the Network’s continued partnership in implementing the Federal and State accountability measures.  OCS appreciates that the shared work of implementing accountability measures, and the new framework for accountability and performance management overall, can be challenging.  To assist States in this change process, OCS and our national providers will provide training and technical assistance through webinars, presentations at conferences, and other communications to provide support on an on-going basis.

In the meantime, if you have questions, please contact an OCS CSBG program specialist.  The list of OCS staff and contact information is posted on the OCS website at www.acf.hhs.gov/programs/ocs/resource/csbg-staff-assignments-by-region.

Together we must insist upon accountability and performance management across the CSBG Network.  The CSBG State and Federal Accountability Measures have the potential to protect and enhance the structural integrity of this national network by assuring that all States that receive CSBG funds, as well as the Federal office responsible for CSBG, are performing as efficiently and effectively as possible to support CSBG’s response to the complex social problems that contribute to poverty.  We look forward to working with the CSBG Network to successfully implement the State and Federal accountability measures and the performance management framework overall.

           /s/ 
Jeannie L. Chaffin
Director
Office of Community Services

Appendices:
Appendix 1: Measuring the Success of Community Action and CSBG
Appendix 2: State Accountability Measures
Appendix 3: Federal Accountability Measures
Appendix 4: American Customer Satisfaction Index and the CSBG State and Federal Accountability Measures