CSBG DCL Federal Accountability Measures Update
U.S. Department of Health and Human Services
Administration for Children and Families
Office of Community Services
Division of Community Assistance
330 C Street, SW, 5th Floor
Mail Room 5425
Washington, D.C. 20201
Community Services Block Grant
Dear Colleague Letter
Re: Federal Accountability Measures Update
Date: May 8, 2018
The Office of Community Services (OCS) has spent the last year analyzing the progress on the federal accountability measures data as a part of the Community Services Block Grant (CSBG) Performance Management Framework. To guide the successful execution and monitoring of the CSBG federal accountability measures, OCS administered activities that focused on four steps: 1) define data requirements, 2) collect data and calculate measures, 3) assess CSBG performance, and 4) practice continuous improvement. Based on these efforts, OCS is providing a status update on the fiscal year (FY) 2017 CSBG federal accountability measures.
State and Federal Accountability Measures — Background
Over the last several years, OCS and the CSBG Network — comprised of CSBG eligible entities, State CSBG Lead Agencies, State CAA Associations, national partners, and others — have worked together to create and implement a new CSBG Performance Management Framework. This framework includes 1) organizational standards for eligible entities, 2) accountability measures for States and OCS, and 3) CSBG outcome measures (National Performance Indicators). These elements are designed to increase accountability across all three levels of the network (federal, state and local) and enable us to make better program decisions that are based on data. The CSBG Performance Management Framework is based on the National Community Action Theory of Change.
Starting in 2013, with guidance and assistance from a CSBG Performance Management Task Force, OCS developed the state and federal accountability measures. Tied to the critical roles and responsibilities of OCS, and, where applicable, aligned with the state measures, the federal accountability measures indicate OCS’s effectiveness and efficiency as well as OCS’s impact on improving the performance of State CSBG Lead Agencies. The federal accountability measures addresses efficiency and effectiveness characteristics such as timeliness, accuracy, standards, and stakeholder satisfaction in the following critical activities:
- CSBG State Plan review and acceptance
- Distribution of funds
- Grant monitoring and corrective action
- Data collection, analysis, and reporting
- Organizational standards
- Training and technical assistance
The federal Accountability Measures are described in CSBG Information Memorandum (IM) 144 State and Federal Accountability Measures and Data Collection Modernization. See Appendix 3 of IM #144 for the federal accountability measures.
Data Collection Methods
In FY 2016, OCS began collecting federal accountability measures data using three mechanisms: 1) a nationally administered survey, 2) automated state plans and reports, and 3) other federal grant and information systems.
OCS uses the nationally administered survey, American Customer Satisfaction Index (ACSI) as the methodology for surveying states’ perceptions of OCS performance. The ACSI methodology is the “gold standard,” and allows for the collection of consistent, uniform information. It has provided OCS with actionable insights to assure strong working relationships at all levels of the CSBG network and, ultimately, boost program results. In 2015, OCS launched the ACSI survey seeking feedback from each State CSBG Lead Agency and eligible entities. The results of the ACSI survey were used to guide OCS’ training and technical assistance efforts and provide baseline data to the federal accountability measures. For more information about OCS’ use of the ACSI see Appendix 4 of IM #144.
OCS uses the automated Online Data Collection (OLDC) system for the submission and review of the CSBG State Plan and CSBG Annual Report. In addition to producing timelier, more accurate and consistent performance data, OCS utilizes these automated systems to collect data pertaining to federal accountability measures. For example, in order to collect data for OCS performance in the State Plan review process, OCS uses the federal OLDC system to track timeframes for OCS review and acceptance of state plans.
Additionally, OCS employs various federal grant and information systems to collect and monitor the implementation of the federal accountability measures. These systems are utilized continually and have been ingrained in the division’s operation procedures.
Continuous Improvement Approach
In an effort to monitor the implementation of the federal accountability measures, OCS engaged in the following activities:
- Establishing Baselines — Using available data sources, OCS defined its data collection methodology to calculate the baseline for each Federal accountability measure. For those measures where no data was available (e.g., Measure 1Fa-3), OCS identified improvements to the applicable process to enable future data collection efforts. The baseline measures serve as starting points in which OCS sets targets.
- Setting Targets — Considering the baseline, program resources, and CSBG Network expectations, OCS set targets for each federal accountability measure with an established baseline. The targets serve as the guideposts for monitoring whether progress is being made and promote transparency and accountability by making available information on whether results have been achieved or not over time.
- Measuring Progress — OCS plans to measure its progress on an annual basis. OCS developed two types of documents to support its monitoring and to share the results of its performance to the CSBG Network:
- Dashboard: CSBG Federal Accountability Measures (Appendix A), a high-level document showing whether OCS met the target for each measure.
- Report: CSBG Federal Accountability Measures, a more detailed document showing data in the dashboard, as well as each measure’s data source and baseline.
The FY 2018 Dashboard is included with this Dear Colleague Letter for reference.
During the course of the last two years, OCS has created several internal processes to closely monitor the successful execution of the federal accountability measures. The automation of the CSBG State Plan and Annual Report, in addition to the use of other federal grant systems has afforded OCS the opportunity to track the successful review and responsiveness to required annually submitted reports. As such, during FY 2017, OCS exceeded the FY 2016 target metric for the following federal accountability measures:
- 1Fa-1: OCS reviewed and provided a response for “x” percent of State plans within 45 calendar days of receipt of the submitted State plan.
- 1Fa-2: OCS accepted “x” percent of State plans within 60 calendar days of receipt of the submitted State plan.
- 2Fa: “x” percent of States with accepted State plans received funding within 21 calendar days of OMB/Department of Health and Human Services (HHS) appropriation of funds.
- 4Fa: OCS reviewed and provided feedback on “x” percent of State annual reports (and any required quarterly or semi-annual reports) within 60 calendar days of receipt of the report.
ASCI 2017 Highlights
Since its initial implementation, OCS has been committed to administering the ACSI survey biennially and has administered the survey in FY 2015 and FY 2017. By obtaining the baseline data from 2015, OCS is able to utilize the FY 2017 ACSI results to track overall grantee satisfaction throughout the years and to identify areas of training and technical assistance needs.
- OCS’ Customer Satisfaction Index (CSI) score improved from 58 in 2015 to 62 in 2017. This improvement was based on higher scores for all three customer satisfaction metrics: Overall satisfaction (+5) Satisfaction compared to expectations (+3) and Satisfaction compared to the ideal grant awarding agency (+4).
- Scores for Grant Monitoring improved from 47 in 2015 to 62 in 2017. Clarity of feedback (+18), Relevance of procedures (+16), and Usefulness of visit (+16) were the specific areas showing the greatest improvement.
- Performance for Training and Technical Assistance provided by OCS Staff improved five points with a score of 60. Ability to direct grantees to useful resources and Ability to answer questions improved eight and nine points, respectively.
- Grantees report a greater level of Trust in OCS; up six points to 66.
Areas for Improvement
In response to the 2017 OCS Federal Accountability Measures Report, OCS has identified top priority areas of improvement to ensure organizational efficiency and effectiveness. While the overall trend in the ACSI results showed improvement, OCS has specifically noted many areas for improvement, including concerns related to timeliness and consistency of monitoring results and technical assistance. Indicated in the findings from the ACSI survey and other federal grant systems, OCS is committed to improving efforts in the areas of grant monitoring, training and technical assistance provided by staff, communication, and state plan review and acceptance. OCS staff are currently reviewing results and will be developing action plans in each of these areas with the goal of improved customer satisfaction.
We look forward to the Network’s continued partnership as we implement and monitor the federal accountability measures. This effort will continue to protect and enhance the structural integrity of CSBG efforts and ensures that OCS responds to the complex social problems that contribute to poverty in an efficient and effective manner. Rooted in the work we do on a daily basis, OCS will continue to utilize the federal accountability measures to implement process improvement solutions with a focus on customer service.
J. Janelle George Seth Hassett
Acting Director Director, Division of Community Assistance
Office of Community Services Office of Community Services