CSBG IM #150 Use of the American Customer Satisfaction Index (ACSI) to Improve Network Effectiveness

U.S. Department of Health and Human Services
Administration for Children and Families
Office of Community Services
Division of State Assistance
330 C Street, SW, 5th Floor
Mail Room 5425
Washington, D.C. 20201

COMMUNITY SERVICES BLOCK GRANT PROGRAM
Information Memorandum

Transmittal No. 150

Date: August 22, 2016

TO: State and U.S. Territory Community Services Block Grant (CSBG) Administrators, Eligible Entities, and State Community Action Associations

SUBJECT: Use of the American Customer Satisfaction Index (ACSI) to Improve Network Effectiveness

RELATED
REFERENCES: Community Services Block Grant Act, Title VI, Subtitle B, of the Omnibus Budget Reconciliation Act of 1981, Public Law 97-35, as amended; Human Services Amendments of 1994, P.L.103-252; the FY 1996 CSBG Appropriation Legislation, P.L.104-134; CFR Title 45, Part 96; Coats Human Services Reauthorization Act of 1998, P. L. 105-285; Department of Health and Human Services Block Grant Regulations and Current Poverty Income Guidelines.

In fall 2015, the Office of Community Services (OCS) used the American Customer Satisfaction Index (ACSI) to obtain feedback from CSBG eligible entities about services provided by the state CSBG Lead Agencies, as detailed in the new State Accountability Measures. OCS also obtained feedback from state CSBG Lead Agencies on services provided by the federal agency, as outlined in the new Federal Accountability Measures. Both OCS and state CSBG Directors received their state survey results in February 2016.

OCS Expectations for States

States are expected to share the survey results with their local agencies and use the survey information, in collaboration with those agencies, to identify improvement strategies for meeting specific State Accountability Measures. Each state will report on its planned improvement strategies, along with the ACSI overall satisfaction score and what target score is being set for next year’s ACSI satisfaction score, in the FY 2017 State Plan, due September 1, 2016. In the CSBG Model State Plan (Question 3.5), labeled “Eligible Entity Overall Satisfaction,” applicants are asked to provide the state’s target for eligible entity Overall Satisfaction during the performance period.

As part of their FY 2016 Annual Report submission, due March 31, 2017, states will also report on how they considered feedback, including ACSI survey results, from their eligible entities in addressing the State Accountability Measures; what activities they undertook in response to that feedback; their most current ACSI overall satisfaction score; and, what target they are setting for next year’s ACSI score. Because the state’s Model Plan may cover two fiscal years, annual updates related to the eligible entities’ satisfaction will be provided in the Annual Report. Appendix 1 crosswalks State Accountability Measures with the State Plan and Annual Report requirements. As illustrated in this Appendix, information about the satisfaction score and future target are requested in both the State Plan and the Annual Report.

Going forward, OCS will continue to use the ACSI as part of its enhanced Performance Management Framework. Use of the ACSI will allow OCS to collect consistent, uniform information from eligible entities across the country, and will provide the states with actionable insights to improve their customer experience and boost program results. This is in keeping with our enhanced emphasis on using data for analysis and decision-making to continually make program improvements.

OCS expects to conduct the ACSI survey of eligible entities on at least a biennial basis. Under current plans, the next survey is expected to be administered during the second quarter of FY 2017 (January-March 2017). The survey results will be available to the states for use in developing the FY 2018 State Plans. Appendix 2 illustrates the Survey-Planning-Reporting Cycle.

OCS will also continue to use the ACSI to obtain feedback from the states about services provided by OCS, as detailed in the Federal Accountability Measures. OCS will share the survey results with the states and use the survey information to identify improvement strategies for meeting the Federal Accountability Measures.

Background on the ACSI

The American Customer Satisfaction Index (ACSI) is the national indicator of customer evaluations of the quality of goods and services available to U.S. residents. It is the only uniform, cross-industry/government measure of customer satisfaction. ACSI has measured more than 100 programs of federal government agencies since 1999. This allows benchmarking between the public and private sectors and provides information unique to each agency on how its activities affect the satisfaction of customers. The effects of satisfaction are estimated, in turn, on specific outcomes (such as trust). Additional information on the ACSI can be obtained on the following web address: http://www.theacsi.org/

How ACSI Survey Results Will Be Used

The ACSI “cause and effect” model is helpful in identifying specific improvements that will have the greatest impact on customer service. This model will enable OCS and the states to use the results from the surveys to learn about performance in several areas addressed in the Accountability Measures, and to better focus training and technical assistance efforts. The survey results will not be used to make policy or funding decisions. The focus is to improve the efficiency and effectiveness of OCS and the states.

ACSI survey data is integral to the implementation of the State Accountability Measures as specified in the Model State Plan. Reporting on ACSI results is also part of the Annual Report, specifically the State Administration Module, Section B: Statewide Goals and Accomplishments. As such, OCS anticipates providing the states with timely survey results as well as feedback on the states’ use of the survey data.

Setting Targets for the ACSI

In thinking about the ACSI Overall Satisfaction target, the key is to ask “what is attainable?” Here is some guidance from our colleagues at CFI Group, the organization that administers the ACSI, to keep in mind:

• There are no “perfect” scores of 100--different customers have different expectations, and some are easier to satisfy than others;
• For ACSI scores currently in the 80s, a 1-point gain might be realistic;
• For ACSI scores in the 70s, a 2-3 point gain is reasonable;
• For ACSI scores in the 60s, 3-5 points may be attainable; and
• For ACSI scores in the 50s, 5-10 points should be possible.

Also keep in mind that not all program improvements have the same impact on overall satisfaction. Program improvements made in the “high impact” areas recommended in the individual state ACSI Reports will most likely result in higher Overall Satisfaction scores than improvements in areas of less impact. For example, if Linkages and Communication had an impact of 2.5, you would expect overall satisfaction to increase by 2.5 points when the Linkages and Communication score increases 5 points. Hence, focusing on the high impact areas where scores are comparatively lower should result in a higher Overall Satisfaction score.

Other things to consider when setting ACSI targets:

• Ask what percentage of the customer base will be affected by any improvements undertaken. If you improve the use of discretionary funds, for example, but only 25% of customers are affected in a given year, it might have a limited effect on ACSI (even though it might also be the right thing to do). But, also remember that any positive changes can also affect the customer experience, which can have successful “spillover” effects.

• Use benchmarks to set realistic targets. Compare yourself to other states similar to yours, based on geography or type of population served. To assist states in these benchmarking efforts, OCS intends to provide the State CSBG Directors with a “Top-5 List” detailing the states that received the highest scores in each of the areas surveyed.

• It is important to think about “long-term” goals when working with the ACSI. ACSI scores typically do not jump overnight. It is best to track trends over time — up two points one year, down two points the next is not success. Focused, continuous improvement is the key to success. Sharing best practices across states—knowing what works and what does not—in terms of the State Accountability Measures can be key to long-term success. The “Top-5 List” may also be useful in identifying practices that are considered effective by the eligible entities.

• Knowing that ACSI scores might not move immediately and provide only a snapshot, look to other internal metrics as well.

• Finally, working collaboratively with the local agencies to make improvements and communicating what improvements are made will likely also have a positive impact on the ACSI score.

OCS will provide training and technical assistance to the states, as necessary, to guide the development of ACSI targets.

The ACSI and State Accountability Measures

As detailed below, seven of the State Accountability Measures specify use of ACSI survey data and other feedback, as appropriate, to adjust the State Plan to improve performance. As noted earlier, Appendix 1 cross-walks all the Accountability Measures with the State Plan and Annual Report requirements. Outlined below are the specific State Accountability Measures related to ACSI measurement.

Development of the State Plan

1Sb. Using data from a nationally administered survey[1] of eligible entities, and feedback from OCS and other sources, the State adjusted its plan (in the next State Plan submission), as appropriate, to improve performance regarding:

i. The extent of eligible entity participation in developing the State Plan; and

ii. How well the State Plan reflects the input of the eligible entities.[2]

Implementation of the State Plan

Distribution of Funds

2Sb. Using data from a nationally administered survey[3] of eligible entities and feedback from OCS and other sources, the State adjusted its plan (in the next State Plan submission), as appropriate, to improve the quality of grant administration.

Use of Remainder/Discretionary Funds

3Sb. Using data from a nationally administered survey[4] of eligible entities, and feedback from OCS and other sources, the State adjusted its plan (in the next State Plan submission), as appropriate, to improve its use of remainder/discretionary funds.

3Sd. Using data from a nationally administered survey[5] of eligible entities, and feedback from OCS and other sources, the State adjusted its plan (in the next State Plan submission), as appropriate, to improve the Training and Technical Assistance provided to the eligible entities.

Grantee Monitoring and Corrective Action

4Sb. Using data from a nationally administered survey[6] of eligible entities, and feedback from OCS and other sources, the State adjusted its plan (in the next State Plan submission), as appropriate, to improve its monitoring activities.

State Linkages and Communication

7Sb. Using data from a nationally administered survey[7] of eligible entities, and feedback from OCS and other sources, the State adjusted its plan (in the next State Plan submission), as appropriate, to improve its communication efforts.

Eligible Entity Satisfaction

8S. The State achieved/maintained an Overall Satisfaction Score of “x”.[8]

Conclusion

OCS is committed to meeting the Federal Accountability Measures and improving the services and tools it provides the states. State feedback—and ambitious but attainable targets for increased performance—are critical elements to our ability to learn and improve. We look forward to working with states and CSBG eligible entities to assure accountability and responsiveness throughout the CSBG Network.

             /s/
Jeannie L. Chaffin
Director
Office of Community Services

Appendix 1 — Accountability Measure Crosswalk

Appendix 2 — Survey Planning and Reporting Cycle