Phase II Findings: Error Rate Methodology Pilot
Executive Summary, I. Introduction, II. Methodology, III. Measuring Improper Payments in Nine States, IV. Findings and Next Steps, V. Recommendations, Appendices: Appendix A: Arkansas, Appendix B: Colorado, Appendix C: Illinois, Appendix D: Ohio, Appendix E: Record Review Worksheet, Appendix F: Data Entry Form, Appendix G: Telephone Discussion Guide, Appendix H: Arizona, Appendix I: California, Appendix J: Kansas, Appendix K: Nebraska, Appendix L: New Hampshire
Executive Summary
Table of Contents Executive Summary |
The Child Care and Development Fund (CCDF) is a block grant that provides funds to States, territories, and tribes to support child care subsidies for low-income working families. During Fiscal Year (FY) 2004 the CCDF, managed by the U.S. Department of Health and Human Services (HHS), Administration for Children and Families (ACF), Child Care Bureau (CCB) totaled $8.5 billion. This includes $4.8 billion in subsidized child care funds and an additional $3.7 billion in Temporary Assistance for Needy Families (TANF) and Social Services Block Grant (SSBG) transfer funds. During FY 2003, the Child Care Program served an average of 1,751,300 children each month.
The CCDF block grant allows maximum flexibility for States to set critical policies such as establish eligibility criteria, define administrative structures that allow maximum choice for parents, and establish fiscal management approaches. Because of the discretion given to States, eligibility criteria, rates, regulation of child care providers, and payment mechanisms vary widely among jurisdictions. This flexibility makes it difficult to develop common approaches for identifying and measuring improper payments.
During FY 2002, the CCB developed an Erroneous Payment Assessment plan. The purpose of the plan was to identify and reduce improper payments in relevant HHS programs in response to the Improper Payment Information Act of 2002 and related Office of Management and Budget (OMB) guidance. In July 2003, the CCB began preparation to conduct a pilot project to determine whether there is a cost-effective approach or methodology for estimating improper payment amounts or rates in the Child Care Program.
The CCB contracted with Walter R. McDonald & Associates, Inc. (WRMA) to develop and pilot test a common methodology that States could use to compute an error rate and work with States to validate existing protocols or develop new approaches to address improper payments and fraud. Specific requirements, including statistically valid sampling, guided the conceptual design of this study. The CCB, in consultation with a number of “partner States” chose to focus the analysis on eligibility error to measure an element that is common to every State and to mitigate some of the variation among State definitions encountered during the Phase I pilot study. The methodology—conducted in Arkansas, Colorado, Illinois, and Ohio—had four main components.
- The study team assisted each State to select a random sample of up to 150 cases (children), using a sampling frame of all children in the State who received child care services during October 2004. The study team designed the sample size to produce a statistically valid estimate of erroneous payments.
- States developed a Record Review Worksheet to reflect child care policies in the State. The State used this instrument to guide a record review of the sampled cases to identify administrative errors in eligibility determination. States collected data regarding the number of cases with errors and whether the errors led to an improper payment. Although all of the worksheets contained common elements, the definitions pertaining to those elements varied from State to State.
- The study team conducted site visits and provided technical assistance to the State representatives who conducted record reviews and collected data. The study team computed the error rates using the data submitted by the States.
- The study team also conducted telephone interviews, using a consistent protocol to gather additional information about improper payment activities in five States. These States were Arizona, California, Kansas, Nebraska, and New Hampshire.
Definitions of the error rates calculated for this study and the relevant findings are as follows:
- Percentage of Cases with a Potential Error—This percentage is based on the number of sampled cases with any administrative error divided by the total number of valid cases in the sample. The estimated percentage of sampled cases determined to contain administrative errors ranged from 12 percent to 38 percent.
- Percentage of Cases with a Potential Improper Payment—This percentage is based on the number of sampled cases with an administrative error that resulted in an improper payment divided by the total number of valid cases in the sample. The percentage of sampled cases with administrative errors that resulted in an improper payment ranged from 6 percent to 27 percent.
- Percentage of Payments Potentially Made in Error—This percentage is based on the total amount of child care payments that were made in error divided by the total amount of child care payments for the sample. The estimated percentage of payments made in error in the four pilot States ranged from 4 percent to 14 percent.
An initial error rate measure provides both the States and the CCB with a baseline against which to compare future findings. The pilot findings confirm the value to States of regularly conducted compliance reviews to improve administrative practice. The pilot methodology was also useful for States to gain evaluative information on potential areas for improvement in administrative policy. Administrative protocols dealing with improper payments vary greatly from State to State. This is particularly true with threshold amounts that are used in pursuing recoupment. Within some States thresholds vary greatly from jurisdiction to jurisdiction—for example, one State will pursue a debt of any amount, while another requires a minimum of $5,000. Improvements to administrative practice, such as independently confirming employment and training or reconfirming participation and attendance, can reduce administrative error.
The four study States that participated in this pilot plan to reexamine their monitoring processes, provide training, or clarify policy and procedures in those counties where they identified problems or issues. The CCB will gather and compile information from those States as they continue to take action based on the pilot test findings. Colorado extended its record review process to explore additional areas of error and examined client and provider errors for a subset of the sample. This small subset indicated that the study methodology could include an external verification of both provider and client errors, both of which are important areas of risk for mitigating improper payments.
This pilot study demonstrated and reinforced that improvements to administrative practice to reduce administrative error can result in preventing or deterring improper payments caused by clients as well as providers. Independent confirmation of employment or training and intermittent reconfirmation of participation, as well as comparing attendance or sign-in and sign-out sheets with claim forms, can help ensure that recipients participated in the activities for which services were provided. As a result of this study, each State has planned action steps or has implemented several new systematic changes to improve monitoring or reduce improper payments.
<< Table of Contents | Introduction >>