State Assessment of Internal Controls Final Report, May 2007

Download Guide in Word (1,622 KB) or PDF (676 3KB) format.

EXECUTIVE SUMMARY

This report describes the development and pilot of the State Internal Control Self-Assessment Instrument (Instrument). This report highlights the background and objectives, the potential benefits to States and Federal Staff, and the methodology used to develop and implement the pilot in Kansas, the initial pilot State. The major part of this report focuses on nine additional pilot States and includes lessons learned and associated costs from implementing the Instrument.

A. Background

In response to the Improper Payment Information Act of 2002, the U.S. Department of Health and Human Services (HHS), Office of the Inspector General, Administration for Children and Families (ACF), as well as the Office of Management and Budget (OMB) developed an Erroneous Payment Assessment Plan. The plan identified the Child Care Program as being at high risk of errors that could result in improper payments caused by mistakes, inadequate controls, fraud, waste, or abuse.

Since 2003, the Child Care Bureau (CCB) has taken a systematic approach to addressing the issue of improper payments in the Child Care Development Fund (CCDF). The CCB has examined the policies and practices in 11 partner States, conducted an error rate pilot in nine pilot States, and analyzed associated costs. The current pilot examines the feasibility of implementing a State Internal Control Self-Assessment Instrument (Instrument) to assist States in identifying potential gaps and problems within the Child Care Program.

In 2004, the Government Accountability Office (GAO) issued a report describing the strategies implemented by 16 States to address improper payments in the CCDF and Temporary Assistance for Needy Families (TANF) block grant programs. The GAO studied what States were doing to manage improper payments and how the U.S. Department of Health and Human Services (HHS), which oversees the TANF and CCDF programs, helps States identify and address improper payments in these programs. The GAO concluded, “HHS lacks adequate information to assess risk and assist States in managing improper payments.” ¹

The GAO report, The Improper Payments Information Act of 2002, the President’s Management Agenda goal of “Improved Financial Performance,” related OMB guidance and recommendations from the President’s Council on Integrity and Efficiency² all stress that Federal and State agencies should prioritize the creation of an effective control environment within their programs using resources and incentives to support this effort. According to GAO, HHS needs mechanisms to gather information on State internal control activities, including steps that States take to minimize or eliminate risks, in order to detect and minimize errors that could result in improper payments.

B. Objectives for Developing a Process to Examine State Internal Controls

The Government Accountability Office (GAO) identifies assessing internal controls as an important strategy to assist States in efforts to minimize erroneous payments. Assessment of risk through an internal control self-assessment process is an activity that entails a comprehensive review and analysis of program operations to determine where risks exist, identify those risks, and then measure the potential or actual impact of those risks on program operations. The information gathered from this assessment can determine the nature and type of corrective actions needed, and provides baseline information for measuring progress in reducing errors that could result in improper payments.

In response to the GAO directive, the CCB organized a Federal Project Team that included staff from Regional offices, the State of Kansas, the CCB central office and Walter R. McDonald & Associates, Inc. (WRMA) to draft an approach to address internal controls, using GAO’s Internal Control Management and Evaluation Tool as a model. ³

C. Potential Uses of the Self-Assessment

The CCB views the State Internal Control Self-Assessment Instrument as a promising tool for both State and Federal managers. The Instrument can provide the CCB, Regional Offices, and States with a systematic method for reviewing and documenting the adequacy of a State’s internal control system, identifying internal control weaknesses, and providing documentation of findings and possible corrective actions.

ACF Regional Office program specialists and grants officers along with representatives from nine States/Territories and WRMA were an integral part of the development and piloting of the Instrument. The State and Federal Project Teams who participated in the pilot identified the following benefits of the Instrument implementation process:

1. Benefits to States

• Enables the States to evaluate their internal controls against GAO standards;
• Can be adapted for use in other program areas;
• Combines into a single document a wide variety of information that can be used to monitor internal controls and prevent errors;
• Documents areas needing ongoing monitoring or other corrective action activities;
• Can become a “virtual document” that continually updates through hyperlinks to the State’s Internet or Intranet site each time the State changes a supporting materials;
• Addresses the core processes affecting the child care program, when used in conjunction with the State Plan;
• Documents when necessary internal controls exist and function adequately to minimize the risk of errors that could result in improper payments; and
• Can be adapted to meet the unique requirements of a particular State and adapted to other State programs.

2. Benefits to ACF Staff in Regional Offices

• Assists States in performing oversight and supporting the CCDF program;
• Identifies areas for technical assistance with States, individually or in regional meetings;
• Highlights areas that States identify as weaknesses, which when combined with the State Plans, audits and other reports, can develop and support corrective actions; and
• Provides a vehicle for sharing of best practices both within and across Regions.

D. Conceptual Design of the Instrument

The purpose of the Instrument is to provide a supplemental guide for States to assess the adequacy of internal controls and to identify issues that may contribute to errors in the administration of the Child Care and Development Fund (CCDF). This Instrument provides a starting point for States to modify to fit the circumstances, conditions, and risks relevant to each agency. States can review their internal controls and procedures using the relevant factors and elements in the Instrument and document responses as appropriate to their particular agency circumstances.

Using the GAO tool as the base, the State Internal Control Self-Assessment Instrument has five sections corresponding to the five standards for internal controls:

• Control environment
• Risk assessment
• Control activities
  • Common categories of control activities
  • Control activities specific to information systems general
  • Control activities specific to information systems application control
• Information and communications
• Monitoring

Each section of the Instrument contains a list of major elements or criteria for States to consider when reviewing internal controls as they relate to each particular standard. By reviewing each element, States can first determine the applicability of the item to local circumstances. If applicable, the Instrument provides standards against which States can assess performance, identify any resultant gaps or weaknesses, and determine the extent to which the element influences the agency’s ability to achieve its mission and goals. The results of this assessment can guide States in developing processes, procedures, or other measures to improve internal controls in order to meet the intent of the IPIA and ACF's need to document improvements. State assessment results can help provide a framework for ongoing technical assistance, when shared with ACF staff in Regional Offices.

E. Instrument Development

The Federal Project Team, consisting of Federal CCB staff from Central Office, Region VII and WRMA, worked with Kansas as a partner State in the review, modification, and initial pilot test of the Instrument. Alice Womack, State Child Care Administrator, agreed to lead the State Self Assessment of Internal Controls Project Team, which included staff representing Program, Fiscal, Policy, Human Resources, Information Technology, and Auditing Divisions, at both the central office and field office levels.

Federal Project Team members visited Kansas in November 2005 and conducted a two-day working session to orient the Kansas Project Team to the project and work together to tailor the Instrument for use in reviewing the child care program. The Kansas and Federal Project Teams reviewed every section, element, and criterion in the initial draft Instrument and, through consensus, modified and tailored the Instrument. The Project Teams deleted several of the elements and their associated criteria from the original document. The Project Teams came to consensus on all modifications to the Instrument before making any changes. The Project Team members chose to delay further refinement and modification of the Instrument until after State pilot implementation. 

F. Initial Pilot State: Kansas

Kansas participated in the development of the Instrument and agreed to be the first State to pilot the Instrument from November 21, 2005 to January 13, 2006. Following completion, the Federal Project Team conducted a debriefing call to discuss the Kansas experience. After considerable discussion, the Kansas Project Team made several recommendations for successful implementation of the Instrument, including:

1. Composition and Responsibilities of the State Project Team

• Select one overall project coordinator.
• Have a high-level management staff member recruit team members and act as organizational sponsor.
• Form a broad-based team. See Appendix B for a listing of potential team member’s functional assignments.
• Designate adequate support/clerical staff to be involved from the beginning of the project, especially for tracking and compiling data and information
• See “Suggested Section Assignments,” Appendix C for more guidance on forming the State Project Team.

2. Orientation Meetings and Ongoing Communication

• Hold an initial team meeting to explain the purpose, process, and benefits of participation and uses for future work within the agency.
• Hold additional orientation meetings, if staff members who will complete sections of the Instrument are not at the orientation meeting.

3. Gathering and Recording Information

• Discuss the procedures for gathering documentation, including: the use of hyperlinks to both the State Internet and Intranet Web sites; and how hard copy attachment options are to be coordinated with all team members.
• Provide all team members with an electronic version of the document to use to enter all information and send to the coordinator who can then assign support staff to compile into one document. More than one team member may complete some sections, which will require integration by the coordinator.

4. Finalizing and Submitting the Completed Assessment

• Have support staff edit the completed Instrument for errors, grammar, and writing style consistency.
• When completed, hold a wrap-up meeting with all team members to share completed documents and receive comments and feedback; make changes to document as agreed upon in discussion.
• Have high-level management staff/team review the completed Instrument to ensure that it is consistent with agency mission and can be integrated with agency strategic or business plan.

G. Training of Federal Staff in ACF Regional Offices

Upon completion of the Kansas pilot, the CCB convened a one-and-a-half day training session in Kansas City, MO for Regional grants officers and program specialists from CCB Central Office and all ten ACF Regional Offices. Members of the Kansas Project Team also attended the training.

The purpose of the training was to involve the ACF Regional Offices in provision of technical assistance to the States during the Instrument pilot and to demonstrate how they could use the results for monitoring purposes. At the conclusion of the session, the CCB invited an additional nine Regions and States or Commonwealths to participate in the pilot, including Maine (Region I), Puerto Rico (Region II), Kentucky (Region IV), Illinois (Region V), Arkansas (Region VI), Kansas (Region VII), Montana (Region VIII), Nevada (Region IX), and Washington (Region X).

The Federal Project Team conducted site visits in eight of the nine jurisdictions; Maine chose not to have a site visit due to timing issues. Instead, the Federal Project Team conducted a conference call with the Maine Project Team to answer questions and prepare team members for completing the Instrument.

H. Conclusions

Based on the self-assessment process, the nine pilot States offered the following conclusions for future pilots: 

A successful Instrument implementation process needs involvement across program areas. States realized that the scope of the self-assessment process was broad, requiring the involvement of common supporting areas that cross walk several different programs, such as Human Resources and Information Technology. As a result, the commitment of all parts of the organization is critical to the success of the self-assessment process.

High-level management support is important for optimal implementation. For those States without the necessary organizational support, the self-assessment process was a greater challenge. The States that found the self-assessment process most rewarding were those States that had a high level of commitment from the top State organizational level.

Redundancies in the Instrument did not serve a useful purpose. All nine pilot States expressed a need to streamline the Instrument and eliminate the redundancies. States also requested creating a section specific to the child care program. WRMA revised the Instrument as explained in the section below.

States need to allow adequate time to complete the self-assessment process. States recommended establishing an agreed upon timeframe to ensure timely and thoughtful completion of the Instrument. Most States indicated that a 90-day timeframe for Instrument completion was adequate. Several States recommended scheduling the self-assessment process to occur within a timeframe that does not conflict with other major agency activities. Most States recommended updating the Instrument every two years.

States may use hyperlinks to refer to documents on the State’s Internet and/or Intranet Web site to ensure that the Instrument stays current by linking dynamically to continuously updated documentation. Most States indicated that combining all of the documentation in a single document was helpful. Using the hyperlink rather than printing the complete set of documentation was beneficial because the link will route to the most current version of a document. This is important as many policy documents are on a regular schedule for updates. The Illinois Project Team recommended establishing an area on the CCB Web site for States to provide links for sharing internal control documentation, which is not available on either the Internet or the State agency Intranet.

Agency allocation of an overall project coordinator is critical to the self-assessment process. The project coordinator needs to have top-down management support, sufficient authority to set deadlines, availability during the entire process to provide clear instructions, answer questions, consult with team members and coordinate with top agency leadership.

I. Estimated Costs

Despite the scope and complexity of the Instrument, the costs of conducting a State Internal Control Self-Assessment are relatively modest. Kentucky and Maine estimated costs of approximately $3,000, while Montana and Washington estimated between $8,000 and $9,500. The range for all States was $3,032 through $9,445. The costs for the Commonwealth of Puerto Rico were significantly higher, due to several factors. Puerto Rico involved the greatest number of personnel in the site visit and in the Instrument implementation process. Puerto Rico also expanded the scope of the self-assessment process to examine areas needing attention as identified in a prior audit.

J. Revisions to the Instrument

The nine pilot States recommended elimination of the redundancies and the tailoring of certain elements to the child care program. Based on the advice of the participating States and ACF staff in Regional Offices, WRMA modified the Instrument. The objective of the modifications was to reduce the redundancy, clarify the instructions, and tailor Section III to include elements specific to the child care program. In order to involve all pilot States in the process of Instrument modification, WRMA shared the revised Instrument with the States and the ACF staff in the Regional Office who drafted the initial Instrument. Following receipt of all of the comments, WRMA made further revisions to the Instrument. Appendix M contains the revised Instrument.

The modifications to the Instrument included:

• Eliminating or combining 23 elements in Section I and II (Control Environment and Risk Assessment) for a reduction of 23%;
• Reviewing and clarifying all instructions for each section of the Instrument;
• Revising Section III: Common categories of control activities to specifically focus on the child care program; and
• Making minor changes to Sections IV and V.

The modifications did not affect the integrity of the Instrument.

K. Recommendations

The Instrument can provide the CCB staff in the Central and in Regional Offices and States with a systematic method for reviewing and documenting the adequacy of a State’s internal control system, identifying internal control weaknesses, and providing documentation of findings and possible corrective actions. Recommendations include:

• Obtain the commitment of all parts of the organization prior to initiation of the process. A successful Instrument implementation process needs involvement across program areas. States realized that the scope of the self-assessment process was broad, requiring the involvement of common supporting areas that cross walk several different programs.
• Market the Instrument to programs in addition to the child care program. Some States adapted the Instrument to meet other needs beyond the child care program. Puerto Rico used the self-assessment process to re-examine fiscal areas highlighted in a previous audit, and included an action plan with the Self-Assessment findings. Maine intends to strengthen its Performance Evaluation System (PER) by including a section focusing on ethics and personal integrity.
• Consider involving an external entity to review the accuracy and validity of State responses. Arkansas recommended enlisting an external entity to validate the self-assessment responses from the agency. Puerto Rico and Maine also recommended developing a verification or validation process to ensure that responses were accurate and appropriate. Several States recommended the involvement of the Regional Offices to assist with the validation process.
• Consider integrating the self-assessment process with the agency’s strategic or business plans. High-level management support is important for broader implementation of the self-assessment process. States may choose to involve senior management staff initially to determine the applicability of the self-assessment process with the broader agency mission or strategic planning process. Within the child care program the Instrument can be combined with the CCDF State Plan to serve as a biannual update of State efforts towards the prevention of errors that could result in improper payments. Senior management staff can review responses to ensure consistency with agency mission and strategic and business plans.  

L. Recommended Steps for Successful Implementation

Based on the first pilot State experience, Kansas offered several logistical steps that can maximize the effectiveness of the methodology process. The following steps provided useful guidance for the nine pilot States’ implementation experience and support the above recommendations. The steps include:

1. Select one overall project coordinator. It is critical that the project coordinator is available during the entire process to answer questions, consult with team members and coordinate with the CCB or WRMA.

2. Select a full range of State Project Team members. The following list includes potential County and Contractor staff for States where that may be appropriate. Appendix B provides a table listing potential team members and their functions. Potential team members may include:

a. High and Middle Level Management;

b. Audits;

c. Human Resources;

d. Information Technology;

e. Program Staff including those working on systems for programs if different than IT staff;

f. Quality Assurance;

g. Eligibility staff (State, County or contractors);

h. Licensing Staff;

i. Finance/Accounting;

j. Operations;

k. Public Information Officer;

l. Legislative Liaison; and

m. County management staff if County Administered.

3. Hold an initial team meeting to explain purpose, process and sell the project. If possible, have a senior level management staff person kick-off the project at this meeting. Go over each section of the document and make specific assignments to the team members for each section of the document. States that performed this step reported that it was very important to successful completion of the document.

4. Hold additional follow-up meetings to ensure that all participants tasked with completing sections of the assessment understand the purpose and process.

5. Make sure the identified coordinator is available during the entire process to answer questions, hold conference calls, consult with team members and coordinate with the CCB or WRMA.

6. Designate and involve support staff at the beginning for tracking and compiling data and information, editing for errors, checking grammar, and providing writing style consistency.

7. Provide all team members with an electronic version of the document to use in completing their assigned sections. Have all information entered on the electronic document and sent to coordinator who can then assign support staff to integrate it into one document. If more than one team member completes a section, integration with oversight from coordinator may be necessary.

8. Establish an agreed upon type style and font size so that responses can be inserted into the master document easily.

9. Discuss documentation: Web sites (hyperlink to both Internet and Intranet) and hard copy attachment options with all team members.

10. Hold additional meetings with all team members or sub-groups to share the completed document, receive comments and feedback, and make changes to document as needed.

11. Have senior management staff review responses to ensure consistency with agency’s mission and strategic or business plans.

¹Government Accountability Office. (June 2004.) TANF and Child Care Programs: HHS Lacks Adequate Information to Assess Risk and Assist States in Managing improper Payments. (GAO Publication No. GAO–04–723). Washington, DC: U.S. Government Printing Office. return to text

²Full descriptions of strong internal control environments are provided in OMB Circular A-123, Management’s Responsibility for Internal Control (December 21, 2004), available at http://www.whitehouse.gov/omb/circulars/a123/a123_rev.pdf and Standards for Internal Control in the Federal Government (November 1999), available at http://www.gao.gov/special.pubs/ai00021p.pdf. return to text

³Available at http://www.gao.gov/new.items/d011008g.pdf. return to text

Posted January 31, 2008