Questions and Answers on Pilot Testing the Model Tribal System (MTS)

Question 1. What is a Pilot Test?

Answer 1. A Pilot Test is a structured, controlled test performed over time to find out whether an implemented computer system will fulfill its intended functional purpose as determined by its end-users. With regard to this specific Pilot Test, we expect it to provide a basis for us to ensure the computerized Model Tribal System (MTS) can fully and accurately accept, store, process, calculate, and report on the many different actions, activities, and of course, financial transactions that have to be performed to effectively and efficiently manage a Tribal child support enforcement case.

Question 2. What is a Pilot Test Site, and what time commitment will be required to be the Pilot Test Site?

Answer 2. A successful Pilot Test Site is a Comprehensive Tribal IV-D program that can commit the staff and financial resources necessary to carry out uninterrupted operation of their Tribal child support enforcement program, while concurrently performing full-scale testing of the MTS. The Pilot Test will use active Tribal child support cases, comparing test results from the system with actual case outcomes from daily program operations in the Tribal IV-D office. The Pilot Test is expected to run for approximately three months, with a tentative start date of January 26, 2009 and end date of April 30, 2009.

Question 3. Can a Start-up Tribal IV-D program grantee apply to participate as a Pilot Test Site for the MTS?

Answer 3. No. Applications will only be accepted from current Comprehensive Tribal IV-D programs to be a Pilot Test Site for the MTS project. It is critical that the Tribal IV-D program selected to be the Pilot Test Site be operational and experienced in the day-to-day work involved in the many child support activities associated with the program functions of case intake, locate, establishment, case management, enforcement, collection, distribution and disbursement, reporting, and data privacy and safeguarding. With only an estimated window of three months to deploy and fully test the MTS, it is necessary that it be done in an operational program with experienced, well-trained staff that will understand quickly whether the MTS is meeting the needs, requirements, and expectations for which it was originally designed.

Question 4. Will my Tribe receive any reimbursement for any of the costs we incur in running this Pilot Test of the MTS?

Answer 4. No. Unfortunately, without publication of the Final Rule for Computerized Tribal IV-D Systems and Office Automation, we are unable under current regulations at 45 CFR Part 309.145(h)(1 - 5) to federally reimburse any costs related to the installation and pilot testing of the MTS. These pilot testing costs include travel, salaries, equipment and supplies.

Nevertheless, we are seeking one Comprehensive Tribal IV-D program willing to volunteer to take on the role, responsibilities, and expense of pilot testing the MTS. OCSE will provide training, technical assistance, operational oversight and support during this critical testing process.

Question 5. What costs should we anticipate in running this Pilot Test of the MTS?

Answer 5. Tribal expenses under the Pilot Test of the MTS fall under three general categories: program staff, Information Technology (IT) support, and equipment. We project a minimum of at least two program staff working full-time for three months on the MTS Pilot Test. These salaries for this testing period are not reimbursable with Federal financial participation (FFP), so that is one component of cost. The necessary IT support, whether provided from in-house resources or through contracted services are likewise not reimbursable, so there is the second component of testing costs. In addition, there is the possibility of a need to hire supplemental short-term staffing to provide ongoing Tribal IV-D program services concurrently with the Pilot Test, and if there are any delays in the testing process, these personnel-related costs would continue to be incurred. Lastly, unreimbursable expenditures in the equipment category include computer hardware and software, travel, supplies and any miscellaneous expenses related to installing, operating and maintaining the MTS during pilot testing. We have projected the estimated cost, from beginning to end, to be between $45,000 and $100,000 for the three month period. It is important to note that once pilot testing ends, the MTS will then be eligible for Federal reimbursement of ongoing maintenance and operations costs as it will immediately be considered an "Existing Tribal System" under current regulations.

Question 6. What is OCSE looking for in a Tribal IV-D program that would qualify it as a potential Pilot Test Site? What qualities, capacities, and capabilities will ensure the success of the MTS Pilot Test?

Answer 6. Although many factors can help ensure a positive outcome for the Pilot Test of the MTS, four are foremost in importance. First, the Tribal IV-D program must have the support and commitment of the Tribal government’s executive leadership, as well as the support of all other stakeholders to the Tribal IV-D program. These stakeholders include the Tribal IV-D Director, but may also represent the Tribal Courts, the Tribe’s IT department, and any other participants in and contributors to the success of the Tribal IV-D program. Not only is there a financial cost to the selected Tribe operating the Pilot Test, it will also test the Tribe’s IV-D program resources. Therefore, we believe executive leadership and stakeholder commitment is a critical component to a successful Pilot Test of the MTS.

Second, the Pilot Test of the MTS requires experienced, motivated program staff. Using experienced program staff is important because all manual casework (if the Tribal IV-D program is not yet automated) and any work performed on a State’s automated child support enforcement system has to be duplicated on the MTS. These experienced line staff will quickly be able to review and analyze the system’s inputs, processing, outputs, and reporting for correctness, and in the event we find errors, identify their scope and impact so they can be fixed. This is the purpose of a Pilot Test: to stress and challenge an automated system’s capabilities in real world scenarios using live case data in an operational environment.

Third, the pilot testing the MTS could seriously tax the staff resources of a Tribal IV-D program. Therefore, the ability to quickly supplement the staffing of the Tribal IV-D program, if needed, during the Pilot Test is essential to ensure program operations are not impacted. Not only will all routine child support casework need to continue without interruption, but it will also have to be accomplished with two fewer experienced staff: the staff dedicated to the MTS pilot testing.

Lastly, a Tribe with strong IT support, whether in-house or contracted, is better positioned to successfully support a Pilot Test of the MTS. The MTS is built on a technically sophisticated, modern, open-source architecture designed to help ensure productive service for many years to come. But this sophistication comes at a price: it requires knowledgeable, technically current and proficient IT support services.

Question 7. What is the start date for the Pilot Test?

Answer 7. We are working toward a February 16, 2009 start date for the Pilot Test, as cited herein in question and answer 11. Though this date must be considered tentative, it is provided for planning and estimation purposes.

Question 8. Will my Tribal child support staff be trained to run this Pilot Test?

Answer 8. Yes. We will train Tribal IV-D program staff in the installation, use, and operation of the MTS. Initially, 2 staff will be trained, allowing remaining Tribal IV-D program staff to continue performing their day-to-day child support case work.

In addition, whether the Tribal IV-D program has its own technical staff maintaining and operating its computer systems and networks or acquires these technical services through contract procurement, OCSE will provide training for up to five technical IT (information technology) staff. This technical training will consist of instruction in the system’s operation and maintenance, as well as in the system’s overall design. All initial training on the MTS, for both program and technical staff, will begin with the start of the Pilot Test.

Question 9. How long will training take on the MTS for the Pilot Test?

Answer 9. Although it is too soon to be precise as to the number of hours of classroom training we envision, we estimate caseworker training will require 40 hours of classroom instruction. Supervisors may receive up to two additional days of training during the Pilot on certain specialized system functions, such as: reporting, funds distribution, and document management. The estimated length of technical training will depend on the knowledge, skills, and proficiency of the Pilot Test Site’s IT staff. Technical training will initially concentrate on system configuration, administration, document management, and system operations. IT staff will require at least 40 hours of classroom instruction supplemented with on-the-job training during the Pilot Test period. All classroom instruction will be performed at OCSE in Washington, DC.

Question 10. Will Federal staff (and other project staff involved in the Pilot Test effort) have access to my Tribal IV-D case data during pilot testing? What assurances do I have that our data will be kept secure and confidential?

Answer 10. Because Federal staff will be providing on-site technical assistance in loading data, set-up and configuration, and operational support, they will, by extension, have access to the data loaded to the MTS at the Pilot Test Site. The Tribal IV-D program serving as the Pilot Test Site can be assured that Federal staff will adhere to and comply with all Federal data safeguarding standards at all times. Federal MTS project staff with access to program data will sign Tribal non-disclosure agreements if requested.

In addition to compliance with data safeguarding noted above, all database, network, application, and end-user computing systems used in the Pilot Test employ rigorous security methods and tools to prevent loss or corruption of data.

Question 11. How exactly does OCSE envision the day-to-day deployment, operation, and feedback of the results from the Pilot Test Site being carried out by the Tribal IV-D program?

Answer 11. Our expectations for the timing of various Pilot Test Site tasks and milestones, such as deployment, operation, data gathering, and results analysis have been estimated based on a three month Pilot Test period. The table below displays a high-level list of these tasks and milestones with their respective estimated start dates calculated using a February 16, 2009 anticipated start date for the Pilot Test.

Model Tribal System Pilot Test Tasks and Milestones
Task and Milestones Start Date
OCSE training of Tribal IV-D program technical IT staff on MTS (5 days). 2/16/09
OCSE training of Tribal IV-D program caseworker, legal, financial, and supervisory staff on MTS (7 days). 2/23/09
OCSE and Tribal technical IT staff set-up and configure MTS for Pilot Test Site operations (5 days). 2/23/09
Tribal IV-D program staff begin manual data entry and conversion of a minimum of 200 total Tribal IV-D case records into the MTS installed at Pilot Test Site (estimated tontake 10 consecutive business days). 2/24/09
Tribal IV-D program begins MTS Pilot Test concurrently with ongoing case data entry activities. 3/02/09
Log errors, bugs, and discrepancies identified in Pilot Test by caseworkers, supervisors and technicians. Daily
Perform ongoing comparison of automated case processing results from MTS with results on same cases processed manually by Tribal IV-D staff. Weekly
OCSE follow-up training of Tribal IV-D program supervisory staff on MTS (2 days). 3/23/09
OCSE Pilot Test results evaluation and analysis report to OCSE and Tribal leadership. 4/20/09
OCSE to install new bug fixes, software change requests, and related documentation to MTS in Pilot Test Site. As-needed
OCSE reports on Pilot Test Site status. Monthly
Pilot Test concludes. 5/29/09
Tribal training of remaining Tribal IV-D program staff on MTS (7 days). 5/29/09
OCSE to compile and deliver ilot Test Site summary report for "Go/No-Go" decision to OCSE Executive Leadership. 6/08/09
OCSE announces general release and availability of the MTS (date assumes a "Go" decision from previous Pilot Test task.) 6/22/09

All of the estimated dates above are based on the MTS project meeting its schedule for completion of software application development and acceptance testing. Further, the above estimates for data entry rely on the following constraints and assumptions.

Question 12. During the Pilot Test, will OCSE provide technical assistance, such as "help desk services" for our users and technicians to contact with questions on MTS use and operations?

Answer 12. Yes. OCSE expects to make repeated visits on-site with the Tribal IV-D program serving as a Pilot Test Site and will provide "help desk" services during normal business hours. These help desk services will fall into two categories of assistance: (1) technical assistance to caseworkers and their supervisors on use of the MTS, and, (2) help desk support including remote access directed towards the Pilot Site’s technical IT staff, providing assistance and guidance to those technicians in deploying and managing system operations and maintenance, and rapid installation of bug fixes and software updates.

In addition to this help desk support, we expect to provide a considerable presence on-site. We intend to accomplish this through a series of staggered site visits by numerous project team members during the entire course of the Pilot Test. We are also looking at additional means of support, such as internet-based online chat capabilities between project staff at OCSE and Tribal IV-D program staff at the Pilot Test Site. Finally, we will also be able to support technical IT staff through remote access to assist in performing system restarts, as well as perform software maintenance and install updates during the Pilot Test phase.

Through these various means, we expect to be in constant physical and/or virtual contact and communication with Tribal IV-D staff conducting the Pilot Test. Specific details regarding OCSE’s delivery of technical support during this Pilot Test phase are still being defined, but we expect to have this Implementation Plan for the MTS project completed by December 2008. The Implementation Plan will be shared with any Comprehensive Tribal IV-D program interested in volunteering to serve as a Pilot Test Site.

Question 13. When will the selection of a Pilot Test Site be announced?

Answer 13. OCSE expects to be able to announce the selection of a Pilot Test Site in early January 2009.

Question 14. What are the criteria for selecting the Pilot Test Site candidate?

Answer 14. We developed the following criteria to help us objectively and fairly select the most capable Pilot Test Site candidate. Based on our experience, a Pilot Test Site will need to meet the statistical and organizational requirements, and agree to the commitments listed below.

  1. The Pilot Test Site candidate must be a Comprehensive Tribal IV-D program in full operation. Start-up grantees are not eligible as pilot testing of the MTS requires the performance of the full range of case activities being conducted in an operational Tribal
    IV-D program.
  2. The Tribal IV-D program must have an operational caseload of at least 200 cases. This means that approximately 90 percent of existing Tribal IV-D programs will be represented by the efforts of the Pilot Test Site, and ensures the minimum number of cases needed for testing will be available to be converted into the MTS.
  3. Tribal IV-D program staff must include five or more case and financial workers, etc. This minimum number excludes clerical and management staff.
  4. The Tribal IV-D program must be able to dedicate two experienced, Full-Time-Equivalent (FTE) staff to work on the MTS Pilot Test for a period of up to three months. The use of dedicated staff ensures this commitment does not interfere with the normal operation of the Pilot Test Site’s Tribal IV-D program, and helps ensure retraining of new staff, which would cause delays in the testing schedule, does not occur.
  5. The Pilot Test Site must have the financial means to commit to supporting the MTS Pilot Test. Expenses that could be incurred to operate a three-month Pilot Test are conservatively estimated at between $45,000 and $100,000. These costs are primarily attributable to the salaries of the two program staff performing testing duties during the three-month Pilot Test, as well as the costs of IT-related services and any additional computer hardware, software, services, supplies and travel needed to support the Pilot Test. Another potential cost could be a need to hire supplemental short-term staffing to provide ongoing Tribal IV-D program services concurrently with the Pilot Test. If there are any delays in the testing process, these personnel-related costs would continue to be incurred.
  6. The Pilot Test Site must have access to reliable, proficient IT services, specifically in the areas of systems integration, network operations, and software maintenance in a technically sophisticated, open-source, and web-based application environment. These IT services can be from a Tribe’s own internal IT department or through a contractual relationship with a vendor. However acquired, the Pilot Test Site must commit to ensuring these IT services will be available at the start of the Pilot Test, and throughout the entire testing period. OCSE will work with the Tribe’s IT service provider to jointly support the MTS Pilot Test.
  7. OCSE will be the lead agency responsible for governance, management and operational control of the MTS Pilot Test. The Tribe will agree to support OCSE’s management of the Pilot Test, including ensuring remote access to the installed MTS, facilitating travel for purposes of Pilot Test training of designated program and IT staff, and performing pilot testing tasks under the direction of OCSE.
  8. The Tribal IV-D program selected must commit to installing the MTS, conducting concurrent, simulated program operations using the MTS (pilot testing), and documenting the test data generated by the MTS during the Pilot Test. Activities to be supported by the Pilot Test Site include the following:

Question 15. How should the Tribal IV-D program apply to OCSE to be a Pilot Test Site candidate?

Answer 15. If your Tribal IV-D program has the desire and the necessary staff and fiscal resources to fulfill this important role, please submit the following information.

This information should be accompanied by a cover letter signed by an authorized executive of your Tribal government committing your Tribal IV-D program’s staff time, and physical and fiscal resources for a minimum of three months to be the MTS Pilot Test Site.

Please submit the signed cover letter and requested information no later than December 31, 2008. We will make an announcement regarding the selection of the Pilot Test Site in early January 2009. Please submit your information to:

Commissioner, OCSE
Administration for Children and Families
370 L’Enfant Promenade, SW
Washington, DC 20447

If you have any questions, or would like more information regarding any aspect of volunteering to be the MTS Pilot Test Site, or about the roles and responsibilities of performing the pilot testing, please feel free to contact our Project Director for the MTS development effort, Joseph Bodmer, at (202) 690-1234.