Evaluation to Determine the Effectiveness of the Public Assistance Reporting and Information System – Appendices of the Final Report
Submitted to:
Administration for Children and Families
Office of Financial Services/Division of Financial Integrity
370 L’Enfant Promenade SW
Aerospace Building, 6th Floor-East
Washington, DC 20447
Submitted by:
Health Systems Research, Inc, an Altarum Company
1200 18th St. NW, Suite 700
Washington, DC 20036
June 30, 2007
Table of Contents
I. Overview of Example State and Initial Match
III. Calculating Savings
A. Calculating Savings on the Interstate Match from Unreported Moves
B. Savings from Benefit Reductions as a Result of VA and Federal Match
C. Total Savings in Recoveries
D. Savings from Coordination of Benefits
E. Total PARIS Costs and Savings from Participation in All Three February 2007 Matches
Appendix B.1: PARIS Coordinator Protocol
I. Background
II. Factors Affecting Participation
IV. Staffing
VI. Follow-up
Appendix B.2: PARIS Technical Staff Protocol
I. Background
III. Follow-up
Appendix B.3: PARIS Follow-up Staff Protocol
I. Background
III. Follow-up
Appendix B.4: Financial Questions
I. Background Information on Costs
II. Budget and Expenses for the PARIS Project
III. Calculating the Benefits of PARIS
Appendix B.5: Financial Questions
I. Background
II. Factors Affecting Participation
V. Follow-up
Listing of Tables
Table A. Savings Related to Case Closure of TANF Cases as a Result of Unreported Moves
Table B. Savings Related To Case Closure of Food Stamp Cases as a Result of Unreported Moves
Table C. Savings Related To Case Closure of Medicaid Managed Care Cases as a Result of Unreported Moves
APPENDIX A: SIMULATION OF PARIS COST AND BENEFIT ANALYSIS USING AN EXAMPLE STATE
I. Overview of Example State and Initial Match
The purpose of this sample simulation is to demonstrate how States can use the proposed cost and benefit models to determine the cost benefit of PARIS. For this simulation, we are assuming that the State participates in all three matches, and conducts follow-up activities on all three. The Sample State created has the following features:
- PARIS is administered by the Department of Social Services (DSS), with a PARIS coordinator that serves as a bridge between DSS and the Department of Health Services (DHS) where Medicaid is housed.
- The State Medicaid program uses managed care as its primary method of delivering benefits.
- All initial follow-up activities are conducted locally by caseworkers. Follow-up procedures call for the caseworker to conduct a residency verification on all interstate matches, and to double check all VA and Federal match results to ensure that income was properly reported.
- PARIS procedures require that unresolved local follow-up matches or those that indicate fraud is being committed must be referred to the Office of Investigator General (OIG) for fraud follow-up.
- Upon a determination being made that fraud was committed, a benefit recovery letter is issued to the client and an account set up in the Office of Financial Recovery for action.
- The State Medicaid office has an office of Benefit Coordination, which is responsible for coordination of benefits between Medicaid and other health insurance programs. PARIS matches that show VA or other Federal health coverage are processed through this office.
- The State submitted a total of 456,000 records to DMDC for all three matches. The results that were returned were as follows:
Interstate Match: 2,140
VA Match: 1,770
Federal Match: 264
Total Matches 4,174
II. Calculating Costs
Costs for conducting the PARIS match are divided into four cost center categories, which include:
- The cost of data processing staff to prepare the PARIS files from submission to DMDC through filtering results and submission of the files to local offices for follow-up.
- The cost of local follow-up to resolve PARIS matches
- The cost of OIG and Financial Recovery staff involvement in resolving PARIS matches.
For this simulation the costs in the three areas above were calculated using the following data, which was obtained by averaging actual cost data collected from three study States:
- Data Processing costs. The cost of preparing the PARIS file, filtering the results, and distributing the data were figured on using 25 percent of a Senior Data Processing Administrator's time during the month in which the PARIS match occurs. The salary and benefit cost for this person is $69,600 per year. Therefore the monthly salary and benefit cost is $5800. The total cost for preparing the PARIS data file is $1,450.
- Follow-up costs. The cost of local follow-up have been identified using a figure that each PARIS case requires an average of 30 minutes time to resolve. The average hourly salary and benefit rate for field staff conducting PARIS follow-up activity is $21 per hour. Therefore, filed staff worked a total of 4174 matches, which resulted in 2,088 hours being devoted to follow-up activity at a cost of $43,848.
- Investigative Costs. The average cost per case investigated by OIG and assigned to the Office of Financial Recovery is $3,500. In this case, ten cases were referred for investigation, resulting in a cost of $35,000.
The total cost for conducting this quarter's PARIS match activities is: $80,298
III. Calculating Savings
A. Calculating Savings on the Interstate Match from Unreported Moves
Once the data were returned from DMDC, further filters were run and data for the interstate match was refined to reflect program participation. As a result, the interstate match produced the following results:
Participates in TANF, Medicaid and Food Stamps: 905
Participates in Food Stamps and Medicaid: 890
Participate in Medicaid only: 345
Of these results, follow-up activities were conducted to verify residency of the clients for whom social security numbers matched. The results of this follow-up found the following cases were still showing as active, but the clients had moved and were no longer eligible for benefits:
Participates in TANF, Medicaid and Food Stamps: 401
Participates in Food Stamps and Medicaid: 425
Participate in Medicaid only: 160
Additionally, 10 cases where clients were participating in TANF, Food Stamps and Medicaid were referred to OIG for further follow-up.
TANF and Food Stamp Program benefits are calculated based on family size. Therefore, results from the match were broken down by family size and average TANF and Food Stamp benefits for each group. The average dollar benefit was multiplied by the number of cases closed as a result of the PARIS follow-up, and that product was multiplied by twelve to create the total savings.
| Family Size | Average Benefit | Number of PARIS Match Cases Closed | Total Savings |
|---|---|---|---|
| Two | $260 | 98 | $305,760 |
| Three | $330 | 122 | $483,120 |
| Four | $410 | 166 | $816,720 |
| Five | $479 | 15 | $86,220 |
| Total | 401 | $1,691,820 |
| Family Size | Average Benefit | Number of PARIS Match Cases Closed | Total Savings |
|---|---|---|---|
| Two | $136 | 236 | $385,152 |
| Three | $230 | 268 | $739,680 |
| Four | $312 | 176 | $658,944 |
| Five | $366 | 146 | $641,232 |
| Total | 826 | $2,345,616 |
For calculating Medicaid savings, data were grouped into three separate categories, general, child, and persons with a disability. The average capitation rate for each category was applied to create the savings.
| Medicaid Category | Average Capitation Rate | Number of PARIS Match Cases Closed | Total Savings |
|---|---|---|---|
| General | $168 | 694 | $1,339,104 |
| Child Health | $140 | 256 | $430,080 |
| Disabled | $680 | 42 | $342,720 |
| Total | 986 | $2,111,904 |
Total Savings from the Interstate Match: $6,149,340
B. Savings from Benefit Reductions as a Result of VA and Federal Matches
Case files were checked to determine if benefits from the VA and Federal sources were properly reported. A total of 1,770 VA and 264 Federal matches received follow-up. Of these, improper reporting of income occurred in 119 VA matches and 26 Federal matches. This resulted in an average benefit reduction in TANF and Food Stamps of about $26 per case. Therefore, the total savings as a result of benefit reductions is $44,928.
C. Total Savings in Recoveries
Ten cases were reported for investigation and recovery of benefits paid. Of the 10, eight were determined to have been improperly paid since the last PARIS report. There were therefore eight cases of TANF and Food Stamp benefits averaging $690 paid over 3 months that were issued recovery letters. The value of these savings were calculated to be $16,580.
D. Savings from Coordination of Benefits
Of the 1,770 VA matches, it was determined that 26 were receiving long term care payments from Medicaid that should have been paid by VA. These were referred to the Medicaid office responsible for coordination of benefits, and responsibility for payment of long term care costs were transferred to VA. The average monthly cost of the long term care was $3890. Therefore, the Medicaid savings from transferring these costs to the VA were $280,080.
E. Total PARIS Costs and Savings from Participation in All Three February 2007 Matches
Total cost of PARIS match: $80,298
Total Savings of PARIS Match $6,490,928
Total Benefit after deducting for cost $6,410,630
APPENDIX B.1: PARIS COORDINATOR PROTOCOL
Public Assistance Reporting Information System (PARIS) | Program Evaluation | Interview Protocol | PARIS Project Coordinator | Used in All Participating States
OMB Clearance Number: 0970-0312
Estimates of Burden for the Collection of Information. According to the Paperwork Reduction Act of 1995, no persons
are required to respond to a collection of information unless it displays a
valid OMB control number. The valid OMB control number for this collection is
0970-0312. The time required to complete this information collection is
estimated to average 90 minutes per response, including the time to review
instructions, search existing data resources, gather the data needed, and
complete and review the information collected.
Introduction
Hello, my name is _________, and I am with Health Systems Research, Inc (HSR). The U.S. Department of Health and Human Services' Administration for Children and Families (ACF) has contracted with HSR to conduct an implementation and outcome evaluation of State participation in the Public Assistance Reporting Information System (PARIS).
This evaluation will determine the effectiveness of the program in terms of costs and benefits as well as develop and utilize methodologies to measure the impact on reducing improper payments. Data will be collected to determine factors affecting:
- Program participation
- Relevant PARIS administrative and implementation information
- Challenges in implementation
- Cost of program participation
- Estimated savings through identified and resolved participant matches
To conduct this study, HSR has drawn a sample of both participating and nonparticipating States to be studied. The evaluation will consist of surveys, interviews, and document reviews at different levels of participation.
As you know, we are talking to you because of your State's participation in this program. As part of this process, ACF has requested that HSR administer surveys with key staff from your agency involved in the PARIS project. The information obtained from this evaluation will inform future programmatic decisions at ACF.
Any answers you provide for this study will be kept confidential in the sense that your name will not be identified with any answers you provide. However, reports of such items as best practices or examples of key issues may include the name of the State in which the practice or issue was found.
The estimated amount of time required to complete this interview is one to one and one- half hours. Thank you for taking the time today to talk with me.
Before we begin, do you have any questions?
First, I want to ask some background questions regarding your role in the PARIS project, and how your State became involved.
I. Background
Please describe your role in the PARIS program. In particular, we are interested in how long have you been involved in the program and how you became involved in this program?
What are your specific responsibilities with regard to PARIS? Do you work on PARIS full-time, or do you have other responsibilities? If you have other responsibilities, are they similar to your role with the PARIS program?
How did your agency first hear about the PARIS program? Who within your agency decided that it was worthwhile to pursue participation?
In what agency is PARIS organizationally located? At what level/in what department?
What were the key factors of PARIS that motivated your State to inquire about participating in the PARIS program? What were the benefits you saw from participation?
What programs are currently submitting data to PARIS? Are data centrally collected or must you receive separate eligibility files from each program? If you must collect from each program, are they located within the same Department?
II. Factors Affecting Participation
What process, if any, were you required to follow in order to receive final approval to participate in PARIS? Was this a formal process such as the submission of a proposal to management, or was the decision made informally? Who gave final approval? Was this process similar or different from other decisions you may have made to participate in other (similar) initiatives?
From the time you became interested in participating in PARIS, how long did the participation decision-making process take? Was this a normal amount of time for projects of this nature? If shorter or longer, why did it take that long (Probe for what factors impacted the timeline)
What barriers did you, or those interested in implementing PARIS, encounter in the decision making process? How did you overcome these?
What opportunities, if any, did you or those interested in implementing PARIS identify in the decision making process? (e.g., working with new partners, ability to take advantage of funding from CMS or use Food Stamp reinvestment funds)
Did you speak with other PARIS States prior to your decision to participate in PARIS? What types of information did they share with you? (e.g., information about experiences, protocols, filter algorithms, report layouts) How did these discussions affect your decision to participate?
What other information would have been helpful to facilitate the decision making process (to make the case for participation to final decision makers easier)?
Upon initiation of participation or as part of the program approval process, did you, or your organization, do a speculative cost-benefit analysis and/or write a decision memorandum for approval authority?
Did ACF play an role in your final decision to participate? If so, what role did they play? Were you provided with enough information to fully implement PARIS in your State?
Did you have any problems or issues with regard to signing the PARIS agreement? If so, what were these problems or issues?
Would you suggest that ACF maintain or change its current role in States' decision making processes? If you suggest changes, please provide specifics.
III. Administration of PARIS
How and why was the decision made to place PARIS in the agency and division in which it currently resides? Other alternatives discussed?
What are the advantages of having PARIS organizationally located where it is currently? (Probe for Decision-making? Dissemination of information to Medicaid, TANF, and FSP staff? Enforcement of data standards (formats)? Enforcement of data deadlines? Implementation of post-match activities?)
Are there any disadvantages of having PARIS placed where it is? If so, have you explored options as to how these can they be addressed? (Probe for Decision-making? Dissemination of information to Medicaid, TANF, and FSP staff? Enforcement of data standards (formats)? Enforcement of data deadlines? Implementation of post-match activities?)
What policies and procedures are in place to guide the implementation and administration of the PARIS program in terms of interagency collaboration? How easy or difficult was it to put these policies in place?
Are there any interagency/division workgroups in place to advise on the administration of PARIS and the follow-up of matches? If so, what is the purpose of the workgroup? Who attends the meetings? How often are they held? What topics have been on the agendas?
Have you encountered any communication issues between those that administer PARIS and those responsible for follow-up? If so, what, if any, steps have been taken since initial participation in PARIS to enhance communication?
What level of communication do you have with ACF and DMDC? What has it concerned? How effective is this communication? How could it be improved?
Have you used the PARIS Web-site to support the administration of PARIS in your State? If so, have you found it to be helpful? How so? Could the Web-site be modified to make it more useful (what else could be included)?
What, if any, types of technical assistance could the PARIS project have provided during implementation? Are there any ongoing issues (reading federal match files, follow-up procedures, etc.) for which you would like to have assistance.
IV. Staffing
How many staff members (individuals, not FTEs) are involved with the administration and follow-up from PARIS? Where are they organizationally located?
Are any of these staff members dedicated to PARIS full-time? For those that are not full time, what portion of their time do they spend on PARIS - and how do you estimate this? What are their other responsibilities?
Is your staffing level adequate to meet your needs? Would you do any activities differently (particularly follow-up) if you had access to more staff?
V. Submission of Data
What process is currently used for identifying and selecting the Social Security numbers of clients (SSNs) for the PARIS match? (What conditions need to be met to submit an SSN-time period, active SSN, Valid SSN).
Who decided/decides what criteria should be used for selecting which SSNs to submit?
Have the criteria evolved/changed over time? If so, what were the original criteria? When and why were any of the changes implemented?
How easy or difficult is it for you to submit a file with the required fields? Did special programming have to be done?
Do you include termination dates on your records?
The PARIS project recently proposed to expand the fields to include several optional fields for adding programs later. How easy or difficult will it be if you are required to submit data in optional fields? What would you have to do to make this possible? How useful do you see the optional data as being and why?
Are edits applied to the data prior to submitting to DMDC to check for record accuracy? If so, what are these filters? Why were these selected? Are there programs required to clean data before submitting? If yes, what do they do?
What problems, if any, have you had submitting data to PARIS for matching?
Have you been notified by DMDC or ACF regarding any issues or problems with your submissions to PARIS? (Probe Have you been asked to change your submission format?) Have you been able to make any modifications requested? Why or why not?
How frequently have you missed a PARIS submission deadline that you had wanted to submit for in the past year? By how many days (in each instance)? Why did this occur (systems, people, timing, other)?
VI. Follow-up
Do you conduct follow-up activities at the State-level, the local-level or both?
What happens after the PARIS administrator receives the matched data back from DMDC? Who is responsible for sending matched data files back to the appropriate entities for follow-up? Who is it sent to? What process is in place for follow-up to occur?
What types of filters do you have in place to eliminate false positives or negatives? What other features are filtered prior to submitting the data for follow-up? How/why were they created? Who decided/decides which filters should be used? Have the filters changed over time? If so, when and why were they changed?
If filters are used, what percentage of the matched data are filtered out? Is the remaining number a workable amount?
How useable are the matched data in general as returned from DMDC? What could be done to enhance the usability of the match data?
Do you understand all of the fields included in the various matches? If not, what matches (federal, VA, Interstate) need clarification? Are there other data that you need to make the data you receive more useful?
What is the typical number and percentage of matches returned from DMDC compared to the number submitted?
Have you created special reports to make working with the data more expeditious? What do these include? How could they be enhanced to make them more meaningful and useful?
Do you prioritize data/cases for follow-up? If so, what criteria are used? Are these criteria workable? Have they changed since you started conducting follow-up? Do they differ by program?
Do staff working on follow-up receive training on follow-up procedures? What does this consist of? Are procedures incorporated into field manuals?
Do staff working on follow-up have a time limit in which to conduct the follow-up and resolve a case? What is the average amount of time (by case type) that it takes to conduct follow-up? How does the length of time that follow-up take and the number of matches you receive impact the number of times you submit data to the PARIS match?
What process is in place for individuals from other States to contact you about matches? Do they contact a central office or each local site individually?
For the Interstate match, what issues, if any, have come up that have hampered you in resolving matches with other States (Incomplete data, duplicates, lack of cooperation)? Have any states in particular been problematic? How have you been able to resolve these? Are there still issues or problems you would like to see resolved?
Are programs that submit data required to reconcile or report back to the PARIS administrator about the results of any follow-up?
Is there a specified timeframe in which they have to report back?
What do they report (i.e. number of follow-ups, results of follow-up, dollars saved as a result of follow-up, dollars recouped as a result of follow-up)?
For each program (or for all programs if integrated case management):
Overall, what proportion of matched cases receive follow-up?
What proportion of matched cases that do receive follow-up are referred for further action (administrative or criminal)?
Are there significant differences across the different programs in follow-up and in results?
Do you use the PARIS Web-site to support follow-up activities of PARIS in your State? If not, how could the Web-site be modified to make it more useful (what else could be included)? If so, what has been the most helpful?
Thank you for your time.
APPENDIX B.2: PARIS TECHNICAL STAFF PROTOCOL
Public Assistance Reporting Information System (PARIS) | Program Evaluation | Interview Protocol | PARIS Technical Staff | Used in All Participating States
OMB Clearance Number: 0970-0312
Estimates of Burden for the Collection of Information. According to the Paperwork Reduction Act of 1995, no persons
are required to respond to a collection of information unless it displays a
valid OMB control number. The valid OMB control number for this collection is
0970-0312. The time required to complete this information collection is
estimated to average 90 minutes per response, including the time to review
instructions, search existing data resources, gather the data needed, and
complete and review the information collected.
Introduction
Hello, my name is _________, and I am with Health Systems Research, Inc (HSR). The U.S. Department of Health and Human Services' Administration for Children and Families (ACF) has contracted with HSR to conduct an implementation and outcome evaluation of State participation in the Public Assistance Reporting Information System (PARIS).
This evaluation will determine the effectiveness of the program in terms of costs and benefits as well as develop and utilize methodologies to measure the impact on reducing improper payments. Data will be collected to determine factors affecting:
- Program participation
- Relevant PARIS administrative and implementation information
- Challenges in implementation
- Cost of program participation
- Estimated savings through identified and resolved participant matches
To conduct this study, HSR has drawn a sample of both participating and nonparticipating States to be studied. The evaluation will consist of surveys, interviews, and document reviews at different levels of participation.
As you know, we are talking to you because of your State's participation in this program. As part of this process, ACF has requested that HSR administer surveys with key staff from your agency involved in the PARIS project. The information obtained from this evaluation will inform future programmatic decisions at ACF.
Any answers you provide for this study will be kept confidential in the sense that your name will not be identified with any answers you provide. However, reports of such items as best practices or examples of key issues may include the name of the State in which the practice or issue was found.
The estimated amount of time required to complete this interview is 30 minutes. Thank you for taking the time today to talk with me.
Before we begin, do you have any questions?
First, I want to ask you some questions regarding the background of your role in PARIS:
I. Background
Please describe your role in the PARIS program. How did you come to be involved in this program? Have you been involved since the inception of the program? If not, how long have you been involved?
Where are you organizationally located? Do PARIS staff have direct access to your services, or must they complete work orders or other such documents to obtain your time?
Were you involved in the initial set-up of the PARIS match system? If so, how easy or difficult was it to create the file for match? What were the major technical issues you faced?
Have you been involved in any technical modifications of PARIS programming or software? If so, what has been the purpose of the modifications? Why were these undertaken?
Do you work on PARIS full-time, or do you have other responsibilities? If you have other responsibilities how much of your time do you estimate that you spend on PARIS? Do you have any issues or difficulties meeting requests of the PARIS program?
II. Submission of Data
Are data for the PARIS file located in a single file, or must you merge files from individual programs to create the PARIS file? If located in separate programs, how easy or difficult is it to obtain the files?
What criteria are currently used for identifying those Social Security numbers (SSNs) for which data will be extracted for the PARIS match? (What conditions need to be met to submit the SSNs-time period, active SSNs, Valid SSNs).
Who decided/decides what criteria should be used for identifying those cases for submitting?
Have the criteria evolved/changed over time? If so, what were the original criteria? When and why were any of the changes implemented?
Are there other criteria that you think should be used for case selection? If so, why are they not being used?
Are any types of filters or edit checks applied to the data prior to submitting to DMDC?
What problems, if any, have you had submitting data to PARIS for matching? What have you been told regarding any issues or problems from your submissions to PARIS? Have you been asked to change your submission format or record input? Have you been able to do so? Why or why not?
Have you missed a PARIS submission deadline that you wanted to make during the past year? If so, by how many days (in each instance)? Why did this occur (systems, people, timing, other)?
Are there processes or procedures that would make it easier to meet the PARIS deadline?
III. Follow-up
Is there anything that you are aware of that ACF or DMDC could do to enhance the usability of the match data?
What is the process for handling matches when they come back from DMDC?
Probe for:
Filters: Are any filters subsequently applied to eliminate erroneous matches before turning the matches over for follow-up? (i.e. name and address, dates of eligibility, prior positive matches). Who decided/decides which filters should be used? Have the filters changed over time? If so, when and why were they
Prioritization: Do you have any role in deciding on how to prioritize match data? If so, what criteria are used and what is your role? Are they different by program?
Special Reports: Do you create any special reports that are sent either to program mangers or field staff other than the normal match data? If so, what are these reports and what are they used for?
Do you ever view the summary reports of results of follow-up? What do you like/dislike about the summary reports? How could they be enhanced to make them more meaningful and useful?
What has been the most difficult technical challenge you have faced in providing technical support for PARIS? How was this resolved?
Are there any types of resource limitations that create problems for providing technical support to PARIS? (Probe: Lack of staff, funding, competing priorities, etc?) Have you been able to overcome these problems? If not, do you anticipate these to continue or are there plans for future changes that will help?
Is there any type of technical assistance that could be provided by ACF to help you with your work on PARIS? If so, what topics would be of interest? What type of expertise would you need for assistance?
Thank you for your time.
APPENDIX B.3: PARIS Follow-up Staff Protocol
Public Assistance Reporting Information System (PARIS) | Program Evaluation | Interview Protocol | PARIS Follow-Up Staff | Used in All Participating States
OMB Clearance Number: 0970-0312
Estimates of Burden for the Collection of Information. According to the Paperwork Reduction Act of 1995, no persons
are required to respond to a collection of information unless it displays a
valid OMB control number. The valid OMB control number for this collection is
0970-0312. The time required to complete this information collection is
estimated to average 90 minutes per response, including the time to review
instructions, search existing data resources, gather the data needed, and
Introduction
Hello, my name is _________, and I am with Health Systems Research, Inc (HSR). The U.S. Department of Health and Human Services' Administration for Children and Families (ACF) has contracted with HSR to conduct an implementation and outcome evaluation of State participation in the Public Assistance Reporting Information System (PARIS).
This evaluation will determine the effectiveness of the program in terms of costs and benefits as well as develop and utilize methodologies to measure the impact on reducing improper payments. Data will be collected to determine factors affecting:
- Program participation
- Relevant PARIS administrative and implementation information
- Challenges in implementation
- Cost of program participation
- Estimated savings through identified and resolved participant matches
To conduct this study, HSR has drawn a sample of both participating and nonparticipating States to be studied. The evaluation will consist of surveys, interviews, and document reviews at different levels of participation.
As you know, we are talking to you because of your State's participation in this program. As part of this process, ACF has requested that HSR administer surveys with key staff from your agency involved in the PARIS project. The information obtained from this evaluation will inform future programmatic decisions at ACF.
Any answers you provide for this study will be kept confidential in the sense that your name will not be identified with any answers you provide. However, reports of such items as best practices or examples of key issues may include the name of the State in which the practice or issue was found.
The estimated amount of time required to complete this interview is one to one and one- half hours. Thank you for taking the time today to talk with me.
Before we begin, do you have any questions?
I. Background
Please describe your job duties and, in particular, your role in the PARIS program. How did you come to be involved in this program? Do you work on a number of data or information follow-up activities related to client eligibility? If so, how much of your time is devoted to PARIS?
What process is used to provide you with the PARIS matches for follow-up? How many PARIS matches do you normally process each quarter?
II. Administration of PARIS
How does the organizational location of PARIS affect your ability to receive matches and conduct follow-up activities? (Decision-making? Dissemination of information to Medicaid, TANF, and FSP staff? Enforcement of data standards (formats)? Enforcement of data deadlines? Implementation of post-match activities?) If you have questions regarding PARIS matches, who is your point of contact?
What policies and procedures are in place to guide the implementation and administration of the PARIS program in terms of follow-up procedures? Have these been effective? If not, why not?
Are there any interagency/division workgroups in place to advise PARIS program administrators on issues related to follow-up of any matches? If so, who attends the meetings? How often are they held? What topics have been on the agendas?
(If local offices are used) How is information disseminated about the PARIS project to local to the local level? Do you feel that local staff have a good understanding of the purpose of PARIS and the activities that take place to create the match? How effective is this communication?
What, if any, steps have been taken since initial participation in PARIS to enhance communication?
Do you have any communication with ACF and DMDC? On what topics or issues? How effective is this communication?
What, if any, types of technical assistance could the PARIS project have provided during implementation? Are there any ongoing issues (reading federal match files, follow-up procedures, etc.) for which you would like to have assistance.
III. Follow-up
At what point during the month do you usually receive the PARIS matches for follow-up? Do you have any time limit by which you must complete all of the follow-up activity? What turnaround time on receiving match data would be optimal to make the most use of match data?
In your opinion, what is the quality of the matched data you receive from the PARIS administrator? How effective is the current filtering system in removing cases that do not warrant attention? Do you feel that there are filters that are not being used but could improve the quality of the data? If so, what types of filters?
What is the typical number and percentage of matches your office receives each quarter? What percentage of these matches are pursued? How do you decide how to prioritize match data, if at all? What are the criteria? Have they changed? Are they different by program?
(For Interstate Match)What process or procedures are used once a match identifies and verifies a duplication exists? Do you generally contact the other State yourself to verify duplication? If so, how well does this process work?
(For VA and Federal Matches) Who is responsible for resolving income issues once identified? What process is used to notify the clients of the inconsistency? How successful are you in contacting clients and resolving issues?
How long does it take to resolve an average match case? Does this vary by type of match (Interstate, VA, Federal?) If so, which take longer and why?
Are you required to report back to the PARIS administrator about the results of any follow-up? If so, what is the process?
If you believe intentional fraud has taken place, what process is in place for you to refer the case for investigation? How many of these types of cases do you find each quarter? Are you informed of the results of the investigation?
Are you contacted by individuals from other States to resolve matches? If so, do you feel the communication between you and the workers in other States is effective? If not, what types of problems do you deal with?
How receptive to resolving cases do you find individuals in other States? Are there States that are a particular problem for you? Are there States that you feel are very effective in resolving matches? If so, which States are these?
What has been the most difficult types of case to resolve? What could be done to improve the resolution process for these difficult cases?
Do you feel that the PARIS project is successful in improving program integrity? If not, what could be done to improve the success rate?
Do you have any success stories that you would like to share with us today? If so, please provide the details? Would you like to see local-level success stories published on the ACF PARIS Website?
APPENDIX B.4: FINANCIAL QUESTIONS
Public Assistance Reporting Information System (PARIS) | Program Evaluation | Interview Protocol | Financial Questions | Used in All Participating States
OMB Clearance Number: 0970-0312
Estimates of Burden for the Collection of Information. According to the Paperwork Reduction Act of 1995, no persons
are required to respond to a collection of information unless it displays a
valid OMB control number. The valid OMB control number for this collection is
0970-0312. The time required to complete this information collection is
estimated to average 90 minutes per response, including the time to review
instructions, search existing data resources, gather the data needed, and
complete and review the information collected.
Introduction
Hello, my name is _________, and I am with Health Systems Research, Inc (HSR). The U.S. Department of Health and Human Services' Administration for Children and Families (ACF) has contracted with HSR to conduct an implementation and outcome evaluation of State participation in the Public Assistance Reporting Information System (PARIS).
This evaluation will determine the effectiveness of the program in terms of costs and benefits as well as develop and utilize methodologies to measure the impact on reducing improper payments. Data will be collected to determine factors affecting:
- Program participation
- Relevant PARIS administrative and implementation information
- Challenges in implementation
- Cost of program participation
- Estimated savings through identified and resolved participant matches
To conduct this study, HSR has drawn a sample of both participating and nonparticipating States to be studied. The evaluation will consist of surveys, interviews, and document reviews at different levels of participation.
As you know, we are talking to you because of your State's participation in this program. As part of this process, ACF has requested that HSR administer surveys with key staff from your agency involved in the PARIS project. The information obtained from this evaluation will inform future programmatic decisions at ACF.
Any answers you provide for this study will be kept confidential in the sense that your name will not be identified with any answers you provide. However, reports of such items as best practices or examples of key issues may include the name of the State in which the practice or issue was found.
The estimated amount of time required to complete this interview is one to one and one- half hours. Thank you for taking the time today to talk with me.
Before we begin, do you have any questions?
I. Background Information on Costs
In order to identify the costs of implementing and maintaining the PARIS project, we want to ask a series of questions related to specific cost categories we have identified as being a part of PARIS. However, in order to provide the proper context for asking these questions, we must ask some preliminary questions regarding how PARIS activities are funded, how costs are reported and recorded, and documentation you might have related to PARIS costs. In particular, we are interested in costs associated with central office activities related to submission of the PARIS data and those costs associated with follow-up, whether central office or local. I will be referring to the cost reporting document you completed and submitted back to us. If you are ready, I will proceed with these background questions:
For central office activities, how is the PARIS project funded in your State? Does it have its own separate budget in the department, or is it incorporated into the budget of a larger division, section, or other sub-unit of government? Does it have its own separate charge codes? If not, how are costs coded for reporting purposes?
Where does funding come from to support PARIS? Does it come from the programs involved in the match (Medicaid, TANF, and/or Food Stamps)? If so, is the money allocated as a direct support item for each program, or does each program pay a percentage of the costs? If the programs themselves are not charged directly, what is the source of funds?
Does PARIS have a set number of its own FTEs assigned to the project, or are the PARIS FTEs combined with other FTEs for the office/division/section?
Do you receive any periodic (monthly or quarterly) cost reporting statements from your accounting office documenting expenses for the PARIS project? If so, how helpful are these statements? If not, would such reports be helpful?
Do you have any system to budget and track the costs for PARIS match follow-up? If yes, does the system charge the PARIS program specifically for costs associated with follow-up, or are costs for all types of client case management follow-up combined into a single budget?
Does your cost accounting system record follow-up costs as specific charges to the PARIS program? If so, are there particular cost codes used to identify activities by tasks? Do you track both personnel expenses as well as other direct cost expenses such as communications with other States?
If you have PARIS specific financial reports, can we obtain copies of any PARIS financial reports for the period January 1, 2006 through June 30, 2006?
II. Budget and Expenses for the PARIS Project
In this section, we will ask specific questions regarding the amount of funds budgeted and expended on the PARIS project. We will be asking questions related to staffing costs as well as other direct costs. We have sent you copies of cost-reporting forms for you to complete, and we will walk through these forms as we ask each question. You can submit these forms back to us once you complete them, but we would like them within two weeks of this call if possible. Do you have any questions about this process?
For staffing costs, we are interested in both salary and benefits. However, if you have a single benefit rate for all staff, you can provide that information separately from the salary costs, and we will calculate the total costs. If you have vacant positions, please provide the monthly salary at the mid-step civil service rate and indicate how long the position has been vacant. Do you have any questions? If not, lets get started.
We are interested in the cost incurred by your state to reach the point where you begin the implementation of the PARIS project, which would normally be associate with one-time costs. In particular, we are interested in learning if specific costs associated with implementation of PARIS were documented by your agency, and what those costs were. We have developed four specific areas in which we are interested in obtaining cost information. I will walk through each of these with you, and ask if these costs were recorded/documented, and what these costs were:
The cost of conducting or producing any needs assessment, systems requirement documentation, advanced planning documents, budget change proposals to the department, legislation, or other planning activities.
The cost associated with purchasing any new computer equipment, software development, or systems design work.
The cost associated with developing policies and procedures, including protocols for matching data and follow-up protocols?
Cost of training staff on PARIS activities?
We will now move to the ongoing cost of administering the PARIS Project. For ongoing costs, we have again developed cost categories for different components of the project. However, if your costs reporting system does not break down specific costs as PARIS costs, then we would like your best estimate as to the percentage of the actual costs that could be attributed to PARIS. I will be asking for quarterly costs from the most recently completed quarter, but if you only have them monthly or for some other time period, please provide the information for that time period.
What are the quarterly staff costs for the last complete quarter for administering the PARIS project at the headquarters level? Do these costs include any state-level follow-up costs?
What other direct costs were incurred by the PARIS project for the quarter? Can you break these down to specific cost centers, such as communications, travel, training, etc, or are they grouped?
Are costs for creating the data files at the program level accounted for as a PARIS cost, or are they associated with the specific programs' data processing costs? If they are broken out as PARIS costs, are they included as part of the numbers asked for in questions 2 and 3? If not, what are these costs?
What are the costs associated with the follow-up of PARIS matches? Have you calculated a "cost-per-case" amount? If so, what is that amount, and has it changed over time? If not, can you provide the costs of follow-up activities and the number of matches submitted for follow-up for the last quarter?
Are there costs that are incurred but not associated with PARIS that you feel should be accounted for as PARIS costs? If so, what are these costs? Can you assign a dollar value for these costs for the last quarter?
III. Calculating the Benefits of PARIS
We are interested in calculating the benefits of the PARIS project in terms of both cost savings and cost avoidance to the programs involved. For purpose of this study, we are defining a cost savings and cost avoidance as follows:
a. Cost Savings and Cost Avoidance are savings directly attributable to removal of a client from the public roles in the Medicaid, TANF, or Food Stamp program. The savings includes the direct cost of the benefit being provided and utilized by the client, as well as the savings in staff time from not having the client on the active role. In addition, costs savings/avoidance includes the dollar amount of a benefit level reduction as the result of information provided on the PARIS match, or moving costs associated with medical care, long-term care, or prescription drug costs from the Medicaid program to the Veterans Administration.
b. Cost Recovery is the amount of money recovered from program participants or providers as a result of an improper payment for services or benefits.
If there are other benefits to your state that can be directly attributed to PARIS and can be quantified as a dollar savings, we would also like to hear about these as well, such as reductions in the number of re-certifications for programs or reduced time following up on clients as the result of having matched data.
We will be asking questions related to cost savings/cost avoidance and cost recovery.
Have you developed any cost-benefit calculation systems to document the cost savings or cost avoidance associated with the PARIS project? If so, what types of cost-benefit or cost-saving/avoidance calculations do you do? What are the factors that you include? Where does the information for costs savings come from?
For what period of time are the savings calculated?
Did you consider and reject other methodologies? Why were these rejected?
What percentage of the total PARIS matches result in a client being dropped from a program after each quarterly match? Can you provide me with the total number of matches that were assigned to follow-up during the last quarter for each of the three programs? How many of those matches resulted in a client being removed from the program? How many resulted in a benefit reduction? How many Medicaid matches with VA resulted in a cost shift to the VA program?
Interviewer to complete:
| Program | Matches in Last Quarter | Number of Clients Dropped from the Program | Number of Clients for Which Benefits Were Reduced | Number of Clients Shifted from Medicaid to VA |
|---|---|---|---|---|
| Medicaid | ||||
| Food Stamps | XXXXXXX | |||
| TANF | XXXXXXX |
Have you calculated the average benefit amount for clients that are removed from each of the programs? If so, is this done on an individual basis, or using an average benefit amount per client? What is the average amount for each program?
Medicaid Costs Associated with Capitation Rate __________
Medicaid Cost Associated with Primary Care Case Management __________
Medicaid Costs Associated with Fee For Service __________
Food Stamp Monthly Benefit __________
TANF Monthly Benefit __________
Does your system of follow up identify if program benefits were actually used by clients? If so, for what time period to you track these benefits (certification period, monthly, annual, etc). Can you provide us with these data for the past quarter?
Do you calculated a benefit reduction amount for each case, or do you average the benefit reduction amount over the entire caseload for which benefits were reduced? What is the average dollar amount of benefit reduction for each program?
Medicaid __________
Food Stamp __________
TANF __________
Do you calculate the per-client cost of staff that are administering an individual case for each or any of the three programs? If so, what is that cost? How was it calculated?
Do you have information on the amount of time remaining on a certification period for clients removed from the program? If so, what was the average amount of time remaining once a client is removed? Does this vary by program?
Medicaid __________
Food Stamp __________
TANF __________
Do you calculate the average monthly cost of Medical care, long-term care, and prescription drug benefits for clients moved from Medicaid to VA? If so, can you provide us with these amounts for the past quarter?
Have you initiated any cost recovery efforts as a result of PARIS match follow-up? If so, which organization is responsible for claims issuance and recovery?
What is the dollar value of claims issued for recovery in the last two quarters? What percent of these claims have actually been recovered?
Are there other costs that you identify that should be included in any benefit analysis? If so, what are these costs?
Thank you for your time.
APPENDIX B.5: PROGRAM STAFF PROTOCOL
Public Assistance Reporting Information System (PARIS) | Program Evaluation | Interview Protocol | PARIS Program Staff | Used in All Participating States
OMB Clearance Number: 0970-0312
Estimates of Burden for the Collection of Information. According to the Paperwork Reduction Act of 1995, no persons
are required to respond to a collection of information unless it displays a
valid OMB control number. The valid OMB control number for this collection is
0970-0312. The time required to complete this information collection is
estimated to average 90 minutes per response, including the time to review
instructions, search existing data resources, gather the data needed, and
complete and review the information collected.
Introduction
Hello, my name is _________, and I am with Health Systems Research, Inc (HSR). The U.S. Department of Health and Human Services' Administration for Children and Families (ACF) has contracted with HSR to conduct an implementation and outcome evaluation of State participation in the Public Assistance Reporting Information System (PARIS).
This evaluation will determine the effectiveness of the program in terms of costs and benefits as well as develop and utilize methodologies to measure the impact on reducing improper payments. Data will be collected to determine factors affecting:
- Program participation
- Relevant PARIS administrative and implementation information
- Challenges in implementation
- Cost of program participation
- Estimated savings through identified and resolved participant matches
To conduct this study, HSR has drawn a sample of both participating and nonparticipating States to be studied. The evaluation will consist of surveys, interviews, and document reviews at different levels of participation.
As you know, we are talking to you because of your State's participation in this program. As part of this process, ACF has requested that HSR administer surveys with key staff from your agency involved in the PARIS project. The information obtained from this evaluation will inform future programmatic decisions at ACF.
Any answers you provide for this study will be kept confidential in the sense that your name will not be identified with any answers you provide. However, reports of such items as best practices or examples of key issues may include the name of the State in which the practice or issue was found.
The estimated amount of time required to complete this interview is one to one and one- half hours. Thank you for taking the time today to talk with me.
Before we begin, do you have any questions?
First, I want to ask some background questions regarding your role in the PARIS project, and how your State became involved.
I. Background
In which program are you currently working? (Medicaid, TANF, Food Stamp). Can you please describe the duties of your current position?
Please describe how you came to be involved in the PARIS project? Were you involved in the initial discussions regarding PARIS participation? If so, what was your role?
Can you please describe your current role with the PARIS project? How much time do you normally spend each quarter on PARIS-related activities?
II. Factors Affecting Participation
How did you hear about the PARIS program? Whose decision was it to pursue participation on behalf of the (Medicaid, TANF, Food Stamp) program? Did you work with individuals from the other two programs to make the decision to pursue PARIS participation? If so, how well did that work?
What process was undertaken to make the decision to participate? Who was involved? What approvals, if any, were required?
One you decided to pursue PARIS participation, what was the timeline for the decision and approval process? What, if any, stumbling blocks did you encounter in obtaining the decision? What parts of the approval process worked well?
Did you have any conversations with federal program officials in (CMS, ACF, USDA) prior to your making the decision to participate? Were these discussions helpful?
Prior to making the decision to participate, did you talk with any other program administrators in other States regarding PARIS? If so, were these discussions helpful?
In your opinion, what was the key factor in making the decision to participate in PARIS?
III. Administration of PARIS
How does the organizational location of PARIS affect your working relationship with the program administrators? (Decision-making? Dissemination of information to Medicaid, TANF, and FSP staff? Implementation of post-match activities?)
Have communications between your office and the PARIS administrative unit been acceptable? If not, why? What, if any, steps have been taken since initial participation in PARIS to enhance communication?
What are the organizational advantages of administering PARIS from its current office? What are the organizational disadvantages or barriers?
What policies and procedures are in place to guide the implementation and administration of the PARIS program in terms of interagency collaboration?
Are there any interagency/division workgroups in place to advise on the administration of PARIS and the follow-up of any matches? Who attends the meetings? How often are they held? What topics have been on the agendas?
How is information disseminated about PARIS to your program staff at the State level? At the local level? (Changes in requirements, rules, and deadlines)? How effective is this communication?
Do you have any communication with PARIS staff at ACF and DMDC? If yes, how effective is this communication? What types of issues are usually discussed?
What technical assistance could the PARIS project have provided prior to or during implementation? Are there any technical assistance needs that you have now? If yes, what type of assistance?
How do you, at the program level, receive information about PARIS policies, procedures and issues? How effective is this communication? What other information would be useful to you?
IV. Submission of Data
Are you involved at all with selecting the criteria that are currently used for submitting Social Security numbers (SSNs) for the PARIS match? If so, what is your involvement?
Who has the final decision on what criteria should be used for submitting SSNs?
Have the criteria evolved/changed over time? If so, what were the original criteria? When and why were any of the changes implemented?
What other criteria could have been used? Why were they excluded?
Are you aware of any filters applied to the data prior to submitting to DMDC? If yes, do you feel that these filters are appropriate?
(If eligibility determination is not integrated) What is the process by which the PARIS Coordinator secures data from your program to submit to PARIS
- What are the different steps or activities?
- Who conducts each of the different activities?
- How long do each of the activities take?
- How long does the process take from start to finish?
- Are the programs given deadlines for submission?
- Are the programs required to clean data before submitting to the PARIS coordinator?
Does your program have to undertake any activities related to supplying data for the quarterly submission to the PARIS administrator?
- If yes, What are the different steps or activities?
- Who conducts each of the different activities? Are local offices involved?
- How long do each of the activities take?
- How long does the entire process take?
Are you aware of any problems your State has had submitting data to PARIS for matching? What have you been told regarding your submissions to the PARIS coordinator? Have you been asked to change your submission format? Have you been able to do so? Why or why not?
What would make it easier to meet the PARIS deadline?
V. Follow-up
Of the three types of PARIS matches (Interstate, VA, Federal), which do you see as the most beneficial to your agency? Why?
If your program receives match data, how long does it take for you to receive match data back from the PARIS coordinator? What turnaround time on receiving match data would be optimal to make the most use of match data?
What else could your State PARIS administrators do to enhance the usability of the match data?
Do you know what the typical number and percentage of matches returned for your program are each quarter? What percentage of these matches are pursued?
Are you involved in any decisions as to how to prioritize match data? What is your involvement? Has this level of involvement changed in the last year?
Are you involved in any oversight or quality control role in reporting back to the PARIS administrator about the results of any follow-up?
- Is there a specified timeframe in which you have to report back?
- What do you report (i.e. number of follow-ups, results of follow-up, dollars saved as a result of follow-up, dollars recouped as a result of follow-up)?
For your program:
- Overall, what proportion of matched cases receive follow-up?
- What proportion of matched cases that do receive follow-up are actually fraudulent?
- Are there significant differences across the different programs in follow-up and in results?
For your program:
- Once matched cases are returned to the program, how is follow-up conducted? (at the State level or at the local offices?)
- Are matches prioritized for follow-up?
- How many staff are involved in follow-up efforts?
- Do they receive training on follow-up procedures?
- Do they have a time limit in which to conduct the follow-up?
- Are staff required to report on the results of follow-up?
- On average, how long does it take to follow-up on a matched case?
- What is the rate of follow-up for matched cases?
What proportion of the cases that do receive follow-up are actually fraudulent?
Are you involved at all in the calculation of program benefits? If so, what is your role? What do you see as the greatest benefits of PARIS matching?
(For Medicaid Administrators only) Do you follow-up on VA matches to determine third party liability for such costs as medical and long-term care or prescription drug benefits? If so, what is the process by which you pursue reimbursement from VA? Have you calculated any cost savings from this activity? If so, what are these savings?
(For Food Stamp Program Administrators only) Have you had any problems justifying your participation in PARIS because of food benefits being 100% federal funds? If yes, how have you overcome these problems?
Are you involved at all in any recovery activities for benefits determined to be provided improperly? If so, what is your role (probe: policy, operations, etc.)
Are you involved in referrals for any activity thought to be fraudulent so, what is your role?
Thank you for your time.