Skip Navigation
acfbanner  
ACF
Department of Health and Human Services Administration for Children and Families
          
ACF Home   |   Services   |   Working with ACF   |   Policy/Planning   |   About ACF   |   ACF News   |   HHS Home

  Questions?  |  Privacy  |  Site Index  |  Contact Us  |  Download Reader™Download Reader  |  Print Print      


The Child Care Bureau   Advanced
Search

Measuring Improper Authorizations for Payment in the Child Care Program:
Improper Authorization for Payment Data Collection Instructions [Approved August 31, 2007]

Download editable Word document (1.66 MB) or view in PDF (766KB) in PDF format.

IV. GENERATING A SAMPLE OF CASES FOR REVIEW

States select a statewide random sample of at least 271 active cases. A sample of 271 cases is projected to achieve a State level improper authorizations for payment rate within five percent precision at the 90% confidence level.[2] This Section details steps for the selection process. The steps include three areas in which States have flexibility and which are discussed in more detail below: the decision to sample 271 or 276 cases, how frequently monthly sampling frames will be collected, and the source for the required random numbers. As indicated in Step 1, Exhibit 1, States are required to submit these decisions by email to the Child Care Manager in the ACF Regional Office before the end of October of the calendar year prior to the reporting year.

Review Period

The CCB has designated a 12 month review period, based on the Federal Fiscal Year ending September 30, for the data collection methodology. The purpose of the 12 month review period is to gain a representative estimate of the annualized amount of improper authorizations for payments.

The State estimates of the five error measures will be calculated using all cases examined during the review period. States which choose to sample 271 cases will randomly select 23 cases for each of the first seven months and 22 cases for each of the last five months of the 12 month review period (e.g., October - April, 23 cases per month will be selected and May - September 22 cases per month will be selected). Alternatively States may choose to select 23 cases for each month of the review period, yielding a total of 276 cases. Adding five cases to the annual required total allows States to select the same number of cases for each of the 12 review months.

Sampling Unit

The primary sampling unit for this analysis is an active case (child) for whom a child care subsidy was authorized for payment during a sample review month. An authorization for payment is the subsidy amount authorized during eligibility determination or redetermination for a sample review month, based on the case record documentation.

Each sampling unit will be identified by the following information:

  1. Sequential number;
  2. Child ID;
  3. County of service; and
  4. Sample month.

States determine their own parameters for creating unique Child IDs, adhering to the following criteria when creating the Child IDs:

  • Each child in the sampling frame receives a unique Child ID. If several children exist within a family, case or household, each child will be assigned a unique Child ID.
  • The unique Child ID does not contain identifying information; but rather it is linked to a county or State data system, so that the county or State can pull the case record if the child were selected for the study.

Selecting the Monthly Sample

States create 12 sampling frames of active cases (i.e., one sampling frame per month for each month in the 12 month review period). The review period is the most recent Federal Fiscal Year ending prior to the submission date for the State Improper Authorizations for Payment Report. For example, for a June 30, 2008 submission date, the review period would be October 1, 2006 through September 30, 2007. For a June 30, 2009 submission date, the review period would be October 1, 2007 through September 30, 2008.

Once each sampling frame has been created, States randomly select the monthly sample of 22 or 23 cases (271 (or 276) cases during the review period). Once this process is complete, States randomly select 3 replacement cases for each sampling frame. States may choose to select more than 3 replacement cases for each sampling frame. States use a replacement case only if a case selected does not meet the study criteria for valid reasons. Examples of valid reasons include: natural disaster making the case record unavailable, the case has been referred to a State's fraud investigation unit, or the case is under appeal. Exhibit 3 provides instructions for each of the steps and includes examples using timeframes applicable for the second implementation year. If a State chooses to select more than 3 replacement cases for each sampling frame, the State will replace "(3)" in steps 5, 6, and 7 by whatever larger number of replacement cases the State has chosen to select.

 

Exhibit 3. Selecting the Monthly Sample
Steps Instructions Examples

1. Determine frequency of monthly sample selection
  • States create a sampling frame for each of the 12 months in the review period.
  • States determine how frequently they will create sampling frames.
  • States may choose to create these lists at the end of each month, quarterly, semi-annually, or annually.
  • The choice will be based on how often States choose to pull records and conduct reviews.

Example one (Select 6 monthly sampling frames, two times per year):
For a June 30, 2009 submission date, select 6 monthly sampling frames for the six months. October 2007 – March 2008 in May 2008. Then select the remaining 6 monthly sampling frames for April – September 2008 in November 2008
Example two (Select 12 monthly sampling frames, once per year):
For a June 30, 2009 submission date, select 12 monthly sampling frames for October 2007 – September 2008 in November 2008.

2. Create 12 Monthly Sampling Frames

For each month of the 12 month review period, generate a list of all cases authorized to receive a child care payment during that review month:

  • Sort the list by county & caseload size, listing counties with the largest caseload first to counties with the smallest caseload.
  • List all Child ID numbers sequentially, within each county from smallest to largest.

For each month of the 12 month review period, generate a list of all cases authorized to receive a child care payment during that review month:

  • Sort the list by county caseload size: List counties with the largest caseload first: e.g., County K (2615 cases), County R (995 cases), County W (971 cases), County M (848 cases)

  • List Child ID numbers from lowest to highest: e.g.,233124, 233128, 255320.

3. Calculate Sampling Interval to Select Sample Cases.
  • Calculate a sampling interval based on the size of the sampling frame and the number of cases (23 or 22) to be selected for the sample.
  • Select either 1) 23 cases for seven months and 22 cases for 5 months to yield 271 cases, or 2) 23 cases for each of twelve months to yield 276 cases.
  • Divide the total number of cases listed in the sampling frame (all cases authorized to receive payment during the sample month) by the number of cases to be selected for the sample (23 or 22).
  • When the interval computed is not a whole number, round the end of the first interval to the nearest whole number. If the result is x.50 or more, round up to select the nearest whole number (i.e. x+1). If the result is x.49 or less, round down to select the nearest whole number (i.e. x).
  • Assume 31,286 cases in a sample month.
  • Divide the total number of cases authorized to receive payment during the sample month (31,286) cases, by the number of cases to be selected for the sample (23).
  • Calculate the sampling interval by dividing 31,286 cases in the sample month by 23, the number of cases to be selected in the sample. (31,286/23 = 1360.3)
  • If the sampling interval is a whole number, such as 1360, select every 1360th case from a random starting point defined below.
  • If the sampling interval is not a whole number, e.g., for 1360.3, round down to randomly select a starting case between the 1st and 1360th case or for 1360.6, round up to select a starting case between case 1 and case 1361.

4. Select Sample Cases

  • Using a random number table or software, select a random number within the first sampling interval.
  • Use the random number as the starting point to select the first case in the sample.
  • From the first case, use the sampling interval to select the next case that falls within the second sampling interval.
  • Continue to add the sampling interval to select the next cases until the sample is complete at 23 or 22 cases.
  • When the interval computed is not a whole number, continue to add the exact interval amounts together without rounding to determine each interval from which to round the result to select the next case.
  • If the sampling interval is 1360.3, use a random number table or software to select a random number that falls within the first sampling interval of 1 –1360 (rounded down).
  • If the random number selected is 463, select case 463 as the first case in the sample.
  • If the sampling interval is 1360.3, then select the next case that falls within the second sampling interval, or the case listed 1823 (463+1360.3=1823.3, which rounds to 1823).
  • The next cases selected would be 3184 (1823.3+1360.3=3183.6, which rounds to 3184). Continue until the sample is complete and 23 or 22 cases have been selected.

5. Calculate Sampling Interval to Select Re-Placement Cases
  • After the sample of 23 or 22 cases has been selected, remove the 23 or 22 cases from the sampling frame.
  • Calculate a sampling interval based on the size of the new sampling frame and the number of cases (3)3 to be selected as replacement cases.
  • Divide the total number of cases listed in the sampling frame (after removing the 23 or 22 sample cases) by the number of cases to be selected as replacements. (3).
  • Remove the 23 cases from the total number of cases authorized to receive payment during the sample month. (31,286 – 23 = 31,263)
  • To calculate the sampling interval, divide the remaining total number of cases in the sample month (31,263), by the number of cases to be selected as the replacement cases (3). (31,263/3 = 10,421)
  • If the sampling interval is 10,421, select every 10,421st case from a random starting point defined below.

6. Select Re-placement Cases
  • Using a random number table or software, select a random number within the first replacement sampling interval.
  • Use the random number as the starting point to select the first case in the sample.
  • From the first case, select the next case that falls within the sampling interval.
  • Continue to use the sampling interval to select the next cases until 3 replacement cases have been selected.
  • If the sampling interval is 10,421, use a random number table or software to select a random number that falls within the first replacement sampling interval of 1 – 10,421.
  • If the random number selected is 10, select the tenth case as the first case in the sample.
  • If the sampling interval is 10,421, then select the next two cases that fall within the sampling interval, or the cases listed 10,431st and 20,852nd.

7. Create additional monthly sampling frames for each month of the sampling period.
  • Based on the frequency of monthly sample selection, draw additional sampling frames for each month of the 1, 3, 6, or 12 month sampling period. For each month of the sampling period, repeat steps 2-6, found in column one above.
Example one (Select 6 monthly sampling frames, 2 times per year):
  • If the 6 month sampling period is October 2007 – March 2008, in May 2008, generate one sampling frame per month, starting with the month of October 2007 repeating steps 2 – 6 for each month through March 2008.
  • In November 2008, generate one sampling frame per month, starting with the month of March 2008, repeating steps 2 – 6 for each month through September 2008.
Example two (Select 12 monthly sampling frames, once per year):
  • If the 12 month sampling period is October 2006 – September 2007, generate one sampling frame per month, starting with the month of October 2006 repeating steps 2 – 6 for each month.

 

States then send the original sample list, including the unique child ID and county of service, to the person/office designated to receive them. The designated person/office is responsible for making sure case records are pulled according to the States’ agreed upon review schedule and assigning the case records for review.

2To illustrate the use of confidence limits, two assumptions must be made: (1) the eligibility error rate variable is the percent of clients who exhibit an eligibility error during the review month; and (2) 271 cases are sampled and 50% show an eligibility error (this is the most conservative assumption about the error rate in terms of yielding the largest confidence interval). The confidence limits are calculated as follows: [m = 1.645(SQRT(p(1-p)/n))], where m is the 90% confidence limit (or interval), SQRT is the square root, n is the sample size, and p is the proportion of the sample found to be in error. Following the assumptions, the confidence interval would be calculated as [m = 1.645(SQRT(.5(.5)/100) = .05 (or 5%)]. The 90% confidence interval can be described in two ways: (1) with 90% confidence, the error rate for this sampled population is between 45% and 55%; or (2) with 90% confidence, 50% of the population is in error with a margin of error for this estimate of +/- 5%. Selecting at least 271 cases assures that the confidence limits are +/- 5% or less.

3 If a State chooses to select more than 3 replacement cases for each sampling frame, the State will replace "(3)" in steps 5, 6, and 7 by whatever larger number of replacement cases the State has chosen to select.

V. CUSTOMIZING THE RECORD REVIEW WORKSHEET: >>