Search Type Title Author/Investigator Search Advanced Search

Table of Contents | Previous | Next

Appendix C: Telephone Survey of Frontline Workers: Methodology And Results

This appendix provides detail on the methodology used to select and contact participants in for the telephone survey of frontline workers included in this study. It begins with a description of the purpose and goals of the survey, continues with a description of how the survey was conducted, and concludes by presenting tabulated staff responses to the survey.

The primary goal of the survey of front-line staff was to supplement the case study and administrative data with input from as many case managers and eligibility workers as possible on (1) their perceptions of the role of sanctions, (2) their understanding of sanction policies, (2) their use of sanctions to encourage program participation, and (4) methods of implementing sanctions. We conducted a survey to accomplish this goal for two reasons. First, we did not anticipate that case study interviews alone would be sufficient to provide the relevant information, as interviews were conducted with only a subset of front-line workers involved in the sanction process. And second, the type and amount of administrative data available for analysis in each site is not consistent across sites.

Sample Design

The sampling frame for the survey consisted of 186 workers across eight study sites. In visiting each of these sites, members of the study team collected names, titles, and contact information for all case managers, eligibility staff, and other front-line staff with responsibilities in the sanction process in an effort to draw the most representative picture of implementation of sanctioning practices in each site. When sites had more workers than we planned to interview, we selected a random sample to interview.

Survey Instrument and Pretest

MPR developed the survey instrument in early 2006 in consultation with the Administration for Children and Families. The survey was designed to last approximately 20 to 25 minutes and to be administered by telephone with interviewers recording responses on hard copies of the questionnaire. The majority of questions were closed-ended to minimize the time required to complete the survey and for ease of coding and analysis. Pretests of the questionnaire completed with nine front-line staff persons (four eligibility workers and five case managers) in Washington DC helped to ensure the clarity and purpose of individual questions specifically and the flow and sequencing of the questionnaire more generally.

Survey Data Collection

The survey was fielded over a period of three and a half months, from early November 2006 through mid-February 2007. We began conducting interviews in each site after the completion of a visit to the site by a study team member. After the completion of a site visit and the collection of contact information by the study team, potential respondents were alerted by the MPR on-site contact that they would be contacted by phone for a brief survey.

MPR interviewers contacted frontline staff using the contact information provided by site visitors. Respondents were informed upon contact that their responses would be confidential and would not affect their employment. The names and contact information for 186 staff were submitted to the MPR Survey Operations Center for contact by trained MPR telephone interviewers.  Among this number, 10 were determined to be ineligible based on their responses to some initial screening questions, leaving a sample size of 176 respondents.  The total sample size for each site, as well as the response rate, is reported below in Table C.1.

 

Table C.1. Sample Size and Response Rate for Telephone Survey of Frontline Workers

	Total Sample Size	Nonrespondentsa	Completed Interviews	Response Rate (%)
Suffolk County, NY	29	1	28	96.6
Pima County, AZ	9	0	9	100.0
Duval County, FL	16	1	15	93.8
DeKalb County, GA	29	3	26	89.7
Tarrant County, TX	12	1	11	91.7
Los Angeles County, CA	24	3	21	87.5
Kern County, CA	30	5	25	82.8
Salt Lake County, UT	27	1	26	96.3
Total	176	15	161	91.5

a Nonrespondents include those who chose not to participate in the survey and those who were unavailable during the survey fielding period.

 

Survey Results

To answer the research questions of interest, we completed simple descriptive analyses of survey data in six areas using SAS and Microsoft Excel: (1) contextual or background information; (2) staff knowledge and communication; (3) decision making; (4) implementation; (5) re-engagement; and (6) rates and reasons for sanctions.  Information from tables in these six areas contributed to the analyses presented in the chapters of this report.  The remainder of this appendix is a compendium of analysis tables referenced in the report text, and some additional tables.

 

Table C.2. Staff Roles and Responsibilities

		Percentage
		Among All Staff	Among Case Managers	Among Eligibility Workers	Among Other Staff
Length of time in current position	Less than one year	15.5	13.7	13.3	27.3
	1-2 years	26.7	27.4	20.0	27.3
	3-5 years	23.0	23.4	6.7	31.8
	6-10 years	19.9	25.0	0.0	4.5
	More than 10 years	11.8	7.3	53.3	9.1
Percentage who are personally responsible for	Deciding who is required to meet work requirements	62.7	67.7	73.3	27.3
	Determining what clients must do to meet their work requirements	68.9	82.3	26.7	22.7
	Monitoring a client's participation in work or work-related activities	67.1	82.3	20.0	13.6
	Deciding/recommending whether clients should be sanctioned	68.9	78.2	33.3	40.9
	Carrying out procedures to impose a sanction	73.9	72.6	80.0	77.3
	Contacting sanctioned clients to try to re-engage them in program activities	54.7	61.3	40.0	27.3
Sample size		161	124	15	22

Source: MPR survey of frontline workers.

 

Table C.3. Caseload Size and Composition

	Among All Staff		Among Case Managers		Among Eligibility Workers
	Average	Median	Average	Median	Average	Median
Caseload size	107.0	70.0	91.9	60.0	130.3	150.0
Number of TANF clients exempt from work requirements	31.7	5.0	30.3	4.0	41.2	17.5
Percent of TANF clients exempt from work requirements	17.4	10.0	16.1	7.3	26.8	12.4
Number of clients sanctioned for failing to meet work requirements in the last month	7.4	3.0	5.0	3.0	13.5	4.0
Percentage of clients sanctioned for failing to meet work requirements in the last month	11.6	6.0	10.5	6.1	9.7	2.7
Sample Size	161		124		15

Source: MPR survey of frontline workers. Note: Averages calculated among staff who have a caseload.

 

Table C.4. Knowledge of Exemption Policies

	Percentage of Program Staff
	In Sites Where Clients Are Exempt		In Sites Where Clients Are Not Exempt
	Said Exempt	Said Not Exempt	Said Exempt	Said Not Exempt
Clients with very young children	67.3	30.9	38.7	61.3
Parents working a certain number of hours per week	50.0	49.1	38.7	54.8
Clients with disabilities or certain physical health conditions	89.1	9.1	87.1	12.9
Clients caring for family members with disabilities	90.0	8.2	96.8	3.2
Clients who are beyond a certain point in pregnancy	87.3	11.8	35.5	61.3
Clients with domestic abuse issues	80.0	14.5	54.8	35.5
Clients with severe personal and family challenges	78.2	18.2	67.7	29.0
Sample size	141

Source: MPR survey of frontline workers.

 

Table C.5. Knowledge of Work Requirements

Accurate or Inaccurate Staff Statements in Survey Responses	Percentage
Said correct number of hours required per week	70.2
Said fewer hours than required per week	13.0
Said more hours than required per week	11.2
Said did not know number of hours required per week	5.6
Sample size	161

Source: MPR survey of frontline workers.

 

Table C.6. Knowledge of Sanction Policies Among Staff Responsible for Providing Information About Sanctions to Clients

Issue Related to Sanction Policies	Percentage of Program Staff
	Generally Knowledgeable a	Not Knowledgeable
The effect of first sanction on TANF grant	90.0	10.0
The effect of second sanction on TANF grant	82.3	17.7
The effect of third sanction on TANF grant	76.2	23.9
Minimum time a first sanction must remain in place	49.2	50.8
Minimum time a second sanction must remain in place	66.9	33.1
Minimum time a third sanction must remain in place	63.1	36.9
Length of time clients must participate in activities to end a sanction and receive full benefits again	60.8	39.2
Sample size	130

Source: MPR survey of frontline workers. a Staff who are “generally knowledgeable” understand the policy completely or understand the basic features of the policy if not the details (for example, know that a first sanction reduces the TANF grant by a certain percentage, but do not know what the correct percentage is; or know that there is a minimum period of time a first sanction must remain in place, but do not know what the correct period of time is). Categorizations are based on responses to questions B4, B4a, E1, E1a, E2, E2a, E3, E4, and E4a in the survey instrument (see Appendix B).

 

Table C.7. Knowledge of Sanction Policies Among All Staff

Issue Related to Sanction Policies	Percentage of Program Staff
	Generally Knowledgeable a	Not Knowledgeable
The effect of first sanction on TANF grant	86.3	13.7
The effect of second sanction on TANF grant	78.3	21.7
The effect of third sanction on TANF grant	70.8	29.2
Minimum time a first sanction must remain in place	53.4	46.6
Minimum time a second sanction must remain in place	68.3	31.7
Minimum time a third sanction must remain in place	60.3	39.8
Length of time clients must participate in activities to end a sanction and receive full benefits again	55.3	44.7
Sample size	161

Source: MPR survey of frontline workers. a Staff who are “generally knowledgeable” understand the policy completely or understand the basic features of the policy if not the details (for example, know that a first sanction reduces the TANF grant by a certain percentage, but do not know what the correct percentage is; or know that there is a minimum period of time a first sanction must remain in place, but do not know what the correct period of time is). Categorizations are based on responses to questions B4, B4a, E1, E1a, E2, E2a, E3, E4, and E4a in the survey instrument (see Appendix B).

 

Table C.8. Provision of Information to Clients About Sanction Policies

		Percentage or Average
Percentage of staff who provide information to clients		80.7
	During application process for TANF	41.6
	During work program orientation	62.7
	During assessment	53.4
	During development of employment or individual responsibility plans	60.2
	During work activities	55.3
	During case management meetings or phone calls with clients	72.7
	During home visits	34.2
	During re-certification	35.4
	At noncompliance or after sanction	8.7
	Ongoing	4.3
	Other	7.5
Average number of venues in which staff provide information to clients		4.4
Percentage of staff who never provide information to clients personally		19.3
Percentage of staff who provide information to most clients	Verbally	74.5
	In writing	71.4
	On video tape	5.0
Sample size		161

Source: MPR survey of frontline workers.

 

Table C.9. Personal Challenges Affecting Participation

		Percentage
Percentage of Program Staff Reporting Challenge as One of Two Most Common Challenges Clients Face	Child care problems	42.3
	Motivational issues	36.9
	Transportation problems	29.7
	Mental health issues	16.2
	Physical health issues	16.2
	Homelessness or housing problems	6.3
	Lack of information about program	5.4
	Domestic abuse issues	4.5
	Child behavioral problems	4.5
	Poor education	4.5
	Substance abuse problems	3.6
	Lack of soft skills	3.6
Percentage of staff who believe their agency offers sufficient support services to clients with the problems above	Child care problems	40.5
	Motivational issues	33.3
	Transportation problems	22.5
	Mental health issues	12.6
	Physical health issues	12.6
	Homelessness or housing problems	2.7
	Lack of information about program	3.6
	Domestic abuse issues	4.5
	Child behavioral problems	3.6
	Poor education	4.5
	Substance abuse problems	2.7
	Lack of soft skills	3.6
Sample size		111

Source: MPR survey of frontline workers. Note: Fifty of 161 survey respondents are not responsible for initiating sanctions and were not asked these questions.

 

Table C.10. Personal Discretion in Sanctioning

		Percentage
Percentage of staff with discretion to sanction when clients	Do not attend work program orientation	62
	Do not complete and employment of individual responsibility plan	78
	Do not attend one or more meetings with staff member	75
	Do not respond to sanction notification within a specified period	69
	Participate in required activities but not for enough hours	72
	Participate in required activities for some time, but then stop participating	76
	Do not participate at all in required activities	77
Sample Size		111

Source: MPR survey of frontline workers. Note: Fifty of 161 survey respondents are not responsible for initiating sanctions and were not asked these questions.

 

Table C.11. Personal and Family Challenges Staff Have Ever Considered When Making Sanction Decisions

Reasons for Refraining from Sanctioning a Non-Participating Client	Percent
Mental health issue	75.7
Substance abuse issue	65.8
Physical health issue	78.4
Need to care for a disabled family member	68.5
Domestic abuse issue	72.1
Child care problem	57.7
Transportation problem	48.6
Child behavioral problem	54.1
Homelessness or housing problem	68.5
Legal problem	44.1
Death in the family	1.8
Another reason	5.4
Sample size	111

Source: MPR survey of frontline workers. Note: Fifty of 161 survey respondents are not responsible for initiating sanctions and were not asked this question.

 

Table C.12. Considerations When Deciding Whether to Sanction

		Percentage
Percentage who consider the effect the sanction will have on the family	A lot	23.4
	Somewhat	33.3
	A little	18.0
	Not at all	24.3
Percentage who consider the effort it takes to initiate and carry out a sanction	A lot	26.1
	Somewhat	16.2
	A little	14.4
	Not at all	39.6
Percentage who consider the client's attitude	A lot	18.9
	Somewhat	27.9
	A little	12.6
	Not at all	39.6
Percentage who consider their personal relationship with the client	A lot	24.3
	Somewhat	25.2
	A little	8.1
	Not at all	41.4
Percentage who consider the client's history of participation	A lot	40.5
	Somewhat	34.2
	A little	10.8
	Not at all	13.5
Sample size		111

Source: MPR survey of frontline workers. Note: Fifty of 161 survey respondents are not responsible for initiating sanctions and were not asked these questions.

 

Table C.13. Sources of Information to Make Decisions Around Sanctions

		Percentage
Percentage who rely on written attendance records	A lot	64.0
	Somewhat	24.3
	A little	1.8
	Not at all	9.0
Percentage who think written attendance records are	Very useful	73.9
	Somewhat useful	12.6
	Not useful	2.7
Percentage who rely on computerized participation reports	A lot	42.3
	Somewhat	25.2
	A little	9.0
	Not at all	22.5
Percentage who think computerized participation reports are	Very useful	53.2
	Somewhat useful	20.7
	Not useful	13.5
Percentage who rely on discussions during formal staff meetings or case reviews	A lot	36.0
	Somewhat	31.5
	A little	9.9
	Not at all	20.7
Percentage who think discussions during formal staff meetings or case reviews are	Very useful	35.1
	Somewhat useful	30.6
	Not useful	18.9
Percentage who rely on informal conversations with other staff members	A lot	7.2
	Somewhat	38.7
	A little	19.8
	Not at all	33.3
Percentage who think informal conversations with other staff members are	Very useful	9.9
	Somewhat useful	36.0
	Not useful	36.9
Percentage who rely on personal judgments about the client's circumstances	A lot	34.2
	Somewhat	36.0
	A little	11.7
	Not at all	18.0
Percentage who talk with clients directly about their participation	Daily	23.4
	Weekly	40.5
	Monthly	25.2
	Less than once a month	5.4
	Varies or Depends	4.5
Percentage who do not have enough information about their clients' participation		20.7
Among those who do not have enough information, reasons for lack of information about their clients' participation. a	Don't have time to review information because caseload is too high	4.3
	Don't get attendance or participation reports consistently	17.4
	Information in attendance or participation reports is not useful	13.0
	Staff don't communicate well or at all about client participation	26.1
	Hard to get information from clients	26.1
	Another reason	26.1
Sample Size		111

Source: MPR survey of frontline workers. Note: Fifty of 161 survey respondents are not responsible for initiating sanctions and were not asked these questions. a Category sums to greater than 100 percent because respondents could give multiple answers.

 

Table C.14. Timing of Sanction Decisions

		Percentage
Percentage who believe that sanctions should be used	Soon after detecting nonparticipation as a way of encouraging participation	45.0
	After some time and effort to encourage participation	39.6
	Only as a last resort	14.4
Percentage who are encouraged by supervisors and program administrators to use sanctions	Soon after detecting nonparticipation as a way of encouraging participation	39.6
	After some time and effort to encourage participation	25.2
	Only as a last resort	30.6
Percentage who recommend sanctions, on average	Less than 1 month after detecting nonparticipation	67.6
	1-2 months after detecting nonparticipation	27.9
	3-4 months after detecting nonparticipation	0.9
	More than 4 months after detecting nonparticipation	0.9
Sample size		111

Source: MPR survey of frontline workers. Note: Fifty of 161 survey respondents are not responsible for initiating sanctions and were not asked these questions.

 

Table C.15. Activities Required Before Sanctioning

Activity	Percent of Staff
	Required to Conduct Activity	If Required to Conduct Activity, Frequency of Follow-Througha
		All of the Time	Most of the Time	Half of the Time	Some of the Time	None of the Time
Send a standard sanction notice	62.1	89.0	6.0	1.0	0.0	3.0
Send other letters to clients	49.7	58.8	16.3	15.5	5.0	2.5
Telephone clients	51.6	72.3	21.7	6.0	0.0	0.0
Able to reach clients by phone	--	3.6	19.3	55.4	21.7	0.0
Meet with clients in the office	50.3	67.9	17.3	7.4	3.7	3.7
Able to get clients to attend	--	1.3	25.6	43.6	26.9	0.0
Conduct home visits	37.9b	31.1	16.4	21.3	11.5	9.8
Able to reach clients at home	--	3.3	13.1	31.1	26.2	0.0
Sample size	161

Source:  MPR Survey of frontline workers. a “Most of the time” was defined as 75 percent of the time, and “some of the time” was defined as 25 percent of the time.  For some activities, the frequency does not sum to 100 percent.  The remaining responses were “don’t know.” b Just over 17 percent of staff reported that they themselves were responsible for conducting home visits, and just over 20 percent reported that other staff members were responsible for conducting home visits.

 

Table C.16. Length of Time Required to Impose A Sanction

		Percent of Program Staff
Time it takes staff to make first change in the computer system to impose a sanction	Less than 1 month	74.5
	1-2 months	14.5
	3-4 months	0.9
	More than 4 months	0.0
	Don't know	10.0
Time entire sanction process takes	Less than 1 month	24.5
	1-2 months	92.7
	3-4 months	12.7
	More than 4 months	1.8
	Don't know	14.5
Sample size		110

Source: MPR survey of frontline workers. Note: Fifty of 161 survey respondents are not responsible for initiating sanctions and were not asked these questions.

 

Table C.17. Effort Required to Impose a Sanction

		Percent of Program Staff
Required to provide supervisors with written documentation of efforts to encourage participation before sanctioning		47.8
Among those required, percent who think the documentation process requires	A lot of effort	37.7
	Some effort	41.6
	Not very much effort	16.9
	No effort at all	1.3
	Don't know	2.6
Personally required to change codes in the computer system to impose a sanction		49.1
Among those required, percent who think the process of changing codes is	Very easy	50.6
	Somewhat easy	41.8
	Somewhat difficult	5.1
	Very difficult	1.3
	Don't know	1.3
Required to send a request to someone else to impose a sanction in the computer system		19.3
Among those required, percent who need supervisory approval before making the request		32.3
Frequency with which supervisors approve these requests to sanction	100 percent of the time	60.0
	About 75 percent of the time	20.0
	About 50 percent of the time	0.0
	About 25 percent of the time	0.0
	Never	20.0
Percent who think imposing sanctions is	Very easy	21.7
	Somewhat easy	39.8
	Somewhat difficult	25.5
	Very difficult	7.5
	Don't know	5.6
Among those who think imposing sanctions is difficult, reasons for difficulty	Length of process	20.8
	Bureaucratic hurdles/ required documentation	28.3
	Personal sympathy for client or family	11.3
	Opportunities for conciliation	22.6
	Another reason	28.3
Percentage who have refrained from sanctioning because imposing a sanction was too burdensome	Often	2.5
	Sometimes	6.8
	Seldom	5.0
	Never	82.0
Sample size		161

Source: MPR survey of frontline workers.

 

Table C.18. Re-Engaging Sanctioned Clients

	Among Staff Required to Contact Sanctioned Clients to Re-Engage Them
	A Lot	Some	A Little	None	Don’t Know
Priority that staff place on re-engaging sanctioned clients	53.4	14.8	15.9	11.4	4.5
Staff’s usual telephone contact with sanctioned clients	21.6	37.5	21.6	15.9	3.4
Staff’s usual in-person contact with sanctioned clients	11.4	33.0	27.3	26.1	2.3
Sample size	88

Source: MPR survey of frontline workers.

 

Table C.19. Personal Opinions About Home Visits

		Percentage
Among respondents sites where a home visit is required, percentage who think, in terms of helping to prevent sanctions, home visits are	Very successful	32.8
	Somewhat successful	31.1
	Not very successful	27.9
	Not at all successful	4.9
	Don't know	4.9
Total Sample size		61

Source: MPR survey of frontline workers.

 

Table C.20. Personal Opinions About Sanctions

		Percentage
Percentage who believe sanctions are	Very effective	23.6
	Somewhat effective	49.7
	Not very effective	16.1
	Not at all effective	7.5
Among those who believe sanctions are not effective, reasons why	People become complacent/content with the way things are	39.5
	People don't understand the consequences	5.3
	People decide they don't need cash assistance anymore	13.2
	People's lives are too complicated or they have too many problems to participate regardless of the consequences	5.3
	Policies or consequences not strict enough	28.9
	Another reason	21.1
Sample size		161

Source: MPR survey of frontline workers.

 

Table C.21. Sanction Rates Among Staff With Sanction Responsibility

		Percentage or Average Number of Clients Sanctioned
		Last Week	Last Month
Average among all staff		2.7	8.4
Percentage among all staff	0	45.8	16.8
	1-2	28.0	22.4
	3-5	15.9	21.5
	6-14	3.7	21.5
	15 or more	4.7	13.1
Average by type of staff	Case managers	2.0	6.4
	Eligibility workers	1.2	5.0
Average by caseload size among all staffa	Lower caseload	1.4	5.7
	Typical caseload	3.5	8.6
	High caseload	2.9	10.9
Average by caseload size among case managersa	Lower caseload	1.0	4.4
	Typical caseload	2.7	6.6
	High caseload	1.9	8.2
Average by job tenureb	Short tenure at current job	2.8	8.0
	Longer tenure at current job	2.4	8.2
Average by opinions about effectiveness of sanctions	Believe sanctions are effective	2.5	7.6
	Believe sanctions are not effective	3.1	10.0
Average by opinion about difficulty in imposing sanctions	Believe imposing sanctions is easy	2.5	8.0
	Believe imposing sanctions is difficult	2.9	8.7
Average by opinion regarding timing of sanctions	Tend to sanction soon after detecting nonparticipation	3.1	9.2
	Tend to sanction after some time and effort to encourage participation	2.6	7.8
	Tend to sanction only as a last resort	0.7	4.7
Sample size		107

Source: MPR survey of frontline workers. Note: In some categories, the frequency does not sum to 100 percent. The remaining responses were “don’t know.” a High caseloads are defined as more than 100 cases; lower caseloads are defined as 35 cases or less. Typical caseloads range from 35 to 100 cases... b Shor