2016 Research and Evaluation Conference on Self-Sufficiency (RECS)
25 years of research at ACF
Thanks, Naomi, and thanks to all of you for joining us for the 18th convening of the Research and Evaluation Conference on Self-Sufficiency (RECS) .
I’ve been able to go to most of the prior 17 conferences and in 2018, I’ll look forward to being here again in my personal capacity. But, this will be my last RECS conference as part of ACF leadership, so I appreciate having the opportunity to make a number of points this morning, though there are perhaps a few that I’ll need to hold for 2018.
This year marks the 25th anniversary of the establishment of ACF, and from the beginning, ACF has supported rigorous research and used that research to guide policy. In this Administration, we’ve initiated efforts including but not limited to the Health Profession Opportunity Grant Evaluation, Behavioral Interventions for Advancing Self-Sufficiency, the Subsidized and Transitional Employment Demonstration, our work around TANF-workforce coordination, job search, executive functioning, toxic stress, and more.
A commitment to rigor, relevance, transparency, independence, ethics
In this Administration, we also developed the ACF evaluation policy, which is available on the ACF website and posted in the Federal Register . The evaluation policy highlights our commitment to proceed in accordance with the principles of rigor, relevance, transparency, independence, and ethics. And, while there are a number of aspects of the evaluation policy that I think are important, I want to single out two.
First, we commit to releasing evaluation results regardless of the findings when the evaluation is done, and that’s usually within two months of a report’s completion.
Second, the decisions about approval, release and dissemination are the responsibility of the head of OPRE, not the political leadership of ACF.
I view these as crucial protections for the independence of research, and I’m proud that we’ve made them.
I’m also glad that since the last conference, we’ve elevated Naomi’s position to Deputy Assistant Secretary for Planning Research and Evaluation. In doing so, we communicated to all staff our vision of ACF as an evidence-based organization dedicated to continuous learning, and Naomi and her colleagues are actively pursuing a set of steps to bring that effort forward.
ACF as a learning organization
It perhaps doesn’t sound like a bold statement to say that we’re committed to being an evidence-based organization, but, I want to take a few minutes to say what seems to me straightforward about this, and what seems challenging
We’re committed to supporting and drawing from research because we want to improve the effectiveness of our efforts and we don’t want to do things that don’t work. And, from the research, we’re able to learn that sometimes our best judgments about these questions don’t turn out to be right, or turn out to be only partially right, or sometimes turn out to be sort of right but for the wrong reasons. So, having a rigorous research agenda is a crucial way to learn, to confirm or refute hunches, and in a highly partisan political environment, to try to get beyond politics when there’s actual evidence that bears on a question.
However, sometimes this gets framed as an effort to determine “what works,” but I think that in important ways, that’s not a helpful way to think about what we’re trying to do. There may be times when research unequivocally demonstrates that a program had no impact, or that its costs clearly outweighed its benefits, and the clear implication is we need to stop doing whatever that was. But, lots of research findings aren’t like that, particularly when we see that a program had some impact for some groups in some settings. If that’s what we see, then it really shouldn’t follow that our response should be “it worked, let’s replicate it,” and tell administrators they can only do the thing that was evaluated and had an impact. That shouldn’t be our response because even if there was an impact, it may or may not have been large, we may or may not have confidence about what caused the impact, there may be challenges of replicability, particularly in different contexts or with different populations, and even when we are confident that we’ve seen a sizeable impact and know why, we recognize that we’re still not at the end-point of knowledge and it may be possible to do better. So, one challenge in being evidence-based is to ensure that we’re always asking how this set of research findings contributes to our overall efforts to strengthen the effectiveness of programs, and to not over-simplify by dividing the world into things that work and things that don’t.
Second, in saying we’re an evidence based organization, we need to ask what does it mean to be evidence-based; and do we want to be evidence-based all of the time? At ACF, we place a strong emphasis on the importance of supporting and learning from experimental, random assignment studies. At the same time, our commitment to being evidence-based is guided by the recognition that evaluation involves more than experimental studies and learning involves more than evaluation. Our evaluation policy says “A learning organization with a culture of continual improvement requires many types of evidence, including not only evaluation but also descriptive research studies, performance measures, financial and cost data, survey statistics, and program administrative data.” While we recognize the crucial role of experimental findings, we also understand that policy officials and program administrators face hundreds of questions each week, only a small fraction of which can be answered by findings from experimental studies and we need to proceed with the humility that comes from the recognition that research almost never answers a policy question completely and unequivocally.
We also need to ensure that being evidence-based is not the enemy of innovation. There are times when it’s suggested that we’re only confident that something works when it has been experimentally evaluated, and we should only or primarily fund those initiatives that have been experimentally evaluated. As I’ve discussed a few minutes ago, one problem with this approach is that it divides the world of practice into things that work and things that don’t in ways that are far too simple to be useful for the complicated world we live in. But, I think another danger of this perspective it that can impose a potentially impossible burden of proof on efforts to try something new. Even with a fairly broad definition of evidence-based, there’s a case to be made that much of the status quo is not evidence-based. To improve, we have to try new things and that requires going beyond what we already know. At ACF, we’re committed to ensuring that we never treat the interest in building an evidence base as being in opposition to innovation.
20 years after the enactment of TANF
So, that suggests the broader context for our work, and for this year’s conference.
This year’s conference coincides with the 20th anniversary of the enactment of the Personal Responsibility and Work Opportunity Reconciliation Act, and while we ultimately decided against doing a formal 20 year retrospective, the TANF experience informs the content of the conference in several important ways.
First, as was the case 20 years ago, it still is true that most poor children live in homes with a single mother, and when we look at trends in employment rates for single mothers, it’s clear that the dramatic progress that began in the early 1990s ended in 2000. And, it’s also the case, that too often, the welfare employment research and TANF practice has substantially relied on findings that are now more than 20 years old, reflecting a very different, pre-internet labor market. So, in this Administration, we’ve supported research to find more effective ways of helping families enter and sustain employment and gain access to better-quality jobs, and there are sessions during the conference on career pathways research, the potentials for sectoral strategies in health care, stronger coordination with the rest of the workforce system, better use of labor market data to guide employment strategies, and the role for subsidized employment.
Our Administration has also emphasized the fundamental importance of ensuring that children receive the financial and emotional support of both parents, and you’ll see sessions in the conference focusing on strategies for supporting both the employment and engagement of fathers in their children’s lives.
And, while the research is still at an early stage, we are encouraged by the innovative and more comprehensive two generation strategies that a number of states and communities are exploring; we’re committing research dollars to exploring these efforts, and we’ve got a session highlighting the potential for two generation strategies. .
It’s also clear that one major part of the TANF story is that the sharp decline in provision of assistance to poor and eligible families has been an important contributing factor in the growth of extreme poverty among families with children over the past 20 years. ACF has supported research to better understand the circumstances and needs of families in very deep poverty, and several of the conference sessions will focus closely on these issues.
Using brain science to inform research, program and policy
If you’ve been to the last several RECS, you know that another major theme in this Administration has been our commitment to looking at and learning from brain science to identify better ways to operate programs and to help families succeed. It’s a principal focus in our early childhood work, and across ACF. The session following this one explores some of the early findings from efforts to apply behavioral economics in the context of human services programs Subsequent sessions will focus on the potential for more systematically addressing and strengthening executive functioning, and on work to bring greater clarity to what it means for human services programs to be trauma-informed.
This work is informed by the research relating to toxic stress, indicating that when children experience strong, frequent, or prolonged adversity without adequate adult support, the prolonged stress can disrupt the development of brain architecture and other organ systems, and have lifelong negative consequences. We think it’s fundamental that public policy seek to reduce the stressors that too often characterize the lives of poor children, families and communities, while strengthening the capacities of caregivers to cope with both current stressors and the continuing effects of past ones. We recognize there are multiple questions about how to best bring this framework to the work of human services programs, and that’s a key part of our research agenda.
We believe our research agenda drawing from the insights of brain science can provide important insights that may ultimately lead to better results in efforts to help adults succeed in employment, improve parenting, strengthen capacities to focus on goals and future planning and manage conflict and stress across multiple domains of activity.
So, we view this work as important. At the same time, we want to be clear that structural factors in our economy and public policy, and not deficiencies in executive functioning or the impacts of trauma, are central in understanding why the United States continues to have a child poverty rate exceeding those of virtually all other developed nations, and in understanding how the severity of inequality constrains opportunity and mobility. And, it remains a principal challenge for our research agenda, to better articulate and map out research strategies that contribute to addressing these broader structural factors.
Thank you
In these opening remarks, I’ve only touched on a fraction of what’ll be covered at this conference ---- you’ll also note, among other things, sessions relating to our Tribal research agenda, challenges facing immigrant and mixed status families, sessions relating to family formation, homelessness, disability and other key issues.
In each RECS conference, we seek to have a conference that offers significant value to program administrators, policymakers, and researchers. We thank you for attending, and will look forward to the next three days.