Rigorous Evaluation in Tribal MIECHV: A Series of Briefs

Publication Date: August 26, 2020

Download Report

Download Report PDF (827.36 KB)
  • File Size: 827.36 KB
  • Pages: N/A
  • Published: 2020

Introduction

The Tribal Home Visiting Program, part of the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV), is a federally funded initiative that supports the provision of home visiting services to American Indian and Alaska Native (AI/AN) families and children. The program, also known as Tribal MIECHV, is overseen by the Administration for Children and Families (ACF) in collaboration with the Health Resources and Services Administration and was authorized under Section 511 of Title V of the Social Security Act.

Each Tribal Home Visiting grantee funded from 2010 to 2017 was required to develop an evaluation of its program that was driven by community questions and met the ACF established criteria for rigor. This process optimized the likelihood that findings would be meaningful to the program and the local community and would also contribute to the general knowledge base about successful implementation of high-quality evidence-based home visiting services in AI/AN populations. For more information about the grantees’ evaluation plans, please see Grantee-Led Evaluations in the Tribal Maternal Infant, and Early Childhood Home Visiting Program: A Compilation of Grantee Evaluation Plan Profiles. This brief series tells the story of the Tribal Home Visiting Program grantees’ experience designing and implementing evaluation studies in their communities.

The brief series includes –

Purpose

Five evaluation briefs have been developed to share the story of the grantees’ rigorous evaluations and provide recommendations for those who oversee evaluations with tribal communities or are seeking to support evaluations with tribal populations. The Tribal Home Visiting Evaluation Institute (TEI) created the brief series using information provided by grantees (n=23) through evaluation plans and evaluation final reports submitted to ACF.

The primary audience for this series of briefs is federal program staff and leadership who work with tribes or tribal organizations or manage tribally focused grant programs. Each brief concludes with takeaways for program staff and leadership. The information presented may also be of interest to policy makers, researchers, and academics working in the broader human services field.

Key Findings and Highlights

The series of briefs holistically examined the grantees’ experiences in designing and conducting rigorous evaluations of their home visiting programs. Through this examination, key findings and takeaways were identified for each brief. Some of those key takeaways are listed below.

Federal program staff and leadership working with tribal communities on evaluation may benefit from the following lessons learned:

  • Understand the history and context of evaluation in tribal communities. Allow ample time and ensure a collaborative, iterative process to arrive at an appropriate, flexible evaluation plan.
  • Recognize that the diversity of tribal communities may require flexibility in evaluation designs and questions.
  • TA providers with experience and skill supporting tribal communities in program evaluation can support grantees to meet evaluation requirements in locally meaningful ways.
  • Individuals designing grant requirements for projects with tribal communities should be familiar with the importance of community engagement.
  • Build in time for grantees to complete IRB and other local review processes, which often take longer than anticipated.
  • Federal staff overseeing evaluation studies in tribal communities should be familiar with the contextual factors that may influence how study findings are interpreted and set evaluation expectations appropriately.

Methods

TEI reviewed and coded the grantees’ evaluation plan profiles and evaluation final reports to develop the briefs in this series. Coding identified grantee study designs, topics of focus, outcomes studied, and evaluation findings and implications. TEI also reached out to four Tribal Home Visiting Program grantees to request their participation in case studies presented in briefs 3 and 4. Grantees reviewed the case studies summarizing their evaluation experiences and provided edits as needed. The five briefs were written in collaboration with the Office of Planning, Research and Evaluation (OPRE) at ACF.

Citation

Roberts, E., Morales, J., Buckless, B., Geary, E., & Lyon, K. (2020). An overview of local evaluations: Rigorous evaluation in Tribal Maternal, Infant, and Early Childhood Home Visiting (OPRE Report No. #2020-47). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Meyer, A., & Denmark, N. (2020). Supporting Tribal Home Visiting grantees in meeting the MIECHV evaluation requirements: Rigorous evaluation in Tribal Maternal, Infant, and Early Childhood Home Visiting (OPRE Report No. #2020-48). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Buckless, B., Morales, J., Roberts, E., Geary, E., & Atukpawu-Tipton, G. (2020). Engaging tribal communities in evaluation: Rigorous evaluation in Tribal Maternal, Infant, and Early Childhood Home Visiting (OPRE Report No. #2020-49). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Buckless, B., Roberts, E., Morales, J., & Geary, E. (2020). Addressing evaluation challenges: Rigorous evaluation in Tribal Maternal, Infant, and Early Childhood Home Visiting (OPRE Report No. #2020-50). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Morales, J., Roberts, E., Geary, E., & Atukpawu-Tipton, G. (2020). Evaluation findings and implications: Rigorous evaluation in Tribal Maternal, Infant, and Early Childhood Home Visiting (OPRE Report No. #2020-51). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Last Reviewed Date: