
For OPRE, transparency is very important. It’s one of ACF’s five research principles of rigor, relevance, transparency, independence, and ethics. To work towards greater transparency, here are some of the activities we already do (and, of course, there’s always more we can do):
- Making information about planned and ongoing evaluations easily accessible on OPRE’s website, where we share project descriptions and link to OMB packages (example). The OMB packages include instruments (e.g., surveys, in-depth interview guides), data collection plans, and analysis plans.
- Pre-registering by publishing most study plans in advance in registries such as:
Pre-registering your study, no matter what type it is (e.g., qualitative, secondary analysis), is an important part of transparency in research and evaluation. Pre-registering can also keep researchers and evaluators from overfitting to their data (that is, making decisions about data analyses and conclusions that are too specific to a particular sample or study) and using questionable practices like p-hacking, cherry picking results, or hypothesizing after results are known. Pre-registering helps with testing theories, too. In addition to the above, OPRE also practices:
- Comprehensively presenting results, including favorable, unfavorable, and null findings. For example, a large-scale evaluation of career pathways interventions with 10 different programs found a combination of favorable, unfavorable, and null findings across the each of programs. All of the results are reported in detail on OPRE’s website.
- Broadly communicating research and evaluation results in a timely manner. In fact, ACF’s evaluation policy states that OPRE must publish results within about two months of the report’s completion .
- Archiving and sharing datasets (e.g., the child and family data archive , national data archive on child abuse and neglect ) to allow external researchers and evaluators to conduct secondary analyses. OPRE also funds researchers to do this. For example, OPRE has funded grants to conduct secondary analysis of its data collected through the Pathways for Advancing Careers and Education (PACE) Project, Health Professions Opportunity Grants (HPOG) Impact Study, HPOG National Implementation Evaluation (NIE), Family Strengthening work, and its Early Care and Education portfolio.
Striving for greater transparency is a small part of a much larger movement in science, open science, which was precipitated by a need to grow trust in the scientific enterprise after issues like the replication crisis , or the idea that the results of many scientific studies can’t be reproduced. The Credibility Revolution video clip below describes this movement.
Kriti Jain is a Senior Social Science Research Analyst whose work at OPRE focuses on domestic/intimate partner violence, ongoing research methods training, healthy marriage, and responsible fatherhood.
Marie Lawrence is a Social Science Research Analyst whose work at OPRE focuses on improving family well-being through research and evaluation projects related to employment and training, child support, and the applications of behavioral science to human services.
Jenessa Malin is a Senior Social Science Research Analyst whose work at OPRE research and evaluation projects related to child welfare and early care and education programs.