At OPRE and ACF, we work every day to help build evidence that is used to support policy and decision making. We are proud to guide our work by the five principles of the ACF Evaluation Policy: rigor, relevance, transparency, independence, and ethics.
The Department of Health and Human Services (HHS) recently finalized a new scientific integrity policy sharing a similar commitment to these principles in our Evaluation Policy. The new HHS policy adopts a broad definition of science (PDF)and scientific activities that is inclusive of all the activities OPRE engages in day to day. The new policy builds from the Official Federal Definition of Scientific Integrity:
Scientific integrity is the adherence to professional practices, ethical behavior, and the principles of honesty and objectivity when conducting, managing, using the results of, and communicating about science and scientific activities. Inclusivity, transparency, and protection from inappropriate influence are hallmarks of scientific integrity.
The HHS policy outlines seven areas:
- Protecting scientific processes;
- Ensuring the free flow of scientific information;
- Supporting policymaking processes;
- Ensuring accountability;
- Protecting scientists;
- Professional development for government scientists; and
- Federal Advisory Committees.
These seven areas are all in the service of supporting a culture where scientists are protected and can conduct work that is free from outside interference, where issues of equity, diversity, inclusion, and accessibility are paramount, and where research is conducted responsibly and ethically.
As we review this new policy, we take a moment to look back at our Evaluation Policy, and to reflect on what that policy means for our research and on ways the principles are reflected in our work.
Rigor
The new HHS policy supports practices that safeguard the quality of research and other scientific activities. ACF is committed to using the most rigorous methods available that are suitable for the evaluation questions, populations, settings, and circumstances at hand. This policy can help dictate decisions, including: which analytical methods are used; who is conducting the research; and whether any interferences made are well-founded and not over-generalized. Through our annual Methods Meeting , we explore innovative statistical methods to help ensure our research uses the most rigorous methods available.
Relevance
The new HHS policy applies scientific integrity to participatory modes of science, inclusion of the voices of lived experience, and the recognition of multiple forms of evidence, including Indigenous Knowledge. Evaluation and evidence-building do not exist in a vacuum. Evaluation priorities should take into account legislative priorities, policy-making needs, and the types of questions that are meaningful to the diverse populations that ACF programs served. Simply conducting the research is not enough; it also must be disseminated to all who might find it useful in ways that are clear and accessible. For example, our recent blog post examines efforts to co-create evidence and evidence-based programming with American Indian and Alaskan Native (AIAN) communities. It highlights AIAN Family and Child Experiences Survey (FACES), sharing examples of the ways in which co-created data is meaningful and useful for program directors, researchers, and staff.
Transparency
The new HHS policy highlights practices such as peer review and open science as important for supporting scientific integrity. ACF makes information about planned and ongoing evaluations easily accessible, typically through posting information on our website. Study plans are published in advance, and results are shared regardless of the findings. One key way to promote transparency is by pre-registering a study, as described in this blog post. Pre-registering, the practice of deciding on a research and analysis plan and sharing it publicly before research begins, can prevent researchers from overfitting to their data and from employing other researcher practices, such as p-hacking, cherry picking, or hypothesizing after results are known.
Independence
The new HHS policy emphasizes the importance of not suppressing, delaying, or altering scientific findings and products for political purposes, and ensuring that they are not subjected to political interference or inappropriate influence. Independence and objectivity are critical factors that must undergird all research. To insulate evaluations from undue influence and bias, ACF protects independence in the design, execution, analysis, and reporting of evaluations. One way we do this is by conducting studies through the competitive award of grants and contracts to external experts who are free of conflicts of interest.
Ethics
The new HHS policy defines ethical behavior in science as including honesty, lawfulness, equity, professionalism and adherence to relevant laws, regulations, and policies. ACF-funded research must be conducted in a way that is ethical and equitable and promotes, honors, and safeguards the dignity, rights, safety and privacy of participants. OPRE’s commitment to incorporating more equitable approaches into our research, evaluation, technical assistance, and data practices is embodied in our Racial Equity Research and Resources page. Here we highlight work that supports these more equitable practices.
At OPRE, we are proud to move forward in building evidence in a way that continues to embrace and build upon these five principles. We are excited about the new HHS policy and the principles underlying it. We join with other HHS programs, departments, and offices in conducting science that works to meet the highest standards of scientific integrity.
Lauren Supplee is the Deputy Assistant Secretary for Planning, Research, and Evaluation at ACF.