What does it mean to pre-register, and why is it worthwhile?
Pre-registration is the practice of deciding your research and analysis plan prior to starting your study and sharing it publicly, like submitting it to a registry. OPRE held a meeting on open science — or the movement encouraging scientists to transparently share all phases of the research process - in 2019 , where Dr. Katie Corker gave a talk titled Pre-Registration: What and Why (slides (PDF), video). After her talk, Ryan Moore of the GSA presented on Pre-registration, Pre-analysis, and Re-analysis at GSA’s Office of Evaluation Sciences (slides (PDF)).
There are many reasons to pre-register studies. Pre-registering may prevent researchers from overfitting to their data or, in other words, making analysis decisions that are too specific to a particular sample or study. Pre-registering may also prevent the use of questionable research practices, like p-hacking, cherry picking, or hypothesizing after results are known (sometimes called “HARKing”). Pre-registering can also increase the transparency and rigor of research and evaluation, which, in turn, may help to bolster public confidence in the results of federally funded research and evaluation.
Pre-registering also gives researchers and evaluators a chance to test theories — specifically, if an investigator comes up with a theory-derived hypothesis, they can test it and prove themselves wrong. But if that investigator doesn’t come up with their hypothesis before doing the analysis, they lose the chance to test it.
There are also several common myths related to pre-registration, which are shown in the diagram below (source of diagram, source of content depicted (PDF)):
So how do you do this well?
“Pre-register your study” is easy to say but hard to do well. As Ryan Moore shared in his talk at the 2019 methods meeting (PDF), there can be a lot of pieces to create a thorough preregistration, but a minimal one can be simple. A basic pre-registration is a read-only document that specifies when it was created and describes the study and analysis. Ideally, this document is public-facing and includes information such as variables that the researchers and evaluators will be using to assess outcomes, how the researcher/evaluator will analyze the data (including considerations like treating missing data), what hypotheses they will test, the standard errors they will use, how those standard errors will be estimated, checks for covariate balance and adjustments, and what other exploratory analyses they plan to do (e.g., subgroup analysis). Depending on the research question and study design, not all of these pieces of information will be relevant. There are many example templates available, such as this one from the Center for Open Science .
Kriti Jain is a Senior Social Science Research Analyst whose work at OPRE focuses on domestic/intimate partner violence, ongoing research methods training, healthy marriage, and responsible fatherhood.
Marie Lawrence is a Social Science Research Analyst whose work at OPRE focuses on improving family well-being through research and evaluation projects related to employment and training, child support, and the applications of behavioral science to human services.
Jenessa Malin is a Senior Social Science Research Analyst whose work at OPRE research and evaluation projects related to child welfare and early care and education programs.