Fact Sheet – How Does the Prevention Services Clearinghouse Rate the Design and Execution of Studies?, Handbook of Standards and Procedures, Version 2.0

Publication Date: November 15, 2024
Fact Sheet – How Does the Prevention Services Clearinghouse Rate the Design and Execution of Studies?, Handbook of Standards and Procedures, Version 2.0

Download Brief

Download PDF (1,439.16 KB)
  • File Size: 1,439.16 KB
  • Pages: 3
  • Published: 2024

Introduction

Research Questions

  1. How often do studies meet the Clearinghouse’s study design and execution standards?
  2. What are the most common reasons that studies receive low ratings on the design and execution standards?
  3. Can design and execution issues be addressed, and if so, how?

The Prevention Services Clearinghouse conducts objective, independent, and transparent reviews of research on programs and services intended to provide enhanced support to children and families and prevent foster care placements. This Fact Sheet provides an overview of how the Clearinghouse rates the design and execution of the studies it reviews. Studies are assigned a rating of high, moderate, or low support of causal evidence, based on the extent to which they meet the Clearinghouse’s design and execution standards. This Fact Sheet has been updated to reflect the Handbook of Standards and Procedures, Version 2.0. 

Purpose

The Fact Sheet aims to address frequently asked questions about how the Clearinghouse rates the design and execution of studies it reviews, describe common reasons why studies may receive low ratings on the design and execution standards, and lay out possible approaches to design and execute studies in alignment with the standards. 

Citation

Jackson, C., Wilson, S. J., & Glenn, M. (2024). How Does the Prevention Services Clearinghouse Rate the Design and Execution of Studies?, Handbook of Standards and Procedures, Version 2.0, OPRE Report 2024-XXX, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. 

Glossary

Analytic Sample:
The analytic sample is the sample of participants included in an analysis of the impact of the program or service on an outcome. Studies may have multiple analytic samples because the number of participants available for analysis may differ for different outcomes and different time points within a study.
Intervention Condition:
The intervention condition is the set of services offered to study participants, inclusive of, but not necessarily limited to, those related to the program or service under review.. Studies may have more than one intervention condition.
Minimal Intervention Comparison Condition:
Minimal intervention conditions are those in which participants are offered minimal or limited services. These individuals may receive handouts, referrals to available services, or similar nominal interventions.
No Intervention Comparison Condition:
No intervention comparison conditions are those in which the participants are offered no services. Participants may be placed on a waiting list for future services or be offered no services as part of the study.
Quasi-Experimental Design (QED):
A study in which units are assigned to intervention and comparison conditions via a non-random process.
Randomized Controlled Trial (RCT):
A study in which units are assigned to intervention and comparison conditions via a random process (e.g., a lottery).
Study:
One research investigation of a defined subject sample, and the interventions, measures, and statistical analyses applied to that sample. Sometimes study results are reported in more than one document, or a single document reports results from separate studies.
Treatment as Usual Comparison Condition:
Treatment as usual comparison conditions are those in which the participants are offered or are free to seek out the usual or typical services available for the population in the study.
Head-to-head Comparisons:
Participants are assigned to another intervention that is not a variant of the program or service under review and does not meet the criteria for treatment as usual.