Assessing Effectiveness

Efficacy or effectiveness of an intervention can be measured in many ways, depending on the phase of implementation (planning, implementing, evaluating) and the data sources available. In the RE-AIM framework, efficacy and effectiveness encompass more than just a test of whether a primary outcome was affected by the intervention; other key considerations include broader outcomes of importance to stakeholders (e.g., quality of life, unexpected outcomes, negative outcomes), robustness of effects across subgroups, attrition and differential rates by subgroups, and data helping to explain efficacy and effectiveness outcomes.

EXAMPLE METHODS OF ASSESSING EFFICACY/EFFECTIVENESS (From the RE-AIM Model Dimension Items Checklist)

Effectiveness
Measure of primary outcome with or w/o comparison to a public health goal (e.g. HP 2020 goals, exercise 30 min/day; eat 5 Fruits &Veggies)
Measure of broader outcomes (e.g., other outcomes, measure of QoL or potential negative outcome) or use of multiple criteria
Measure of robustness across subgroups (e.g. moderation analyses)
Measure of short-term attrition (%) and differential rates by patient characteristics or treatment condition

Note that effectiveness may or may not be assessed in every project or application of RE-AIM, depending on phase of the project (i.e., planning, implementation, evaluation) and the questions of interest. However, even in projects that do not directly measure effectiveness, participant and stakeholder perceptions and expectations of effectiveness can be assessed using multiple methods (e.g., surveys, interviews, focus groups), informing planning for subsequent projects.