Assessing Maintenance

In the RE-AIM framework, maintenance of an intervention occurs at two levels: maintenance of individual-level of effects of an intervention (i.e., long-term effectiveness), and maintenance of the intervention at the setting-level (i.e., long-term implementation). 

As with assessment of efficacy/effectiveness, assessment of individual-level maintenance of intervention effects includes not only the long-term effects of the intervention on a primary outcome, but also long-term effects on outcomes of importance to stakeholders (e.g., quality of life, unexpected outcomes, negative outcomes), robustness of long-term effects across subgroups, long-term attrition and differential rates by subgroups, and how and why individual-level effects were maintained (or not maintained). The length of time over which maintenance may be assessed depends in part on the specific behaviors and outcomes targeted by a specific intervention.

Assessment of setting-level maintenance of an intervention includes whether the setting continues to deliver the intervention (usually focusing on at least 6 months after research funding or support ends, but time frames of interest may vary), if and how the intervention was adapted long-term, alignment and sustainability of the intervention with an organization’s mission and business model, and how and why an intervention was (or was not) institutionalized for long-term maintenance at the setting level.  

EXAMPLE METHODS OF ASSESSING ADOPTION (From the RE-AIM Model Dimension Items Checklist)

Maintenance – Individual Level

 

Measure of primary outcome (with or w/o comparison to a public health goal) at ≥ 6mo follow-up after final intervention contact

 

Measure of broader outcomes or use of multiple criteria at follow- up (e.g., measure of QoL or potential negative outcome) at follow- up

 

Robustness data – something about subgroup effects over the long- term

 

Measure of long-term attrition (%) and differential rates by patient characteristics or treatment condition

 

Use of qualitative methods data to understand long -term effects

 

Maintenance- Setting Level

 

If program is still ongoing at ≥ 6 month post study funding

 

If and how program was adapted long-term (which elements retained AFTER program completed)

 

Some measure/discussion of alignment to organization mission or sustainability of business model

 

Use of qualitative methods data to understand setting level institutionalization

 

Note that maintenance may or may not be assessed in every project or application of RE-AIM, depending on phase of implementation (i.e., planning, implementation, evaluation) and the questions of interest. However, even in projects that cannot directly measure maintenance due to short time-lines, participant and stakeholder perceptions and expectations of individual- and setting-level maintenance can be assessed using multiple methods (e.g., surveys, interviews, focus groups), informing planning for sustainment.