Use During Implementation

A. Decide who is on the team- (this process will ask a moderate amount of them- on an ongoing or periodic basis; though not all have to be at all meetings) -should include all the key implementers -would be good if there is at least one patient, or intended recipient who is not bashful -would be good to have decision maker(s) about program adoption and sustainment- at least at the initial and final meeting (about adaptations for sustainment) -designate a local RE-AIM process coordinator (this is often the overall program manager, but needs to be a person who has good overall knowledge of the program and sufficient time and support to complete the tasks below)

B.      Optimally, define RE-AIM outcomes and goals at the outset, during planning process

a.      Use the new 10-16 item questionnaire (*REF LINK BELOW) for definitions to specify RE-AIM measures and goals for each of the 5 dimensions for their project (see Appendix A )- or review/confirm for funded projects what was previously specified.  Refer to www.re-aim.org as needed for guidance and questions

b.      Establish initial priorities: It is important to have goals and reasonable expectations for outcomes on all RE-AIM dimensions, but it is often not possible to focus on everything at once. Initially, it is usually important to focus on Adoption- to have all key implementers and their supervisors on board; and on Reach- what channels will be used to enhance equitable and high reach”)

c.       Decide on your research or evaluation design- will you implement for all settings at same or different times; do a comparative study- (iterative RE-AIM compared to what?): just use iterative RE-AIM as/when needed; use an iterative RE-AIM minimal contact process initially and then incorporate more intensive support- (or other implementation strategies) as needed?

C.      Decide approximately how often will do the periodic RE-AIM assessments/adjustments

a.      For most projects, unless very short term, we recommend conducting first one at approx. 6-8 weeks; then approximately every 2 months; then one month to 6 weeks prior to the end of funding/close of supervision support .

b.      Adjust to your context- e.g., team experience; length of project; results you feel best

D.     Prepare for and plan periodic evaluations of the overall process-(the format and content of meetings; team members, forms and materials used, etc.)

a.      Work with project manager to set up meetings- and track time involved

b.      Gather any ‘quantitative data’ available- e.g., on reach or implementation, prior to each meeting

c.       Have the entire team read either or both the article in Appendix B and the ‘Iterative RE-AIM Guide Tip sheet’ (Make your own variation on this document here) prior to Team meeting #1

A.      Conduct Iterative Meeting # 1 (note if team has not yet set RE-AIM goals, then this will actually be meeting #2.   Also, you should check with your IRB to see if you need any IRB approval for any of these surveys/activities?)

If possible, have this be part of an existing team meeting that has high regular attendance instead of having to schedule a separate meeting.   This should take approximately 30 min. if you do not need to start from the beginning with introducing RE-AIM and what the dimensions are (it will take probably 50-60 minutes otherwise)

B.      Briefly overview and summarize the Iterative RE-AIM Guidance Process: 1) team agrees on specific RE-AIM goals and how to assess progress; 2) each team member independently rates both progress and importance- at that point in time- on each RE-AIM dimension; 3) collect and summarize measures using ‘gap chart’ figure (see example here- Appendix D); 4) team reflects upon and discusses these results;  5) identify 1 or 2 RE-AIM dimensions based on this discussion to prioritize for next time period; 6) formulate specific goals and action plans for these goals for next implementation period;  and 7) specify who will do what, when and how will evaluate results

C.      Review or create specific definitions for this project of the 5 RE-AIM dimensions- distribute this list to all for use when doing ratings below and if not previously set, provide a draft of these for meeting participants rather than starting from scratch (see www.re-aim.org for examples under ‘Guidance’ section)

D.     End the meeting by distributing a rating form  (Appendix E) to all team members to complete, without talking to others, as soon as possible after the meeting. Instruct people to answer each question based on both any data that are available to them (e.g., number attendees; implementation checklists; list of participating staff) as well as their subjective impressions. They should return completed ratings and comments to designated person within 2-3 days

E.       This person should prompt those who have not provided ratings by agreed upon time; and then average and integrate comments and ratings for next meeting using template in Appendix D.

F.       Conduct Meeting #2 – and following meetings (these sessions usually take an entire hour, at least the first couple times)

a.      Distribute and briefly summarize the results of the RE-AIM ratings using Gap Analysis report and comments (example in Appendix D)- explain the information summarized (including indication of the range of scores)- and give participants a few minutes to reflect individually upon these results.

b.      Instruct each team member to focus especially on the ‘Gap’ between importance and progress for each dimension.

c.       State that the goals of the upcoming open discussion in which you want to hear from each person are to:

                                                                                      i.      discuss and clarify current progress and importance (with emphasis on dimensions on which there are differences of opinion);

                                                                                     ii.       select one or two (definitely no more than 2) RE-AIM dimensions to prioritize for attention over the next X months

d.      Conduct an open ended discussion, facilitated by your process coordinator-making sure that each person gets chance to share their perspective and that no one dominates the discussion. Conclude by agreeing on the 1-2 RE-AIM priority dimensions

e.      For each of these, specify as much as possible the specific goal to attain using the SMART (Specific, Measurable, Achievable, Realistic, Timely) goal setting process (see Appendix F here).

f.        For each goal, develop a specific action plan (see template in Appendix G here) that details who will do what, by when, and how progress will be tracked and adjustments made

G.     Repeat steps 5 and 6 above at intervals identified (usually every 2-3 months). Make sure to schedule a final iterative review session approximately 4-6 weeks prior to ending of the ‘active support’ phase of the project that is focused on maintenance or sustainability.

a.      In this final meeting, focus on Maintenance, as either the sole focus or one of the two goals, and discuss what adaptations are necessary to maintain attention to and success on RE-AIM outcomes long-term.

If possible, have this be part of an existing team meeting that has high regular attendance instead of having to schedule a separate meeting.   This should take approximately 30 min. if you do not need to start from the beginning with introducing RE-AIM and what the dimensions are (it will take probably 50-60 minutes otherwise)

B.      Briefly overview and summarize the Iterative RE-AIM Guidance Process: 1) team agrees on specific RE-AIM goals and how to assess progress; 2) each team member independently rates both progress and importance- at that point in time- on each RE-AIM dimension; 3) collect and summarize measures using ‘gap chart’ figure (see example here- Appendix D); 4) team reflects upon and discusses these results;  5) identify 1 or 2 RE-AIM dimensions based on this discussion to prioritize for next time period; 6) formulate specific goals and action plans for these goals for next implementation period;  and 7) specify who will do what, when and how will evaluate results

C.      Review or create specific definitions for this project of the 5 RE-AIM dimensions- distribute this list to all for use when doing ratings below and if not previously set, provide a draft of these for meeting participants rather than starting from scratch (see www.re-aim.org for examples under ‘Guidance’ section)

D.     End the meeting by distributing a rating form  (Appendix E) to all team members to complete, without talking to others, as soon as possible after the meeting. Instruct people to answer each question based on both any data that are available to them (e.g., number attendees; implementation checklists; list of participating staff) as well as their subjective impressions. They should return completed ratings and comments to designated person within 2-3 days

E.       This person should prompt those who have not provided ratings by agreed upon time; and then average and integrate comments and ratings for next meeting using template in Appendix D.

F.       Conduct Meeting #2 – and following meetings (these sessions usually take an entire hour, at least the first couple times)

a.      Distribute and briefly summarize the results of the RE-AIM ratings using Gap Analysis report and comments (example in Appendix D)- explain the information summarized (including indication of the range of scores)- and give participants a few minutes to reflect individually upon these results.

b.      Instruct each team member to focus especially on the ‘Gap’ between importance and progress for each dimension.

c.       State that the goals of the upcoming open discussion in which you want to hear from each person are to:

                                                                                      i.      discuss and clarify current progress and importance (with emphasis on dimensions on which there are differences of opinion);

                                                                                     ii.       select one or two (definitely no more than 2) RE-AIM dimensions to prioritize for attention over the next X months

d.      Conduct an open ended discussion, facilitated by your process coordinator-making sure that each person gets chance to share their perspective and that no one dominates the discussion. Conclude by agreeing on the 1-2 RE-AIM priority dimensions

e.      For each of these, specify as much as possible the specific goal to attain using the SMART (Specific, Measurable, Achievable, Realistic, Timely) goal setting process (see Appendix F here).

f.        For each goal, develop a specific action plan (see template in Appendix G here) that details who will do what, by when, and how progress will be tracked and adjustments made

G.     Repeat steps 5 and 6 above at intervals identified (usually every 2-3 months). Make sure to schedule a final iterative review session approximately 4-6 weeks prior to ending of the ‘active support’ phase of the project that is focused on maintenance or sustainability.

a.      In this final meeting, focus on Maintenance, as either the sole focus or one of the two goals, and discuss what adaptations are necessary to maintain attention to and success on RE-AIM outcomes long-term.

A.      After the first 2-3 meetings, survey and discuss with the entire team the usefulness of the Iterative RE-AIM process (see optional form and questions here  in Appendix H) and any adaptations that need to be made to the process, persons involved, or frequency of iterative RE-AIM meetings

B.      The process coordinator and other decision makers relevant to your project (e.g., supervisor; PI,  administrator with budget authority) should meet to discuss these results and adaptations needed, fit with other organizational priorities, etc.

C.      Decide whether to continue the Iterative RE-AIM process and whether or not, and if you have the resources and infrastructure and processes (e.g., audit and feedback) needed to maintain (and continue to improve) project outcomes (see PRISM survey, Appendix I and article.)