Tuesday, June 4, 2019
Hi, we’re
Jonathan Scaccia of the Wandersman Center and
Paul Howard of the Institute for Healthcare Improvement (IHI), and we’d like to share a method that we’ve been using in our evaluation work for the past 4 years: The Critical Moments Methodology.
Lessons Learned: Community psychology puts a strong emphasis on participation and inclusion in all stages of a project; from design, to implementation, to evaluation. It’s natural that those who identify as community psychologist gravitate to participatory and Empowerment Evaluation methods.
Spreading Community Accelerators for Learning and Evaluation (SCALE), a Robert Wood Johnson Foundation-funded, IHI-convened initiative, aimed to help healthy community coalitions develop readiness to transform how they approached health improvement projects. There was A LOT of data, and it was sometimes hard to know how to make sense of it. Because SCALE was a formative evaluation, we needed a process to deeply delve into of emerging results on an ongoing basis.
Hot Tip: To do so, we had periodic “Critical Moment” Sessions. The Critical Moment method uses a structured narrative approach in which the evaluator poses questions to elicit participant emotions, reactions, and beliefs on topics of interest. Reflective activities that involve stakeholders can be used contextualize the ongoing narrative of a project and ascribe meaning to experiences (e.g., What has been the most memorable moment or experience on this project? What about it made the moment so memorable? How did the moment affect you?)
We conducted these in group settings to allow all relevant stakeholders, from the implementation team, to the funder, to community representatives, the opportunity to process together. First, we sent out an “inquiry” question to participants ahead of time, which we then synthesized and grouped all responses. We shared this grouping during the session (see visual below; yup, it is crowded, there were a lot of comments!)
After allowing everyone to voice what they thought were the key moments, we held a voting process to focus discussion around 2-3 moments to explore in more depth.
This general process was so successful, we conducted over ten critical sessions over the course of the 4-year project, and saw the methods adopted by community participants to us in their own settings. The process really tapped into the spirit of community psychology putting the meaning-making into the hands of the people tasks with making program changes.
Rad Resources: Check out this online paper from MIT understand and dive into this powerful tool.
http://web.mit.edu/cil/web_scripts/www/Critical%20Moments%20Methodology%20Brief%20CoLab.pdfThe American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
About AEA
The American Evaluation Association is an international professional association and the largest in its field. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products and organizations to improve their effectiveness. AEA’s mission is to improve evaluation practices and methods worldwide, to increase evaluation use, promote evaluation as a profession and support the contribution of evaluation to the generation of theory and knowledge about effective human action. For more information about AEA, visit www.eval.org.