Thursday, April 18, 2019
I’m
Stephanie Chasteen, an external evaluator on NSF-funded projects aiming to improve STEM education at the university level. This is Part 1 of a 2-part series on Theory of Change. Today, I’ll share the process I use to help clients develop Theories of Change (ToC). Tomorrow, I’ll share how I’ve been able to work with an evaluation champion within the organization to develop organizational capacity in Theory of Change development.
The National Science Foundation (NSF) has required Theories of Change as part of many educational proposals for some time, but there has been quite a bit of confusion among my clients as to what exactly is meant by a ToC or why it would be useful.
Lesson Learned:I’ve learned not to back down when a client doesn’t want to develop a ToC (or take the time to do it well). When I have not done so, target audiences and outcomes were usually muddy. One client fought me the whole way, feeling it was confusing exercise without practical value. Later, however, she related to me, “Now I know why you had me do that, and it was totally useful. Now I’m developing one on my own for this new project and it was so much easier.” Most clients who develop Theories of Change go on to create (very good!) ToCs to guide other projects, and this gives me pride.
Hot Tip:Developing a Theory of Change
is confusing the first time around. Emphasize with clients that boxes represent outcomes, not strategies. Get your hands dirty with post-it notes; write down outcomes, move them around, and ask questions about how they connect. (The image is our in-progress diagram; see tomorrow’s post for the final version.) Give clients permission to do it their way: The test of a good TOC is that it is
useful, not that it is perfect or follows a prescribed model. My clients have been very creative in using color coding or organizational elements to help their diagrams.
Questions I like to ask are:
- How do you want the world to be different as a result of your project? (Project vision)
- Name a few changes that would need to happen to achieve your vision? (Long-term outcomes)
- What changes would need to happen to achieve those outcomes? (Short-term outcomes)
Thus, we essentially “work backwards” from the project vision. Don’t skimp on the project vision! It is such a valuable clarification and inspiration moment for most leaders.
During this process I keep my eye open for “miracles” – steps which seem to “magically” lead to the next step. A common example is “High quality materials are developed” and “People use those materials effectively.” Wow, magic! The missing steps might be “People find the materials and see that they are valuable” and “People learn how to use the materials.” Such insights lead to valuable additions to the project strategy.
Rad Resources:Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
About AEA
The American Evaluation Association is an international professional association and the largest in its field. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products and organizations to improve their effectiveness. AEA’s mission is to improve evaluation practices and methods worldwide, to increase evaluation use, promote evaluation as a profession and support the contribution of evaluation to the generation of theory and knowledge about effective human action. For more information about AEA, visit www.eval.org.