Home > NewsRelease > SIM TIG Week: The most important impacts are not measurable—does that skew things? by Simon Dixon
SIM TIG Week: The most important impacts are not measurable—does that skew things? by Simon Dixon
American Evaluation Association (AEA) American Evaluation Association (AEA)
For Immediate Release:
Dateline: Washington , DC
Friday, May 17, 2019

I am Simon Dixon, Co-Founder of Kwangu Kwako Limited (KKL), which builds and sells concrete panels to build safer, more secure affordable homes. Like most social enterprises and those who work in this sector, impact is very important and the drive behind why KKL and I do what we do. Metrics are useful but they must be used intelligent—especially when comparing very diverse businesses and sectors.
During our journey we have encountered statements like: How many unique customers do you have? And, how many beneficiaries do you have? Not unreasonable questions.
My favorite statement was, “We only invest in enterprises that will impact a million beneficiaries.” I asked, “Do you apply that metric whether we have an app, solar light/cook stove, a health clinic or a house?” The answer was yes. While I respect that organizations have to have some criteria for how they select partners, I feel that using such triggers directs funds toward a certain type of intervention. It can distort support away from slower, higher capital with longer-lasting, broader-based impact. Yes, anyone who has looked at our website and knows what we do will see that I could be a bit biased! But I am however trying to be objective, as this is a real and wider concern beyond our focus.
Lesson Learned:
One of the best pieces of advice at this time was from an investor. He told me, “Simon, if they are talking crazy metrics for your intervention, you are talking to the wrong investors. The right investors will understand.” That really helped motivate us and streamline our selection of investors. An investor is an important partner; they must understand or be willing to take the time to understand the impact and form of impact you have.
Some interventions, like ours, have far more qualitative and wider-ranging impact than a mere quantitative assessment would suggest. Again, reliance solely on metrics without a broader and balanced perspective—and yes, maybe, just plain judgement and instinct—can miss a great opportunity.
The real impact of products like houses, improved roads, and bridges, is subjective—how do we measure dignity, self-esteem, and pride? How do we measure the lifelong impact on a child of moving from a shack to a permanent home? Do grades improve? Do they earn more due to increased confidence?
Amid launching and scaling a social enterprise, it is next to impossible to know the answers to such questions, let alone within budget constraints. But we could all miss out on that opportunity if we rely solely on quantitative metrics.
We have however seen a shift when engaging with stakeholders over the past year, which is promising. If anyone has the answer to how to measure the more subjective impacts of social enterprises interventions, and/or would like to help us with this challenge, we would love to know. Please get in touch.
The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

About AEA

The American Evaluation Association is an international professional association and the largest in its field. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products and organizations to improve their effectiveness. AEA’s mission is to improve evaluation practices and methods worldwide, to increase evaluation use, promote evaluation as a profession and support the contribution of evaluation to the generation of theory and knowledge about effective human action. For more information about AEA, visit www.eval.org.

American Evaluation Association
Washington, DC
Other experts on these topics