For Immediate Release:

Friday, January 11, 2019

My name is Tatia Prieto, President of Prismatic Services, and we focus on PreK-12 evaluations. With clients, sometimes it seems our biggest challenge is helping them understand why and when to collect data. I’m a data geek who thinks in terms of process maps and spreadsheets, so it’s easy to forget that program implementers are the ones in the day-to-day fray, fighting against being lost in the proverbial weeds. As many people are engaged in New Year’s resolutions, it strikes me that effective evaluators share something in common with popular diet programs like Weight Watchers: a culture of accountability.
One of the biggest reasons why programs like Weight Watchers work well for people is that they establish a “community.” Back in the 1970s, I remember my grandmother complaining about going to meetings and weighing in. Being a child, I was confused as to why she would pay to be part of something she didn’t like, but I also noticed she didn’t snack as much on the days before the meetings. Why? It turns out that those meetings provided her with external expectations (accountability), spurring her to make better choices for herself.
We once had a client hire us halfway through their funding cycle. In our first meeting, I asked how often they collected data from their sites and they responded, “two weeks before the evaluation is due to the state.” By waiting until the state deadline was looming, they left little time to correct errors or omissions. They were also missing the chance to make data-informed course corrections throughout the year. I persuaded the client to conduct more frequent data collections. Sure enough, the first time we collected data, we found data errors and holes. One set of errors was serious enough that the client reassigned data collection duties within their organization. Did I provide the client with something earth-shattering or that required my advanced statistical skill set? Nope, but it was equally valuable. I provided the necessary accountability that enabled the client to make better choices for themselves.
Lesson Learned: The ability to create accountability was not something I would have considered as a top 10 evaluator skill back when I was starting out. Now I know better.
Evaluation should make sure that accountability is part of the program. If the program implementers have lost sight of it, evaluators should put accountability back into focus.
Rad Resources: Insights from Rubin’s Four Tendencies have informed my practice. Rubin sorts people into a 2×2 based upon how they handle internal and external expectations. Knowing my client’s type (an obliger, rebel, upholder, or questioner) helps me design accountability measures that are likely to suit them.
The Basecamp.com tool is an organizational and communications game changer. We use it to keep tabs on deadlines and to support internal accountability – who is responsible for doing what.
Here’s a screenshot of Basecamp’s project page for one of our evaluations:

The American Evaluation Association is celebrating Southeast Evaluation Association (SEA) Affiliate Week with our colleagues in the SEA Affiliate. The contributions all this week to aea365 come from SEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.