Working out whether or not your sustainability program is having the impact you intend is more complex than you might think. There’s heated debate among academics and practitioners around how to do it best, much of it pointing out that designing a perfect measurement methodology in theory is a lot more straightforward than applying it in practice. That said, most can agree that it’s a necessary exercise. With this paper, we add the Cocoa Life perspective to the debate.

Impact is important for many reasons. One recalls the quote often attributed to management guru Peter Drucker: “if you can’t measure it, you can’t manage it.” This is just as applicable in development interventions as it is with corporate performance. Only by measuring and evaluating what’s working and what is not can those responsible for a program gain the information needed to fine-tune the program to make greater effect on target communities.

In a similar vein, impact reporting can be essential for justifying the money spent to funders who are eager to know they are putting their money behind something that works. Achieving and reporting on impact is an important factor in ensuring the sustainability of funding for most programs. For Cocoa Life, Mondelēz International’s cocoa sustainability program, we measure and evaluate the impact of our interventions so we can understand what is working and how to improve for our own purposes. But just as important as the internal reasons is sharing our insights with others, to help them learn from our experiences and overcome the same challenges we faced.

The central challenge in evaluating the impact of any program is working out what would have happened in the absence of the intervention. This is called the counterfactual. By its very nature, you can never know what it actually is. Thankfully, economists and statisticians have developed some clever ways of estimating the counterfactual. Once you have this estimate, you compare it with the data for people who took part in your program. The difference between the two can be attributed to the effects of your program.

Cocoa Life works with Ipsos, our expert learning and evaluation partner, to run impact evaluation in cocoa origin countries. We have trialled a range of approaches, from the experimental randomized control trial (RCT) – considered by many as the gold standard for impact evaluation – to quasi-experimental methods, such as propensity score matching, for occasions when it’s too resource-intensive to run a successful experimental evaluation.

As anyone experienced in impact evaluations will acknowledge, a lot can go wrong during the process. To help others learn from our experiences, we have published a short report, containing two case studies on M&E projects in cocoa communities: an RCT in Ghana and propensity score-matching in Lampung, Indonesia. We’d love to hear your thoughts on this report or if you’ve had similar experiences with impact evaluation, or solutions to common challenges – contact us at CocoaLife@mdlz.com if you have anything to add or to discuss!