Co-Creating the Way: Measuring Impact

Elena  0:00  

We received a very interesting question. Of course, it’s very central you touched upon it Sybil: data, the importance of measuring success. And it seems that systems change, of course, is non-linear, it is a feedback loop and so much more difficult to, you know, to measure. But there are ways there are ways of doing it. So the question reads, The international development community is, of course, increasingly focused on measurable impact. And using evidence, of course, to prioritize resource allocation, as you were saying Sybil. But systems change is, in a sense, hard to measure. And how can we push for systems change approaches and being in this context? How can we measure the impact? How can we address the challenges of showing success and evidence for that success? I think, Sanjay, I think you’ll have opinions on that. Can you take that question, please? 

Sanjay  0:59  

Sure. I would like to certainly start by quoting a gentleman called Albert Bartlett who said the greatest shortcoming of the human race is our inability to understand the exponential function. We as a species, find it very difficult to understand anything which is exponential in nature. And that is why measuring systems change is so difficult because it is not a linear curve, mathematically speaking, so the data that we collect, it takes time to build and then it has its own tipping points it has its own inflections whereas, the measurement and evaluation methods that we normally use in general are designed for a very short distance between cause and effect and mostly in a linear fashion. And that is why it’s so hard to go back with data and prove that systems change is happening or just the time – I’m sure all of you, we are all in the midst of a total disaster with what’s going on with COVID. And I’m sure that all of you have developed a deeper understanding of exponential curves looking at how the COVID moves, right. So that’s how systems change even works, right? It’s a very similar pattern, which is because this is network effects and this is how it works, nothing is happening and then suddenly things start happening and all of a sudden lots is happening. And I think, how to deal with this. My short term experience has been to look at it from three lenses. One is look for whether we are, how do you measure progress of the innovation that we’re doing? That it is working that it is proving itself to be right? Second, are the different actors engaging and more and more people co-creating. And third is the scale machinery actually adopting and deploying things at scale. Whereas the reflex is to go and say what is the outcome? And in systems change outcome is not necessarily a very short term factor. It’s a very long term factor. So I think it needs a new paradigm in measurements, which has to follow our ability to understand exponential change, rather than our ability to understand linear change. Because, and, and the way the funding systems work or the way the sector works most of the time is based on linear outcomes, rather than exponential outcomes. And that’s why it is so hard. So I am not giving you a very straightforward answer, but I’m saying that that’s the task ahead of us. How do we make people who work with us who support us our co creators or collaborators understand the exponential nature of the change that we’re working with? And not oversimplify by proving it with certain linear functions?

Elena  3:58  

Thank you for that, Sanjay. Sybil, would you like to…

Sybil  4:06  

Yes, thank you. Thank you, Elena and Sanjay. I fully agree with your thinking and definitely see it playing out too, you know, in our days, especially as we’re thinking of easing lockdowns and things like that we’re gonna see exponential in a new type of way. Maybe the only thing I would add is that I do think that, and we’re seeing this, and have created a body of work of strategy at the foundation is looking at ensuring that we have better gender data. I think this is one area that is a huge gap worldwide. So you know, having very nuanced and detailed information about women’s labor force participation Participation in agriculture, enterprises etc. There’s a lot of data that’s just missing. And then, you know, when when you talk about the norms that are impacting women that play into some of these other indicators around women’s participation in the economy, those have to be considered as well, yet those aren’t being collected. So there’s a gender data gap. And I do think that there’s a lot of entities, including the foundation that is playing a role in really strengthening better data systems across the world. And there’s also, you know, call for the use of that data, because it’s only through the use of that data, then, that more informed decisions and policies and engagements can be made across a multitude of actors. So I do think that there’s a lot of things that need to be strengthened. And in terms of data and measurement in order to, especially within the gender perspective that I’m bringing to this conversation that can really help drive some of the systems change.

Elena  6:19  

Thank you so much Sybil for these remarks, and thank you to all of you for sending such an interesting question. Many are coming and unfortunately, we cannot take them all but there are wonderful contributions within your questions. So thank you so much for sending them.