By: Samantha Hautea, GREAT Communications Specialist
Evaluating the effectiveness of any program is complex, and particularly challenging for projects like GREAT, which seek to build skills and shift mindsets. For more insight into evaluation and impact assessment from the perspective of a donor, we spoke with Vicki Wilde, Senior Program Officer for Agricultural Development and Women’s Economic Empowerment at the Bill & Melinda Gates Foundation. Vicki works with public and private sector partners to ensure women have equal access to agriculture opportunities that can boost crop and livestock productivity and improve nutrition and incomes for millions of impoverished farming households in sub-Saharan Africa and South Asia. Before joining the Gates Foundation in 2014, Vicki was the founder and executive director of African Women in Agricultural Research and Development (AWARD.
Vicki Wilde, Senior Program Officer,
Agricultural Development and Women’s Economic
Empowerment at Bill & Melinda Gates Foundation
Wilde added that documenting indirect change was important. “There has to be clarity about what you can control directly: I like to start with quality and relevance. Is this going to change how you do your work? With GREAT’s model, you’re looking at field implementation as part of the learning process so it’s not just theoretical. You’re creating clusters of researchers who will continue to learn from each other and support each other to make bold innovations over time.”“Impact assessment for a program like GREAT is challenging for a number of reasons,” Wilde explained. “You’re wanting to build skills and awareness among individuals and institutions who in turn may adjust how they work in order to develop technologies that may have more impact on the ground. You’re having to evaluate an indirect impact, so you have to use proxies, and those can be hard to define. It’s not as clear cut as measuring direct impacts. It starts with how people think, design, innovate, and prioritize. The whole concept of gender-responsive crop breeding requires a new set of analytical skills, and then they have to put it all into action in a way that may go against the grain.” Why is this important? There is a risk of deepening existing inequalities if technology and innovations aren’t adapted to meet the needs of both women and men.
Donors like the Bill & Melinda Gates Foundation, too, are changing how they support these types of projects. “Alongside our more recent gender grants we’ve invested also in independent evaluations so that we build rigorous evidence as we go,” Wilde said. In addition to providing the funding for a project, she explained, they allocate additional funding for monitoring and evaluation. This isn’t about a lack of trust in their grantees to follow through. “It’s more about the pervasive gender evidence gaps that hold back our goals. There are so many holes in the field of gender and agriculture and we all need to learn more about what works.”
As an example, Wilde explained, GREAT aims to identify the most effective way to incorporate gender responsiveness into crop breeding programs; subsequently we’ll all want to understand what difference this makes in the lives of women and men smallholder farmers. Though GREAT cannot directly be held accountable for the latter part of this equation, it is an indirect outcome that will be important to capture.
With her extensive experience in the field of development and gender, Wilde also had a few reflections to offer:
On growing support for gender programs:
A growing number of the organizations we work with take gender issues more seriously today than ever before. There’s clarity on the inter-relationship between inequalities and poverty, and therefore more intentionality to drive for reducing the gender gaps. I see momentum toward filling the gender data gaps, evidence gaps and figuring out the innovation and implementation models that would benefit women and men both. We also see this intentionality in the agriculture strategies of governments, such as Ethiopia, India, Bangladesh. There’s a long way to go, but the momentum is genuine.
On convincing donors to invest in capacity development:
I hear reluctance amongst many donors – including the Gates Foundation – to invest in capacity development because it’s difficult to show what’s achieved. I’ve heard it said many times: ‘it’s a bottomless pit.’ There’s so much change that needs to happen and –it can be difficult to answer: what is the return on investment? Answering this question will always be important to the donor community.
On the importance of data to making the case for investment:
When we’re talking to governments and other development partners we’re doing it with data. Yes, there are risks and shortcomings to an over-reliance on data. Any economist or social scientist knows those shortcomings, but data is the most common language amongst policymakers. What we need is high quality data.
Stories are important too. I love impact stories, especially from people who can explain why and how they’ve made a change and how they’re influencing the community around them to do so as well.
For example, in the case of GREAT, it’s helpful to have documentation of stories such as those from the RTB and wheat teams. It is important to a donor to hear about how they are making changes in their research or that they are willing to pay for GREAT courses for more of their team members. GREAT’s collaboration with the CGIAR’s Gender & Breeding Initiative to develop new protocols and guidelines for gender-responsiveness are important global public goods that are good to see too.
On failure and the realities of project implementation:
Documenting failure is extra important. When things fail we want to know about it, and we want to understand why. We can learn from that failure going forward. We’re always skeptical when reports are positive only.” We know this work is hard and we don’t expect everything to go as planned. Learn and adjust. And repeat.
