USAID's Measuring Impact Closeout Event

“Functioning natural systems are critical to human survival itself,” said Carrie Thompson, Deputy Assistant Administrator of the Bureau for Economic Growth, Education and Environment at USAID, at a recent Wilson Center event on USAID’s efforts to increase the effectiveness of its biodiversity conservation programs. Measuring Impact was designed to help support USAID’s 2014 Biodiversity Policy, which is “grounded in a recognition that human well-being and progress are dependent on the health of biodiverse systems and [that] durable development gains are not possible unless these systems are valued and safeguarded,” said Cynthia Gill, Director of USAID’s Office of Forestry and Biodiversity.

Collaborating, Learning, and Adapting: A Framework for Innovation

The “Collaborating, Learning, and Adapting” approach is a “way to operationalize organizational learning and adaptive management at USAID” to facilitate the design of “development solutions that are appropriate to the local context, responsive to local priorities, and adaptive to the changing circumstances of the places where we work,” said Melissa Patsalides, Acting Deputy Assistant Administrator from USAID’s Bureau for Policy, Planning and Learning.

“Too often,” said Patsalides, “accountability was the overwhelming focus of monitoring and evaluation efforts. This squeezed out the space for learning, and left implementing partners and USAID staff averse to taking risks.” Instead, Measuring Impact prioritized institutional learning and encouraged innovation through programs like the Cross-Mission Learning Program, which established collaborative learning groups of environment officers from different USAID missions, said Shawn Peabody, Senior Learning Specialist from Environmental Incentives.

Measuring Impact also encouraged adaptive management, which allows programs to adjust course throughout the implementation period based on new information or changing circumstances. Rebecca Butterfield, Director of Monitoring, Evaluation and Learning at Tetra Tech, cited positive experiences with “Pause and Reflect” sessions prior to annual work planning to discuss ”what’s working, what hasn’t worked  in the last year, how have our assumptions changed, how have risks changed.” She noted that performance-based contracts are not very conducive to adaptive management; for example, if a contract stipulates that you should train 10,000 people, but “you learn after Year One that training isn’t the way to get to the end result you want, you still have to train 10,000 people, and that is just a bad use of resources,” she said.

Adaptive management and learning should be included in contract language whenever possible to make agreements more flexible, said Olaf Zerbock, Biodiversity and Forestry Advisor at USAID.

Evidence Is Not Enough

An evidence-based approach to conservation can help amplify the impacts of investments, said Andrés Gómez, lead of Measuring Impact’s evidence team and Senior Biodiversity Researcher at ICF. “We should strive to use evidence not as an end in itself, but because evidence helps us make better decisions…reduce risk, and improve outcomes.”

However, “it’s not enough just to have the evidence,” said Diane Russell, formerly a biodiversity and social science advisor at USAID; it also “requires working the system.” In Cameroon 27 years ago, for example, Russell’s research found that certain techniques favored at the time by the research and donor communities were not the best way to discourage slash-and-burn farming and encourage more permanent cultivation methods. “But despite our meticulous documentation of the system and the parameters farmers used to make decisions, we could not get the senior scientists working on this technology to consider our recommendations,” she said.

“Power can trump evidence,” said Russell, “but less so if there is a system in place that encourages, incentivizes, or even mandates that decision-makers consider evidence in structured ways.”

The quality of evidence also matters. Russell called for impartial review systems, such as peer review or working with independent evaluators, in order to ensure a neutral body of evidence about conservation programs. So much rides on a project’s success, said Russell, that “there’s a huge pressure to give greater weight to positive findings.”

Resources for Missions on the Ground

USAID mission staff responded favorably to Measuring Impact’s approach of “learning by doing, building on success, and learning from failure,” said Zerbock, which included producing a wide array of resources, tools, and how-to guides.

“Mission staff and implementing partners really seemed to appreciate having a concrete process and set of tools that they could use to do adaptive management,” said Vinaya Swaminathan of USAID’s Measuring Impact contract and a Senior Program Officer at Foundations of Success, such as a toolkit for combating wildlife crime or the “Evidence in Action” resource, which helps practitioners use evidence to design and implement conservation programs. “It’s like a self-help manual to build skills in critical thinking and being an evidence champion,” said Russell.

The evidence-based approach is already very common in other sectors, such as health and education, said Colin Holmes of USAID’s Office of Forestry and Biodiversity. “We’re not boldly going where no one’s gone before. Environmental evidence is something that’s up and coming; there’s no reason we can’t continue to build that evidence base and improve our approaches,” he said.

Written by Rebecca Lorenzen and edited by Meaghan Parker.