If you have a headache, you can take an Advil. In an ideal word, all projects would be conceived to address a specific ache with a “treatment” that we know will ease the pain. But the aches of countries in crisis and conflict are complex, multifaceted, and closely interlinked. How do we find an Advil for that?
We can begin by documenting what works (and what does not!) – by amassing evidence in a systematic way. This is one of the objectives of USAID ECCN, to define what can be called “evidence” and aggregate it in a way that makes sense and is actually helpful to those endeavoring to bring peace and stability. While most reliable evidence of intervention effectiveness might come from impact evaluations, plenty of other useful data resides with those who are direct participants of interventions – implementers, funders, and stakeholders. It is the challenge that’s incumbent upon all development practitioners – to fill the gap in the knowledge by contributing micro-evidence borne out of direct observation and experience to the community of practice.
At the upcoming CIES conference, the USAID ECCN research team along with International Initiative for Impact Evaluation (3ie) will talk about evidence of what works in crisis and conflict-affected areas (details here). 3ie researchers have assembled evidence and constructed maps for a variety of topics to show visually what interventions have documented their impact on the outcomes of interest, and where we still see gaps. Such “evidence gap” maps are useful for policy makers as well as donors and practitioners because they detail specific relationships rather than general assertions.
One evidence gap map that USAID ECCN is creating, using 3ie’s model, is focused on Safer Learning Environments in conflict and crisis environments. The research team has – through literature review and consultations – identified and categorized various types of SLE-specific interventions to try to make learning safer, which are often used (globally). We then collected and categorized evidence of varying degrees of rigor related to the many types of threats to safer learning environments (broadly categorized as Environmental Threats, External Threats, and Internal Threats), and put that evidence onto the map. We cast a wide net for evidence here, and still we see many gaps.
Another map in development is based on the Theories of Change utilized in USAID Goal 3 programming. The USAID ECCN research team analyzed sixteen Goal 3 projects’ results frameworks (as presented in solicitation documents) released between 2011 and 2014. The map was populated with supporting evidence for where outcome(s) and intervention(s) intersected based on the knowledge shared within the solicitation documents, and further supplemented with evidence presented in the exceptional 2015 INEE report “What Works to Promote Children’s Educational Access, Quality of Learning, and Well-being in Crisis-Affected Countries” (Burde et al, 2015), which provided an up-to-date review of academic and grey literature.
Evidence gap maps are useful for resource allocation by donors as well as national governments. Based on information they provide, programming which has been proven to be effective can receive more funding while prompting researchers to turn their attention to those relationships that, so far, have little to no data. Investments to eradicate evidence gaps will pay many times over in improved efficiency of resource allocations and ultimately better development oucomes.
What is your experience with using evidence gap maps? If you have never used them, do you think they would be a valuable resource for programming and/or prioritizing research? If so, what sort of thematic focus would you like to see on future maps?
For more information on the Identifying Data Gaps and Building Evidence for Theories of Change in Education in Conflict and Crisis Environments session at CIES, please click here. USAID ECCN recently launched a CIES Event page where you can access information about USAID ECCN hosted presentations and featured member organization presentations.