Telling the Goal 3 Story: Are we capturing the impact of our work?

To answer this question, the ECCN support team recently conducted an analysis of 26 Performance and Evaluation Monitoring Plans (PMPs) from current USAID Goal 3 (G3) programs. The analysis investigated current efforts to measure the impact of our work towards increasing equitable access to education in conflict and crisis-affected areas. As a community of practice we are working in diverse contexts, implementing innovative approaches, and surely making a difference in the lives of our beneficiaries. But the indicators that we use could tell the story better. Here is what we found:

  • Equity, Conflict Sensitive Education, and School Safety indicators are underrepresented.These three concepts are critical to address to be successful in achieving Goal 3. Many G3 programs do indeed address these topics programmatically. However, few indicators are used to measure progress toward them. The graph below shows the percent of programs with at least one indicator for each of nine topics. As the graph shows, less than half of programs measured progress toward equity, few programs measured progress toward school safety, and one in ten programs measured progress toward conflict sensitive education.

Bar graph: Percent of programs with at least one indicator per category

  • Most indicators measure outputs, not outcomes. We need to find ways to measure the impact of our work beyond counting beneficiaries of trainings or number of textbooks delivered. Examples of outcome indicators that would be useful to include in PMPs are, number of schools who meet local safety criteria, or increase in number of students from marginalized populations who can access education.
  • The connection between program outputs and outcomes frequently is unclear.
    The connection between program output indicators (e.g., teacher training, materials provision, data systems strengthening) and desired outcomes is frequently unclear. While results frameworks link outputs to outcomes, it is important that these links be further described and supported by evidence.
  • Few indicators are standardized. While some outcomes are measured using standardized indicators (e.g., F-Indicators for counting number of new students and number of teacher trained), indicators related to retention, equity, school safety, and conflict sensitive education are not standardized. The lack of standardized indicators prevents aggregating or comparing data across projects. It is important to have the right balance between customized indicators that are program and context-specific and standard indicators that can be used to look at progress in aggregate.

Despite these challenges to capturing the impact of Goal 3 programs, the future of M&E efforts for Goal 3 programs is bright! There are several ongoing initiatives which promise to help the field capture impact particularly in areas of safety, conflict sensitivity, and equity:

  • New standard indicators: The Education Office at USAID is developing new standard or F-indicators related to Goal 3. Specifically, they include indicators to measure progress toward increased equitable access to education, making learning environments safer, and increasing educators’ knowledge and skills related to conflict sensitivity. In the past few weeks many of you in the community of practice provided feedback to these indicators (thanks!), which will help to make these indicators more relevant and applicable to our work. Stay tuned in the coming weeks and months to hear more about the rollout of these standard indicators.
  • Education Equity Research Initiative: A new initiative, led by FHI 360 and Save the Children, is bringing together stakeholders from the international education world to inform policy and programming on the effective ways of strengthening equity in and through education systems. One of the areas of focus of this initiative will be to develop better ways to measure equity as part of education programming, including in the context of conflict and fragility. FHI 360’s Carina Omoeva and Save the Children’s Eric Eversmann will discuss the plans and proposed set of priorities for the initiative in an upcoming blog in late February.  In addition, Charles Gale will discuss some of the technical aspects of equity measurement during the CIES panel: Evaluating Education in Conflict and Crisis-Affected Regions.
  • New approaches to measuring safety: Two ECCN members—FHI 360 and EDC recently developed new safety indicators. FHI 360 piloted a school safety index in South Sudan that considers physical and psychosocial health on school campuses and surroundings. You can learn more about their work at the CIES panel: Conceptualizing and Measuring Safe and Healthy Learning Environments. In addition, EDC is piloting a new tool to measure efforts to increase school safety in the Gao region of northern Mali, building on Save the Children’s Quality Learning Environment (QLE) framework. To learn more about the latter, come hear EDC’s Gabriel Montero present on this new measurement tool at the CIES panel: Evaluating Education in Conflict and Crisis-Affected Regions.

These are just some examples—we know that there are other initiatives to develop better indicators and tools to document the impact of education programs in conflict and crisis-affected environments. Are you developing or applying any new indicators or tools? What other approaches can we use to better tell our story? Share your thoughts and experiences below.

Posted in Cross-Cutting and Management Activities

Leave a Comment