On Friday, March 25, the Centre of Excellence for Development Impact and Learning (CEDIL) closed its 2022 conference with a panel session titled Transferring evidence between contexts. CEDIL’s four-day long conference convened researchers and experts from around the world to discuss strengthening evidence use during the pandemic and beyond.
The COVID-19 pandemic continues to present challenges that require expeditious access to high quality evidence at both global and local scales. Thus, the topic of research and evidence transferability across humanitarian and development interventions and contexts remains especially critical to informing policy responses.
With this ongoing demand for evidence in mind, the panelists explored the conditions and methods necessary for transferring evidence between different contexts, and the role of theory, evidence base, and systematic reviews.
The theories above were “the means by which evidence from other contexts informed our interventions in a novel and urgent context,” said Prof. Bonell. Although the SPI-B group assessed the face validity of the theories, rather than their evidence base, “this approach resonates with increasing interest in evaluations informing broader policy,” he added.
Ms. Kullenberg highlighted IRC’s overarching approach to transferring evidence between contexts. A key component of her presentation was a demonstration of IRC’s Outcomes and Evidence Framework tool, established in 2016, which delivers key information on programmatic interventions and outcomes in the context of available evidence. The goal was to provide “an assessment of the evidence, the strength of the evidence, and the quality of the evidence around these interventions that are linked to the outcome of concern,” she explained.
The tool has since evolved to a prototype of “Evidence Cards” that share and bring evidence together by incorporating “impact findings, both qualitative and quantitative,” including a “confidence level” indicator by IRC based on the quality and inferences that can be made from the available evidence, Ms. Kullenberg added.
Dr. Okwen discussed his experience utilizing meta-analysis to predict the transferability of evidence from global contexts to local decision-making using a mathematical model based on his work in education programs in the Lake Chad Basin. A key component of this approach is engaging with stakeholders to assess evidence transferability relevance, cost, complexity, impact, and feasibility. This step is followed by a mathematical scoring of transferability using five measurements: transferable; transferable with minor modification; transferable with moderate modification; transferable with major modification; and non-transferable.
“The more complex the program we’re designing, the more important it is we have evidence that applies to that type of program,” said Dr. Brown. “We need to make sure that we have evidence that is showing a change in one thing leads to a change something else; and finally, when we’re using middle level theories, we have evidence supporting those theories from the same intervention in one study,” she added.
Dr. Brown echoed the input of fellow panelists, noting the role of theory in transferring evidence, operationalizing quantitative and qualitative evidence in interventions, and the use of meta-analysis. She also discussed how the conversation about evidence transferability is a key element of the growing focus on locally led development, which is informing new ways of making decisions about the design and implementation of programs. Thus, it is critical to keep in mind “the challenge of how we can keep bringing evidence into these news ways of making decisions,” she noted.
Watch the panel below learn more about CEDIL’s conference sessions here.