For the past six years, I’ve had the privilege to serve as Research and Evaluation Specialist for an integrated development project called ASPIRES – Accelerating Strategies for Practical Innovation and Research in Economic Strengthening – focused on household economic strengthening for orphans and vulnerable children (OVC) and other populations made vulnerable by HIV. When we began in 2013, there was recognition among practitioners that even the best-designed HIV-related behavior change intervention could not succeed if intended beneficiaries were so poor they couldn’t participate in or enact the new behaviors. And having worked on the socio-behavioral side of HIV prevention for several years, I shared the felt need to address poverty and economic vulnerability within HIV prevention and treatment programs. We knew – from this combination of practice wisdom and the literature – that economic need was among the issues impeding HIV prevention and treatment adherence. What we weren’t sure of was how best to address this need.

It was in this environment that USAID and PEPFAR set out to support the integration of evidence-based household economic strengthening (HES) into programming for HIV vulnerable populations through ASPIRES. Their investment was not meant to fund direct field implementation, but rather to provide evidence-based technical support to ongoing programs and to assess and build the evidence base for economic strengthening approaches in combination with other HIV-related interventions.

As we prepare for our close-out event of ASPIRES, I wanted to take this opportunity to describe our approach to developing evidence across multiple projects. In some cases, that meant pulling questions, ideas or activities apart; in others it meant piecing together data or literature.

Let’s get real(ist): The evaluation approach

USAID and PEPFAR were focused on how HES could most effectively be used to:

• improve the capacity of families to provide for the wellbeing of children in their care, especially OVC; and
• mitigate the influence of poverty and inequality as structural risk factors for HIV transmission, especially among adolescent girls and key at-risk populations.

Our interdisciplinary ASPIRES team of HES and HIV experts generated a corresponding research and learning agenda at the outset of the project. We noted that foundational to this focus was the (reasonable) assumption that providing some type of economic strengthening support to vulnerable families could positively affect economically-related HIV behaviors (e.g., reducing transactional sex, reducing cost-related barriers to HIV testing, increasing antiretroviral therapy [ART] adherence). Yet because many of the programs attempting HES-HIV integration were reality-driven – responsive to immediate observable needs on the ground – rather than theory-driven, few integrated HES-HIV projects had a clearly articulated theory of change to guide research and evaluation efforts.

In response, we adopted a realist evaluation approach, one that views programs as theories incarnate (Leeuw, 2016). Taking HES-HIV integration for vulnerable populations as a program writ large, we sought to articulate underlying theories of change and then empirically test them, to learn about how context and mechanisms interact to produce outcomes for different populations. This approach allowed for a nested set of questions, first to clarify which population(s) face which problems and for what reasons, then to describe how we think an intervention should work, and ultimately to assess whether it does work, and in what context.

The appropriate (or feasible) questions to ask about a specific program or intervention necessarily varied based on which questions had already been answered and, sometimes, which had programmatic or strategic priority. With each USAID- or Mission-led request or opportunity, we considered what kinds of information a particular program could contribute to the evidence base and then designed a research or evaluation activity accordingly.

Starting with who

For example, the USAID Mission in Côte d’Ivoire wasn’t (yet) interested in knowing which HES intervention might best support their beneficiaries, but rather, how best to define and determine relative “vulnerability” among the populations they served. At its heart, this was a “who” question – How do we know whom to engage with HES-HIV activities? – and one that initiated a productive series of learning activities.

My colleague Whitney Moret collected and compared a range of commonly used vulnerability assessments to assess the types of indicators included and the purposes for which they are used. Then, using an instrument based on this information, Holly Burke and colleagues conducted a cross-sectional survey of 3,749 households in five health regions of Côte d’Ivoire with high prevalence of HIV and large numbers of OVC. Burke et al. aimed to create a locally-relevant index for targeting interventions to vulnerable populations by identifying sets of correlated vulnerabilities. But they demonstrated instead that households face numerous unique pathways to vulnerability that cannot be readily summarized by a scorecard.

Later, similar efforts in Uganda and South Africa furthered the suggestion that broad vulnerability metrics are not sufficiently sensitive to the level of variation within local contexts to allow for use as a population segmentation/prioritization tool. With this caveat that instruments would require further local adaptation and tailoring, Moret synthesized findings from the primary and secondary data to produce an interactive handbook to assist practitioners in the selection of vulnerability assessment approaches for monitoring and evaluating economic strengthening interventions.

Clarifying the what and why

Other ASPIRES projects also started out with a “who” – a particular key population or group – and sought to clarify the “what” and “why” of their experience to improve program design and activity selection.

For instance, at the outset of ASPIRES we could say based on existing literature that financial strain is a common motivator (“why”) for women’s participation in sex work (“what”), but financial strain is a rather amorphous concept around which to design an intervention. The Key Populations Technical Working Group supported formative research in Côte d’Ivoire that allowed us to understand the financial lives of female sex workers in Abidjan through ethnographic walk-alongs and financial diaries before we set about designing an HES-HIV curriculum and intervention. We found, in short, that these women had erratic but not insubstantial cash flows and adequate food security, which had implications for the HES activities we recommended (look for a follow-up post on this topic).

Similarly, my colleagues Lara Lorenzetti and Mandy Swann are just completing a study of women accessing prevention of mother-to-child transmission (PMTCT) HIV care in Malawi that involves financial diaries, in-depth interviews and focus groups, and clinical record review to detail household economic conditions and their relationship to women’s adherence to ART and retention in care. The findings will allow selection of HES-HIV activities that align with the population’s needs and the interventions’ (theorized) mechanisms of effect.

ASPIRES work funded by the USAID’s Displaced Children and Orphans Fund (DCOF) also focused on generating evidence to help match HES activities to a household’s needs, though in the context of preventing separation of children from their families and of supporting family reintegration of previously separated children.

Within this DCOF workstream called Family Care, we had the opportunity to fund and evaluate two learning projects, both in Uganda, that offered a range of HES activities to approximately 1,400 families at risk of family-child separation or who had been reunified with a previously separated child. In addition to identifying associations between different HES activities and indicators of family-child well-being, our multi-method feasibility and impact assessment has helped to clarify the primary drivers of family-child separation in the Ugandan context, which will help us better identify at-risk families in the future and better match them with HES activities responsive to their needs. A survey of practitioners who are implementing integrated family strengthening and economic strengthening programming helped to generate a broader view of the types of activities being offered in the field, implementers’ understanding of the role of HES activities in addressing family-child separation, and practical lessons learned to be incorporated into future guidance.

Evaluating whether integrated HES-HIV interventions work

The South Africa Mission was similarly interested in learning about integrated development, and in disentangling effects from individual elements of the program. With their investment in a full-factorial randomized controlled trial (RCT) in Pretoria, South Africa, they provided a rare opportunity to test for combined effects.

This RCT, led by Holly Burke, was designed to test whether providing both HIV prevention education and financial management training for adolescents had any greater effects on HIV-related outcomes than either activity by itself. A related longitudinal qualitative study in KwaZulu Natal led by Greg Guest followed a cohort of adolescents through the training curricula to gain participant perspectives on behavior change and the activities related to them. Though the results of both studies are under embargo as of this writing, a paper discussing the RCT evaluation design provides thoughtful discussion of the challenges of implementing this “gold standard” approach in an integrated development context.

There are few such RCTs in this space and the overall scope and strength of existing evidence varies widely, as Mandy Swann found in a comprehensive evidence review (summarized here) linking a wide range of HES activities to HIV prevention and treatment outcomes. A key takeaway from the ASPIRES work in general – and from an expert consultation to define priorities for the field of HES-HIV going forward – is that we need to sufficiently understand the context of a situation to accurately understand relationships among causal factors, select appropriate HES Interventions and, ultimately, to connect and demonstrate cause and effect.

Future HES-HIV programming

Though I’ve highlighted only a few examples, the ASPIRES studies show how we can ask questions of programs at different points in the program cycle, using different methodologies, and what we can learn from them using a realist evaluation approach. As the field continues work to improve the capacity of families to provide for the wellbeing of children in their care and to mitigate the influence of poverty and inequality as structural risk factors for HIV, these types of research and evaluation activities – even if not billed as such – will be critical for improving the design, selection, implementation, and effectiveness of HES programming in the context of OVC and other vulnerable populations.

Learn more and discuss these and other ASPIRES HES-HIV projects at our June 17 event, “Learning from ASPIRES: Reality-driven economic strengthening for OVC & other vulnerable populations”, at the FHI 360 office in Washington, DC (the morning session will be live-streamed for those who cannot attend in person). Registration is free; we hope to see you there!

Photo credit: ASPIRES

This blog post was produced under United States Agency for International Development (USAID) Cooperative Agreement No. AID-OAA-LA-13-00001 and was made possible by the generous support of the American people through USAID and the United States President’s Emergency Plan for AIDS Relief. The contents are the responsibility of FHI 360 and do not necessarily reflect the views of USAID or the United States Government.

Sharing is caring!