Over the past several decades, the wealth gap in the U.S. has been steadily widening and is particularly pronounced along racial lines. Historically, systemic racism and discriminatory policies have resulted in a significant disparity in wealth between white households and Black households. The average Black household earns about half as much as the average white household and owns about 15 percent as much net wealth (Federal Reserve, 2021). The Greater Washington Urban League (GWUL), with funding support from the United Way, implements the Center for Financial Inclusion (CFI), which is designed to address the historic and systemic causes that perpetuate economic inequality.

We recently completed an evaluation of the CFI using a framework that emphasizes collaboration and capacity building. This approach provides guidance to researchers, developers and implementers of similar adult financial programs on strategies for developing internal evaluation capacity and increasing reliability and usability of evaluation findings. In this post, we reflect on our experience evaluating GWUL’s CFI, share our approach and methods for designing a collaborative evaluation framework and briefly discuss key findings.

Innovation in Adult Financial Education

GWUL’s financial empowerment approach acknowledges the Black community’s historic struggles with institutional racism and discrimination in housing, education and the labor market, which have contributed to limited resources and long-term poverty. Society tends to blame individuals from vulnerable populations and doubt their ability to manage assets, causing financial trauma among members of those populations. This is why GWUL’s CFI takes a long-term trauma-informed approach that focuses on financial therapy. This approach consists of a series of webinars that help clients grasp the underlying factors influencing their financial behaviors, including the neuroscience of decision-making. Financial coaches and therapists assist clients with changing their behaviors, and personalized one-on-one counseling helps clients develop a “financial freedom plan” that focuses on accumulating assets and building generational wealth.

Throughout each phase of our evaluation design, we worked closely with GWUL and CFI program staff to ensure its usefulness, relevancy and the accuracy of findings. During the planning phase, we met with GWUL’s chief administrative and financial officer, the associate director of programs and strategic partnerships and the CFI project director to review program offerings, design initial evaluation questions and logic models and align our data collection to program activities. As we implemented the evaluation, we shared data collection responsibilities with the project director and program staff. During the completion phase, we collaborated with GWUL and CFI program staff to clean and analyze data, building GWUL’s capacity to evaluate its programs. And during the final dissemination phase, we worked together on a visual report and co-hosted a launch event to share findings with clients, partners and funders.

Our collaborative framework was beneficial to GWUL in that is created a customized design that reflected their current needs and capacity. As a new program, many components were in flux or under development as staff were brought on or as demands changed (e.g., onset of COVID-19).  The evaluation design’s flexibility allowed us to adapt and revise data collection to respond to changes in program implementation over time. GWUL continued to adjust and improve the CFI to meet their clients’ needs and maintained rigorous data collection and evaluation at the same time. By working so collaboratively, we were able to build buy-in from staff, clients and community members, which helped increase the validity of our findings.

Methods

Our collaborative evaluation design utilized a mix-method approach that shared responsibility for data collection with GWUL. FHI 360 supported GWUL in developing a plan to track and collect data and to ensure accuracy and completeness of records. Quantitative data collection built upon existing CFI data collection efforts such as GWUL registration forms, planned event surveys and existing financial wellness scales and metrics. Led by FHI 360, qualitative data collection consisted of focus groups and interviews with participating clients and a series of follow-up interviews. Together, these quantitative and qualitative approaches focused on developing an impact narrative that would emphasize the client journey/experience. We used the following sources of data for this evaluation:

• Administrative data were collected by GWUL. Participant activities, outputs and outcomes were tracked by GWUL using service data, notes and administrative records. This includes client registration information and demographics, the number of people participating in each activity and the number of activity offerings.
• Participant surveys were collaboratively designed by FHI 360 and GWUL to measure short-, mid- and long-term outcomes (e.g., how past trauma shows up in financial decision-making, understanding relevant financial terms and achieving financial stability). CFI program staff administered the surveys online at the completion of each webinar and as an annual survey.
• The Consumer Financial Protection Bureau’s Abbreviated Financial Well-Being Scale assessed clients’ current financial well-being, tracked progress over time and helped show how program activities were affecting clients’ financial well-being. GWUL administered the scale at registration and key points throughout the year at the completion of program activities (e.g., counseling).
• Participant focus groups were conducted by FHI 360 to understand client experiences in CFI program offerings and financial freedom journeys. Analysts sought common themes and unique experiences to highlight as findings related to the client experience and key takeaways that might inform future project implementation.
• Serial interviews with five identified clients were conducted by FHI 360 to better understand client experiences and financial wellness journeys and to provide rich, in-depth client case studies.

Given the need for timely results to inform programmatic design, FHI 360 conducted ongoing descriptive analyses during the data collection period through the development of a Microsoft Power BI data dashboard. Service data, outputs and outcome measurements collected by the GWUL provided the necessary inputs for the dashboard, which FHI 360 updated on an ongoing basis. Thematic analysis of qualitative data identified emerging themes and patterns and emphasized what brought clients to the CFI, what successes and challenges they encountered and what their plans were for continuing to learn and progress financially.

Findings

Evaluation data showed that overall, 94 percent of clients reported improvement in at least one area of their financial situation. Eighty-nine percent were committed to building assets by reducing debt, improving their credit, boosting savings, buying insurance and owning a home. Eighty-one percent believed they have the right to accumulate assets and build generational wealth, and 80 percent reported understanding the root causes of their relationship with money. Additionally, five client case studies documented participants’ financial wellness journeys and emphasized the importance of prioritizing healing their emotional relationship with money before helping others, working toward financial stability and a secure retirement, identifying the root causes of financial trauma, finding support in the community and becoming accountable.

Conclusions and Implications

Our collaborative approach to the evaluation of GWUL’s CFI involved regular communication and collaboration to facilitate the sharing of insights, data and feedback. The goal was to foster a sense of ownership and engagement among all participants, making the evaluation more credible, relevant and likely to lead to actionable recommendations for improvement. By involving GWUL, their partners, funders and clients in the evaluation process, our collaborative approach enhanced transparency, built trust and created a supportive environment for continuous improvement that strengthened GWUL’s overall capacity to evaluate the CFI and its other programs.

Sharing is caring!