Since the United States’ summer of racial reckoning in 2020, I have been reflecting on how I, as a white woman, have supported racist structures, especially in my career as an education researcher. I threw myself into learning the history behind many of the standard research practices I, and my colleagues, use every day. What I found was troubling.

As a result, I am joining a scholarly movement to identify and implement ways to conduct quantitative research that actively work to remove biases and reduce racist practices in what is broadly defined as “critical quantitative research.” Compared to traditional methods that view all numerical data and statistical models as objective or neutral, critical quantitative work – such as that of Stage, Rios-Aguilar, and Gillborn – highlights how humans (and their biases) influence data collection, analysis, and interpretation. Based on my reading of the works above and those from Eduardo Bonilla-Silva, Dessica DeCuir-Gunby, Verónica Vélez, and Nichole Garcia, I have identified, and present here, 10 actionable strategies education researchers can use to center the work on the societal contexts – and identify and reduce the racist structures within it –before, during, and after their studies.

The troubling history of statistics

Establishing data-driven, unbiased scientific processes are the foundation of scientifically based research practices. However, it seems that even scientific practices are prone to biases. For instance, the idea of a result being “statistically significant” can be traced directly to Francis Galton, the founder of eugenics, and Karl Pearson and Ronald Fisher, who were also proponents of using science for the advancement of the white race. “Statistical thinking and eugenicist thinking are, in fact, deeply intertwined, and many of the theoretical problems with methods like significance testing—first developed to identify racial differences—are remnants of their original purpose, to support eugenics,” (Clayton, 2020).

Part of the problem is that proponents of eugenics often use statistics to claim causation rather than observation or correlation. Correlation, which is the process of objectively analyzing data in ways that establish a relationship or connection can be a slippery slope into the fallacy of causality (Clayton, 2022. Los Angeles Review of Books). And that is often used in ways that elevate whiteness. For instance, data that shows students who perform at higher levels on standardized tests are white can wrongly be interpreted as a causal relationship that white people are inherently smarter than their Black or Latino peers. Or, because data shows that Black students are subject to disciplinary action at higher rates than their White peers might lead to incorrect causal inferences that Black students are more disrespectful, reactionary, or violent. Yet, unfortunately, education researchers continue to use statistics and significance testing as part of their methodology.

10 actionable strategies

While more research will need to be done to validate and refine these practices, the strategies described here are all based on emerging methodologies and practices that are showing promising results within a larger diversity, equity, and inclusion landscape.

1. Acknowledge systems of oppression. Nearly every education research study can benefit from a careful look at how racism, sexism, classism, homophobia, or other structural forms of domination influence outcomes. We know that race, class, and even zip code relate to student achievement. So, in studies on students’ educational achievement, researchers could discuss how racist policies such as school segregation and unequal funding impact students’ attendance, school and teacher quality, and resources that affect test scores. Rather than blame students’ attitudes or ability as the sole cause of achievement differences, critical quantitative researchers should focus on how systemic forms of discrimination play a part in creating the conditions that lead to these outcomes. One way to acknowledge these systems would be to incorporate theoretical frameworks that model how these systems of oppression affect the independent and dependent variables.
2. Involve communities of focus. When a study involves people from the same racial or ethnic group or economic class, researchers should center the work on and work with people from those groups in authentic ways instead of keeping them removed as study subjects. A participatory research partnership can provide more context to whether people within the specific demographic groups actually want or need the study to be conducted, inform variables or model considerations that should or should not be included (e.g., culturally relevant variables), support interpretation of results in a way that honors their assets and experiences, and share findings with the community in actionable ways. with can ensure that the community actually wants or needs the study to be conducted, inform variables or model considerations that should or should not be included (e.g., culturally relevant variables), and support interpretation of results in a way that honors their assets and experiences.
3. Reflect on positionality. When researchers do not acknowledge how their identities are relevant to the study, they may not realize the need to ask for insight from those with lived experiences or help the reader understand how their background shaped the study in question. To illustrate, Caroline Criado Perez in her book Invisible Women: Exposing Data Bias in a World Designed for Men documents how research conducted by men can affect and has negatively affected women in a variety of ways – from making incorrect claims related to car safety and heart attack symptoms to considering what research even gets conducted. All researchers have biases and so, through acknowledging them, we can recognize that the research team may need team members with different perspectives or help readers interpret the conclusions when the team diversity is limited.
4. Strive for (racial) equity and policy change. Critical researchers aim to make change, large or small, toward a more racially just world. For quantitative researchers, this may mean that their work helps the field understand the inequities in the first place or includes inclusive and equitable policy-relevant variables that could be used as levers for change. Additionally, this work should not exist solely within a journal or on a shelf but be deployed to change agents whether through social media posts, infographics, policy memos, or in conversations with groups who could improve that context based on the results. If conducting research on teachers’ well-being, for example, researchers should share effective interventions with principals or school district leaders whose responsibility it is to create positive and supportive working conditions for teachers.
5. Avoid recreating supremacy with your reference groups. Whether because of tradition or because they are the largest sample size, researchers will often choose white males as their reference group – the group others are then compared to. This holds white males up as a standard and can, whether intentionally or unintentionally, promote white supremacy. Instead, researchers should ask themselves if it makes sense to compare all other groups to one? Very often it does not. They can also run separate models to limit comparisons and avoid holding up whiteness as the standard.
6. Don’t “control for” race, gender, and other core identity variables. When we “control for” identity variables, we are statistically shelving them to look at how outcomes would change if everyone were on equal footing. But when those variables are directly related to systems of oppression – variables like race and gender – it’s impossible to look at everyone as equal. We must acknowledge how race, gender, and other core identity traits relate to outcomes because they are directly related to the way students experience education in the United States. Researchers might consider running multiple models by subgroup and removing “control for” when discussing core identity traits.
7. Show heterogeneity in the data and results. Showing heterogeneity — the differences within subgroups — can highlight the humanity and diversity within any group. For example, researchers often use “Asian” as a racial category. But Asia is comprised of 48 countries and cannot be reduced to one monolith. Doing so can perpetuate the “model minority” myth that all Asian students have high achievement when, in reality, achievement varies greatly within this one group. Researchers should examine heterogeneity within groups and describe the differences in their statistics and charts. This will not only show a more accurate picture of the results but will also foster a deeper understanding of each group’s experience.
8. Report more than the p value. When examining race and ethnicity, sample sizes may be too small to have enough power to detect statistically significant effects. However, just because a sample size is “too small” does not mean they do not matter. Statistics like effect sizes, or the strength of the relationship, can add valuable context. Researchers need to interpret effect sizes for practicality and policy implications by looking at the evidence specific to each community with small sample sizes to understand whether the resulting effect size “matters.”
9. Honestly discuss limitations. Critical quantitative researchers turn away from the concept of objectivity and acknowledge–and clearly state–the limitations with their data, modeling approach, and their team so that readers have a full picture of the results. For example, if the data does not allow for ideal modeling, such as if the data conflates race and ethnicity to where a participant can only be Black or Hispanic but not both. Some dataset creators do not include culturally relevant variables or did not oversample small communities for sufficient sample sizes and so stating these issues can serve as a call to action for funders and researchers to correct these issues in future data collection efforts. Or, if the research team is comprised of people without lived experience similar to the group being researched, that poses limitations in their understanding.
10. Disseminate to and with the community. Working with the communities who best know where the pain points are and who live amongst those who could make the measurable change can facilitate that change more quickly. Additionally, when presenting results to policymakers who may not know the community, having community members share their experiences alongside the numerical results can have a larger impact than sharing numbers alone.

An illustration of how to effectively work with a community can be seen in a mixed-methods evaluation with six American Indian and Alaska Native communities. FHI 360 evaluators collaboratively planned the work with the Tribes, centered tribal members’ voices in the report, discussed their initial results to the Tribes first, and then created a final report that reflected changes made from tribal discussions.

Ultimately, perfection is not possible in critical quantitative work, but researchers should try to improve when and where they can. We should all take time before, during, and after we conduct a study to consider our roles in changing systems and structures for racial equity.

Sharing is caring!