In an earlier blog post on sample sizes for qualitative inquiry, we discussed the concept of data saturation – the point at which no new information or themes are observed in the data – and how researchers and evaluators often use it as a guideline when designing a study.

In the same post, we provided empirical data from several methodological studies as a starting point for sample size recommendations. We simultaneously qualified our recommendations with the important observation that each research and evaluation context is unique, and that the speed at which data saturation is reached depends on a number of factors. In this post, we explore a little further this “it depends” qualification by outlining five research parameters that most commonly affect how quickly/slowly data saturation is achieved in qualitative inquiry.

1. The degree of instrument structure. The more structure embodied in the instrument, the sooner saturation will be reached. As more structure is added, the range of relevant responses is narrowed. For studies using an unstructured instrument, or no instrument at all, saturation may never be reached (the questions may be constantly changing!).
2. The degree of sample homogeneity. The more homogeneous the sample, the quicker saturation is achieved. Groups that are alike on various dimensions are more likely to think and behave in similar ways and have similar life experiences.
3. The complexity and focus of the study topic. For more complex and intricate topics, it will take longer to reach saturation than simpler and more targeted topics.
4. Study purpose. Finding high-level common themes across a sample will generally require fewer sampling units than identifying the maximum range of variation within a sample. If you’re interested in finding the big issues, a smaller sample is often sufficient. Conversely, if your study objectives require the comprehensive documentation of all the minutia your study population can provide, you’ll need to sample substantially more than the modal recommendations suggest.
5. Analyst categorization style. Some folks are “splitters.” They tend to see detail in everything, and create codebooks accordingly. On the other end of the continuum are “lumpers”— individuals who like to group things into a few large conceptual categories. Codebooks created by splitters will invariably include a lot more codes than codebooks created by lumpers. The fewer the codes (representing themes) in the analysis, the quicker saturation will be achieved.

To summarize, and provide an example, let’s imagine a context at one end of the spectrum. If a research study: 1) is assessing a complex topic, 2) has a highly heterogeneous sample, 3) aims to identify a large range of variation, 4) uses an unstructured interview guide, and 5) the analytic style is to split (code with a lot of granularity), you may never reach saturation!

Hyperbolic example 1

Your research objective is to understand the full range of views among Americans on the related concepts of democracy and governance. You decide to split your sample along party lines to compare perspectives from Democrats and Republicans, but within each sub-sample you seek to maximize the variability of viewpoints captured by including individuals representing diversity in race/ethnicity, gender, age, geography, socioeconomic and education levels. For your study, you want to ensure that participants answer in their own words, and to minimize any constraints on how they might answer. For this reason, you use a highly unstructured ‘guide’ that contains just a few overarching concepts to guide questioning and probing; questions are not scripted.

The data analyst you hire for the project is very detail-oriented, and doesn’t feel comfortable merging (in their view, losing) data points to create larger concepts/themes.

Alternatively, if your sample consists of very similar individuals, your instrument uses verbatim questions, your topic is focused and simple, and you’re looking for high-level themes that cut across participants, as few as six interviews or two focus groups will likely be enough to achieve your objectives.

Hyperbolic example 2

Your research objective is to describe what Latino Republican men living in El Paso, Texas, think about building a wall between the U.S. and Mexico. You sample a relatively homogenous group of individuals. Eligibility criteria include: being male between the ages of 18 and 40, self-identifying as Latino, living in the city of El Paso, and being a registered and self-identified member of the Republican Party. For your study, you are interested in identifying broad themes/perspectives that are shared among participants in your sample. You employ a semi-structured interview guide that contains a series of scripted, but open-ended, questions.

The data analyst you hire for the project is very good at summarizing and synthesizing multiple types of information and enjoys the squishiness of working with narratives and themes.

From these examples, you can see how the principle of saturation and the subsequent nature of your sample is directly tied to your study design, research objectives, study topic and the data collection instrument. They are all inter-related and should be considered at the onset of study design. Remember also that saturation is usually discussed or assessed per sub-population of interest. In Hyperbolic example 1, though the study is split into sub-populations of Republican and Democrat, there are so many sub-strata within those categories that saturation would require many more than the modal recommendations per sub-population.

One final caveat about saturation and recommended sample sizes: regardless of what makes methodological sense, your audience may have different standards. Many journals (and this will vary across disciplines) are less likely to accept qualitative manuscripts with smaller sample sizes, despite the evidence regarding the adequacy of small sample sizes for many types of qualitative inquiry. Likewise, PhD committees, clients and funders may often insist on larger sample sizes than are really necessary. So although increasing your sample size might not be necessary from a scientific perspective, you need to consider your audience’s predilections. In most cases, it doesn’t hurt to increase your sample size if you have the resources.

Sharing is caring!