My team recently encouraged a colleague who is collecting and analyzing data in one of our projects to submit the analysis and results to a journal. His response was “why?” In our first attempt to answer his question, we pointed him to David Evans’ recent post about why people might want to submit short (economics) papers as “letters” to journals that publish letters. David says, “If you believe there is value in expanding the audience, in the quality improvement, and if the citations are useful for you professionally, then it may be worthwhile.”

My colleague argued that his audience is not reading journals, that there are other mechanisms for getting feedback, and that since he is a technical person and not a researcher, he doesn’t need citations. He believes publishing in other ways is equally valued. In other words, he wasn’t convinced.*

This post is my attempt to convince him, and you, that practitioners who are conducting research should publish in journals.

Before giving you my reasons, I think it is important to define what I mean by research. I have in mind a slightly altered version of the US Department for Health and Human Services definition. Mine is: a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to knowledge. My alteration is that I deleted the word “generalizable” before knowledge. To me, a contribution to knowledge means it needs to be more than just information about a specific project useful only to those working on that project. But I’ve found that people can get caught up in what the requirements are for “generalizability” or “externally validity”, so I am avoiding those more loaded terms here. My interest here is when people carefully analyze data or other information to produce new knowledge that can be helpful to others.

So why should practitioners go to the effort of submitting an article to a journal instead of just self-publishing and writing a blog post about it? I see three reasons: because it increases the credibility of the research, it increases the credibility of the practitioner, and it puts the knowledge in the permanent, searchable record. Spoiler alert: I think the third is the most important.

To understand what makes research credible, it is important to recognize that where people want to get their information is not the same thing as where they want the information to come from. I like to learn about research from blog posts as much as anyone, but we all know you can’t believe everything you read on the internet. What makes research findings reported in blog posts credible is where they come from, and the most credible source is a journal article. Here’s a great example that hit my inbox while I was writing this: an ICTworks post about an article published in PLoS One. Even if our desired audience never reads the journal articles, they still want to know that the information they get from blogs, briefs, websites and conferences is credible, and a link or reference to the source journal article lends credibility.

We know that the US Agency for International Development (USAID) and other funders place a premium on evidence from peer-reviewed journals. From my own experience producing evidence products for USAID (see for example the work on STIP), I know that many at USAID do look for whether evidence is from a peer-reviewed journal. In fact, USAID launched its own peer-reviewed journal, Global Health: Science and Practice to make this medium more accessible to both the producers and users of research. Here is a statement from the Gates Foundation highlighting the importance of peer-reviewed publishing.

It is a reasonable question whether the process of publishing in a peer-reviewed journal indeed improves the quality of the research. Many have debated this question (for example here). I personally do not believe that peer review guarantees the quality of the research published in journals, which is why I think that replication is so important. But I firmly believe that the average quality of studies published in peer-reviewed journals is higher than the average quality of studies published elsewhere, even if those published elsewhere are subject to some kind of peer review. You can find empirical evidence to that effect by looking at systematic reviews that do risk-of-bias assessments on included studies that come from various publication sources.

My claim about the relationship between journal publication and quality is a claim of correlation not necessarily causality. Submission to a peer-reviewed journal is an important signal, both that the researcher feels the work is of a standard to be published in a journal and that she is willing to undergo rigorous external peer review. So even if the peer review process itself doesn’t improve the quality (although more often than not it does), the work that was done to get the manuscript journal-ready probably did.

Publication in journals also benefits the careers of those who publish, even technical specialists who publish infrequently. Journal publications are a strong signal of thought leadership, a quality that organizations and donors like to see in technical specialists as well as researchers. The articles themselves are evidence that the author is contributing to the knowledge base, and the motivation to publish signals that a person engaged with the current advancements in her field. Conference organizers prefer speakers with publication records, even for panels that are not about research, as they want speakers who are recognized thought leaders. Over my career I have recruited for hundreds of short-term and long-term positions for USAID and other funder proposals and projects, and while a record of journal publication is rarely a necessary credential, it is often a favored credential.

The USAID Bureau for Global Health commissioned this guide on peer-reviewed publishing, which encourages its own staff to publish and states, “One way to show and develop technical leadership is by publishing, especially in peer-reviewed journals.”

Many of you who have made it this far in the post are asking (perhaps emphatically) “if USAID cares so much about journal publication, why do they refuse to pay for it??” Good question. Very good question. It is definitely true that many USAID projects—even some with research as a primary component—do not support journal publication. I think the concepts of time inconsistency and public goods help us to understand this, but I’ll leave that discussion for another post. What I’ll say here is, we need to keep gently encouraging USAID to let their projects contribute credible research to the knowledge base upon which they themselves rely.

To make my third point, about the permanence of journal articles, I’ll start with an anecdote. A few weeks ago, we celebrated the milestone of having more than 10 posts in our new R&E Search for Evidence blog. As I opened the web page to post number 11 and then scrolled backward through the earlier posts, I saw that the post number 1 had disappeared! In turns out the blog was initially programmed to include only 10 posts at a time. We have fixed this, but it drove home the point that blog posts can be ephemeral.

Even if a post lives forever on the internet, the chance that someone looking for research findings is going to find a blog post published a year ago, much less five years ago, is slim. Anyone who is doing a literature search more serious than just typing a few words in Google starts by searching indexes (e.g. EconLit and PubMed). Blog posts, policy briefs and webpages are not typically indexed, neither are most institutional reports. Systematic searches will include databases as well as indexes, and databases often include many working papers and institutional reports. But database searches are very labor intensive, and they still do not uncover blog posts. That means that research not published in journals has, at best, limited long-term impact and, even worse, is overlooked in literature reviews and evidence syntheses that are designed to find the knowledge that is generalizable from large bodies of research.

I put the credibility reasons for publishing in journals first, because I think they appeal more to individual incentives. But if the mission of our work is ultimately social impact, then the third reason is the most important. If the question is salient enough to do research, and the research findings are credible enough to disseminate publicly, then shouldn’t we make sure they enter the permanent, searchable, public record?

I will concede one point to my colleague: citations are not such a big deal to technical specialists. Citations reflect whether other researchers are using your research when they write journal articles. If the objective of your research is to inform policy or practice directly, a journal article summarized in a blog post or policy brief may have a big impact without garnering a lot of citations. I do know that people reviewing CVs pay some attention to the quality of journals in which candidates publish, but I have never heard of a case where someone reviewing CVs for a technical position has looked up the number of citations that the candidate has.

Not all practitioners conduct research, so I am not arguing that all practitioners should be publishing in journals. But there are many times when technical specialists do conduct research as part of their project work. Ideally, that would happen more often than it already does, as the need to learn is infinite and our projects are a better source of data than we realize. It is true that publishing in journals is not easy and takes time (another topic for another post) so it is not always feasible. But I think practitioners can and should be publishing in journals more than they currently do.


*In fairness to David, he was not making a case for publishing overall, just for publishing letters, so we misapplied his post a bit in trying to make the initial case to our colleague.

Photo credit: ksandrphoto/Freepik

Sharing is caring!