Like many areas of research, education research must also reckon with the replication crisis (Hedges, 2017). To learn from and work with other education-specific researchers on increasing awareness about the open science movement, I recently attended the Open Scholarship in Education convening sponsored by the Center for Open Science (COS). The point of the convening was to increase the understanding and use of open science practices within the field of education.
In this blog post, I share insights from a session by Brian Nosek, founder of the Center for Open Science, during which Nosek highlighted the results of a survey that collected education and social science researchers’ beliefs and behaviors regarding open science. The results, discussed below, focus on the specific practice of sharing data, but remain consistent across all open science practices (e.g., study pre-registration sharing materials).
- Knowledge: Researchers did not feel like they knew how to make sure the participant data was anonymous. They also struggled with how to share the data directly or find a place to disseminate it.
- Motivation: Researchers did not believe that sharing data was important or relevant for their project and did not want other researchers to use their data despite believing in the practice of sharing data generally. Additionally, other motivations within the education field relate to the following questions: who gets to participate (democratizing access, inclusion, flattening resource distribution), what’s being done in research, (transparency, sharing), and how we can do it better (rigor & reproducibility, facilitating self-correction).
- Pluralistic ignorance: Researchers did not think that there was support for open science, even though they and the majority of their colleagues favored the practice of sharing data.
- Infrastructure: make the uptake of open science practices possible through tools, platforms, and guided workflows. One of the main platforms that education researchers can use is the Open Science Framework (OSF) website. OSF helps researchers in all parts of the research life cycle from discovery and planning to conducting and reporting work. There are also platforms that are specific to education, such as EdArxiv for pre-preprints or the Registry of Efficacy and Effectiveness Studies for a registry of studies that allow for causal inference.
- User experience: make it easy through giving people resources, such as guides and templates. To illustrate, OSF has guides for practices like pre-registrations and registered reports, and there are education-specific resources available on the Open Scholarship Knowledge Base.
- Communities: make it normative where open science behaviors are not seen as unique by incorporating open science practices into trainings, regular discussions and aspects of the process (documenting what practices are done and where they can be accessed). This could be informal through social media conversations or adding open science badge stickers to your water bottle, but could occur through integrating into a specific content community, such as, teacher education, and using resources such as those on the COS STEM education hub.
- Incentives: make using open science practices rewarding so that people receive awards, funding, or publications. For example, registered reports allow researchers to pre-register their study through a journal, receive feedback before conducting the study, and then have a guaranteed publication if they conduct the study as recommended by the reviewers. More informal mechanisms, such as departmental leaders acknowledging this work as valuable and encouraging this work through social accountability.
- Policy: make it required, or suggested, by structures that change behaviors, such as journals, grants, or graduate student training. Many journals have taken up the Transparency and Openness Promotion Guidelines so that editors, researchers, reviewers, and other groups can understand what open science practices could look like within studies.
Photo Credit: Center for Open Science.