The Journal of Social Psychology is a journal that has fully embraced data sharing. Not only do they ask that authors share their data, they also reward it through Open Science Badges, developed by the Center of Open Science. These badges are voluntary notations added to the manuscript that document when authors reach beyond required levels of scientific transparency.
We asked Jon Grahe, Managing Executive Editor of The Journal of Social Psychology, to share his experiences of implementing these badges across the journal, as well as his views on data sharing as an editor.
About Open Science badges
There are 3 badges (Open Data, Open Materials, and Pre-registered) reflecting public sharing of data, materials, or a date stamped registration of the study design and analysis plan before data collection began. We were the fourth journal to adopt these badges, and we were the only journal to ever award “peer review” versions of these badges, which we stopped awarding after a year because they were too costly in time to administer. Instead, we award, “self-disclosure” badges that reward authors for providing data, material, and preregistrations in a permanent location, with a statement about how other researchers can fully reproduce the study using the associated data and or materials.
Rewarding badges in practice
In the first three years of implementation, approximately 11% of our manuscript earned data and/or materials badges. In recent months, that rate has increased to over 50%, possibly due to a new procedure (explained below), or because the American Psychological Association recently stated that all their journals will soon be able to offer Open Science badges, symbolizing a major step in their normalization.
Supporting authors sharing their data
I have observed that authors have not really thought about how best to present their data to make it easy for others to read and use. Variable names and files names should be carefully considered. When authors are interested in data or materials sharing, but don’t have experience, I direct them to some resources I created to make the process easier. In general, authors should consider data sharing as an opportunity to connect a reader of that single study to the larger research agenda. If data are published on a project that also directs readers to a main page where other study data sets are kept, the research can have even greater impact.
Beyond Open Science
It is also worth noting that data sharing is only one facet of Open Science. The Transparency and Openness Promotion (TOP) Guidelines identify eight categories that should be addressed by editors and publishers: (a) citation standards, (b) analytic and methods transparency, (c) design and analysis transparency, (d) preregistration of analysis plans, (e) data transparency, (f) research materials transparency, (g) preregistration of studies, and (h) replication. As a TOP signatory journal, we reviewed our policies and considered our standards of transparency. We were a bit surprised to recognize the discrepancy between our present standards and the expectations from this group of scientists, funders, and publishers. As such we started making other plans to improve our journal’s transparency standards.
Open Materials follows the same protocol for research materials. The Preregistered badges, which we have not yet awarded to any authors, are earned by date stamping the research study design and analysis plan before data collection. Authors are notified about the badges when they log in to submit a manuscript as part of the instructions for authors. They also are reminded of the badges in any revision or acceptance decision letter. Recently, I have started sending one more email about the badges to authors before I process the manuscript for production, and applications for the badges has dramatically increased from 11% to over 50%.
Looking towards Open Data
Until now, we have made it clear that badges are voluntary and that decisions are not associated with badge adoptions. However, from 2018, all manuscripts will need to meet the TOP standards for research materials transparency to be published in the journal. In short, everyone will need to earn an Open Materials badge to be published. We expect to require Open Data in the future as the science norms make it more common for authors to expect to share their data.