January 26, 2015 | Victoria Murphy, Programme Manager at Sense About Science

Why peer review matters when asking for evidence


Every day brings new science claims into public discussion – from news media, policy statements, advertising, websites, campaigns, and social commentary. Making sense of it can be difficult – whether you’re a parent concerned about the dangers of Wi-Fi in schools or a policy maker trying to make sense of consultation responses on nuclear energy. Which claims should be taken seriously? Which stories are “scares” or hype?

When people get in touch with us to ask questions about science, the one thing they have in common is that they don’t want to be taken back to school. People ask us “Is it a scare story?” or “What tests have been done?” or “Is it a majority opinion?” Nobody is asking us for help in relearning the periodic table. They are asking about the status of claims, the scientific process; they want a better understanding of what good evidence is.

So how do you make sense of responses when you ask for evidence, or begin to assess the evidence behind a claim? What constitutes good evidence? Since Sense About Science was set up in 2002, we have been working to popularize an understanding of peer review amongst policy makers, journalists, social influencers, and civic organizations. Peer review may not be a perfect system, but asking if something is peer-reviewed is a good first question in helping people distinguish between science and opinion.

Peer review seems to be a well-kept secret of the research community. When asked about peer review, most scientists think of the paper they have in the system, or the papers sitting threateningly in their inbox awaiting a reviewer's eye. In 2005, when we first spoke to scientists about sharing peer review with the public, they didn’t think anyone would be interested. Through workshops for early-career researchers (with support from Taylor & Francis) and discussions, we’ve encouraged scientists to think of the role that peer review plays in wider society.

We launched the U.K. version of the guide “I don’t know what to believe”  in 2005 and have since received over half a million requests for it. The requests reveal the broad range of people involved in passing on scientific information to the public: health workers, librarians, societies, public-health officials, policy-makers, technology companies, popular writers, educators, parenting groups, and local government.

The guide, now also available in U.S. and Chinese versions, explains how research findings are reviewed for validity, significance, and originality. It helps us to demystify the publication process and to get people in all walks of life asking about where research claims come from. It also explains how editors select reviewers and discusses the importance of ensuring that papers provide information so that others in the field can see how the research was conducted.

Since launching the guide we have seen a change in the way the U.K. media reports on the status and quality of research. The BBC has made it a policy to always reference where research comes from in their articles and most of the national newspapers are doing this now. This starts the public conversation about where science claims are coming from and gives people the opportunity to investigate further and weigh up conflicting stories.

The question “Is it peer-reviewed?” is now shared widely, and equips people to find out more about the status of a claim. There are a number of other resources for the public on understanding evidence available at askforevidence.org that allow everyone to ask probing questions about the responses they get.

Contrary to the fears of some researchers in the U.K., the public seems quite able to understand that peer review is an indicator of scrutiny rather than the final word. So, sharing the question “Is it peer-reviewed?” should be number one on your agenda.

 

Published: January 26, 2015 | Author: Victoria Murphy, Programme Manager at Sense About Science | Category: Front page, In the industry, Information and support, News and ideas, Peer review | Tagged with: