When examining improvement and development opportunities for your journal, editors should aim to enhance and augment the review process. Many editors would agree that the most difficult element of peer review is finding reviewers who are willing and able to evaluate a paper within a few weeks; yet this is such an integral slice of the journal’s day to day running, as it helps to improve both the speed and quality of the peer review process.
It is important to regularly appraise your reviewer list to prevent your reviewer pool becoming out-of-date. If you have a reviewer on your list who has been there for years and yet has never reviewed a paper, you should consider whether or not there is any benefit to keeping them in the pool. In her book Academic Journal Management Best Practices: Tales from the Trenches, Christine Dymek (Senior Managing Editor at journal management consultancy, Kaufman Wills Fusting & Company) notes:
“It’s very important to make sure that the reviewer pool you have is accurate and up-to-date, and that you only have people in it who actually want to review…”
More and more journals are turning to the journal metrics to do this – and both ScholarOne Manuscripts and Editorial Manager have tools on hand to help you.
When you are searching for reviewers on Editorial Manager, the system gives you the reviewer history:
This is just one way of accessing your reviewer’s performance, but it is also useful to regularly examine your reviewer pool as a whole to maximise its potential. In the Reports section of the site (under “Summary and Annual Reports”) there is the “Reviewer Performance Report”:
The Reviewer Performance Report allows you to look at your whole reviewer list to see which reviewers really stand out for returning good reviews quickly. The report also tells you: whether or not the reviewer is a board member; the number of papers they have been invited to review; how many requests the reviewer has accepted or declined; and the amount of time the reviewer has taken to respond to the invitation or complete the review. This report can also be downloaded as an Excel file.
When searching for a reviewer in ScholarOne Manuscripts, again you will see a brief overview of their performance history:
ScholarOne Manuscripts has a “Reviewer Summary” report in the User Performance Reports section:
There is the option to run the report for the whole time the journal has been on the ScholarOne Manuscripts system, or you can filter this to a set timeframe or view the most recently completed reviews. This will give you a wealth of information on the reviewer’s history, as well the reviewer’s name, email address and the following details:
There is also the Reviewer Contact Info and Lifetime Statistics for Export; all of these reports can also be downloaded in Excel to allow you to print them out and examine them in greater detail.
What to take away from this…
- If you regularly assess your reviewers, this will enrich your journal’s peer review process, as your list should start to contain a greater proportion of willing, reliable and able reviewers. This will generate best quality reviews and reduce your review times.
- Regular analysis of your reviewer list allows you see your top reviewers so you they can be rewarded at the end of the year. Run the Performance Report once or twice a year to see who your best reviewers are.
- Weed out the reviewers that never reply or do not return agreed reviews. For these reviewers, the “reviewer” role can even be expired so that they no longer show when searching for Reviewer
- The reviewer pool can be continuously topped up using the Reviewer Discovery tool on Editorial Manager and the Reviewer Locator tool in ScholarOne, which suggests other potential reviewers for a manuscript.
More information on these useful tools can be found here, or by contacting your journal’s Electronic Editorial Systems contact.