On a balmy Monday evening, scholars, academic administrators, research funders, and librarians joined Taylor & Francis representatives at the former Singapore-branch of the Kwangtung Provincial Bank for an evening of ‘conversazione’ around current best practices for promoting and measuring the impact and engagement of research output.
What was discussed?
The session opened with an overview of the ongoing debate in the scholarly community surrounding how to measure impact, with the understanding that it varies between subject area, and by country. Traditional journal metrics such as Thomson Reuters’ Impact Factor remain the most commonly used measurement in Singapore, for example, but altmetrics and knowledge-transfer pieces such as impact statements are gaining traction. Attendees presented their perspectives on best practice, and discussed the challenges they faced.
What were the key takeaway messages?
- The journal plays a key role, not only as a brand but as a trusted platform for discussion. Journals are not just archives of research and a repository for academic career outputs. They serve as drivers of engagement with the major scientific and academic debates of the contemporary world, with the role of a great editor central to quality.
- Metrics are adjuncts to judgement. Measuring impact is difficult. Journal citation metrics are certainly useful to understand the community around a researcher or topic, but should be used as adjuncts alongside judgement to assess research quality. While citations can serve as a first-cut in an evaluation process, they cannot reflect the main ambition of researchers – for their research to reach their target audience and broaden communication within their disciplines.
- Present research in an accessible form for non-academic audiences. In many research-rich countries, we are amidst a ‘mindset change’, moving from simply driving knowledge production to also enabling knowledge translation, translating knowledge in a meaningful way for a multitude of audiences, from practitioner to public. This is especially true for research aimed at the practitioner or policy maker – it is not easy to understand what counts as research impact on the ground. And engagement with different audiences is different – the politician requires clear actionable advice, while the public wants to know how they will benefit from a piece of research. Making research articles ‘actionable’ for practitioners and people outside the academic world can also encourage a platform for academics and non-academics to debate key issues.
- ‘Publishing is now a much messier world, and the world is getting even messier!’ Overall, it is clear that more awareness of alternative metrics is needed in the academic world, industry and government. There was recognition of what journal publishers can do in terms of some of the important initiatives that they can help drive forwards. There may be a role for publishers and libraries to provide training on aspects such as how to understand altmetrics, capture and disseminate research into translatable forms, re-engineer research articles for industry blog posts or media pitches, organize professional development workshops for practitioners, and improve peer review recognition and ensure quality oversight.
The conversation on best practice for measuring impact and engagement is evolving, and we look forward to acting on a number of these initiatives, and continuing to support and engage with our partners as we navigate our new publishing world.