A journal’s Impact Factor tells me how good someone’s research is. Right?

Er, nope … wrong!

We advocate the responsible use of metrics at the University of Exeter – and what’s more – failing to do so could breach our commitments to the San Francisco Declaration on Research Assessment (DORA). This clearly states we should not use a journal’s Impact Factor – a tool originally developed by librarians to assess which journals to buy – when assessing the research of others.

But why does this matter? Fundamentally, failing to use research metrics responsibly has wide and serious ramifications – from the choice of research pursued, to failed career progression. It risks magnifying existing biases and inequalities within research funding and professional recognition. With regard to Impact Factors, such measures are an average of citations for a whole journal over a specific period – they convey nothing about the quality of the research reported in a specific paper, or the nature of an individual’s contribution to it. Responsible metrics are about ensuring fair research assessment and selecting indicators which provide a balanced view of a researcher or organisation’s contribution to research.  In short, not using research metrics with care and in context can lead to the discrimination of particular types of research or researcher.

Exeter’s response to signing DORA

In 2019, the Deputy Vice Chancellor for Research and Impact, as chair of the Research and Impact Executive Committee established a Responsible Metrics Champions Group. The group is made up of academic ‘champions’ across a range of disciplines to embed the principles of DORA in their departments, alongside Professional Services colleagues to help support the culture and policy change needed.

In 2022, the group commissioned a piece of scoping research to better understand what our community understood about responsible metrics; essentially ‘where we are now.’ This highlighted that 61% of colleagues surveyed had not heard about responsible metrics. Over 40% agreed that Journal Impact Factors are a good measure of the quality of research articles, yet nearly 60% couldn’t explain how these are calculated. 67% agreed that they feel under pressure to publish in journals with a “high” impact factor.

As a result, group members have developed a range of resources to help broaden our collective understanding, starting with an agreed set of Guiding Principles to which all University of Exeter colleagues should adhere.

 

But what our Guiding Principles don’t explicitly address is what time-poor colleagues should use instead of short-hand proxy metrics when assessing others’ research. This is where our indicators guidance can be used. Developed by colleagues with expertise in research assessment and bibliometrics, this seeks to guide decision making based on individual circumstances.

“The key takeaway from our Guiding Principles and indicators guidance is that quantitative metrics should be chosen and used with care, and especially that expert or peer judgement based on qualitative evidence should always play a part.” Rob Anderson, Chair of the Responsible Metrics Champions Group.

Integrating Responsible Metrics into our Recruitment, Promotion and Progression policies

This is probably the area which has the greatest potential to positively impact on colleagues; and the Exeter Academic Review provides the opportunity to systematically look at the proposed processes and criteria through a responsible metrics lens.

The Champions Group have fed into the initial stages of this process and will next look to ensure responsible metrics are considered within all areas of our recruitment, promotion and progression through a sub-group. Unpicking what this means in practice will be challenging: on the one hand, ensuring disciplinary differences are considered – and on the other, making sure the information and time demands of new processes are realistic.

Find out more about responsible metrics

Rebecca Euesden
Research and Communications Officer (Responsible Metrics)

Prof Rob Anderson
Chair of the Responsible Metrics Champions Group