Centrality of researchers in reforming research assessment

The Initiative for Science in Europe (ISE) has published a report identifying 18 policy options for stakeholders to improve the ways in which research and researchers are evaluated and rewarded, and to enable a transition to Open Science. This report is ISE’s contribution to current discussions that include governments, research organizations, learned societies, funders, publishers and many others – not least, the researchers themselves – at a time of increasing momentum to reform current systems of research assessment.

In addition to other flaws in the current systems of evaluation, the focus of many governments, universities, and other research performing and funding organisations on journal-level metrics and other misused proxies of research quality create contradictory incentives that have discouraged the research community from fully embracing this transition. These factors together point to the need to reform research assessment and reward systems, with the key to that reform is addressing the removal of pernicious incentives, complemented by the recognition of Open Science practices.  

ISE recognises that this implies a systemic change that might take many years and that requires coordination both within and between stakeholder groups. Inviting research communities to participate in designing the much-needed reforms is critical for the success of this transition. 

In particular, the report identifies four essential principles for a reform of research assessment to be successful:

• Engage researchers in all decisions regarding changes to research assessment.
• End the use of inappropriate metrics and abide by the principles previously outlined in the San Francisco Declaration on Research Assessment and the Leiden Manifesto.
• Agree on appropriate ways of assessing research and researchers by including suitable discipline-specific means of evaluation; establishing an appropriate balance between qualitative and quantitative evaluation; and evaluating which metrics or indicators, if any, are relevant for those assessments.
• Recognise that reforming assessment requires resources, and provide adequate funding to facilitate Open Science practices.

Both from this work and that of others (most recently, the European Commission’s scoping report on research assessment reform – LINK – and LERU’s framework for the assessment of researchers – LINK), it emerges that stakeholders’ roles, responsibilities, and own agendas may begin to amplify each other, potentially resulting in new and improved mechanisms for assessment and academic credit. ISE agrees that the time to act is now, and we must all work together to make this reform a success.

DOWNLOAD THE FINAL REPORT

Toma Susi, chair of the task force that prepared the report, presented the main lines of the report at the Paris Open Science European Conference – LINK – on 4th February 2022. To watch the video of his presentation please follow this link.