Research Impact: Responsible Metrics and Assessment

A collection of materials to facilitate the development of responsible research and researcher assessment policies and practices. (Note that results are filtered to "tools" and "DORA-produced")

HuMetricsHSS supports the creation of values-based frameworks to guide all kinds of scholarly process, and to promote the nurturing of a values-enacted approach to academia writ large.

The programme aims to equip learners with the skills and knowledge required to engage in the use of a range of metrics around research impact and gain understanding of the research landscape.

This document provides guidance on the use of several indicators (sometimes called metrics) used in research assessment: the Journal Impact Factor and other measurements of journals, citation counts, h-index, field-normalized citation indicators, and altmetrics. Five principles guide the use of these metrics: be clear, be transparent, be specific, be contextual, and be fair.

Changing how published papers are displayed could shift the focus from simple metrics to research quality.

Walking the Talk: Toward a Values-Aligned Academy is the culmination of 18 months of research interviews across the Big Ten Academic Alliance (BTAA). Conducted by the HuMetricsHSS Initiative as an extension of their previous work on values-enacted scholarly practice, the interviews focused on current systems of evaluation within BTAA institutions, the potential problems and inequalities of those processes, the kinds of scholarly work that could be better recognized and rewarded, and the contexts and pressures evaluators are under, including, as the process progressed, the onset and ongoing conditions of COVID-19. The interviews focused primarily on the reappointment, promotion, and tenure (RPT) process. Interviewees outlined a number of issues to be addressed, including toxicity in evaluation, scholars’ increased alienation from the work they are passionate about, and a high-level virtue-signaling of values by institutions without the infrastructure or resources to support the enactment of those values. Based on these conversations, this white paper offers a set of recommendations for making wide-scale change to address systematic injustice, erasure, and devaluation of academic labor in order to strengthen the positive public impact of scholarship.

Citations are a key measure of the impact of a researcher’s work. However, research in a variety of disciplines has found that women are less likely to be cited than men; citations contain geographic bias; and the Matthew effect—in which those who begin with advantage continue to gain advantage—leads to more citations for prominent scholars in a specific discipline.