Conferences are a great opportunity to connect with researchers across the spectrum, from Editors to early career researchers . This year, I brought with me a question for delegates: What Metrics Matter The Most To You?

I wanted to know what research metrics were important – what is used to judge the quality of research, to identify worthwhile articles, and how can researchers best measure the success of their work?

To get the conversation started, and to get as much feedback as possible, I used a survey board with the option to leave a comment or to simply post an Agree/Disagree sticker. The aim was to allow people to engage as much as they wanted, and many delegates chose to discuss the (de)merits of the metrics in more detail. Delegates at the European Consortium for Political Research (ECPR), the Academic Association for Contemporary European Studies Annual Conference (UACES), and Remote Sensing & Photogrammetry Society Annual Conference (RSPSoc) participated– providing a diverse range of interactions, across countries and disciplines.

Citations were overwhelmingly identified as the most important metric, yet many researchers commented on how unhappy they were with this. Citations are largely important for institutions as a measure of impact, but some questioned the extent to which citations truly reflect engagement. Is this a fair measure by which to judge output? There was also a clear distinction between citations and Impact Factor, with questions raised at to how relevant Impact Factor is, especially with tools such as Google Scholar freely available.

Social media sharing was the most divisive metric proposed. Some wondered how to effectively track social media use and questioned the level to which social media sharing represented real engagement. Each conference drew out slightly different discussions - at ECPR there was a generational divide, with younger scholars identifying it as a place to find content. RSPSoc’s attendees included industry professionals and government representatives, so it is unsurprising that they felt social media was the place to find useful research. Attendees at both conferences suggested that social media sharing might lead to citations and downloads. UACES delegates gave a mixed response, and although one participant bemoaned the use of Twitter, it was still identified as a “21^st Century Reality”. This sentiment was echoed by most delegates, who thought it would play a bigger role in research metrics in the future.

At both ECPR and RSPSoc, downloads, while acknowledged as important, received little attention. This was not the case for the delegates of the UACES conference, who noted the importance of downloads, and the potential impact they can have on the subject. At both UACES and RSPSoc delegates highlighted the move away from articles to downloading other types of content. As the research community moves further towards an Open Research landscape, more emphasis is placed on the reproducibility and accessibility of data sets – tracking the downloads, references, and replications of data sets was flagged as a big concern for the future.

Some conference attendees went beyond thinking about these three metrics. One key observation made at UACES was the emphasis placed on metrics by different disciplines; in this case, Law researchers observed that neither citations nor Impact Factor were relevant to them, but it raised questions as to how Political Scientists might apply their metric criteria to finding and assessing law research. There were also conversations as to how useful metrics were; is there too much focus on metrics and are they being used fairly? Many delegates at UACES and RSPSoc agreed with these sentiments, but still put their Agree sticker on the citations section of the survey board.

Almost everyone noted that in the modern world, one metric no longer ruled them all. A combination was needed to truly understand the impact of research - however you choose to define it - and to measure engagement.