The credibility of research impact statements: A new analysis of REF with Semantic Hypergraphs

Abstract

When asked to demonstrate the impact of their research, researchers build up statements that have a causal structure. However, as these statements have by nature a historical dimension, their credibility is under question. Historical statements have a genuine causal power only under certain conditions. We derive these conditions from the theory of historical causality and apply them to impact statements in two Medicine and Engineering (STEM) and Social Sciences and Humanities (SSH) areas of Research Excellence Framework. We then process the corpus with a novel text mining methodology called Semantic Hypergraphs. We identify the causal structure of statements and find that it is similar between STEM and SSH, but SSH makes systematically use of a larger number of actors. Making credible statements are more difficult in SSH than in STEM. We derive the policy implications for impact assessment and research policies.