Abstract
It has been argued that due to the growing importance attributed to research impact and forms of its evaluation, an academic ‘culture of impact’ is emerging. It would include certain concepts, values, and skills related to the area of generating and documenting impact. We use thematic and discourse analysis to analyse open answers from 100 questionnaires on research impact submitted by ECRs working in the social sciences and humanities (SSH) in Europe. We explore ECR’s early-career stage positions relative to societal impact and the trade-offs necessary to assure an academic career. The results show how, as the first generation of scholars to be socialized towards value of academic research beyond academia, ECRs are confronted with policy signals that encourage a drive for impact, which are at the same time often in line with respondents’ personal values around impact beyond academia. However, ECRs face a number of competing signals about research value within the evaluation spaces necessary to navigate an academic career. Current evaluative structures often dismiss the achievement of societal impact favouring instead narrower definitions of research excellence. Career structures and organizational realities are often unfavourable to impact-related activity, which has implications for an ECRs’ ability to develop coherent professional positionings.
Dedication to Prof Paul Benneworth
Paul was the driving force of both the ENRESSH (European Network for Research Evaluation in the Social Sciences and Humanities—EU COST (European Cooperation in Science and Technology) Action CA15137 project) and CARES (Careers and Research Evaluation Systems for Societal Impact) project that underpin this submission. It was through our associations with Paul that this research arose and, by extension, this article. He was the life of the party of ENRESSH—he brought us together, and kept us inspired and motivated. Paul had the gift of inspiring curiosity in others. It was always fun working with him, never boring. We are happy that we are able to keep some of Paul’s work alive and to publish it. Paul passed away too early both for his own life, as well as his colleagues and students’ relations with him. He was able to bring together, in CARES and ENRESSH, an enthusiastic group of researchers who made plans for future projects in impact and evaluation. Paul was, amongst many things, a champion for ECRs and it is through bringing these projects to fruition that we wish to honour Paul’s memory.
Introduction
Academia is a complex space to navigate, particularly for Early Career Researchers (ECRs). Growing numbers of PhD graduates must compete for a small number of jobs (Wellcome 2020; Commonfund Institute 2021), while expectations as to the candidates’ profiles become exorbitant: a recent study showed that the entry-level threshold for academic positions is considerably higher than 10 years earlier (Warren 2019). The nature of academic employment is becoming increasingly precarious (particularly at the early stages of one’s career, often characterized by several post doc jobs) (Armano and Murgia 2012; Murgia and Poggio 2018) while workloads become more and more punishing, leading to burnout and poor mental health (Gill 2009; Pereira 2017). These factors put pressure on young scholars who need to negotiate organizational, governmental, and public demands from academics’ work while they work towards research independence and an established academic career. Given limited resources and increasing workloads, ECRs must make choices and prioritize some demands on their time and attention over others. It is within this process that young scholars choose their unique career path and forge their academic identities.
Extra-academic impact (henceforth ‘impact’) as a formalized element of academic reality and increasingly a criterion being adopted globally for the evaluation of research value. Used commonly in coordination with more traditional notions of research excellence, extra-academic impact (societal impact) is a new factor in the landscape of academic realities. It has been adopted in its ex ante evaluations within EU-wide funding programmes (e.g., Horizon 2020 & European Research Council), as well as in many nationally based funding organizations (UK Research and Innovation, National Science Foundation (NSF), National Institute for Health (NIH), Research Council of Norway, etc.). It is also used in ex-post form in formalized research audit frameworks in UK (HEFCE 2014; UKRI 2019), Italy, Poland (Wróblewska 2017), and Norway (Wróblewska 2019) (for an overview of approaches to impact evaluation in different countries see Grant et al. 2009; Donovan 2011; European Science Foundation 2012). While scholars have always engaged in such extra-impact work informally (Hamann and Gengnagel 2014; Pearce and Evans 2018), it was not, until quite recently, explicitly valued as a component of research ‘excellence’ or ‘quality’ (Hessels, Van Lente and Smits 2009). Examples of ‘impactful’ activities which would fall under extra-academic impact include collaboration with industry, social outreach, appearances in media, activism, political activity, etc.
Despite the fast evolution and adoption of extra-academic impact, the research reward cycle as it is currently conceptualized provides little recognition of these new modes of research interaction and engagement. While official institutional communication on the role of science and scientists often highlights the importance of public engagement, outreach, and dissemination, it is achievements related to publications, teaching, and administration which remain crucial for one’s career progression (with publications and grants in particular being associated with academic excellence) (Watermeyer 2015; Hamann 2019). Against the backdrop of conflicting information about excellence, value, and research success (Smith and Stewart 2017), ECRs face the conundrum of resolving conflicting messages of career advancement. ECRs often have to choose between building up credibility by engaging in work that is recognized and rewarded within the current research reward cycle, with altruistic notions of wider community engagement and impact. In this zero-sum game for individual ECRs, time and effort spent on one activity (in this case: public engagement/impact), inevitably drains away time allocated to the other (in this case: obtaining traditional measures of academic esteem). This contributes to an evaluative environment where different and even conflicting notions of academic value are in competition with each-other along with personal positions and aspirations towards using research for public good.
Although this reality is shared by many members of the academic community, this article argues that it is particularly challenging for ECRs as those who are at the stage of building up their academic credentials and therefore have to choose in which area to invest their energies. Additionally, ECRs may lack the experience of their more established colleagues both in engaging in outreach activities and in making strategic use of academic policies. Guidance and mentoring in the area of non-academic impact remains limited and traditional forms of mentoring may not be as reliable, as the entire research community remains uncertain of the definitions, and avenues for achieving successful impact. There is still very little research focusing on how ECRs navigate and resolve their identities within an evolving evaluative landscape, including the introduction and ongoing socialization of ambiguous criteria such as extra-academic impact.
This article explores how early career researchers interpret and work with a range of different signals regarding extra-academic impact in the formation of their academic identity. Specifically, the article explores how ECRs are able to integrate conflicting signals related to extra-academic impact into their career narrative. We argue that reconciling conflicting notions of research excellence, including the notion of impact, is a crucial challenge in the socialization of early-stage academics, one which may have important effects on the construction of their academic identities.
The study below takes a constructivist perspective on identity in adopting the assumption that identities are constructed via discourse (de Fina, Schiffrin and Bamberg 2006; Bamberg, De Fina and Schiffrin 2011). ‘Discourse’ is understood here as a situated social practice of individuals or groups interacting through language (text and talk) and negotiating their identities in social contexts (Angermuller, Maingueneau and Wodak 2014). In the analysis, we strive to look beyond stable ‘inherent’ identities which would solely be reflected in respondents’ words, to the process of discursive negotiation in the formation of academic identities. Hence, this article aims to address the following research question:
How do ECRs discursively position themselves towards the concept of ‘research impact’ in navigating conflicting career signals?
Literature review
The interaction, lived experiences and guidance received at an early stage of an academic career are vital in shaping the development of an academic career. Achieving ‘research independence’ is widely considered to be the objective of the ECR stage of an academic career (Laudel and Glaser 2008). ECRs have been labelled ‘invisible scholars’ (Fenby-Hulse, Heywood and Walker 2019) as this career stage exists within a linear trajectory of striving to achieve research reputation and formal recognition (leading to a stable employment status). Within this liminal space, ECRs continue to establish their identities through achieving small successes. These ‘signals’ (Derrick and Benneworth 2020) can include formal and informal interactions with more experienced and established academic colleagues such as mentors, as well as the blinded feedback received through established avenues of academic governance through scholarly or grant peer review (Derrick, forthcoming). The resolution of this professional identity requires the management of a ‘constellation of influences’ (Laudel and Glaser 2008) in making sense of experiences towards achieving research independence and career stability.
There is a need to understand how personal values and individual desires of individuals overlap with the professional criteria used to assess emerging, and ambiguous criteria for academic success. This is especially important if personal values of the individual resonate with these new criteria. Such is the case with the emerging and still ambiguous (Derrick 2018) impact criteria. As is appropriate for their career level, ECRs are constantly looking for feedback and validation from the academic community, that they are ‘doing well’ and are ‘on the right path’. However, the uncertainty around definitions of impact, and the lack of direct rewards for success in impact, makes it difficult for ECRs to receive this validation. In consequence, the pursuit and achievement of impact become associated more with personal values rather than an effort required to establish a successful career in academia long term.
Discursive positioning and academic identity
Within this constellation of influences, ECRs must also learn about, and learn how to navigate, the explicit and implicit norms and values of academia. How norms and values related to performance metrics are stabilized as they become integrated into routine practices of knowledge production is an area of interest within the disciplines of research evaluation and sociology of science (Müller and de Rijcke 2018). In the traditional apprenticeship model of research careers (Laudel and Glaser 2008), these norms are reinforced through a process of academic socialization (Duff 2010), where future success hinges upon the individual’s ability to demonstrate conformity and achievement relative to the ‘totems’ of the profession (Becher and Trowler 2001) or accomplishment on the established scale of academic values (Lamont 2009). For ECRs therefore, given the precarious nature of employment and career avenues, the pressure to conform to established criteria of research quality and success is strong. McAlpine et al. (2018) argue however for a need to look beyond issues related to work in order to understand individual decisions to remain, or to abandon the academic career. These include life goals, work-life balance, wellbeing, and stress; as well as personal values such as a desire to use research to bring about social change.
As part of the process of academic socialization, ECRs develop their own narratives (stories) about who they are as professionals, what kind of research they do, and what motivates them (Georgakopoulou 2006; Bamberg 2011; Hah 2020). These narratives, when presented to other actors in the social space, or even to oneself, often include instances of what is referred to as ‘positioning’ (Bamberg 1997). ‘Positioning’ is understood as performance-based claims to identity and subjectivity (Davies and Harré 1990), or more generally ‘the way in which people dynamically produce and explain the everyday behaviour of themselves and others’ (Van Langenhove and Harré 1999: 103). Positioning theorists stress how different roles are invoked and enacted in a continuous game of positioning (oneself) and being positioned (by others). In academic contexts positioning may take the form of indexing identities such as ‘scholar’, ‘linguist’, ‘early career researcher’, ‘activist’, etc. (Baert 2012; Angermuller 2013; Hah 2019, 2020). Through such narratives, scholars position themselves in relation to various aspects of their work (such as disciplines, theories, schools of thought, intellectual environments) or possible areas of engagement (the traditional strands being scholarly and organizational activity, and impact a new addition to the mix).
Positioning practices are inherent to the process of navigating the complex signals and requirements of the academic world (Angermuller 2013). It is argued that academics position themselves discursively in relation to science (specialized knowledge), as well as in relation to institutional hierarchy (power) (Bourdieu 1988). Hence a scholar progresses in their career through carving a niche for oneself in terms of their discipline, area of work, topics, and their role in the academic institution (assistant professor, research lead, dean, etc.). It is through this process of occupying discursively constructed positions that one’s identity is shaped (Bamberg, De Fina and Schiffrin 2011).
As a result of the hypercompetitive nature of the research landscape ECRs have to distinguish themselves from other aspiring early career scholars (Christian et al. 2021) are hence may be particularly responsive to incentives and rewards which could allow them to do so. In an evaluative landscape which continues to change and introduce new, less established criteria such as impact, there is a tension between how ECRs access the guidance and signals necessary to navigate a hypercompetitive research career trajectory. In these circumstances, traditionally ECRs would rely on guidance from more senior academics as mentors, however there is still widespread general confusion and mistrust around impact as a concept even amongst more senior academics. This results in the traditional avenues for guidance remaining incapable of providing the type of clarity of progression desired by ECRs in terms of orientating their own professional choices. ECRs therefore are faced with a risk of compromising their career progression by engaging in activities where senior guidance and positive signals are obscure and perceived as less rewarded by the science system as well as where success is far from guaranteed. In many research systems, the pathway to academic prestige is still dominated by traditional indicators of research ‘excellence’. The focus remains on publications as an essential achievement for ECRs to demonstrate. Studies have shown for instance that Spanish scientists who publish before obtaining their PhD reach tenure status faster than scientists who do not (Sanz-Menéndez, Cruz-Castro and Alva 2013) or that German sociologists increase their chance of tenure by 10–15% with every published peer-reviewed journal article or monograph (Lutter and Schröder 2016). ECRs must choose between engaging in activities that will result in achieving traditional indicators of research independence, and activities that are in line with their personal value systems (McAlpine et al. 2018) and that present a motivator for continuing their research career.
Impact as a component of research excellence: tensions in academic positioning
The introduction of both ex-ante and ex-post impact as a formalized criterion in many countries (Wróblewska 2019, 2021) represents the most recent development in the evaluative environment surrounding research assessment. The process of incorporating impact into the definition of research ‘excellence’ and ‘quality’ is under way. Impact is slowly becoming valorized in research practice and formalized in research assessment. Research councils worldwide are increasingly including ex-ante impact criteria which feature either as a formalized section (e.g., Research Council of Norway—Forskningsradet in Norway, Horizon 2020), or are built into the entire proposal throughout the sections (e.g. UKRI). The inclusion of ex-post impact evaluation in the UK’s Research Excellence Framework (REF) is the most prominent example of this process of formalization and socialization for impact. The precise definition of ‘impact’ and the criteria according to which it is assessed will depend on the national and institutional context. And, in each case, they are subject to a process of discussion, negotiation, and refinement (for a reconstruction of the process in the case of REF see: Smith, Ward and House 2011; Derrick 2018; Wróblewska 2018). In this sense the concept of ‘impact’ is constructed through discourse. If and at what pace the academic community will internalize the concept of impact, whether it will accept the definitions and rules present in the policy depends on the local context and the perceived importance of the evaluation in question (De Jong, Smit and Van Drooge 2016; Wróblewska 2019, 2021). It has been argued that an ‘impact infrastructure’ (a dispositif in the Foucauldian sense) is emerging around the area of impact valuation—in effect academics are trained and guided to strive for impact (Wróblewska 2021). Yet, it is still far from being a well-established element of academic excellence on a par with traditional indicators, such as publication and funding capture. Impact is becoming yet another area, alongside that of specialized knowledge and institutional hierarchy, in which one must establish oneself in order to be a successful academic.
This change in evaluative expectations is likely to affect different groups of academics to a different degree. For example, recent evidence has shown how impact is a highly gendered concept both in its generation and evaluation (Chubb and Derrick 2020), inadvertently contributing further to gender disparities in research production and reward. Likewise, the formal inclusion of impact among evaluation criteria will influence senior academics in permanent or less precarious employment situations differently than ECRs who are still attempting to achieve research independence and greater security within their research career. Similarly, the differences in the definition and operationalization of impact between national systems are likely to impact ECRs who tend to be more internationally mobile than more established researchers.
With the increased prominence of impact, areas of activity which were irrelevant or marginal for one’s academic credibility are slowly incorporated into the idea of academic excellence. In navigating this area, ERCs undertake discursive work by positioning themselves in relation to impact, that is by ascribing certain values and meanings to their choices in this area, invoking particular categories and aligning themselves with specific trends in academia.
Methods
This article is based on an exploratory piece of research which used an open survey to address the following research question: ‘how do ECRs discursively position themselves towards impact’. An initial impulse to study this question came from a training school for early career SSH researchers on research impact (Vienna, 30 November 2018) which was part of CARES Project within the European Network of Research Evaluation of Social Sciences and Humanities (ENRESSH). During this session many participating researchers articulated identity tensions around the newly emerging issue of impact.
Survey construction
In an attempt to map this problem more systematically, we used a survey featuring a set of open research questions. The survey was developed using a focus group of participating experts in the topics of ECRs and social impact from the EU COST Action ENRESSH in July 2018 (n = 10). From open discussions a long list of questions was developed and then curated into a questionnaire by the project team led by Prof Paul Benneworth. The survey was then piloted with five respondents who completed and provided direct feedback on the survey instrument. This feedback was then used to refine the instrument as well as anticipate relevant research hypotheses.
The final instrument was distributed amongst early career researchers in order to allow respondents to describe, in their own words, the issues that arose from their experiences with pursuing research impact. The survey acknowledged that definitions of ‘impact’ are contested and differentially linked to evaluation contexts, and therefore did not impose a particular definition or conceptualization of ‘impact’. Instead, respondents were allowed to define what ‘impact’ meant themselves. This approach gives scope for reflection on how definitions differed across various EU-contexts during the analysis.
Survey instrument
The survey included a series of Likert scale questions, as well as free text space for gathering qualitative data. A wide range of ECR participants were sampled through the ENRESSH network, including the Special Interest Group on Early Career Participants, as well as these participants’ personal and professional networks across Europe. A total of n = 111 responses were received from 30 countries, including one from a non-European country, an associate participant in ENRESSH. Below is an overview of the respondents’ geographical spread—in terms of their nationalities (35 in number) and countries of residence (29 in number). As can be seen in Figure 1A and B, ECRs show a fair level of mobility considering their current country of residence, and their nationality. This is important to acknowledge because it shows that many ECRs are highly mobile—this can influence the ways in which they are able (or not) to create impact in the countries where they are undertaking their research (e.g., not knowing the local language may constitute a structural barrier for ECRs’ ability to access local knowledge or participate in national policy debates). Furthermore, ‘impact’ means different things in different countries, so when ECRs move, they also take with them certain ideas of impact that may or may not hold the same value in the host country. For the reasons stated above, and also due to the differences in levels of integration of impact into various national and institutional academic systems of valuation, we do not take a cross-national approach in the analysis.
(A) Nationality of survey respondents (n = 111). (B) Current country of residence of survey respondents (n = 111).
Analysis
For the purposes of this article, only the free-text data from the surveys was analysed. The analysis of the qualitative data stemming from the open questions within the survey included a coordinated approach that utilized thematic analysis encompassing an initially broad approach identifying emerging topics (Guest, MacQueen and Namey 2011) overlaid with a positioning and discourse analysis focused on emerging tensions and conflicts visible in the respondents’ discourse. In the first, thematic part of the analysis, we coded longer fragments (sentences or their sequences) for themes expressing generally positive versus negative orientations towards impact. Coding was carried out collaboratively by two of authors of the article in a bottom-up process informed by grounded theory (Corbin, Strauss and Strauss 2008). The adoption of a team-based approach increases the dependability and trustworthiness of the developed coding (Cascio et al. 2019). In the second, discursive part of the analysis, we took a closer look at how the positive and negative evaluations are intertwined at the level of discourse (e.g., within a single sentence or utterance). This allows us to draw conclusions about the dynamics of the subjects positioning towards the notion of ‘impact’.
We used R (for respondent) followed by a number to identify participants using unique identifiers.
Thematic analysis
This part of our analysis focused on how ECRs discursively ‘make sense’ of impact in the context of their academic careers. The thematic approach was based on an iterative process where two researchers (CB and PB) read all the questionnaires and discussed the potential themes that were emerging in the two parallel analyses. These were classed as expressing broadly positive or negative attitudes towards impact (e.g., ‘positive: there is no point in doing research without impact’, and ‘negative: there is need for academic research to be seen as autonomous and independent’). A coding book was developed from this analysis which allowed for the free text to be coded by one researcher alone (CB).
Discourse analysis
To supplement the thematic analysis, we carried out a closer reading of the survey answers, focusing on how positioning is played out through discourse. ‘Positioning’ is understood here as the discursive construction of personal accounts of one’s stance in the academic community. Discursive resources are used to construct the actions of an individual in a way which is intelligible to herself and others (Tirado and Gálvez 2008). Positioning can take place in discourse through the use of narratives, metaphors, and images. Positioning theory was initially developed as an interactionist approach, i.e. to study social interaction, in particular face-to-face conversation (Davies and Harré 1999; Harré and Van Langenhove 1999). However, the concept of ‘positioning’ has since been used in a variety of contexts, including to describe how individuals take positions vis a vis abstract concepts (Angermuller 2013). Positioning theory stresses both the spontaneous, episodic, and negotiable nature of the process of positioning, but also its long-standing consequences in the socialization of individuals. In this article, we assume that the utterances present in the survey responses constitute a form of conversation—be it a mediated conversation with the ‘audience’ of the survey (its authors) or, perhaps more importantly—with oneself.
Results
Thematic analysis: intrinsic tensions
Several themes emerged from the thematic analysis of the data. Most respondents expressed a positive attitude towards the general concept of research impact and pointed to challenges that prevented them from achieving the impact that they wished to achieve. Participants’ altruistic motivations for doing research, as well as their belief in a society where research and societal impact are closely aligned often conflicted with the reality they faced around achieving independence in their research careers. When discussed through the lens of achieving a successful research career, participants viewed pursuing impact as a detriment to their research careers.
Commitment to impact as a concept
We divided articulations of positive attitudes towards impact into four main themes. The first theme is related to the desire to create positive social change, often through empowering marginalized groups. Here, respondents were motivated by strong convictions of social justice and stressed the role of research in ‘improving people’s lives’ (R54), ‘creating ground for changes in society’ (R48), and ‘improving the society in which we live’ (R11). Respondents also mentioned the importance of ‘empower[ing] people and give[ing] a voice’ to under-represented groups (R4), by ‘bringing to the fore perspectives of vulnerable and marginalized groups’ (R71). Still on this theme, one participant defined impact as ‘initiating social changes that will lead to greater social equality’ (R41).
The second theme revolved around research impact understood primarily as the duty to contribute to maintaining a healthy democracy. This was expressed as the ability to enable the creation of informed opinions among the public. Examples of this are educating the public on how to ‘tackle fake news’ (R94), ‘how to influence public opinion’ (R92), and to ‘create awareness’ (R60) around social issues. Similarly, participants articulated impact as the ‘dissemination of ideas in the public interest’ (R41), as well as ‘inform[ing] and debat[ing]’ (R87).
The third theme conceptualized impact as a duty or a responsibility to give something back to society commensurate with the ambitions of achieving a level of public accountability by research in receipt of public funds. Participants stated that it was their ‘responsibility and civic duty to the public’ (R45) and referred to the need to ‘give something back’ (R18, R58). Interestingly, the terms ‘give something back’ was used by two respondents independently. The final, forth, theme imagined research impact as an integral, and not separate, part of broader society. Participants articulated impact in a way that suggested a belief system in which ‘research should not be dissociated from society’ (R91) since ‘we are all part of a community’ (R93). As one respondent explained: ‘it is impossible to divorce humanities and society; […] humanities are researching society, but they are also part of the same society’ (R69).
Career tensions in pursuing impact at the early stage
Despite mostly expressing positive attitudes towards the concept of research impact (see above), respondents also highlighted challenges related to pursuing impact. These challenges were associated with the tension between the intrinsic motivation to achieve impact, and the reward structure necessary for ECRs to achieve research independence.
Several themes were identified here, including in relation to time constraints such as the absence of time formally allocated to the pursuit of research impact: ‘with the current workload necessary to be more or less successful researcher, it is very hard to spend more time and energy on additional activities’ (R68). Participants referred to research impact as ‘extra’ activity rather than something that was formally acknowledged as part of their role. This created a clash between the time necessary to pursue impact goals, with the time necessary to achieve more formally recognized and measurable research outcomes. A participant wrote:
‘because of time constraints and because my university does not count these activities as part of my workload, I have to be very selective in choosing which of these ‘extra’ activities to participate in’ (R71);
This tension between official requirements of an academic position for ECRs, and the pursuit of impact goals which were expressed by participants as a motivation for doing research, was enhanced by the reality faced by participants in that research independence is a necessary precursor to a successful research career. Achieving this success required producing traditional research outputs such as journal articles which takes time away from engaging in activities that would be classified as impact. The following participant expresses this tension;
‘if I want to move up the steps of the academic ladder then I should write a lot of peer-reviewed articles’ (R65); ‘we must prioritize publishing and teaching in order to meet the requirements for future employment at universities’ (R17).
Likewise, official academic promotion criteria do not formally recognize and therefore incentivize engagement with impact. As one participant explained, ‘national academic promotion procedures or project evaluations usually do not stress the need to create social impact; impact-orientation gives no tangible priority in academic competition’ (R7). Focusing on the particular circumstances of ECRs, where achieving research independence early on is an important marker of esteem (Laudel and Glaser 2008), one respondent maintained that ‘the current career system […] and the precarious working conditions discourage looking for impact’ (R52). Another participant expressed their doubts whether impact is ‘always rewarded or supported appropriately by the university’ (R45). Participants stressed the need for ‘rewards in terms of promotion or salary’ (R67), or ‘valorising press participation/articles in mainstream media’ (R25), as instruments necessary to achieve ‘more recognition [generally] for impact’ (R16), as well as a closer alignment of the reward structures at the ECR-level with more long-term goals related to research impact. One suggestion on how to achieve this was to create more organizational support such as ‘more funding available for dissemination of results to the public’ (R79) or creating a ‘budget for such activities’ (R70)/‘financial support for the visibility and popularization of results’ (R41).
In the same vein, academic publications were still seen as higher valued than impact and hence often prioritized by individuals to the detriment of the latter. As one participant put it, ‘[it is] difficult to make it all work at the same time—publish, boost citation, […] participate in conferences, disseminate and also include wider society’ (R88). This view was also shared by other respondents who often referred to the ‘publish or perish rule’ (R12). One participant stated that ‘incentives at university are directed towards publishing in high ranked journals’ (R42). This by extension affects the choices ECRs make regarding how to allocate their research time. Participants expressed their desire to see formal academic reward structure to alleviate this burden by shifting the focus away from academic publication to the ‘inclusion of societal impact aspects in career promotion procedures’ (R75), or placing ‘less emphasis on publications as a measurement of academic success’ (R60).
Participants also felt insecurity related to their junior status as ECRs and their restricted ability to achieve impact. This represents a further tension experienced by ECRs that belies their intrinsic orientations to impact described in the previous section. For instance, one participant stated:
‘the main problem I have is experience: I am still a junior and I do not feel that I am legitimate to give recommendations to anyone, also because my research results are preliminary and I am not 100% confident in them’ (R7).
A similar view was articulated by another participant: ‘I do not feel sure about my knowledge’ (R30). This insecurity also stemmed from a perceived lack of external validation. For instance, talking about a presentation they gave, one respondent felt ‘considered as just a PhD student doing a presentation rather than something really serious or urgent in the eyes of leaders and decision-makers’ (R61). An insecurity regarding one’s credibility may be interpreted as a specific case of ECR ‘impostor syndrome’. While the concept of ‘impostor syndrome’ has been widely discussed in the literature in relation to scholars’ right to make knowledge-claims (Jaremka et al. 2020), the survey data suggest a parallel phenomenon of impact-related impostor syndrome specific to ECRs.
Geographical mobility was a further impediment to achieving impact, especially local impact that required ECRs to remain in the same place for a long period of time. As one participant noted:
‘We move a lot, we do not have the time to study a situation in a certain society well enough before the research contract is finished and we need to seek new funding for a new project or a new institution. Links with society require time, patience, perseverance and some stability’ (R91).
The precarity of research careers that require ECR researchers to relocate, sometimes internationally, has been discussed in existing studies (Balaban 2018). The survey data reveals another problematic layer to this issue. Not only is the requirement to remain internationally mobile during the initial stages of one’s career detrimental to wellbeing and finances, but it also creates specific and often insurmountable challenges related to generating impact. As the following participant explains, there is little motivation to establish the stakeholder networks necessary to pursue impact successfully during a stay that is not guaranteed to extend for more than a couple of years:
‘As an independent post-doc abroad, I no longer have the platform I used to have ‘back home’. I have to build up a new stakeholder network, which is quite an investment for a temporary stay abroad’ (R10).
To remedy this structural problem and to enable ECRs to develop and maintain the relationships, participants referred to the need for more ‘job stability’ (R16, R17), and in particular ‘security of employment’ (R17), in order to be able to achieve impact in the long term.
Despite expressing a generally positive attitude towards the concept of research impact, based on their view of the socially embedded role of science and their sense of duty to ‘give back to society’, most respondents highlighted challenges on their way to research impact (time, ‘publish or perish’ culture, the perceived low standing of ERCs, high levels of international mobility). The data suggest a conflict between how ECRs align their pursuit of research impact with the realities of an academic reward ladder that values above all traditional markers of esteem. This is compounded by the precarities of the ECR-state (Armano and Murgia 2012; Murgia and Poggio 2018) that inhibit the ability to make the type of long-term relationships necessary to achieving research impact. ECRs must navigate the complex situation by strategically positioning themselves towards the concept of impact. They do this through the means of discourse.
Discourse analysis
Alongside a broad thematic analysis, a closer discourse analysis of selected fragments of open answers to the survey was conducted, focusing on the participants’ positioning towards the concept of ‘impact’. Three fragments have been selected for discussion as they offer insight into some of the tensions played out in the discourse when ECRs discussed impact. While the thematic analysis presented above draws attention to the (often conflicting) institutional requirements recognized by the respondents, the discourse analysis presented below draws attention to their intertwinement. The ‘pros and cons’ of engagement in impact are constantly negotiated by young scholars attempting to establish a position for themselves—both in the academic hierarchies and in discourse. Consider the excerpt below:
Obviously I would like it [impact] to be higher! To be perfectly frank – I am offered no carrot for impact; it is something I do off my own back. And while part of my research (…) has high potential for impact, [in] the other part (…) is much harder [to] generate interest. That said, one of the reasons I moved towards [research area] was to increase my impact work. I have also started working on two projects on [research area], both of which have potential for impact, so I just changed the score from 5 to 6. Basically, [my discipline] it is not the most impactful of research areas (…). So that was part of my reason for pursuing research into other areas after my PhD. If you had asked me this question 10 years ago, as I was finishing my PhD, I would have asked you what impact meant. These days, [scholars] with newly minted PhDs are often much more aware of it. (R8)
Presented below is the coding of this fragment (see Figure 2). Sections have been coded for negative appraisal and positive appraisal in a partial adaptation of Martin and White’s appraisal theory (Martin and White 2003). Negativity and positivity are one of the key dimensions of appraisal theory, namely—polarity. Polarity is the focus of analysis of a rapidly developing field of sentiment analysis (Read and Carroll 2012), often used to explore attitudes expressed in large bodies of text (e.g. social media, online fora posts). In this article, polarity is studied via a non-automated process of coding and incorporated into a broader discourse analytical framework. The colour-coding offers a visual representation of how positive and negative appraisal of concepts and experiences related to impact are intertwined in the respondent’s discourse. Additionally, in the fragment that follows, we have coded for shift in interest or engagement, to draw attention to the dynamic nature of the positioning at play.
Discursive coding of positive and negative appraisal of survey response.
Throughout the fragment above, the respondent’s orientation toward impact both as a concept, and as a relevant attribute of their disciplinary output oscillates between the positive and the negative. The speaker starts their answer with an expression of enthusiasm towards research impact—they would like the impact of their work to be higher. In stating this, they add the booster ‘obviously’, suggesting that scholars strive, or should strive for impact—this can be read as a positioning towards the broader discourse in the community. The participant finishes their sentence with an exclamation mark, which further draws attention to the emotional load of this positive appraisal of impact. Immediately, however, the respondent moves on to the challenges related to impact. In discussing these, they use two phrasal verbs: (1) that they are offered ‘no carrot’; and (2) they have to do impact-related work ‘off [their] own back (sic)’, i.e., on her own, without support. The scholar then proceeds to discuss the potential for impact in their main discipline and the different areas of their work. They explain that their discipline is generally ‘not the most impactful’ but they add that they have progressively been selecting research topics/projects/areas which ‘have potential for impact’. This shift in their interest from less ‘impactful’ areas to those which will allow them to ‘increase [their] impact work’ are in line with their stronger recognition of the concept of impact which the participant suggests was at low levels when they were completing their PhD. There is an interesting convergence here.
The fragment is focused on the respondent’s growing awareness of impact, the general (perceived) awareness of ECRs in their field (‘newly minted PhDs are often much more aware’). This is matched by an apparent increase of awareness during the process of completing the questionnaire (‘I just changed the score from 5 to 6’). For this respondent, the act of talking about impact is clearly a reflexive process. It is through this discursive negotiation of positive and negative appraisals, through taking positions vis a vis the dominant discourse (striving for impact is something ‘obvious’) that the respondent develops and carves out their own conscious positioning towards the concept of impact.
The analysis of this fragment is in line with the hypothesis of a conflict within ECRs generated by the Impact Agenda and the tension it generates for their careers. In the case of the above fragment, conflicting incentives for career choices related to impact are evident. On the one hand, the respondent presents their discipline as ‘not impactful’ as such, and stresses the lack of rewards or support related to pursuing impact (‘no carrot’). On the other hand, they discuss projects with potential for impact which are in line with their personal aspirations and interests. Thus, impact is of increasing importance to this scholar in their career. While the cited fragment contains a narrative of increased importance of impact to an ECR’s professional trajectory, some remarks added outside the context of the narrative (as if ‘meta’ interjections’) speak to the fact that the process of implementation of ‘impact’ into narratives of ECRs careers is ongoing (‘I just changed my score’).
The above fragment can be interpreted as a process of discursive positioning. Most positioning analysis of academic discourse focus on how academics use discursive resources to carve out a space for themselves in the academic community through choices related to publications and institutional positions. Van Langenhove and Harré (1999: 104) draw attention to the fact that comparatively little attention has been paid to the phase of pre-research, i.e., the various activities and choices a researcher must engage in before embarking on an actual research project. These include securing funding, equipment, organizational support, ethical approval, etc. The analysed sequence, along with the thematic analysis, sheds light on the fact that before actually engaging in pre-research a scholar, particularly an ECR, must first select which area in which to invest their energies (traditionally understood scientific enquiry, teaching activities, organization and service, and finally—impact). These choices are simultaneously explained to themselves and to others in a process of discursive positioning towards the established pillars of academic reputation (research projects, publications) and the relatively fresh addition—the hazy (yet somehow ‘obviously’ important) concept of impact. The ambivalent status of impact in the academic value system is mirrored by the many hesitations and adjustments visible in the discourse of a scholar engaging in positioning themselves towards the concept in the fragment analysed above.
Discursive coding of positive and negative appraisal of survey response.
The second selected fragment (see Figure 3) also contains sections which highlight alternatively positive and negative appraisals of impact. The cited respondent stresses that the usefulness of their research in a given field of social activity makes them ‘happy’, and that impact is ‘a real power’. However, pursuing impact has its downsides (risk of being accused of causing negative impact or even ‘destroying’ a given area) and it places constrains on the researcher, as they must consider ‘various ethical principles’ and be prepared to ‘take the responsibility’ or even ‘face the consequences’. In the case of the previously cited fragment (R8, Figure 2) tensions in the requirements related to impact were resolved via ‘shifts’—in the speaker’s career but also in the answer given to the survey question. In contrast, the respondent quoted above (R5, Figure 3) recognizes that the tensions generated by engagement in creating impact are ‘inevitable’. The general tone of this response is one of a resigned awareness when it comes to the realities of ‘impactful’ research. However, we can trace a shade of bitterness or resentment in the opening phrase: ‘nobody told me that having an actual impact is such a great challenge’. We wish to conclude this part of the analysis by citing a telling excerpt from our data: “It is important, but it is impossible sometimes to have a direct impact after one research project” (R79, Figure 4).
A broader consideration of the discourse present in our data, of which we have presented just three representative excerpts, may lead to the conclusion that achieving impact is an ‘important’ but at the same time ‘impossible’ mission for ECRs (see Figure 4). Resolving this tension is a challenge both in terms of strategic decisions regarding allocation of resources and in terms of discursive positioning.
Discursive coding of positive and negative appraisal of survey response.
Discussion
This article offers an initial analysis of the tensions that ECRs currently face when navigating personal positions around achieving extra-academic impact in light of the competing signals about the value these activities have in a successful academic career. Using both thematic and discourse analysis, this research shows the tensions currently experienced by ECRs around the concept of extra-academic impact in relation to their careers. The thematic analysis showed a broad range of conflicting themes related to how ECRs reconcile their intrinsic commitment to extra-academic impact with the realities of a research culture that rewards more narrow and traditional forms of excellence as a prerequisite for achieving research independence. This conflict was mirrored in the discourse analysis where the negative and positive appraisals were intertwined, in a way clearly visible even in short-analysed fragments.
In navigating a contradictory environment where conflicting signals exist around extra-academic impact, ECRs adopt a wide range of positions in relation to impact as an evaluation criterion and career expectation. The role of ‘impact’ in generating tension and as an increasingly non-negotiable norm within this space has contributed to the invisibility of scholars at the ECR-level (Fenby-Hulse, Heywood and Walker 2019). Thus, early-career researchers rely on discursive strategies to negotiate their position vice versa the concept of extra-academic impact. While in this study most respondents are keen to create positive change in society (echoing the findings of other studies: Marsh and Klima 2022; Friesike, Dobusch and Heimstädt 2022; John et al. 2023), many of them are also mindful that extra-academic impact is not a top priority in the context of their career progression. Although personally committed to the idea of extra-academic impact, many ECRs cannot ‘afford’ to pursue it, as that would mean dedicating their limited resources to activities that are not likely to help them advance in their careers. Vanholsbeeck (2022) referred to this, along with open science policies, as the ‘triple bind’ of academic career requirements and direct implication for the first generation of scholars through open science policies and the impact agenda (Vanholsbeeck 2022). In addition, ECRs who responded to our study noted the lack of power necessary to change the system, leaving little choice but to abide by the rules as they are currently constructed. This dis-incentivizes the pursuit of their intrinsic motivations to achieve extra-academic impact, creating difficulties in incorporating these concepts of excellence that exist beyond traditional notions of research excellence and value into ECRs’ strategies for career progression.
Resolving the tension around the position of extra-academic impact in ECR careers requires the concept to become more institutionalized. It should be an integral part of academic socialization that is embedded, in its most inclusive form, in career progression structures. Friesike, Dobusch and Heimstädt (2022) adopted a reflective essay to examine 5 main concerns for ECRs aiming to use their research to have impact. Many of these tensions, specifically: the opportunity cost of impact work, the ‘fuzziness’ of impact measurement, the challenge of translating small results and the need for role models, and the career-risk associated with publicly communicating results have been mirrored in this article. However, institutionalizing impact for ECRs and their careers requires not just delivering training as part of their development, but also providing ECRs the space and long term support necessary to track, collect, engage, and transform on their research findings to evolve into evidence and become useful beyond academia.
In addition, changes to the evaluative structures for ECRs could involve developing clear guidelines for the recognition of impact achievements, as well as providing necessary incentives and rewards built into the hiring and promotion processes at the early stages of a career. This would act to balance the appreciation of extra-impact achievement against the weighting of other academic activities, such as publications, citations, grant capture and teaching. In addition, the adoption of similar frameworks for the appreciation of extra-academic impact across institutional or national contexts, would also favour the continuation of partnerships and capacity necessary to realize impact in the long term.
While this study did not code data relative to the evaluative context of respondents (countries, type of research organization, field, etc.), further research is recommended that adopts a case study approach which is sensitive to this level of contextualization. Future studies should map challenges particular to next cohorts of ECRs socialized into a culture of impact which continues to develop and consolidate, as well as shedding light on the specific situation of scholars in different academic cultures. This would enable a better alignment between extra-academic impact with professional and personal realities, thereby allowing for the development of more coherent academic identities in forthcoming generations of academic scholars.
Conclusion
Policy solutions are recommended to address the current conflicts between ECRs’ intrinsic orientation towards achieving extra-academic impact and the requirements for successful career progress that favour traditional notions of research value and excellence. Policy-makers should concentrate on aligning research career structures and scholars’ personal aspirations with ongoing policy discourse around the centrality of extra-academic impact as part of research’s societal contract, rather than concentrating on the provision of impact training programmes. There is a need to create supportive academic environments, where the pursuit of extra-academic impact is presented as an integral part of researchers’ professional ethos and not as an optional avenue that presents more risks to an ECR’s career than it does opportunities. Formally including extra-academic impact in recruitment and promotion criteria (as is relevant for each country context) as well as championing positive examples of scholars who have had success with extra-academic impact, would aid the incorporation of impact into broader constructions of research excellence. The implementation of these recommendations would ease the conflict currently evident in the uneasy discursive positioning explored in this article.
Directions for future research include a further monitoring of development of impact evaluation policy in cross-national contexts. Specifically, scholars should look at aspects of policy-borrowing, the centre—periphery dynamics, the convergencies and contradictions between national and international impact evaluation policies. In addition to describing the policy regulations, scholars should study their practical implementation and the support offered by institutions. The reception of these policies amongst ERCs should also remain an object of study. A study similar to the one described in this article could be repeated in regular intervals to monitor the positive, negative, planned, and unexpected effects of impact evaluation policies and their reflection in academic discourse.
Acknowledgements
The team would like to thank European Network for Research Evaluation in the Social Sciences and the Humanities (ENRESSH) for funding CB’s short-term scientific mission (STSM) to the Centre for Science and Technology Studies (CWTS) at Leiden University. It is where a good part of the qualitative analysis was conducted, under the guidance of Prof. Paul Benneworth. Marta Wrwska wishes to acknowledge the support of National Science Centre, Poland. Part of the work presented in this paper was funded under grant number 2022/47/B/HS6/01341.
References
Christian, K., Johnstone, C., Larkins, J. A., Wright, W., and Doran, M. R. (
McAlpine, L., Pyhältö, K., and Castelló, M. (
