Abstract
We present results of a survey conducted with researchers and practitioners involved in a Swiss National Research Programme on steering energy consumption. We analyse what motivates practitioners and researchers to engage in a collaborative research project, their perception of the collaboration intensity in different project phases, and the extent to which the research project provided useful results for practitioners. Our analyses demonstrate that the intensity of collaboration is a key driver of successful collaboration as it fosters trust between researchers and practitioners. Thereby, it increases the usefulness of the research project for practitioners and their perceived contribution to the success of the research project. Research programmes should thus (1) foster trust through incentivising collaboration between research and practice; (2) facilitate the development of a shared understanding of researchers’ and practitioners’ respective roles; and (3) support the inclusion of practitioners in the project development phase through financial support during the proposal-writing phase.
1. Introduction
In the face of complex sustainability problems, science policy and research funding programmes increasingly ask researchers to co-create knowledge with policymakers, businesses, or civil society actors—that is, with practitioners. This demand generally rests on the assumption that the participation of practitioners in the research process results in societally relevant and robust knowledge that is more likely to contribute to societal transformations. The call for involving practitioners has become an integral part of the sustainability discourse in research policy and funding (Cundill et al. 2015; Hessels et al. 2018; Schmidt and Pröpper 2017; Schneider et al. 2019; Spangenberg 2011; Van der Hel 2016).
At the European and international levels, this call for the inclusion of diverse societal actors in knowledge production is reflected, for instance, in the ‘Science with and for Society’ stream of the European Commission’s Horizon 2020 programme (2019) or in global initiatives such as Future Earth (Future Earth 2019). At the national level, both public funding agencies and private donors have set up funding schemes that call for the integration of different actor groups into research processes. In the European context, these include the German funding priority programme ‘Socio-Ecological Research’ (SöF) by the Federal Ministry for Science and Education (BMBF), the Austrian ‘Cultural Landscape Research Programme’ (KLE, ‘Kulturlandschaftsforschung’) by the respective Federal Ministry of Science and Research, or the National Research Programmes (NRPs) by the Swiss National Science Foundation (SNSF), to name but a few.
This growing policy relevance and the public expenditures it entails imply the need for systematic assessments of transdisciplinarity (TD)-related research practices and their added value. This concerns in particular the relation between TD processes and the practical relevance and usefulness of the outputs which they produce (Newig et al. 2019). Several studies have analysed individual or small sets of TD projects (Binder et al. 2015; Fritz et al. 2019; Hansson and Polk 2018; Wiek et al. 2014) and, more recently, funding programmes (Hessels et al. 2018; Hoffmann et al. 2017; de Jong et al. 2016) with regard to the links between collaboration processes and their societal effects. These studies suggest that the forms and features of the collaborative research process involving researchers and practitioners affect its potential for producing relevant results and contributing to societal impact. Process features that are discussed in this regard include the intensity of researcher–practitioner interactions (Schneider and Buser 2018; Walter et al. 2007; Wiek et al. 2014), the timing of practitioner involvement (Lux et al. 2019), the methods of knowledge integration used (Newig et al. 2019), as well as process dynamics such as the development of mutual trust (Fritz et al. 2019). In addition, few studies discuss the link between process features and practical relevance of TD research against the background of funding structures, pointing to the powerful role of science policy and funding bodies in creating a favourable environment for fruitful research–practice collaborations (Fritz and Binder 2020; Lux et al. 2019; Lyall et al. 2013; Mitchell et al. 2015). Comprehensive empirical studies accounting concomitantly for many of the aforementioned process features as well as the structural conditions in a given funding context are, thus, needed to better understand how the practical relevance and potential for societal impact of researcher–practitioner collaborations can be fostered.
For a long time, practitioners’ perspectives on collaborative and participatory research processes constituted blind spots in TD scholarship (Bracken et al. 2015; Fritz and Binder 2018; Schmidt and Pröpper 2017; Zscheischler et al. 2018). It is only recently that they have been receiving increasing attention. Qualitative studies have begun to capture the expectations and goals, experiences and perceptions of practitioners regarding the TD process and its (desired) effects (e.g. Binder et al. 2015; Bracken et al. 2015; Di Giulio et al. 2016; Fritz et al. 2019; Schmidt and Neuburger 2017; Schmidt and Pröpper 2017). Insights gained in these qualitative studies, often based on a single case, point in similar directions: practitioners’ motivations and expectations regarding participation in and the effects of TD projects are likely to differ from those of the researchers (see also Section 1.1.1). A recent study showed that aspects perceived as possible side effects by researchers (e.g. network effects), were a core expectation and indeed a motivation for practitioners to participate and vice versa. Researchers’ and practitioners’ perspectives might, furthermore, differ not only regarding the kinds of effects which they expect, but also regarding the pathways towards achieving these effects (Fritz et al. 2019). Likewise, the few quantitative studies of both researchers’ and practitioners’ motivations and perceptions of successful collaboration point to researchers’ and practitioners’ differing perspectives (Thompson et al. 2017; Zscheischler et al. 2018). In order to better understand TD processes, their practical relevance, and potential for societal impact, greater knowledge of practitioners’ motivations and perceptions is, thus, needed.
In summary, despite the growing uptake of TD approaches in science policy, larger-scale systematic analyses of the real-world practice of TD at the level of funding programmes have been scant. So far, most reflections on TD practices are single-case studies written from researchers’ standpoints, with little more aggregated evidence of how forms and perceived features of the collaboration process relate to the perceived practical and scientific relevance of its results. Empirical investigations into how different process features (such as timing, form, and intensity of practitioner involvement) link to the perceived practical usefulness and societal relevance of results are largely missing. In light of the high expectations of these research modes, deeper empirical insights into both researchers’ and practitioners’ perceptions of relations between effects, process features, and factors of participation in TD research projects are, thus, needed.
In this article, we investigate how features and factors of participation as perceived by researchers and practitioners relate to the perceived relevance and usefulness of the research project and its results. In so doing, we aim to (i) elicit differences between researchers’ and practitioners’ motivation for and perception of their collaboration (e.g. timing and intensity of involvement); (2) assess which factors are perceived to shape the collaboration process itself; and (3) understand how features of the collaboration process are linked to the perceived benefit of the project (practical and scientific). Finally, we reflect on the role that research programme structures and funding instruments can play and how they could contribute to fostering societally relevant and useful research. We base ourselves on a quantitative survey with researchers and practitioners who participated in projects funded by a major Swiss National Research Programme.
Before presenting the methodology and results of this study, the following section introduces the core concepts and theoretical claims in current TD scholarship that this article builds on.
1.1. Conceptual background: discourses on science–practice interactions in TD scholarship
In this study, we build on core concepts and issues underpinning current discourses on science–practice interactions in TD scholarship. This section reviews selected state-of-the-art literature on the conceptual building blocks that pave the way for the empirical foci chosen in this study.3 This section is partly based on Fritz (2020). These include (1) ideal-typical features of TD processes, (2) factors shaping real-type TD processes, and (3) claims about the practical relevance of TD process results.
1.1.1. Ideal-typical features of TD processes
First, a considerable body of literature elaborates on the question of what TD should be and how such collaborative research processes should be governed. TD being a ‘fuzzy and contested field’ that is formed by heterogeneous conceptions of science and expertise (Hirsch Hadorn et al. 2008: 27), in this article, we follow actor-orientated approaches because they have found their way into various funding programmes and informed science policy. In these approaches, TD is seen as a mode of research that: is rooted in lifeworld problems and creates knowledge that is solution-oriented and has a high practical relevance; integrates different scientific disciplines; and includes practitioners in the production of knowledge and fosters mutual learning (e.g. Defila and Di Giulio 2015; Hirsch Hadorn et al. 2008; Jahn et al. 2012; Lang et al. 2012; Novy et al. 2008). In an attempt to guide the implementation of these features, ideal-types of what a TD processes ought to look like have been developed. In the following, we elaborate on features particularly salient for researcher–practitioner interaction, neglecting other features of TD such as interdisciplinary collaboration (see also Fritz and Meinherz 2020).
1.1.1.1. Interlinked process phases
Ideal-typical TD processes are frequently distinguished in three interlinked phases: (1) the problem framing and development phase; (2) the knowledge (co-)production phase, (3) the dissemination and ‘bringing results to fruition’ phase (Bergmann et al. 2005; Jahn et al. 2012; Lang et al. 2012; Pohl et al. 2007; Schneider and Buser 2018). Throughout these process phases, both inputs by researchers and practitioners and benefits for them appear as balanced (see e.g. Lang et al. 2012). Design principles such as ‘shared control’, ‘co-leadership’, and interactions on ‘equal footing’ (Lang et al. 2012; Luthe 2017; Scholz and Steiner 2015a) should support the realisation of ideal-typical features throughout the process.
1.1.1.2. Intensities of involvement
One of the core concepts that has been operationalised in order to categorise and assess researcher–practitioner collaboration, is the intensity of involvement. Alluding to one of the most popular conceptions of participation—Arnstein’s (1969) ‘ladder of participation’—Stauffacher et al. (2008), for example, analyse the degree of practitioners’ involvement along the axes of information, consultation, cooperation, collaboration, and empowerment. Schneider and Buser (2018) assess the intensity of involvement, which is the most promising for different types of research projects. Despite a strong normative flavour of conceptions based on Arnstein’s ladder, suggesting ‘the more participation, the better’ (Chilvers and Kearnes 2015), it is increasingly acknowledged that researcher–practitioner interactions can but do not necessarily take place across all phases and that their intensity can vary throughout the research process (Stauffacher et al. 2008). Empirically speaking, the relation between intensities of involvement in different phases and the success and/or practical relevance of research projects remains unclear.
1.1.1.3. Diversity of actors and multiplicity of objectives
TD processes, as understood here , are knowledge (co-)production processes that include heterogenous actors holding diverse issue-specific expertise: researchers from across the disciplinary spectrum, at different career stages, holding diverse epistemological and ethical values as well as practitioners such as policymakers, politicians, public service providers, private sector actors, third-sector actors, or individual citizens (Fritz and Meinherz 2020). Being primarily knowledge production processes, practitioners’ participation in TD research is first and foremost associated with epistemic objectives (Defila and Di Giulio 2019). Yet, in sustainability research that has a transformative ambition, epistemic and social or democratic objectives often mingle (Lamine 2018). Frequently mentioned objectives that researchers pursue by involving practitioners include: increasing the relevance and legitimacy of research outputs (e.g. Schmidt et al. 2020); facilitating the implementation or application of the research outputs (e.g. Kaufmann-Hayoz et al. 2016); enabling transfer and dissemination of project results to a wider context (e.g. Mitchell et al. 2015); or empowering the participating practitioners (e.g. Blackstock et al. 2015). Objectives that motivate practitioners to engage in collaborations with researchers include: building new networks and stronger ties to fellow practitioners and/or researchers (e.g. Fritz et al. 2019); strengthening their knowledge base or capacities for decision-making (e.g. Hansson and Polk 2018); as well as their status, visibility, or legitimacy in the practice context (e.g. Binder et al. 2015). While the heterogeneity of actors and the multiplicity of objectives they pursue are acknowledged as a key feature of TD processes (Scholz and Steiner 2015b), practitioners’ motivations and objectives in TD processes are little documented and different objectives are often implicitly assumed to be commensurable and symbiotic (Fritz 2020).
In this study, we build on these ideal-typical process features (timing and intensity of involvement, multiplicitly of underlying objectives) and scrutinse how they are implemented in TD practice in the selected projects and affect the latter’s perceived practical relevance.
1.1.2. Factors shaping TD processes in practice
A second strand of literature focuses on real-type TD and the unfolding of researcher–practitioner interactions in practice. Many of these empirical investigations reveal discrepancies between the ideals of TD and the lived practice (Zscheischler and Rogga 2015), putting in question amongst others ideals of ‘shared control’ (Brandt et al. 2013), joint problem framing and early involvement of practitioners into the research (Wuelser and Pohl 2016).
This growing body of scholarship emphasises the ‘making of participation’ and has been enquiring into the conditions, structures, and factors that forge practitioners’ involvement in TD practice (Fritz and Binder 2018; Klenk and Meehan 2017; Schmidt and Neuburger 2017). Such work suggests that ‘successful’ knowledge co-production involving researchers of multiple disciplines and practitioners with various backgrounds cannot be taken for granted, but depends on a variety of factors—residing both within the project realm and its context. A recent systematic review of empirical and case-study based TD literature identified key factors shaping participation processes in knowledge production: (1) (in)coherences with the reference system in research and practice, (2) availability and distribution of financial and time resources, (3) timing of involvement, (4) compatibility and fulfilment of expectations and objectives, (5) mutual trust, (6) worldviews, values and joint problem understanding, and (7) power relations (Fritz and Binder 2018).
In this study, we take up several of these so-called participation factors identified in the literature and subject them to empirical scrutiny in the case of a major Swiss research programme and the projects funded therein. We furthermore aim at substantiating and extending them by explicitly accounting for the practitioners’ perspective, underrepresented in current scholarship.
1.1.3. Results of TD processes: practical relevance and intended societal impact
A third core element of TD discourse informing this article concerns the intended real-world impacts of the knowledge produced. Mirroring one of its defining features, a central claim of TD is that practitioners’ participation improves the implementation and applicability of the knowledge produced (Gross and Hoffmann-Riem 2005; Hansson and Polk 2018; de Jong et al. 2016).
This claim builds on the expectation that practitioners’ participation in the production of knowledge enables them to continually ‘test’ new knowledge. This testing is expected to make the knowledge produced socially robust, ensures that it is relevant for practitioners and, consequently, more likely to guide their actions (Demeritt 2010). Besides the importance of the societal context’s receptivity for TD results (Nagy et al. 2020), several process features are assumed to be pivotal for enabling such ‘testing’ and fostering the practical relevance and potential for societal impact of the research.
A process feature frequently addressed with regard to enhancing practical relevance is the intensity of researcher–practitioner interactions (Schneider and Buser 2018; Walter et al. 2007; Wiek et al. 2014). Arguably, producing knowledge that is practically relevant, requires close collaboration and continuous interaction between researchers and practitioners. Higher intensities of involvement are, thus, commonly assumed to contribute to a higher practical relevance of results (de Jong et al. 2016; Newig et al. 2019).
Another process feature discussed with regard to enhancing practical relevance is the timing of practitioner involvement (Lux et al. 2019). Projects that involve practitioners early on, in particular at the stage of framing the problem and research questions, are seen as more likely to integrate practitioners’ ideas and demands in a meaningful way all along the process (Enengel et al. 2012). Early involvement is, thus, associated with increased legitimacy and relevance of the research project (Robinson and Tansey 2006) and higher chances of its results being implemented (Renner et al. 2013).
Further process features and participation factors discussed with regard to practical relevance include the use of structured methods for knowledge integration (Newig et al. 2019), as well as process dynamics such as the development of mutual trust (Fritz et al. 2019), assuming that the prevalence of trust among actors enhances ownership of the process and commitment to implementing its results. Despite being a central claim in TD scholarship, the potential of specific process features and factors for enhancing practical relevance and intended real-world impact of TD collaborations, are still poorly understood and only recently studied empirically.
In this study, we put the claim of enhancing practical relevance and usefulness through TD collaboration centre-stage and examine its potential determinants, as perceived by researchers and practitioners in selected projects. In doing so, we mobilise the aforementioned process features and participation factors.
Taken together, the conceptual elements and ongoing issues outlined in this section constitute the backbone of the quantitative empirical research presented in this article. Investigating different perceptions of researcher–practitioner collaboration processes that are situated within the same funding structures, the study contributes to clarifying conceptual issues in TD scholarship.
2. Material and methods
In order to empirically scrutinise the assumed connections between different features of researcher–practitioner interactions and the perceived practical relevance of TD research projects, we conducted a quantitative analysis of seventeen energy research projects funded within a major Swiss research programme. This section outlines the research context and case study chosen as well as the collection and analysis of the data.
2.1. Research context and case study: the Swiss National Research Programme 71
This study is based on an investigation of transdisciplinary researcher–practitioner interactions within a publicly funded National Research Programme (NRP) in Switzerland. The NRPs are among the foundational mechanisms of the SNSF—Switzerland’s main public institution promoting scientific research.1 The NRPs are the main instrument of the SNSF to promote application-oriented research through thematic funding programmes with a long tradition (Freiburghaus and Zimmermann 1985). Despite a tradition of promoting application-oriented research through thematic funding programmes (Häberli and Grossenbacher-Mansuy 1998), Swiss science policy only recently started to explicitly call for and foster TD research, particularly through SNSF’s NRPs or the Swiss Programme for Research on Global Issues for Development. In setting up its TD-oriented funding programmes, Swiss science policy could rely on the rich theoretical and practical contributions by Swiss researchers strongly involved in shaping TD discourse and building an international TD community. SNSF benefited from relevant expertise of two institutions belonging to the Swiss Academies of Arts and Sciences: the Commission for Research Partnerships with Developing Countries and the Network for Transdisciplinary Research (Schneider et al. 2019).
The research carried out by NRPs consists of targeted research that contributes to solving contemporary problems of national concern. Therefore, NRPs attach great importance to cooperation between scientific research and practitioners from business, society, and public administration. NRPs are distinguished by the following characteristics: they are solution-oriented and close to the practical realm; they are interdisciplinary and transdisciplinary; the research projects of an NRP jointly pursue an overall goal; knowledge transfer and the communication of results are valued highly.2 A team of consultants supports the transfer of knowledge and technology into practice. Each NRP has a steering group of experts from the relevant scientific fields which is responsible for the selection and supervision of projects, as well as for the integration and consolidation of the findings. Submissions for NRP projects can only be made in response to a public call for proposals by the SNSF.
The NRP 71 ‘Managing Energy Consumption’ focuses on socio-economic questions related to energy consumption. It aims to develop basic scientific knowledge and practice-oriented approaches that facilitate regulatory decisions and help set the course of Swiss energy policy. Special attention is given to areas with a high savings potential, like commercial and private electricity consumption, residual heat in residential buildings, and private transport. The call for NRP 71 was announced in 2013. International scientific experts reviewed the proposals. The main reviewing criteria were: scientific quality and originality; concordance with the goals of NRP 71 and relevance for the Swiss ‘Energy Strategy 2050’; clear application, transfer, and utilisation strategy; and adequate personnel and infrastructure. Out of more than 100 proposals, the steering committee decided to recommend that the National Research Council approve nineteen projects. Most of the projects started in 2014 and ended in mid-2018.
Because of its aims, its funding criteria, and centrality in the Swiss funding landscape, the NRP 71 constitutes an ideal empirical basis for studying transdisciplinary interactions between researchers and practitioners and their relevance for addressing complex problems; in this case, Switzerland’s energy transition. The quality of TD in the Swiss NRPs as well as the factors that determine this quality and the ensuing contribution of research to the economy, politics, and society have been the subject of several studies (Freiburghaus and Zimmermann 1985; Kolarz et al. 2018; Sager et al. 2019). However, the present study is the first to systematically include the experiences not only of all researchers but also of all practitioners involved and to link the answers of researchers and their partners in practice.
2.2. Data collection and analysis
The basis of this article is an online survey conducted in Spring 2018. In this section, we describe how the survey was developed and implemented, introduce the sample, and give insight into the structure of the survey and its relation to the conceptual elements of TD.
2.2.1. Development and implementation of the survey
Members of the steering committee, in cooperation with representatives of the SNSF and a member of the knowledge and technology transfer team of the programme, developed the questionnaires in a multi-stage procedure. A separate questionnaire was developed for researchers and practitioners. The questionnaire for the researchers was available in German and English, and the questionnaire for the practitioners in German and French.
The questionnaire was made available to all researchers and practitioners who were involved in projects under NRP 71. The knowledge and technology transfer team had a list of addresses of researchers and practitioners involved in the respective research projects. The project leaders were informed about the survey and the practitioners (related to their projects) to whom the survey would be administered. They were invited to check and complete the list of practitioners.
The survey was programmed using the Qualtrics online tool. An individual link to the survey was sent to researchers and practitioners via e-mail. Persons who had worked on more than one project were contacted only once. They were asked to refer to the most important project for them. Table 1 shows how many researchers and practitioners were contacted and how many took part in the online survey. In total, 72 of the 99 researchers contacted (72 percent) and 61 of the 120 partners from practice contacted (51 per cent) participated in the survey. For the analysis of the data, the SPSS statistics programme was used.
Dispatch and response of the online survey.
| Survey steps | Researchers | Practitioners |
|---|---|---|
| Cleaned lists | 100 | 124 |
| Invalid addresses | 5 | 12 |
| Additional addresses received | 4 | 12 |
| Persons included twice | 1 | 4 |
| Total valid addresses | 99 | 120 |
| Return (absolute) | 72 | 61 |
| Return (relative) | 72% | 51% |
| Survey steps | Researchers | Practitioners |
|---|---|---|
| Cleaned lists | 100 | 124 |
| Invalid addresses | 5 | 12 |
| Additional addresses received | 4 | 12 |
| Persons included twice | 1 | 4 |
| Total valid addresses | 99 | 120 |
| Return (absolute) | 72 | 61 |
| Return (relative) | 72% | 51% |
Data source: Balthasar et al. (2018).
Dispatch and response of the online survey.
| Survey steps | Researchers | Practitioners |
|---|---|---|
| Cleaned lists | 100 | 124 |
| Invalid addresses | 5 | 12 |
| Additional addresses received | 4 | 12 |
| Persons included twice | 1 | 4 |
| Total valid addresses | 99 | 120 |
| Return (absolute) | 72 | 61 |
| Return (relative) | 72% | 51% |
| Survey steps | Researchers | Practitioners |
|---|---|---|
| Cleaned lists | 100 | 124 |
| Invalid addresses | 5 | 12 |
| Additional addresses received | 4 | 12 |
| Persons included twice | 1 | 4 |
| Total valid addresses | 99 | 120 |
| Return (absolute) | 72 | 61 |
| Return (relative) | 72% | 51% |
Data source: Balthasar et al. (2018).
2.2.2. Description of the sample
Researchers and practitioners from all nineteen projects were contacted. Of the seventy-two researchers who took part in the online survey, more than half were affiliated to a university and a further 11 per cent were at a university of applied sciences. Sixty-seven (93 per cent) of the participants had direct contact with practitioners within the framework of the research project. They represented seventeen out of nineteen projects. Three of the participating researchers participated in more than one project within the framework of NRP 71.
Sixty-one practitioners took part in the online survey. Twelve of them worked for private companies, eleven for the federal government, ten for industry associations, nine for municipalities, six for cantonal authorities, and six for energy service providers. The remaining six persons worked for an NGO, association, or society. Around 40 per cent of the practitioners were located in the German-speaking part of Switzerland. A further 40 per cent were active in the whole of Switzerland. Between 3 per cent and 6 per cent worked in the French-speaking or Italian-speaking part of Switzerland or abroad.
Thirty-six of sixty-one practitioners (59 per cent) who answered the questionnaire knew that they were listed as partners on a research project. They were partners in thirteen of the nineteen projects. Three practitioners collaborated with more than one project under NRP 71. Fourteen practitioners had been involved in previous NRPs. Twenty-five practitioners (41 per cent) and five researchers (7 per cent) did not know that they were listed as a partner or as having a partner from practice. These persons were asked only the introductory and final questions. They were not further included in the sample.
2.2.3. Structure, content, and measures of the survey
Guided by the conceptual elements outlined in Section 1.1, the main aim of the survey was to assess (1) how core features of TD processes were implemented in the context of NRP 71 and how they were perceived by the actors involved, (2) which participation factors researchers and practitioners perceived as pivotal in their collaboration, and (3) the results of the collaboration regarding their perceived practical relevance and usefulness. The survey was structured accordingly along these three main themes informed by state-of-the-art TD scholarship:
Features of the collaboration process between researchers and practitioners: The first part of the survey concerned the form of practitioners’ participation in the research project and the intensity of the exchange. These survey items are based on ideal-typical features of TD processes (see Section 1.1.1).
Success factors for the collaboration: The second part of the survey dealt with factors that researchers and practitioners considered important in order to achieve a fruitful collaboration. These survey items take up participation factors derived from empirical literature on TD practice (see Section 1.1.2).
Results of the collaboration: The third part of the survey addressed the assessments of both groups regarding awareness, relevance, applicability, and usefulness of the results. It also included questions concerning the fulfilment of expectations regarding the collaboration and the benefit that the collaboration has created for the research project and practitioners. These survey items put core assumptions about increasing practical relevance made in TD scholarship centre stage (see Section 1.1.3).
First, regarding features of the collaboration process, the survey asked how practitioners had cooperated with researchers, using the same set of participation formats (e.g. financial participation with cash contribution, provision of data, provision of equipment and/or software) in both questionnaires. Motivations were elicited using actor-adapted sets of possible sources of motivation for researchers (e.g. to secure the practical relevance of our project; to enhance the credibility of the results; to enhance the applicability of the results) and practitioners (e.g. to develop a concrete product or process; staying on top of research; finding solutions to a specific problem; supporting researchers).
The frequency of contact between researchers and practitioners was assessed separately for the three phases of the (1) development of the research question, (2) processing of the research question, and (3) dissemination of results, using the levels 1 = no contact, 2 = very little contact (1–2 times), 3 = occasional contact (3–5 times), and 4 = frequent contact (more than 5 times) with the additional possible response ‘I do not know’. An integrative measure for the overall frequency of partner contacts was computed based on the sum score of the three separate measures (three levels: 1 = very little contact (sum < 6), 2 = moderate contact frequency (sum score 5–9), 3 = frequent contact (sum > 9)).
Secondly, both the researchers and practitioners had to rank different participation factors according to their importance for the success of collaboration between research and practice (1 = most important reason, 2 = second most important reason, etc. to 5 = least important). The five participation factors which were derived from the literature considered here were: (1) researchers have sufficient time, (2) practitioners have sufficient time, (3) practitioners have sufficient financial resources, (4) there is a consistent perception of the problem to be addressed, and (5) there is geographic proximity to the partners.
Thirdly, various aspects of the actual process of collaboration, its perceived relevance, and usefulness were addressed in the questionnaires. Researchers were asked, for example, ‘whether the role of the partners in the project was always clear’, and whether they incorporated practitioners’ input into the project (four-point response scale: no, rather no, rather yes, yes; with the additional response option ‘I do not know’). Practitioners were asked for example whether ‘There were moments when I didn't realize what my role in the project was’, whether their ‘knowledge and expertise found appreciation in the project’, and whether they ‘were able to establish a trustful relationship with the researchers’. An additional open question asked researchers and practitioners which ‘other factors favoured or hampered their exchanges with their partners (practitioners or researchers)’.
To assess the usefulness of the research project for practice from the perspective of the practitioners, a composite usefulness indicator was computed as an average from their valid responses to the four items ‘Are these results relevant to your work?’, ‘We can apply the results directly’, ‘The project will influence our work’, and ‘The results give legitimacy to our work’. The resulting scale of this usefulness indicator accordingly ranged from 1 (= low usefulness) to 4 (= high usefulness). The perceptions of the researchers in relation to the usefulness of the research–practice collaboration for practice were assessed by the responses of the researchers to the single item ‘Do you think the results are of relevance for the activities of your partners?’ (response options: ‘No’, ‘To some extent’, ‘Yes’, and ‘I do not know’).
To investigate practitioners’ contribution to the research project, they were asked whether they think that they contributed to the success of the research project (response scale from 1 = no to 4 = yes; with the additional response option ‘I do not know’). With the same aim, the researchers were asked how well the processing of the research questions would have progressed in their view without collaboration with the practitioners (response options: 1 = better, 2 = as well, 3 = worse; with the additional response option ‘I do not know’).
The researchers were furthermore asked whether their expectations of collaborating with practitioners were met in overall terms (four-point scale: no, rather no, rather yes, yes with additional response option ‘I don't know’) and practitioners were asked in a similar way ‘whether their expectations for the research project have been fulfilled so far’. The final questions on the survey asked for the type of research institution and for the type of organisation where the practitioners were based and both practitioners and researchers were asked whether they had additional comments.
3. Results
3.1. Perception of key features of the collaboration process
3.1.1. Researchers’ motivation to engage in the collaboration process
Figure 1 shows the researchers’ motivation for involving practitioners in their research projects. The two most frequently named motives were (1) to ensure the practical relevance of the project and (2) that the results should be applicable. Further important reasons included (3) to validate the research performed; (4) positive experiences with collaboration in the past; (5) to increase the legitimacy of the research; and (6) to enhance the credibility of the findings. About 12 per cent of the researchers stated that they had not themselves decided to pursue collaboration with practitioners and that the involvement of practitioners had been a requirement of the programme management. Eleven researchers (16.4 per cent) specified other motives that were not eligible in the questionnaire; four of these researchers explained that collaboration with practitioners was an integral part of the research or that the project would not have been feasible without the involvement of practitioners. Two researchers mentioned access to data as an important reason for involving practitioners.
Researchers’ motivations for involving practitioners in the research project (percentages of researchers selecting different motives; n = 67 researchers responded; more than one option could be selected). Source: Balthasar et al. (2018).
3.1.2. Practitioners’ motivation to engage in the collaboration process
Regarding practitioners’ motivations, the five most frequently named reasons for participating in the research project were (1) the development of a solution for a concrete problem; (2) an interest in generating new knowledge together with researchers; (3) to support researchers; (4) to establish a network and contacts; and (5) to keep up with research (Fig. 2). These five reasons were named by majorities in the range of 53 per cent and 67 per cent, showing their importance. Only two practitioners each selected the ‘development of a concrete product or process’ and ‘previous experience as a practitioner in a project’, and two other practitioners specified other reasons, namely to ‘receive fundamental knowledge’ and to ‘get an evaluation of a planned intervention at low cost’.
Practitioners’ motivations for participating in the research project (percentages of practitioners selecting different motives; n = 36 practitioners responded; more than one option could be selected). Source: Balthasar et al. (2018).
The main motivations of researchers are similar to those of practitioners to participate in the research project. Researchers want either to secure the practical relevance of the project or the applicability and validity of the project results. Similarly, practitioners aim at finding solutions to a specific problem and generating new knowledge together with researchers. This indicates the importance of the practical relevance of the research as a motivation for researchers and practitioners to collaborate with each other. Interestingly, researchers frequently mentioned positive experiences with practitioners in earlier research projects as an important motivation for including practitioners, whereas this aspect was mentioned as motivation by only two practitioners in the sample. Practitioners mentioned the relevance of establishing a network or new contacts, which was not highlighted by the researchers.
3.2. Perceived forms of collaboration
3.2.1. Researchers’ perspective
When looking at the ways in which collaboration took place, researchers mentioned practitioners’ participation in workshops most frequently (58 per cent), closely followed by their assistance in access to information (e.g. data, clients, stakeholders, role as door opener) (57 per cent), the direct provisioning and transferring of data (54 per cent), participating in interviews (54 per cent), and participating in accompanying groups (e.g. steering or advisory) (51 per cent). Researchers also stated that practitioners provided support in the communication of the results (45 per cent). Financial support in the form of cash contributions (15 per cent) or as services, or analyses performed by practitioners (22 per cent) was mentioned rather rarely by the researchers and provisioning of software and equipment was not mentioned at all (Fig. 3). Other forms of collaboration described by the researchers included the cooperative design of interventions.
Percentage of researchers and practitioners who have perceived different forms of collaboration (n = 66 researchers, n = 36 practitioners, multiple answers possible; Percentages of n = 36; more than one option could be selected). *p < 0.05; +non-significant tendency with p < 0.1 (Chi-square tests); Source: Balthasar et al. (2018).
3.2.2. Practitioners’ perspective
Practitioners perceived their involvement in the research projects as follows. They considered their participation in accompanying groups (67 per cent) and workshops (64 per cent) as the most frequently used forms of collaboration, followed by their availability for interviews and assistance in access to information (role as door opener, e.g. for access to data, clients, participants) both with 39 per cent. Provisioning of data and support in the communication of results were both mentioned by 28 per cent of the practitioners.
The relative frequencies of the different categories named by researchers and practitioners were compared using a Chi-square test. By and large practitioners and researchers perceived the forms in which they collaborated in similar ways. The largest difference between their perceptions was found for the provisioning of data, which was acknowledged much more often by the researchers than by the practitioners (54 per cent; χ2 = 6.38, df = 1, p = 0.012). This was also the only significant difference between the two perspectives in relation to the frequency of a certain collaboration format (at the level of p < 0.01). In addition, there were three forms of collaboration where the differences were actually non-significant but where a substantial difference was observed (0.05 < p < 0.1). These three forms of collaboration, namely (1) provision of access to information or stakeholders (role as door opener), (2) financial participation with contribution in kind, and (3) support in communicating results were all perceived substantially more often by the researchers than by the practitioners. A reason for these differences could be that practitioners might recall primarily regular forms of participation, while researchers also remember and mention one-shot services, such as the transfer of data.
3.3. Perceived frequency of interaction in the three phases
Another key feature investigated relates to the perceived frequency of researcher–practitioner interactions for the three phases of the research project. The distribution of the respective responses from researchers and practitioners is displayed in Fig. 4.
Ratings of the frequency of the contact between researchers and practitioners in the phases of (1) the definition of the research questions, (2) the scientific working phase aiming to answer the research questions and the (3) the communication of the findings. *p < 0.05; **p < 0.01; ***p < 0.001 (Mann–Whitney U-tests); Source: Balthasar et al. (2018).
For statistical comparisons with significance tests, the responses were considered as a rank scale from 1 to 4 to compare responses of researchers and practitioners (via Mann–Whitney U-tests) and to compare the three phases (via Wilcoxon tests). Three non-parametric Mann–Whitney U-tests showed that the perceived higher contact frequency reported by the researchers compared to the practitioners was significant for the design phase (Mresearch = 2.3; Mpractice = 1.5, P < 0.001), the active research phase (Mresearch = 3. Mpractice = 2.6, P = 0.034) as well as for the dissemination phase (Mresearch = 2.8, Mpractice = 2.1, P = 0.002). Apparently, researchers perceived the research–practice collaboration in all three phases to be more intense than the practitioners involved (Fig. 4).
There were also significant differences among the three phases. According to researchers’ ratings, the contact during the phase of developing the research question was significantly less intense than during both the phase of work on the research question (Wilcoxon test, P < 0.001) and the phase of communicating the findings (p = 0.004). A similar picture emerged from the analysis of the practitioners’ responses. They perceived the contact during the phase of developing the research question as significantly less intense than during the work on the research question (p < 0.001) and the communication of the findings (p = 0.006). In addition, the practitioners also perceived the collaboration during the work on the research question as more intense than during the communication of the findings (p = 0.037).
3.4. Perceived importance of participation factors for a successful collaboration
The analysis of the participation factors, which the literature states should support a successful collaboration, shows that both researchers and practitioners share the same perception regarding the importance of the five participation factors selected (Table 2). Both researchers and practitioners consider the common perception of the problem to be solved to be the most important factor for a successful collaboration. The second most important factor, according to the researchers, is that practitioners have enough time and, conversely, for practitioners that researchers commit to spending time on the collaboration. Both groups consider sufficient time on their own side to be the third most important factor. Furthermore, both groups consider sufficient financial resources for the practitioners to be the fourth most important factor. Finally, geographical proximity to each other is rated by both groups as the least important factor.
Assessment of the importance of different factors for successful collaboration between research and practice (researchers’ and practitioners’ perception).
| Perception of researchers | Perception of practitioners | ||
|---|---|---|---|
| Rank | Factors | Rank | Factors |
| 1. | Shared perception of the problem to be solved | 1. | Shared perception of the problem to be solved |
| 2. | Practitioners have sufficient time | 2. | Researchers have sufficient time |
| 3. | Researchers have sufficient time | 3. | Practitioners have sufficient time |
| 4. | Practitioners have sufficient financial resources | 4. | Practitioners have sufficient financial resources |
| 5. | Geographical proximity to practitioners | 5. | Geographical proximity to researchers |
| Perception of researchers | Perception of practitioners | ||
|---|---|---|---|
| Rank | Factors | Rank | Factors |
| 1. | Shared perception of the problem to be solved | 1. | Shared perception of the problem to be solved |
| 2. | Practitioners have sufficient time | 2. | Researchers have sufficient time |
| 3. | Researchers have sufficient time | 3. | Practitioners have sufficient time |
| 4. | Practitioners have sufficient financial resources | 4. | Practitioners have sufficient financial resources |
| 5. | Geographical proximity to practitioners | 5. | Geographical proximity to researchers |
Data source: Balthasar et al. (2018).
Ranking on the basis of the total number of points = sum of the awarded points over all interviewed researchers or practitioners; scoring: most important reason = 5 points; second most important reason = 4 points; third most important reason = 3 points; fourth most important reason = 2 points; fifth most important reason = 1 point.
Assessment of the importance of different factors for successful collaboration between research and practice (researchers’ and practitioners’ perception).
| Perception of researchers | Perception of practitioners | ||
|---|---|---|---|
| Rank | Factors | Rank | Factors |
| 1. | Shared perception of the problem to be solved | 1. | Shared perception of the problem to be solved |
| 2. | Practitioners have sufficient time | 2. | Researchers have sufficient time |
| 3. | Researchers have sufficient time | 3. | Practitioners have sufficient time |
| 4. | Practitioners have sufficient financial resources | 4. | Practitioners have sufficient financial resources |
| 5. | Geographical proximity to practitioners | 5. | Geographical proximity to researchers |
| Perception of researchers | Perception of practitioners | ||
|---|---|---|---|
| Rank | Factors | Rank | Factors |
| 1. | Shared perception of the problem to be solved | 1. | Shared perception of the problem to be solved |
| 2. | Practitioners have sufficient time | 2. | Researchers have sufficient time |
| 3. | Researchers have sufficient time | 3. | Practitioners have sufficient time |
| 4. | Practitioners have sufficient financial resources | 4. | Practitioners have sufficient financial resources |
| 5. | Geographical proximity to practitioners | 5. | Geographical proximity to researchers |
Data source: Balthasar et al. (2018).
Ranking on the basis of the total number of points = sum of the awarded points over all interviewed researchers or practitioners; scoring: most important reason = 5 points; second most important reason = 4 points; third most important reason = 3 points; fourth most important reason = 2 points; fifth most important reason = 1 point.
Subsequently, researchers and practitioners were asked about other beneficial and hindering factors. The researchers mentioned that it was beneficial for the success of collaboration if the practitioners had already been involved in research projects in the past or if they were genuinely interested in research or scientific thinking. Furthermore, they stated that it was important to agree on the questions and procedures. This is supported by regular exchange and collaboration with an already known person. One hindering factor was the difficulty of finding time for joint appointments.
In addition, practitioners considered it beneficial if the persons already knew each other, had shared interests, or spoke a ‘common language’. Five practitioners mentioned that it was more difficult to collaborate if researchers themselves had little or no experience in the market sector.
3.5. Perceived usefulness of the research project
3.5.1. Researchers’ perspective
Five researchers responded to the question ‘Do you think the results are of relevance for the activities of your partners?’ with ‘I do not know’. They were thus excluded from the following analysis. Of the remaining sixty-two researchers, 43.5 per cent responded with ‘Yes’ and 56.5 per cent with ‘To some extent’. No one responded with ‘No’. Consequently, the indicator of usefulness for practice resulted in a dichotomous variable with only two levels due to the restricted response range of the participants. Therefore, instead of correlations, Mann–Whitney U-tests were used to investigate the relationships between various features of the research collaboration and relevance of the project results for the practitioners as perceived by the researchers.
The significant findings emerging from these analyses are shown in Table 32. Similarly to the aspects mentioned by practitioners, those researchers who perceived a high practical relevance of the research findings (‘yes’ group) (1) had a significantly higher contact frequency with the practitioners involved, (2) perceived the role of the practitioners in their project to be more clear, and (3) incorporated inputs from their partners more often into their projects, compared to researchers who were less sure about the relevance of the research findings to their partners (‘to some extent’ group).
Significant differences (according to Mann–Whitney U-tests) in responses of the researchers perceiving a high relevance of the research findings for the involved practitioners (‘yes’ response) and researchers perceiving it only ‘to some extent’ in relation to important aspects of the collaboration.
| Collaboration aspects (items) | N | Mscale | Mranks | Significance p |
|---|---|---|---|---|
| Frequency of contact with practitionersa | 26 | Myes = 2.3 | 34.4 | 0.032 |
| 32 | Mpartly = 1.8 | 25.5 | ||
| At times, the role of the partners in the project was unclear to me.b | 27 | Myes = 3.7 | 36.2 | 0.044 |
| 35 | Mpartly = 3.3 | 27.9 | ||
| Have you incorporated inputs from your partners into the project?c | 26 | Myes = 1.2 | 25.8 | 0.018 |
| 35 | Mpartly = 1.5 | 34.9 |
| Collaboration aspects (items) | N | Mscale | Mranks | Significance p |
|---|---|---|---|---|
| Frequency of contact with practitionersa | 26 | Myes = 2.3 | 34.4 | 0.032 |
| 32 | Mpartly = 1.8 | 25.5 | ||
| At times, the role of the partners in the project was unclear to me.b | 27 | Myes = 3.7 | 36.2 | 0.044 |
| 35 | Mpartly = 3.3 | 27.9 | ||
| Have you incorporated inputs from your partners into the project?c | 26 | Myes = 1.2 | 25.8 | 0.018 |
| 35 | Mpartly = 1.5 | 34.9 |
Data source: Balthasar et al. (2018).
Low frequency = 1, moderate = 2, high = 3.
Yes = 1, rather yes = 2, rather no = 3, no = 4.
Yes = 1, rather yes = 2, rather no = 3, no = 4; ‘I do not know’ = excluded from the analysis.
Significant differences (according to Mann–Whitney U-tests) in responses of the researchers perceiving a high relevance of the research findings for the involved practitioners (‘yes’ response) and researchers perceiving it only ‘to some extent’ in relation to important aspects of the collaboration.
| Collaboration aspects (items) | N | Mscale | Mranks | Significance p |
|---|---|---|---|---|
| Frequency of contact with practitionersa | 26 | Myes = 2.3 | 34.4 | 0.032 |
| 32 | Mpartly = 1.8 | 25.5 | ||
| At times, the role of the partners in the project was unclear to me.b | 27 | Myes = 3.7 | 36.2 | 0.044 |
| 35 | Mpartly = 3.3 | 27.9 | ||
| Have you incorporated inputs from your partners into the project?c | 26 | Myes = 1.2 | 25.8 | 0.018 |
| 35 | Mpartly = 1.5 | 34.9 |
| Collaboration aspects (items) | N | Mscale | Mranks | Significance p |
|---|---|---|---|---|
| Frequency of contact with practitionersa | 26 | Myes = 2.3 | 34.4 | 0.032 |
| 32 | Mpartly = 1.8 | 25.5 | ||
| At times, the role of the partners in the project was unclear to me.b | 27 | Myes = 3.7 | 36.2 | 0.044 |
| 35 | Mpartly = 3.3 | 27.9 | ||
| Have you incorporated inputs from your partners into the project?c | 26 | Myes = 1.2 | 25.8 | 0.018 |
| 35 | Mpartly = 1.5 | 34.9 |
Data source: Balthasar et al. (2018).
Low frequency = 1, moderate = 2, high = 3.
Yes = 1, rather yes = 2, rather no = 3, no = 4.
Yes = 1, rather yes = 2, rather no = 3, no = 4; ‘I do not know’ = excluded from the analysis.
3.5.2. Practitioners’ perspective
The practitioners’ responses show that they consider the results of the research as quite useful for them. The mean value of the corresponding composite usefulness score which assessed the usefulness of the research project for practice from the practitioners’ point of view was, with M = 2.7 (SD = 0.6), in the upper half (> 2.5) of the four-point scale (from 1 = low usefulness to 4 = high usefulness). Table 4 shows the aspects that were significantly related to the usefulness of the research project for the practitioners. A successful and trustful relationship with researchers (r = 0.474, P <0.01), a higher frequency of the contact with the researchers (r = 0.398, P < 0.05), and the perception that their own knowledge and expertise were highly valued by researchers (r = 0.356, P < 0.05*) were significantly correlated with the perceived usefulness of the research project for the practitioners. Interestingly, in addition to these correlations found (Table 4), the trustful relationship with researchers is highly correlated with the frequency of the contact between practitioners and researchers (r = 0.536**), so that the two most important factors for the usefulness of the research for practitioners are themselves correlated with each other.
Significant correlations between aspects of collaboration as perceived by the practitioners and the ‘usefulness of the research for practice’ (composite score).
| Variable | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.474** | 0.005 | 33 |
| Frequency of contact | 0.398* | 0.018 | 35 |
| Perceived estimation of knowledge and expertise | 0.356* | 0.042 | 33 |
| Variable | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.474** | 0.005 | 33 |
| Frequency of contact | 0.398* | 0.018 | 35 |
| Perceived estimation of knowledge and expertise | 0.356* | 0.042 | 33 |
Data source: Balthasar et al. (2018).
n = 33–35. *P < 0.05, **P <.0.01
Significant correlations between aspects of collaboration as perceived by the practitioners and the ‘usefulness of the research for practice’ (composite score).
| Variable | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.474** | 0.005 | 33 |
| Frequency of contact | 0.398* | 0.018 | 35 |
| Perceived estimation of knowledge and expertise | 0.356* | 0.042 | 33 |
| Variable | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.474** | 0.005 | 33 |
| Frequency of contact | 0.398* | 0.018 | 35 |
| Perceived estimation of knowledge and expertise | 0.356* | 0.042 | 33 |
Data source: Balthasar et al. (2018).
n = 33–35. *P < 0.05, **P <.0.01
3.6. Perceived contributions of the practitioners to the success of the research project
3.6.1. Researchers’ perspective
Researchers were also asked how the research project would have progressed without the participation of the practitioners (response scale: ‘better’ = 1, ‘the same’ = 2, ‘worse’ = 3; additional option ‘I do not know’). Five researchers responded ‘I do not know’ and one did not respond at all. Of the remaining 61 researchers 14.8 per cent selected ‘better’, 13.1 per cent ‘the same’ and a clear majority of 72.1 per cent responded with ‘worse’, indicating that the research benefited substantially from the practice collaboration is by far the majority of the projects (M = 2.6, SD = 0.7). It was further investigated whether researchers’ judgement of the various aspects of the collaboration correlated significantly with the perceived usefulness of the practitioners’ input for the research project. However, no significant relationships emerged. Still, considering different forms of collaboration, it turned out that researchers evaluated practitioners’ contribution to the success of the research process significantly higher if their partners assisted in access to information (e.g. data, clients, stakeholders) and/or acted as door opener (M = 2.7 vs. M = 2.4 if not; see Table 5).
Significant differences (according to Mann–Whitney U-tests) in researchers’ assessment of how the processing of the research question would have progressed without the participation of the practitioners depending on the form of collaboration.
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders)/role as door opener: Yes | 35 | Myes = 2.7 | 34.1 | 0.046 |
| No | 26 | Mno = 2.4 | 26.9 |
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders)/role as door opener: Yes | 35 | Myes = 2.7 | 34.1 | 0.046 |
| No | 26 | Mno = 2.4 | 26.9 |
Data source: Balthasar et al. (2018).
Item/ scale: How would the research project have progressed without the participation of the practitioners? (response scale: ‘better’ = 1, ‘the same’ = 2, ‘worse’ = 3; the response option ‘I do not know’ was excluded from the analysis).
Significant differences (according to Mann–Whitney U-tests) in researchers’ assessment of how the processing of the research question would have progressed without the participation of the practitioners depending on the form of collaboration.
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders)/role as door opener: Yes | 35 | Myes = 2.7 | 34.1 | 0.046 |
| No | 26 | Mno = 2.4 | 26.9 |
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders)/role as door opener: Yes | 35 | Myes = 2.7 | 34.1 | 0.046 |
| No | 26 | Mno = 2.4 | 26.9 |
Data source: Balthasar et al. (2018).
Item/ scale: How would the research project have progressed without the participation of the practitioners? (response scale: ‘better’ = 1, ‘the same’ = 2, ‘worse’ = 3; the response option ‘I do not know’ was excluded from the analysis).
3.6.2. Practitioners’ perspective
Of the practitioners, 80 per cent stated that they perceived having contributed (‘yes’ or ‘rather yes’) to the success of the research project. The average rating of practitioners on the corresponding four-point agreement scale was M = 2.8 (SD = 0.6). Most of them (94 per cent) also stated that they could bring in their experience into the project (‘yes’ or ‘rather yes’). Practitioners perceived having a higher contribution to the success of the research project if (1) they were able to build up a trustful relationship with the researchers (r = 0.484**); (2) it was clear to them what their role in the project was (r = 0.371*); (3) they had frequent contact with the researchers (r = 0.579**); and (4) they perceived the research project to be useful (r = 0.377*, see Table 6). In addition, considering the different forms of collaboration, practitioners assisting in providing access to information, data, or acting as door openers perceived a significantly higher contribution to the success of the research project (M = 3.1) compared to practitioners who stated not having acted as e.g., door openers (M = 2.7, see Table 7).
Practitioners’ view on the factors affecting the ‘perceived contributions to the success of the research project’ (bivariate correlations between contribution ratings and perceived collaboration aspects).
| Collaboration aspects (Spearman rank correlations) | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.484** | 0.004 | 33 |
| There were moments in which it was not clear what my role in the project is | −0.371* | 0.031 | 34 |
| Frequency of contact | 0.579** | <0.001 | 35 |
| Usefulness of research for practitioners (composite score) | 0.377* | 0.026 | 35 |
| Collaboration aspects (Spearman rank correlations) | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.484** | 0.004 | 33 |
| There were moments in which it was not clear what my role in the project is | −0.371* | 0.031 | 34 |
| Frequency of contact | 0.579** | <0.001 | 35 |
| Usefulness of research for practitioners (composite score) | 0.377* | 0.026 | 35 |
Data source: Balthasar et al. (2018).
Practitioners’ view on the factors affecting the ‘perceived contributions to the success of the research project’ (bivariate correlations between contribution ratings and perceived collaboration aspects).
| Collaboration aspects (Spearman rank correlations) | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.484** | 0.004 | 33 |
| There were moments in which it was not clear what my role in the project is | −0.371* | 0.031 | 34 |
| Frequency of contact | 0.579** | <0.001 | 35 |
| Usefulness of research for practitioners (composite score) | 0.377* | 0.026 | 35 |
| Collaboration aspects (Spearman rank correlations) | R2 | Significance p | n |
|---|---|---|---|
| Trustful relationship with researchers | 0.484** | 0.004 | 33 |
| There were moments in which it was not clear what my role in the project is | −0.371* | 0.031 | 34 |
| Frequency of contact | 0.579** | <0.001 | 35 |
| Usefulness of research for practitioners (composite score) | 0.377* | 0.026 | 35 |
Data source: Balthasar et al. (2018).
Significant differences (according to Mann–Whitney U-tests) in practitioners’ assessment of how the processing of the research project would have progressed without their participation - depending on the form of cooperation.
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders) / role as door opener: Yes | 14 | Myes = 3.1 | 20.2 | 0.039 |
| No | 21 | Mno = 2.7 | 14.6 |
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders) / role as door opener: Yes | 14 | Myes = 3.1 | 20.2 | 0.039 |
| No | 21 | Mno = 2.7 | 14.6 |
Data source: Balthasar et al. (2018).
Item/ scale: Responses of the practitioners to the question whether they think that they have ‘contributed to the success of the research project?’ (response scale from 1 = no to 4 = yes; without consideration of the additional response option ‘I do not know’).
Significant differences (according to Mann–Whitney U-tests) in practitioners’ assessment of how the processing of the research project would have progressed without their participation - depending on the form of cooperation.
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders) / role as door opener: Yes | 14 | Myes = 3.1 | 20.2 | 0.039 |
| No | 21 | Mno = 2.7 | 14.6 |
| Form of collaboration (items) | N | Mscalea | Mranks | Significance p |
|---|---|---|---|---|
| Assistance in access to information (e.g. data, clients, stakeholders) / role as door opener: Yes | 14 | Myes = 3.1 | 20.2 | 0.039 |
| No | 21 | Mno = 2.7 | 14.6 |
Data source: Balthasar et al. (2018).
Item/ scale: Responses of the practitioners to the question whether they think that they have ‘contributed to the success of the research project?’ (response scale from 1 = no to 4 = yes; without consideration of the additional response option ‘I do not know’).
3.7. Fulfilment of the expectation of the collaboration
Reflecting on the collaboration process and its results, researchers and practitioners were asked whether their initial expectations were met. Of the researchers, 90 per cent stated that their expectations regarding their collaboration with practitioners had been fulfilled or partially fulfilled, 5 per cent said that their expectations had (rather) not been met, and 5 per cent had chosen the option ‘I don't know’. Disaggregated findings for different types of practitioners and researchers respectively show that the researchers at universities of applied sciences stated particularly often that their expectations had been fulfilled (75 per cent) or rather fulfilled (25 per cent). The situation is similar for researchers working for private companies. They also stated that their expectations were met (69 per cent) or rather fulfilled (31 per cent). The perspectives of the university researchers are less enthusiastic: only 33 per cent stated that their expectations were met and 61 per cent that they were rather met. For around 5 per cent of researchers at universities, their expectations were not met at all. A Kruskal–Wallis test showed that the difference between the fulfilment of the expectations of researchers from universities, universities of applied sciences, and private companies was significant (df = 2, p = 0.014; excluding ‘I don't know’ responses).
The higher the frequency of the contact with practitioners was, the more likely was it that the expectations of the researchers for the collaboration were fulfilled. The corresponding rank-correlation between the two variables was with r = 0.33 clearly significant (p = 0.011).
Practitioners’ assessment of the fulfilment of their expectations regarding the collaboration resembles the one of the researchers: 54 per cent of the practitioners stated that their expectations had been fulfilled. For 40 per cent, their expectations were partly fulfilled and for 6 per cent they were not fulfilled. Here the findings likewise indicate that the frequency of contact with the researchers plays a crucial role for meeting the practitioners’ expectations, even though the correlation slightly missed the 5 per cent significance level (r = 0.31, P = 0.067). These results show that researchers and practitioners are not monolithic entities and that there is a need to study them as diverse and heterogeneous entities.
4. Discussion
In this article, we analysed researchers’ and practitioners’ motivations for engaging in a collaborative research project, their perspective on the relevance of factors for a successful collaboration, their perceptions of the forms and intensity of their collaboration, their assessment of how practitioners contribute to the research project as well as of the latter’s practical usefulness. The results were derived from a survey with researchers and their partners from practice from a National Research Programme on energy research in Switzerland (Balthasar and Roose 2018). It is one of only a few quantitative studies analysing the perceptions of both researchers and practitioners who participated in such a programme.
In the following, we discuss the results obtained in this study along three main lines: first, we present an overall synthesis of the results and their implications for theory and practice; secondly, we derive policy recommendations for large funding programmes that seek to have a societal impact. Lastly, we discuss limitations of this study and identify promising avenues for future research.
4.1 Overall synthesis and implications for theory and practice
4.1.1 Determinants affecting the usefulness of collaboration for practitioners
One central claim of TD approaches scrutinised in this study is that practitioners’ participation facilitates the implementation and applicability of the knowledge produced in the research project (Gross and Hoffmann-Riem 2005; Hansson and Polk 2018; de Jong et al. 2016). Figure 5 summarises the significant determinants of the usefulness of the research project for practitioners, bringing together the analysis of both the practitioners’ and the researchers’ perspectives. To achieve comparable indicators for both perspectives, we use biserial rank correlation coefficients as measures of the strength of influence of the aspects, according to the researchers’ perspective (instead of comparing the differences in means as shown in Table 2).
Significant aspects related to the usefulness of the research project for practitioners according to the analysis of the responses of researchers and practitioners (* p < 0.05, **p <.0.01).
For both, practitioners and researchers, the perceived frequency of contact is a significant determinant of an increase in the project’s usefulness for practitioners. In the case of practitioners, the frequency of contact is also related to establishing a trustful relationship with researchers, thus confirming that this is a key factor for a successful TD collaboration (Elzinga 2008; Renner et al. 2013; Shdaimah and Stahl 2012).
The second determinant (Figure 5) is similar for practitioners and researchers: ‘perceived appreciation of their own knowledge and expertise’ (practitioners) and ‘level of integration of practitioners’ input into the research project’ (researchers). This determinant suggests that the better the practitioners’ inputs can be integrated into the research project, the higher the perceived appreciation of their knowledge and expertise is likely to be—leading to an increasing usefulness of the research project for practitioners. This second determinant could potentially be linked with the perceived frequency of contact in the different project phases (see Section 4.1.3).
The third determinant differs between researchers and practitioners (Figure 5). For researchers, the perceived clarity of the practitioners’ role in the project is pivotal for ensuring the practical usefulness of the research. For practitioners, the results indicate that a trustful relationship is a key determinant of the practical usefulness of the research. The fact that the frequency of contact was also correlated with mutual trust suggests a connection between these three factors. One could envision a dynamic relationship like the one postulated by Ostrom (1998: 13), who states: ‘If initial levels of cooperation are moderately high, then individuals may learn to trust one another, and more may adopt reciprocity norms. When more individuals use reciprocity norms, gaining a reputation for being trustworthy is a better investment. Thus, levels of trust, reciprocity, and reputations for being trustworthy are positively reinforcing.’ This indicates that a perceived higher intensity of collaboration increases trust, which, in turn, leads to a higher usefulness of the research for practice and finally, also to a higher perceived contribution of practitioners to the success of the research project (see Figure 6).
Significant aspects related to the ‘perceived contributions to the success of the research project’ according to the analysis of the responses of researchers and practitioners (* p < 0.05, **p <.0.01, ***p < .0.001).
4.1.2 Determinants affecting the usefulness of collaboration for researchers
The analyses of researchers’ and practitioners’ responses are consistent in relation to (only) one determinant which statistically significantly affects the perceived contribution of practitioners to the success of the research project. This determinant includes the assistance of practitioners in providing access to information for researchers, as well as in opening the door to the practice field (Fig. 6). From the practitioners’ perspective, furthermore, the perceived frequency of contact, a trustful relationship with researchers, the perceived clarity of their own role within the project, and the usefulness of the research for themselves were relevant aspects for their perceived contribution to the research. This is well in agreement with their motivations for participating in the research project. Practitioners wanted to contribute to solving a problem, co-create knowledge with the researchers, and support the latter. This finding underlines the importance of eliciting and discussing the motivations of both researchers and practitioners for getting involved and collaborating in a research project. This would allow for clarifying the role of the practitioners in the project and negotiating how and when they should and would like to be involved (see also Fritz et al. 2019; Fritz and Binder 2020).
4.1.3 Frequency of contact as crucial variable
One core process feature of TD research that has been scrutinised in this study is the intensity of the collaboration, here measured as frequency of contact. Our findings suggest that the perceived intensity of collaboration is a key variable for (1) ensuring the usefulness of the research project for practitioners; (2) having a higher perceived contribution to the success of the research project (practitioners), (3) fostering trust between researchers and practitioners; and (4) ensuring the relevance of the results for practice. Our findings confirm the pertinence of the intensity of collaboration as an indicator of a fruitful collaboration (Schneider and Buser 2018; Stauffacher et al. 2008), but add a word of caution to interpreting them. The perceptions of intensity were found to largely diverge, with researchers assessing intensities higher than practitioners. Thus, using intensities as an indicator for fruitful collaboration requires due consideration of both researchers’ and practitioners’ assessments.
Indeed, our results show that researchers and practitioners perceived the intensity of their collaboration differently, with practitioners perceiving their degree of involvement as being significantly lower than researchers. This holds true for all three research phases. The difference in the perception of the collaboration intensity might be related to the perceived form of collaboration. For specific activities, such as assistance in access to information (p < 0.05), provision of data (p < 0.01), support in communication of results (p < 0.05), and financial contribution (p < 0.05), researchers perceived a significantly higher level of interaction than practitioners. These activities constitute, however, weak forms of collaboration, better captured in terms of one-way transfer of information or services. The chosen type of collaboration might be related to the character of the funding instrument, in which only researchers are funded and marginal funding is available for practitioners (in the form of remuneration for assistance in workshops and steering boards). In contrast, even though not statistically significant, practitioners perceived their involvement to be mainly in the form of participation in steering groups and workshops, which can be considered a stronger form of collaboration (Stauffacher et al. 2008).
Another key finding relates to the variance of intensities of collaboration over the project course. Both researchers and practitioners consider a common problem understanding as the most important factor for a successful collaboration (see Table 2). As presented above, typically this common understanding is reached in the first phase of the research, that is, the development of the research question. In this phase, however, the perceived intensity of collaboration was significantly lower than in the other two phases. These results corroborate qualitative studies on TD in practice, which found that often initial project ideas originate in an academic context, with researchers then approaching practitioners whom they consider relevant to be involved. The latter are frequently only taken on board after the funding proposals have been accepted (Felt et al. 2012, 2013; Lang et al. 2012; Wuelser and Pohl 2016). Some scholars have explained this phenomenon with the incompatibility of co-design of project ideas and proposal-based research funding mechanisms (Schmidt and Pröpper 2017; Talwar et al. 2011; Whitman et al. 2015). A number of scholars argue that whoever initially formulates the problem owns the process (Webber and Ison 1995). In this perspective, the lack of practitioners’ involvement in the initial phases of a project leads to an unbalanced problem ownership (Lang et al. 2012; Röckmann et al. 2015; Siew et al. 2016; Stauffacher et al. 2008; Talwar et al. 2011) and is assumed to lower the legitimacy of the research (Robinson and Tansey 2006) and the chances of its results being implemented (Renner et al. 2013). Surprisingly, however, in the present study, considerable success of research–practice collaboration was observed even though the frequency of interaction in the early stages of the research projects, where a common problem understanding is developed, was much lower than in subsequent phases of the projects.
Three explanations might elucidate this seemingly inconsistent observation: The first is that the problems addressed in the research programme related to the energy transition and thus might be clear for both researchers and practitioners, since it has been a prevalent issue during recent years . In this case, there might be less need to obtain a common understanding than in a less popular topic. Secondly, the motivations for researchers to include practitioners and for practitioners to participate in research projects are similar. Both aim at contributing to solving a real-world problem by performing and contributing to research (Figure 2). Thirdly, the results suggest that practitioners’ involvement in the form of workshop participation and participation in steering boards gives them sufficient possibilities to bring forward their views and thus contribute to a common understanding of the research project.
Finally, the relationship between the intensity of collaboration and trust merits attention. Our results suggest that trust is a key factor for the perceived practical usefulness of the research project, as well as for the perceived contribution of practitioners to the success of the research project. The centrality of trust found in this study corroborates previous qualitative findings on the role of trust in participatory and collaborative research projects (Elzinga 2008; Renner et al. 2013; Shdaimah and Stahl 2012).
4.2 Policy implications: the role of funding programmes
The findings of this study point to some key issues to be considered when developing funding programmes that ask applying projects to involve practitioners and tackle real-world problems. As shown below, funding programmes can play a significant role in fostering the collaboration between researchers and practitioners as well as improving the quality of their interactions.
4.2.1 Fostering trust through the intensity of collaboration
The role of trust emerged as one of the key determinants for increasing the usefulness of the results for practice and the relevance of the input of practitioners for research. In participation scholarship, prior collaborations and building on existing relationships are seen as one way of ensuring trust amongst participants (Maclure and Bassey 1990; McKee et al. 2015). A second way of ensuring trust, which is confirmed by our findings, is through the project and joint activities within it (Dewulf et al. 2005; Maclure and Bassey 1990). Additionally, if the collaboration has been successful, practitioners have stated that they are prepared for a collaboration with researchers and are likely to engage in future research collaborations (see also Binder et al. 2015). Our results complement these findings by adding the component of the intensity of interaction between practitioners and researchers. Hence, at the programme level the interaction between researchers and practitioners can be actively promoted. This is confirmed as 12 per cent of the researchers stated that if they had not been asked to interact with practitioners as a programme requirement they would not have done so. Thus, funding programmes might want to foster the intensity of collaboration by having programme meetings to which practitioners are invited or explicitly ask for meetings between researchers and practitioners in the proposal writing phase. In this way, according to our results, trust between researchers and practitioners could be enhanced. Finally, funders should recognise that building trust and co-producing knowledge in a collaboration that encompasses many perspectives can be time-intensive (also addressed by Lyall et al. 2013)
4.2.2 Including practitioners in early phases of the research project
The participation factor stated as the most important one by researchers and practitioners alike was a shared perception of the problem and the agreement on the research question. This common agreement is usually obtained in the beginning of the research project. However, both researchers and practitioners perceived the intensity of their interaction to be the lowest in this project phase. Thus, it should be further explored when the early involvement of practitioners in the design phase is necessary. Our results suggest that by including practitioners in workshops early on to critically reflect together on the problem to be addressed might lead to a common understanding of the research question and aims. These, in turn, are essential for practitioners to share the data relevant for the research project and act as a door openers..
4.2.3 Invest in a common understanding on what is meant by ‘partner from practice’
One important result of the study is that researchers and practitioners did not always share the same understanding of their role in the collaboration. However, a common interest and understanding of the other side’s point of view are important prerequisites for the success of research–practice collaboration. Our results suggest that a clarification of the role of practitioners within the research project could be key for increasing practitioners’ contribution to the latter. For this reason, roles must be clarified at an early stage so that mutual expectations coincide. Funders might support the clarification of roles, by providing professional support for working on eliciting the mutual expectations of researchers and practitioners. A common workshop at the beginning of the project where explicitly expectations are shared and potential conflicts are identified could further enhance the collaboration and, thus, benefit both, researchers and practitioners.
At the programme level, this information would also allow for providing targeted support regarding the type and intensity of interaction between researchers and practitioners. In support of previous research, our findings suggest furthermore that at the programme level, also the motivation of practitioners for increased networking among themselves and with researchers should be considered—an intended effect highlighted in TD literature (Binder et al. 2015; Fritz et al. 2019; Hansson and Polk 2018; Spaapen and Van Drooge 2011). Such dynamics could be stimulated through joint activities at the programme-level.
4.3 Limitations and future research
While the quantitative analysis of research–practice interactions performed in this article offers valuable insights into diverse perceptions of TD practices in a major Swiss research programme, its analytical scope remains partial.
A first limitation of the study could be the institutional context within which this study was conducted. It is possible that an eventual bias of the results was caused by the fact that those responsible for the NRP conducted the study. However, there were no dependencies between the practitioners involved and the NRP, for example, in the form of financial transfers, which could have influenced the answers.
Secondly, another important limitation of this study relates to its focus on a single funding programme in the Swiss science policy context. The resulting lack of a comparative assessment circumscribed the extent to which we could capture the influence of specific funding conditions on the process, its perceived success, and practical relevance. Funding bodies and funding conditions which set the leeway for researcher–practitioner interactions (e.g. through resources provided by the funding programme, contract formats, or proposal requirements) (Fritz and Binder 2020; Lux et al. 2019; Newig et al. 2019), thus, merit greater attention in future research on features and effects of participation processes in TD research. Conducting similar surveys to the one proposed in this study in a wider sample of research programmes would allow to build such a broader knowledge base on the role of funding bodies and the conditions they set in TD research.
Thirdly, the translation of multi-faceted concepts that convey multiple meanings into quantitative survey items comes with certain limitations. While the analysis performed allowed us to pinpoint key drivers for a successful collaboration between researchers and practitioners, it does not allow for conclusions on what constitutes ‘success’ in the context of TD projects. The very definition of a ‘successful’ collaboration might vary and different actors make use of different collaboration features to qualify it as (un)successful (Zscheischler et al. 2018). The ways in which actors think about success and ‘failure’, respectively, thus requires closer examination. Similarly, our study did not account for the fact that ‘usefulness’ and ‘practical relevance’ might be subject to fundamentally ambiguous interpretations and can carry different meanings (Schikowitz 2019).
Our research also opened avenues for further research: firstly, a key aspect of the collaboration between researchers and practitioners is the added value to research and practice. However,, there might be multiple interpretations as to when a collaboration is successful and an output useful. Thus, a qualitative in-depth assessment of the diverse and equally-valid ways in which researchers and practitioners conceive ‘success’, ‘usefulness’, and ‘practical relevance’ would add crucial insights to this study, in particular as to how to shape this type of research programmes. Secondly, we pointed out that the frequency of contact fosters the usefulness of the project for practitioners. Thus, one should further investigate to what extent the different forms of involvement (e.g. workshops, steering boards) might be related to the perception of practitioners that their own ideas and knowledge feed into the research project. Thirdly, this study found trust to play a key role in the collaborative research projects analysed. Besides the key role of trust in the ongoing participatory and collaborative research project (Elzinga 2008; Renner et al. 2013; Shdaimah and Stahl 2012), scholars have argued that, if individuals have had positive earlier experiences in collaboration with practice or science, they bring their positive experience into the setup of the new collaboration, potentially accelerating the development of mutual trust (Fritz et al. 2019, Binder et al. 2015). Future research should, thus, investigate the role of earlier forms of involvement on trust and on the willingness to participate and engage in collaborative research projects.
5. Conclusions
This article presented the results of a survey with researchers and practitioners of a large Swiss national research programme on steering energy consumption. We analysed the motivation of practitioners and researchers for engaging in a collaborative research project, their perception of the intensity of the collaboration in the different project phases, and their perception of how practitioners contributed to the research project and of the extent to which the research project provided useful results for practitioners. Our analyses showed that in this programme, the main motivation for both researchers and practitioners was to contribute to research that addresses real-world problem and that can later be implemented. One key finding was that the intensity of collaboration constituted an important driver for a successful collaboration. The perceived intensity of collaboration fostered trust between researchers and practitioners and increased the practical usefulness of the research project as well as the perceived contribution of practitioners to the success of the research project. Secondly, both, researchers and practitioners, stated that the shared perception of the problem to be solved was the most important factor for a successful collaboration. However, we found that the interaction during the first phase of the project, where the problem is defined was the lowest. One open question to be further investigated concerns the role of earlier forms of involvement for building trust and their influence on the willingness to engage in collaborative research projects.
Finally, from our point of view, research programmes have an important role to play. They should (1) foster the trust relationship through developing mechanisms of exchange between research and practice; (2) support researchers and practitioners in developing a shared understanding of their respective role in the collaboration; and (3) enable the inclusion of practitioners already in the development of the project, for example, through providing seed money during the proposal-writing phase, in order to ensure that closer cooperation in research projects evolves from the beginning on.
Funding
The Swiss National Foundation’s National Research Program 71 on Managing Energy Consumption supported the realisation of the survey financially and administratively. We thank Swiss Mobiliar for their generous support.
Conflict of interest statement
The authors declare no conflict of interest. The survey was carried out anonymously.
Endnotes
This section is partly based on Fritz (2020).
Acknowledgements
We would like to thank the steering committee of the Swiss National Science Foundation’s National Research Program 71 on Managing Energy Consumption who agreed to realise the survey with researchers and practitioners on the effects of their collaboration. We thank our reviewers for their excellent comments. We also thank Simon Liebi for formatting and Hannah Roberson for language editing.
References
European Commission (
Future Earth. (
