General practice research training: impact of the Australian Registrar Research Workshop on research skills, confidence, interest and involvement of participants, 2002-2006

Background. An intensive 3-day training programme, the ‘Registrar Research Workshop’ (RRW), has aimed to build research capacity among Australian general practice registrars since 1994. Objectives. To investigate the impact of the RRW on participants' skills, confidence, interest in research and research activity. Methods. Cross-sectional postal survey in 2006 of five groups of registrars who participated in the annual workshop in 2002–2006 (response rate: 64%; 77 of 121). Outcome measures included research experience and skills prior to and after the workshop; impact of the workshop on capacity, confidence, attitude and interest in research; and research involvement as measured by publications and grant funding. Results. Self-reported research skills increased over time for the whole group (two-way analysis of variance: P = 0.047), most significantly for registrars with little or no research experience (P < 0.001) and research project participants (P = 0.003). The impact of the workshop on capacity, confidence and interest in research was rated highly (mean 3.5–4.0 ± 0.1 on a five-point scale). Two-thirds of the survey respondents had been research active, 34% presented their findings at conferences, 25% published in peer-reviewed journals and 31% received research funding. Eighty-four per cent of respondents indicated a high interest in undertaking research in the future. All survey respondents recommended the workshop to other registrars. Conclusions. The RRW provides a useful model for effective research training for interested general practice trainees. Such training has the potential to increase knowledge of research methods, which might augment future research activity in general practice.


Introduction
It is widely acknowledged that research in general practice is vital for the improvement of patient health care outcomes. 1,2 Compared to other clinical disciplines, however, general practice has produced significantly less published research. 3 The development of capacity for primary care research is therefore crucial.
Recently, there has been considerable investment in research capacity building in Australia. [4][5][6] Incorporating research in the process of vocational training is one way of developing research capacity in a clinical discipline. The Royal Australian College of General Practitioners (RACGP) acknowledges the need for research in its training programme curriculum with a 'critical thinking and research' statement, 7 and now some regional training providers include compulsory research projects as part of general practice vocational training in Australia. 8 The Registrar Research Workshop (RRW) is a national initiative to build research capacity among general practice trainees. This event is an intensive, annual, 3-day workshop which involves approximately 25 GP registrars, who are chosen via a competitive process. Applicants must demonstrate a special interest in learning about research, but existing knowledge or experience of research is neither a prerequisite nor an exclusion criterion. The workshop was initiated by the RACGP in 1994 and has been organized by General Practice Education and Training (GPET) since 2002. The event relies on experienced members of the GP research community, who offer presentations about the research process. Interweaved with these presentations are small group sessions in which registrar teams are guided to develop a research proposal. Beginning with ideas from clinical practice, registrars are led to develop a research question, identify research methods, plan a budget, address ethical concerns and develop a presentation on their project to share with their peers. 9 Although participants' immediate evaluations of the workshops had been positive, no long-term data existed on the participants' impressions of the workshop or their subsequent research behaviour. In this study, we aimed to measure the impact of the workshop on participants' skills, confidence and interest in research and their subsequent research involvement.

Methods
We undertook a cross-sectional survey in mid-2006 of 5-year groups of workshop participants (n = 121, 2002-2006), for which contact details were available from GPET. Our questionnaire collected data on participants' demographics, qualifications, research experience prior to the workshop and current research expertise and involvement. We also gathered participants' opinions on the workshop programme, the impact of the workshop on personal research capacity and their anticipated future participation in research. Our questionnaire was modified from an existing questionnaire used in previous studies. 5,6 Participants categorized their research involvement level using a previously described four-tiered model. 10 The validated 'research spider' 11 was used to measure experience in 10 core areas of research skills. A mixture of categorical data, Likert scales and free text responses were used for other items.
Contact information was retrieved by the GPET member of the research team from GPET databases, supplemented occasionally by information from regional training providers and state and territory medical boards. Participants were informed about the study via email and posted an introductory letter, the questionnaire and reply-paid envelope.
Confidentiality of responses was ensured, as deidentified coded questionnaires were sent directly to the University of Adelaide research team for data entry and analysis. Received questionnaire codes were reported to the GPET researcher for follow-up of non-responders. Non-responders were sent a reminder email and a second copy of the questionnaire a month after the first mail-out. A third and final email reminder was sent to non-responders.
Quantitative data were analysed in SPSS version 13.0 and SAS version 9.1. Data of all five cohorts were combined to increase sample size and therefore power and generalizability. We determined statistical differences in response rates between cohorts using chisquare analysis and used log binominal regression modelling to analyse trends in publication and grant application. Two-way analyses of variance (ANOVAs) were fitted to investigate self-reported changes over time in research skills using the research spider model. 11 Bowker's test of symmetry was applied to compare changes over time in self-reported research involvement. Thematic analysis was performed on qualitative data but not all qualitative data are reported in this article.
Information about non-responders was of poor quality, particularly for early cohorts, limiting comparisons between responders and non-responders. However, we compared gender distribution between the two groups and performed a sensitivity analysis to examine the potential effect of non-response on publication and funding outcomes.

Research experience and skills
We assessed participants' level of involvement in research using a previously described four-tiered model 10 : non-participants (Category 1, not participating Family Practice-an international journal in research activities), participants (Category 2, participating in research as member of a team), clinician researcher (Category 3, managing own project) and academic (Category 4, experienced researcher).
Our survey took snapshots of registrars' research involvement at two time points. Table 2 gives an overview of the research categories which survey participants considered themselves to belong to 'prior to' the workshop (time point 1) and at time of the survey 'after' the workshop (time point 2). Time point 1 varied between cohorts (2002-2006), while time point 2 gave a cross-sectional view at time of the survey. Table 2 indicates that two-thirds (66%, 51 of 77) of survey participants had undertaken research (categories 2-4) at some time, 42% (32 of 77) prior to the workshop and/or 51% (39 of 77) at the time of the survey. Forty-two per cent (19 of 45) of non-researchers (Category 1) at time point 1 increased their research involvement. Among all participants, a third (32%, 25 of 77) increased, 44% (34 of 77) sustained the same level and about a quarter (23%, 18 of 77) reduced or stopped their research involvement by time of the survey. However, 13 of the latter 18 indicated their plans to pursue research in the next 5 years. Confounding life events included, for example, maternity leave. While a number of individuals moved up or down the research category scale during the investigated time interval, Bowker's test of symmetry revealed no statistically significant change in research categories for the group as a whole (P = 0.082). Figure 1 illustrates respondents' self-reported mean research experience levels in 10 core skill areas of research and compares skill levels of respondents prior to the workshop with skill levels at the time of our survey. These range from low levels of experience for 'publishing research' [mean = 1.6 ± 0.1 standard error (SE)] and 'applying for research funding' (mean = 1.7 ± 0.1) to 'some experience' for 'finding literature' (mean = 3.2 ± 0.1) and 'critically reviewing literature' (mean = 3.0 ± 0.1).
Two-way ANOVA tests were conducted to analyse skill levels over time for the whole group and by research category. Research skill levels increased over time in all 10 research skill areas for the whole group Importance and impact of the RRW The majority (over 80%) of past workshop participants rated the general structural aspects of the workshop, such as presentations by experts to the whole group, the small group work and networking opportunities, 'important' or 'very important' with nearly half of the respondents (46%) stating that 'networking with experts and facilitators' had been very important (Box 1).
We assessed the impact of the RRW on participants' self-reported capacity (ability/knowledge), confidence and interest in research using a five-point Likert scale. Survey respondents felt that the workshop had 'moderate impact' (means = 3.5-4.0 ± 0.1 SE) on their capacity, confidence and interest in undertaking research in the future, but felt that the workshop had slightly less impact (mean = 3.3 ± 0.1) on their confidence and capacity for 'applying evidence-based research in clinical practice' (Fig. 2). For example, of 45 respondents in this study who were not research active before the workshop (Category #1), 19 (8 + 10 + 1) became research active after the workshop (Category #2-4). Six of the eleven respondents who considered themselves to belong to Category #3, clinician researcher, remained research active on this level, while four decreased their research activity and one was not in involved in any research at time of the survey. The majority of surveyed workshop participants (75%) reflected positively on the workshop's influence on their perception of research; stating a better understanding of the research process; raised awareness of its relevance, scope and importance in general practice; and realization that research is 'doable'(details of thematic analysis available from authors). Table 3 provides an overview of all survey respondents' research involvement and outcomes, such as publications and grants, as well as details of those respondents who were not involved in research prior to the workshop Category 1 (non-participants). A quarter of all respondents had published their research in a peer-reviewed journal and a third had been successful in receiving grant funding (Table 3, columns 2 + 3). Similarly, almost a quarter of respondents who were not research active prior to the workshop had published or submitted a paper for publication at time of the survey, and 22% had received grant funding ( Table 3, columns 4 + 5).

Registrars' research involvement and outcomes (publications and grants)
Because non-responders to our questionnaire might be less research active than responders, we undertook a simple sensitivity analysis to counter for nonresponse. Even if all our non-responders had been completely inactive in research, a total of 21% of all workshop participants would have presented research at conferences, 16% would have published in a peerreviewed journal and 20% would have successfully received grant funding.
Since time frames for applying for research funding, conducting and disseminating research differed between cohorts, we analysed research outcomes by cohort (year of participation in the RRW) using a log binominal regression model. The analysis revealed a significant increase over time available in the number of research grant applications made (P < 0.009) and successful receipt of grant funding (P < 0.008); 1-year difference was associated with a 27% increase in receipt of grant funding. Time trend analysis on publication rates was borderline significant (P < 0.06), with some suggestion that a greater percentage of ar-

Participation in future research
Registrars and GPs who had participated in the RRW and taken part in our survey were generally very enthusiastic about pursuing research in the future. About 80% indicated being highly interested in conducting further research, publishing research and attending further research training. Enthusiasm was a little lower for preparing grant applications (57%). Encouragingly, two-thirds of past workshop participants planned to integrate research into their career in general practice, mainly as clinician researcher (57%).

BOX 1 Examples of participants' comments on the workshop programme
The best aspect of the workshop was having access to senior GP academics-as role models-and meeting early career researchersas reassurance.
Role modelling from academics and research-minded registrars was influential. It was good to discover that academics were not scary!
The general enthusiasm amongst researchers was infectious on us registrars!
The small group work brought alive what we learnt in the presentations and was a vital part of the workshop. Perceived barriers to future research involvement matched previously identified themes 12 : lack of time and having other priorities were mentioned by the majority of respondents (69%), followed by lack of funding (53%), appropriate supervision and support (13%), research opportunities (8%), rural isolation (8%) and lack of collaborators (7%).
The future of the workshop All 77 respondents of the surveyed five workshop cohorts agreed that they would recommend the RRW to other registrars. Consistent themes emerging from responses included the workshop's uniqueness as a model of research training in general practice and the non-threatening and supportive nature of guidance offered by senior researchers at the workshop. Furthermore, some registrars emphasized the importance of continuation of the workshop for future registrar cohorts and suggested that the workshop be made available to more registrars (Box 2).

Discussion
Our study demonstrates that the RRW had a positive impact on participants' capacity, confidence and interest in research. Self-reported research experience and involvement, publication and grant application rates increased over time in the surveyed group.
Our study has several limitations. First, because the participants in the workshop are a self-selected group of registrars interested in research, the benefits suggested in our study may not be generalizable to less motivated groups. Some degree of responder bias is likely to be present despite high response rates, and our limited knowledge of the demographics of nonresponders prevented us from assessing whether responders were typical of all potential participants. It is possible that more research-motivated participants were more likely to have completed the questionnaire. Due to our retrospective study design and our questionnaire's reliance on self-reporting, recall bias might have influenced our study results. Also, several of the authors were known to participants from their involvement in past RRWs; this might have introduced a social desirability bias. Finally, changes in skill levels cannot be causally attributed to the workshop, due to the lack of a control group.  The RRW is an important foundation stone that will underpin future clinical research that can be used to provide better health outcomes in the Australian community.
The workshop is a much needed gateway to furthering interest and capacity in GP research and academic career options. Family Practice-an international journal Despite these limitations, our study is of importance. Since the inception of this annual workshop in 1994, this is the first systematic evaluation of the views of multiple cohorts of participants on the effectiveness and impact of this educational model in general practice research training. We included multiple cohorts in our study to increase sample size and minimize bias in relation to individual workshop characteristics such as group dynamics (between participating registrars, presenters and small group facilitators), with the purpose of increasing generalizability. Furthermore, the literature on research activities of registrars and early career GPs is sparse, and our study addresses this gap.
Our study established that the majority of surveyed clinicians were pursuing or planning to pursue their interest in research in general practice. Our 77 respondents reported a total of 31 peer-reviewed published articles. While our data do not allow exact calculation of yearly publication rates, this level of publication in our early career population seems to compare favourably to the wider general practice publication rate in the 1990s of one article per 1000 GPs per year. 3 In addition, research grant funding was received by a third of surveyed workshop participants.
Our results concur with previous findings regarding perceived barriers to research involvement such as lack of funding, supervision, opportunities and collaboration. 12 Some of these barriers might be addressed by opening research channels between academics and medical students or registrars, such as finding suitable academic mentors, involving students in established research projects and building in feedback loops on grant applications.
Our study is timely given the current Australian Government interest in development and evaluation of research capacity-building programmes in general practice and primary health care. This interest is exemplified by the Primary Health Care Research Evaluation Development Strategy, funded by the Australian Government Department of Health and Ageing since 2000, 13 which aims to nurture primary health care research.
Our findings imply that research-training initiatives such as this workshop can positively influence GP registrars' interest in and knowledge of research. At the same time, such initiatives provide a useful means of introducing registrars to a network of academics and GP clinician researchers. There is scope to harness these groups of research-active junior doctors to advance academic general practice.
Clinical researchers in general practice are much needed. How best to educate such clinicians and enable them to become active in research is a worthy subject for further research. Such research could include long-term follow-up of a group of clinicians, a suitable control group and ideally a randomized experimental design. While such evidence is awaited, the workshop we describe may serve as a useful model of an educational intervention for GPs or other health professionals.