Development and impact of a massive open online course (MOOC) for antimicrobial stewardship

Abstract

Background

The University of Dundee and the BSAC developed a massive open online course (MOOC) to address the global need for education to support antimicrobial stewardship in low- and middle-income countries.

Methods

An interactive course, Antimicrobial Stewardship: Managing Antibiotic Resistance, was developed and delivered via the FutureLearn^© platform. The course ran over four 6 week periods during 2015 and 2016 supported by educators and was evaluated via data on uptake and feedback from learners on impact on clinical practice.

Results

In total, 32 944 people, 70% of them healthcare professionals, from 163 countries joined the course from Europe (49%), Asia (16%), Africa (13%), North America (9%), Australia (8%) and South America (5%). Between 33% and 37% of joiners in each run completed at least one step in any week of the course and 219 participants responded to a post-course survey. The course was rated good or excellent by 208 (95%) of the participants, and 83 (38%) intended to implement stewardship interventions in their own setting. A follow-up survey 6 months later suggested that 49% had implemented such interventions.

Conclusions

The MOOC has addressed a global learning need by providing education free at the point of access, and learning from its development will help others embarking upon similar educational solutions. Initial quantitative and qualitative feedback suggests it has engaged participants and complements traditional educational methods. Measuring its real impact on clinical practice remains a challenge. The FutureLearn^© platform offers flexibility for MOOCs to be sustainable through modification to remove educator facilitation but maintain active participant discussion.

Introduction

The WHO has warned that without urgent action, we will return to an era where common infections will kill once more due to antimicrobial resistance (AMR). Professional education and training is one of seven key areas for action within the UK Five Year Antimicrobial Resistance Strategy 2013 to 2018.¹ The recently updated Cochrane review of interventions to improve antibiotic prescribing practices for hospital inpatients² confirmed that education interventions had a positive effect on patient outcomes. However, although education is a key element of antimicrobial stewardship (AMS), and specific competencies³ have been developed, evidence suggests the knowledge of medical students and practising clinicians about antimicrobial prescribing is suboptimal.^4–7

A key issue is the lack of standardization of AMS content in medical undergraduate training. Education of other clinical staff such as pharmacists and nurses involved in stewardship is also critical.^8–10 While educational programmes are available at national and local level in the USA,¹¹ Australia¹² and many European countries,¹³ this is not the case in many low- and middle-income countries where AMR is more prevalent. However, the global success¹⁴ of the generic WHO Guide to Good Prescribing¹⁵ provides a source of good practice.

Traditional education models of face-to-face teaching or structured programmes using printed media cannot meet the learning needs of healthcare professionals working across a diverse range of healthcare settings. The use of online learning has developed over the past decade and offers a flexible and cost-efficient method to provide education to large numbers of learners.¹⁶ This includes global web-based platforms offering massive open online courses (MOOCs), generally offered free of charge with content developed and funded through collaborative working with academic institutes and professional and commercial organizations. Academic experts on the topic area are commissioned to develop content supported by pedagogic and technical staff. MOOCs have many advantages, such as global reach, cost-effectiveness and appeal to individuals with a range of learning styles, as well as some potential disadvantages, such as minimal student–teacher interaction, which may make learners feel isolated and discourage course completion.¹⁷

AMS requires an urgent multinational, multidiscipline one-health approach and therefore is an ideal topic for a MOOC allowing experts and participants to interactively share and disseminate their knowledge and experience with colleagues working in all economies, including low- and middle-income countries. Stewardship leaders have stressed the demand and need for open access education resources for front-line clinicians in resource-limited settings¹⁸ and the BSAC is working to address this need through development of a variety of resources,¹⁹ including the Antimicrobial Stewardship: Managing Antibiotic Resistance MOOC.²⁰ This resource, hosted by FutureLearn^©, was developed as a joint venture by the University of Dundee and the BSAC with the objective of providing high-quality accessible education on AMS for healthcare teams across the globe.

In this article we aim: (i) to describe the challenges of course development and delivery; (ii) to present the initial quantitative and qualitative evaluation we have received for the first four facilitated runs of the MOOC; and (iii) to discuss the value and potential impact of the MOOC on global AMS.

Methods

Course development

The concept for the course was developed by the educational organizations (University of Dundee and BSAC) who applied to host the MOOC on the FutureLearn^© platform. Development of the MOOC was funded through the establishment of a consortia stakeholder group that supported a project team based at the University of Dundee. The topic areas for each week and the overall framework of the MOOC were proposed by the course lead and the goal was to provide a comprehensive programme covering AMR and AMS including practical applications that would be suitable for a multiprofessional audience. The ethos of the content and structure was to engage practitioners with little or no experience in stewardship with a particular focus around implementation, behaviour changes and learning from global case studies as a foundation for seeking more advanced learning. The course educational faculty were known professionally to the course lead (D. N.) and were infection specialists, microbiologists, pharmacologists and pharmacists from the UK, Spain, South Africa, India and Switzerland. All had experience of education development and delivery within their field and could utilize educational content they had on file to develop material for the MOOC. In addition, all tutors used current published literature and their personal clinical practice to inform content using novel formats to encourage learner interaction.

The development process included regular meetings of the course lead, educators and a central project team via e-mail and conference calls. Content was developed iteratively, starting with a clinical scenario relating to hospital prescribing practice and where possible relating each week back to this scenario. Each week’s content was agreed by consensus initially by the two topic educators, then with the rest of the faculty to ensure consistency of educational quality and message, avoid unnecessary repetition and maintain agreed timelines. This approach was intended to lead the learner through an introduction to AMR and AMS and provide the necessary basic knowledge ‘scaffolding’ for implementing AMS. Further learning focused on measurement, education, behaviour change and problem solving, all related to clinical practice to allow learners to build upon and apply the basic knowledge.

The course was promoted by the BSAC to its membership, through allied healthcare organizations and globally via engagement with the British High Commission Foreign and Commonwealth Offices, primarily in South Africa, India and Russia, and national professional organizations. Consortia sponsors promoted launch campaigns, provided by the BSAC, to their global company and customer networks. The University of Dundee contacted alumni and current undergraduate and postgraduate students, and used its intranet to promote the course. In addition, FutureLearn^© promoted the course to its extensive network of learners and the course was also promoted via social media.

Course delivery

The MOOC was launched in September 2015 as a live, facilitated course with an anticipated 3 h of study per week for 6 weeks and was accessed via a website with a secure log-in facility allowing learners to work at their own pace and track their progress. Learners had unlimited access to the course content after the 6 week period.

The course opens with a ‘setting-the-scene’ video of a fictitious clinical scenario set in a hospital in Scotland where two patients have died of an antibiotic-resistant infection following routine urological surgery (https://vimeo.com/211659768—Part 1, https://vimeo.com/211663005—Part 2).

Each week of the 6 week course covered different areas (Table 1), facilitated by a different pair of educators with weekly content presented as a series of short steps utilizing a range of educational formats and techniques: articles to read, videos to watch, short audio pieces, case studies, discussion steps, quizzes, and submission of written pieces (Figure 1) for peer review.

Table 1.

Course weeks, topics and content

Week of course	Topic	Content
1	Welcome to the course	Introduction to the topic Introductions to the educators Glossary of key terms The fictitious video sets the scene Resistance—burden, genetic basis Prescribing and resistance
2	Antimicrobial stewardship	What is antimicrobial stewardship? How and where to start Who should be involved Strategies for implementation Effective strategies to try
3	Measurement	Why measure antibiotic use? How and what to measure Measuring the quality of antibiotic use Feedback of results
4	Emerging strategies for antimicrobial stewardship	Seizing the opportunity to diagnose the cause of infection Novel diagnostic tests and their implementation Biomarkers The role of computers Pharmacokinetics and pharmacodynamics
5	Behaviour change	Whose behaviour needs to change? Culture and context and team dynamics Behaviour change techniques The impact of power distance and uncertainty avoidance
6	Quick wins from across the globe	A quick-win strategy from South Africa: Implementing a model for stewardship without infectious diseases resources and measuring the impact A quick-win strategy from India: Use of technology in antibiotic guidelines

Figure 1.

Assignment and guidelines from week 4 of the course.

Participants were set tasks in order to reflect on practice in their own clinical setting or to develop ideas for discussion with educators and other participants via an online forum. This forum could also be used to ask questions of the educators, leave comments for discussion with other learners, or provide acknowledgements and feedback about the content. The relevant educators joined the MOOC site daily to answer queries and contribute to the online participant discussion forums supported by the central project team. Most of the educators adopted a thematic approach to respond to questions as opposed to each single question specifically. Social media and the internet were also utilized to host Twitter chats and Google hangouts and provide learners with additional live, interactive educational opportunities, which enabled educators to highlight key issues and comments posted by learners (https://www.youtube.com/channel/UC1-dFZXTczge-4gS2ux2mSg).

Following the first run of the MOOC the content was reviewed, based on feedback from participants, and the experiences of educators and the project team, to address issues relating to learner understanding, level of difficulty, usefulness and time taken to complete each week’s programme. A further three facilitated courses ran in February, May and September 2016 and required minimal revision of content. The course was initially provided in English, but has been translated to unfacilitated versions in English, Simplified Chinese, Spanish (Latin American) and Russian. Participants could purchase a course completion certificate as proof of learning. There was a Certificate of Achievement for completing at least 90% of the steps in the course, or a Statement of Participation for completing at least 50% of the steps.

Course evaluation

Each run of the MOOC was evaluated via a post-course, online feedback survey circulated to all participants by FutureLearn^©; this included a question about what learners intended to implement in their own setting. In addition, participants could post feedback each week via the online discussion board. A second survey about implementation of learning was circulated 6 months after course completion to assess participants’ perceptions of the impact of the course on stewardship activities and clinical practice in their own setting. Only data from runs 1 and 2 of the course were available at the time of writing.

Results

Course development

The content for each week of the course and details of the topics covered during each of the 6 weeks are shown in Table 1. Topics included were chosen by the course lead to provide multiprofessional clinical teams with an understanding of AMR and give them practical tools to develop stewardship. Planning the course took ∼6 months. Development of content took a further 6 months with an additional 2 months to collate, check and test the online resources prior to launch. Key challenges were ensuring that content met the requirements for a MOOC using a blended learning approach to keep the course interesting and ensuring each week was broken up into small, distinct steps. Educators overcame these challenges through regular discussions with the project team, and as development progressed they became familiar with the structure and format required. Minor changes were made following feedback received after the first run of the MOOC and included removal of some calculations and Google hangout sessions and rearranging of steps to improve flow.

Course uptake

A total of 32 944 people from 163 countries registered, defined as ‘joiners’, for the first four runs of the course with 16 143 (49%) registrants from Europe, 5271 (16%) from Asia, 4283 (13%) from Africa, 2965 (9%) from North America, 2635 (8%) from Australia and 1647 (5%) from South America. This includes course educators and administrators and the majority of these ‘joiners’ (23 061, 70%) were healthcare professionals. Details of the specific professions and demographics of the remaining 9883 (30%) of ‘joiners’ were not captured via the registration process. For runs 1–4, there were 16 697 ‘learners’, defined as those who registered and viewed at least one step in any week of the course, and 16 247 ‘joiners’ who did not access the course content; 5036 (30%) of these ‘learners’ did not progress to completing any course elements and the remaining 11 661 (70%) were defined as ‘active learners’ (33%–37% of joiners for each run) by completing at least one step in each week of the course. In terms of sustained learning, there were a total of 3027 ‘fully participating’ learners who completed all steps of each of the 6 weeks (26% of active learners). Sustained interaction with the course was demonstrated by 1043 learners who submitted week 4 assignments and 663 learners who submitted week 5 assignments. Details for each of the four runs are shown in Table 2.

Table 2.

Level of participation in MOOC for runs 1–4

Criteria	Number run 1	% run 1	Number run 2	% run 2	Number run 3	% run 3	Number run 4	% run 4
Joiners	15 571		6407		5585		5381
Active learners	5496	35% of joiners	2380	37% of joiners	1848	33% of joiners	1937	36% of joiners
Fully participating learners	1506	27.4% of active learners	628	26.38% of active learners	427	23% of active learners	466	24% of active learners
Week 4 assignments completed	419	7.6% of active learners; 28% of fully participating learners	258	10.8% of active learners; 41% of fully participating learners	162	9% of active learners; 38% of fully participating learners	204	10.5% of active learners; 43.7% of fully participating learners
Week 5 assignments completed	260	4.7% of active learners; 17.2% of fully participating learners	162	6.8% of active learners; 25.8% of fully participating learners	102	5.5% of active learners; 23.8% of fully participating learners	139	7% of active learners; 29.8% of fully participating learners.

Course evaluation—participant feedback

Only 219 participants (1.9% of active learners; 0.7% of joiners) from runs 1–4 of the MOOC responded to the FutureLearn^© post-course survey. Feedback and comments were largely positive as shown in Figure 2. There were a few negative comments in the post-course survey: the peer-reviewed 500 word assignments in weeks 4 and 5 were not popular with everyone, and some found the concepts of DDDs and biomarkers difficult to grasp. In addition to the survey, there were also a range of comments posted on the online forum during each run of the course; 44 134 comments from both learners and educators were posted throughout the four runs of the course.

Figure 2.

Respondents’ feedback on course (n = 219).

Course evaluation—impact on practice

An implementation survey performed 6 months after completion of the first two courses was completed by 409 participants (1.2% of joiners, 3.5% of active learners) from 41 countries. Of these respondents, 325 (79%) were healthcare workers (including 37 pharmacists, 34 clinicians, 28 nurses, 18 microbiologists and 43 ‘other’ professionals) and 160 (49%) of them reported that they had implemented stewardship interventions since completing the course. Demographic details of the remaining 84 (21%) respondents who were not healthcare workers were not captured. The reported interventions that they had implemented correlated well with those reported in the post-course ‘intention to implement’ survey, but a limitation was that those who stated an intention were not necessarily those who responded to the implementation survey.

Discussion

Findings

The MOOC contained a mix of basic principles of good prescribing, principles of the structures and processes required for good stewardship, and practical advice on implementation. This was based on the experience of educators actively involved in stewardship at local, national and international levels, and provided opportunities for participants to discuss with these educators and other learners the challenges they had faced and ideas to improve stewardship relevant to their own setting. In order to encourage staff not already involved in stewardship and those early in their careers, along with those experienced in stewardship, to participate, the content was developed for the level of experience of a final-year medical student/first-year post-qualification doctor. Educators recruited to support each week were from a variety of clinical backgrounds, nationalities and environments to ensure that content was accessible and relevant to a global multiprofessional audience. All of the educators are specialists in their field and had demanding clinical and other roles to maintain, and so adherence to the course development timelines was, at times, challenging. An ongoing commitment from all educators was also essential during each run of the course to ensure that learners were supported during each week, any arising questions were answered, and educators actively participated in discussion forums. Although the MOOC is hosted on an innovative, interactive learning platform, content development was undertaken using mainly low-technology methods with educators submitting Word and PDF documents, PowerPoint presentations with audio files and short video clips generated using smartphones. This approach ensured that educators required minimal technical support and production costs were low. The most resource-consuming component of the material was the development and filming of the fictitious outbreak scenario.

During development and delivery of the course, we discovered several important things to consider when developing future education programmes of this type. It is important not to underestimate the time required to develop a course such as this, especially in the early stages when many aspects of course design, and the host platform, are unfamiliar to the project team. The high quality of production standards required by FutureLearn^© also impacted on the time required to develop and upload content, but were vital to ensure that the course was designed and produced to a standard that attracted and kept learners engaged.

After the first run of the course a few minor amendments were made to content, but the course required minimum updating. This level of editorial maintenance will continue to ensure the course remains fit for purpose, but future runs will be non-facilitated as it was recognized during the early delivery of the course that ongoing educator interaction was not sustainable. However, social interactions between learners do provide peer support and educational opportunities as observed throughout facilitated runs of the MOOC and this will continue to be available. We have learned that learners have a wide range of previous knowledge and exposure to AMS, and some participants are in a position to educate others through sharing of their own views and experiences. As the aim of the course was not only to encourage participants to completion, but also to empower them to undertake AMS in their own location, quantitative and qualitative feedback was a key indicator of success. The presented quantitative data suggest the MOOC was successful in terms of the overall high number of registrations and active learners, the global reach and the multidisciplinary nature of professional roles of participants. This scope and range of participants could only have been achieved using a globally available, free-to-access online course. The number of learners (those who complete at least one step in any week) was about half of those who registered, which is the average level of engagement for FutureLearn^© courses. The number of active learners (completing at least one step in each of the 6 weeks) relative to the number of learners (those who view at least one step in any week) was 70%, which is slightly below the average of 81% for all FutureLearn^© courses, but may reflect the highly specialized academic content of this course.

The ultimate aim of the MOOC was to improve the knowledge and skills of participants to facilitate engagement with their colleagues, in order to improve prescribing within clinical teams. Success in achieving this is difficult to measure. Qualitative feedback from the post-course survey suggested that some participants were planning to use the discussed strategies and felt empowered to translate their new knowledge and skills into action. Results from the subsequent implementation survey support these findings. Measurement of antibiotic use, both quantitatively and qualitatively, were the most common anticipated/implemented post-course interventions, suggesting the course provided useful practical advice in this area, including for low- and middle-income environments. On occasions, the posts of some learners during courses suggested an immediate impact on antimicrobial prescribing and stewardship within their own organization. However, the number of respondents to both the post-course feedback and implementation surveys was very small compared with the number of active learners. Despite the low level of feedback, our data suggested the course was well designed, of the correct duration, and highlighted that week 5 (behaviour change) was perceived to be the most valuable for clinical practice. This feedback supports the importance of behaviour change evidence to support clinical practice as highlighted by a recent publication.²¹

Limitations

The level of response to both the qualitative online feedback and the survey on learner experience was low and this is a major limitation in demonstrating the impact of the MOOC. The evaluation was constrained by the feedback process within the FutureLearn^© system and it was not possible to make feedback mandatory. A key learning point was that when developing an online education resource it is essential to scope what the delivery system can support in the way of data on user experience. It is acknowledged that voluntary user feedback surveys are likely to have a low response rate so are an unreliable method for demonstrating impact. To confirm impact, further surveys or other means of engagement with a larger number of learners are required to assess how participants’ stewardship plans for intervention have progressed. If technically feasible it would also be helpful to embed requests for feedback within the course content, providing opportunities for participants to provide ‘real-time’ information as the course progresses. It may also be useful to revisit the global survey of stewardship programmes that was performed in 2015²² to assess progress with implementing stewardship as a means of evaluating the impact of the MOOC along with other stewardship-related educational initiatives by various organizations.

There was low uptake of the Certificate of Achievement and Statement of Participation, partly due to the low number of learners who completed all steps and also perhaps as these required payment of a fee, which may not be feasible in resource-limited settings. Accreditation of the course by relevant professional bodies may increase participation rates and completion of all elements by rewarding learners with a recognized qualification without the need to pay a fee.

Only having an English version of the MOOC limited access for some potential learners. The unfacilitated English and Simplified Chinese versions were launched in 2017 and the Spanish (Latin American) and Russian versions are planned for release in 2018, and the BSAC has also commissioned several short complementary modules on specific aspects of stewardship (e.g. point prevalence surveys) to facilitate more in-depth practical education to help learners progress with their stewardship ambitions.

Conclusions

We have demonstrated the feasibility and potential of a MOOC to fulfil a global learning need with the key advantages of providing free education and virtual group learning while acknowledging the challenges for implementation in resource-poor settings. The initial quantitative and qualitative feedback suggests the course was successful in engaging participants and supporting their practice where relevant, but whether it changes practice is uncertain and in need of further research. The MOOC contributes to a recognized wealth of available training, but ensuring these are sustainable and used in a way that ensures accessibility to all, improves knowledge and skills, changes clinical practice and, ultimately, reduces AMR remains a major challenge for the future.

Acknowledgements

We thank the other course educators for their input to development and delivery of the resources: Professor Neil Woodford, Lead for Applied Molecular Bacteriology, Public Health England and Imperial College London, UK; Dr Michael Cooper, Consultant Microbiologist, Director of Infection Prevention and Control, Newcross Hospital, Wolverhampton, UK; Dr David Jenkins, Consultant Medical Microbiologist and Infection Prevention and Control Doctor, University Hospitals of Leicester NHS Trust, UK; Mr Mark Gilchrist, Consultant Pharmacist in Infectious Diseases, Imperial College Healthcare NHS Trust, London, UK; Mr William Malcolm, Pharmaceutical Advisor, NHS National Services Scotland, UK; Dr Benedikt Huttner, Infectious Diseases Physician, Geneva University Hospitals, Switzerland; Dr Jose Ramon Pano Pardo, Infectious Diseases Specialist, Hospital Clínico Universitario Zaragoza, Spain; Professor Peter Davey, Infectious Diseases Specialist and Quality Improvement lead, University of Dundee, UK; and Ms Esmita Charani, Research Pharmacist, Imperial College, London, UK.

 We also thank members of the development team from the University of Dundee and FutureLearn: Ms Natalie Lafferty, Lecturer in e-learning, University of Dundee; Mr Jonny Luckett, IT design and production, University of Dundee; Mr Lester Milbank, Cambridge Film and Television Production Ltd; Mr Richard Banks, Senior Content Producer, FutureLearn; and Ms Shelaine-Fraser-Robertson, IT design and production, University of Dundee.

Funding

This work was hosted on the FutureLearn^© platform and was supported by the British Society for Antimicrobial Chemotherapy (BSAC) through donated time and unrestricted educational grants secured by the BSAC under a consortia funding arrangement contributed to by five commercial stakeholder partners (Alere, Bayer, Pfizer, MSD and Pharma AG).

Transparency declarations

None to declare.