Managing the production of a Cochrane systematic review

Background

Undertaking a Cochrane systematic review can be an incredibly rewarding experience. It is however a challenging and time-consuming task. The Cochrane Handbook for Systematic Reviews of Interventions¹ provides an essential resource to help reviewers navigate the often complex methodological issues of systematic review research. Additional guidelines have been developed for those undertaking reviews of public health topics,² and Cochrane Centres throughout the world offer invaluable training opportunities. This emphasis on training and methodological rigour has helped Cochrane reviews become one of the most respected sources of synthesized research available.

Even with the assistance available, however, many authors with good intentions register titles and prepare protocols but fail to publish the completed review. Data extracted from Cochrane's Information Management System (Archie) in June 2010 showed that there were 1,301 titles registered more than two years ago that have not been published as a full review.³ Of these registered titles, 697 have had protocols published (25 are no longer active) while 604 have not even progressed to this stage (154 are no longer active). There are also 146 protocols that have been published for more than two years without being converted into completed reviews. These registered titles and protocols that have not yet progressed to a completed review represent a significant amount of time and energy invested by review authors, Cochrane editorial staff and, in some cases, external referees.

While there may be a range of potential reasons for a title not to progress to a completed review, a recent study identified time and author communication as the major barriers hindering the completion of Cochrane reviews.⁴ To help overcome these obstacles we suggest looking to the tools and techniques of project management and have also highlighted some aspects specific to the review process which will contribute to enhanced efficiency, and the timely completion of these reviews.

Project management ‘is a discipline of planning, organizing and managing resources to bring about the successful completion of specific goals’.⁵ The purpose of project management is to foresee or predict risks, and to plan, organize, and manage activities so that projects are successfully completed in spite of the risks. There are various approaches to project management; however the critique of these is not the purpose of this paper. Nor is it our purpose to rewrite the processes of conducting a review. Rather we hope to highlight some tools available and to promote the consideration of project management techniques to improve review management, collaborator communication, stakeholder engagement and ultimately the prospects of review completion.

Using project management tools and resources in the review process

The process of producing a Cochrane review may be divided into four phases of project management: concept, planning and definition, implementation and finalization. The concept phase includes assembling the authorship team and registration of the proposed title with the appropriate Cochrane Review Group. The planning and definition phase of the review includes protocol development. From the protocol, the components and tasks for each stage of the review should be identified and clearly defined. The responsibilities for each task can be allocated to members of the author team. At this stage, a plan for training and resource assessment may be useful. A budget may be formulated which should include the time and resources each author can commit to the review process. It is also important to agree on subsequent publications, commitment to involvement in updating the review and the order of authorship during this phase. The implementation phase meshes with all aspects of review production as outlined in the published protocol. The finalization stage is the write-up and publication of the review itself, managing feedback from the published review and making plans for future updates.

Project management packages and software are available for purchase, although many tools can be downloaded free from the internet. In Table 1 we offer some suggestions of simple project management tools and templates that authors may find useful. Much of the information captured on these templates can be used when planning and conducting the review.

Table 1

Template and tools for project management

Template/Tools	Description
Task assignment template	Used to itemize tasks and allocate them among the review team
Project schedules and Gantt charts	Indicates what activities will be undertaken and when they will occur
Issues register	Documents issues that arise and the action required. These may include changes to the protocol and communications with editors
Process documentation	Describes how information and processes will flow (e.g. the process from searching to screening for included studies)
Learning registry	Documents the important learnings of a review. This may later be used in the discussion section of the review or even subsequent publications
Communications strategy	Describes how the team will communicate, including frequency and mode of conference calls, and the contact details of authors, editors and advisory group members. This is discussed further below.

Establishing effective communication strategies

Cochrane reviews have some specific challenges for which the application of project management techniques and tools may help to overcome. For example, reviews often involve authors from multiple settings, disciplines and countries, working in ‘virtual’ teams. It is common for author teams to include individuals who have never met in person, but have been introduced by the editorial team, or through the authors sourcing out experts in the field (through the internet or via professional mailing lists, for example). For the team to function, an enormous amount of good will, trust and commitment is required by all involved, along with the capacity to communicate effectively.

It is important to establish effective communication strategies at the onset of the project. It may be useful to develop a documented communications plan. This helps prepare for the challenges that having review authors in different time zones, distant locations and even different languages, brings.

While the tools of project management provide documentation that serve as valuable records of decisions, personal communication between collaborators is a necessary and essential part of any team. Email provides a quick means of communication, but has recognized limitations. Often it may be more effective to discuss issues verbally. The advent of resources of voice over internet protocol and video services, such as Skype video, offer cheap means of communication. Video call functions increase the interaction between users. Conference calls help set dates for work to be done and if managed effectively, can be used to build team comraderie. The annual Cochrane Colloquium also provides convenient meeting opportunities.

It is also useful if members of the team have access to the project's management resources and templates. File sharing software such as Lotus Quickr, supported by some universities, provides a place where project documentation, endnote files, included data extraction forms and quality assessment tools can be readily accessed by team members. Table 2 provides examples and links to some of the resources available. Additionally, Robin Good's Collaborative Map (www.mindmeister.com/12213323)⁶ identifies online resources for collaboration.

Table 2

Resources for facilitating good communication between team members

Resource	Examples
Meeting planning tools	www.agreeadate.com, www.timeanddate.com/worldclock/meeting.html
Voice and video communication	Skype, www.skype.com
	Google Talk, www.google.com/talk
Sharing files	Drop.io (free file sharing)
	www.dropbox.com (file sync/file sharing/online backup)
	www.5.sendthisfile.com
	Other: Lotus Quickr (replaces Lotus Quick Place)
Remote control of a PC to demonstrate or fix a problem	www.logmein.com, www.tightvnc.com, www.netviewer.com (also online conferencing)

Stakeholder engagement—management of a Review Advisory Group

In addition to tools that assist with time management and communication, it is worth considering aspects of the review process itself that are likely to enhance efficiency and streamline activities. Benefits of stakeholder engagement include increased relevance of review topic and methodology to potential end-users. An excellent way of involving stakeholders is to establish a Review Advisory Group (RAG) at the outset of protocol development. Members of a RAG can help author teams outline the parameters of their proposed review, alerting reviewers to potential methodological issues, assisting with prioritization of outcomes for analysis and refinement of inclusion criteria for studies and populations. Careful consideration of such issues will help save time over the long term, particularly for reviews of complex interventions, such as those in public health. RAGs should comprise members from different parts of the world to ensure the review has relevance globally and could include a combination of consumers, content experts, policy-makers and practitioners. To ensure effective management and utilization of a RAG, it is recommended to formalize the process at the outset, considering aspects such as roles and responsibilities, anticipated workload, method of communication, frequency of consultation and timelines and ensuring agreement of these with the authors and RAG members. Communication strategies as mentioned previously can apply when managing information sharing and feedback amongst the RAG members and the authorship team.

Conclusion

The preparation of a Cochrane review is a complex and time-consuming process, and thus the need for initial planning by authors is imperative. Project management principles provide a good basis for both planning and managing the production of the review. Well-considered planning, management, effective communication and stakeholder engagement has the potential to improve timeliness and completion rates of reviews, as well as enhancing the review experience of the author team.

Funding

The Cochrane Public Health Review Group acknowledges the support of the Victorian Health Promotion Foundation (VicHealth) and Queensland Health who provided the authors time to undertake Cochrane Collaboration activities. No endorsement is given to the resources identified in this paper.