The mandate of Human Reproduction Update involves several roles: (i) to provide a synthesis of evidence that can aid scientists and clinicians in their daily work; (ii) to help reproductive specialists understand concepts from related disciplines; and (iii) to summarize current knowledge generated by basic science as the foundation of future scientific and clinical advancement. Given that review and synthesis are central to good scientific and clinical practice, and that a grasp of the current state of knowledge is a prerequisite to designing new studies, it is pertinent to ask which reviews are most likely to fulfil the needs of readers. A related question concerns whether systematic reviews meet the needs of all review topics and all readers.
Summarizing evidence or knowledge is a difficult problem in reproductive medicine, as in other branches of science and medical care (Eddy et al., 1992). For each question there may be multiple studies that use different designs and inclusion criteria. For clinical questions, the interventions, outcomes and measures of effect may vary: the effect measures in treatment studies include odds ratios, relative risks and absolute differences. For scientific questions, the experimental species, models and designs may differ. Moreover, it is always uncertain whether all of the relevant evidence has been evaluated. Even when the search has been exhaustive, there are no simple guides on how to interpret conflicting results and whether to accept apparently outlying studies. The choices that the reviewer makes to address the variable conditions and uncertainties may be conservative, strict and exclusive, or liberal, open and inclusive. The decisions made by the reviewer may not be consistent throughout and these choices may or may not satisfy the reader who seeks out the review to address a clinical or research question. Faced with uncertainty and doubt, readers nonetheless must form an impression of the evidence and synthesize the state of knowledge in order to address the clinical or research question that stimulated their interest in the review. We argue that the reader is better served when the choices made in the review, regardless of whether they are strict or open, should be explicit, transparent, clearly stated and reproducible by interested readers.
This list of objectives for reviews is more easily satisfied by systematic reviews, which use explicit methods to methodically search, critically appraise and synthesize the available literature on a specific issue. The question or issue need not be clinical: indeed, the concept evolved primarily in psychology studies (Light and Pillemer, 1984). The systematic review attempts to reduce reviewer bias through the use of objective, reproducible criteria to select relevant individual publications and assess their validity. A systematic review may include a meta-analysis or statistical summary of the individual study results: the aggregate of effects from several studies yields an average treatment effect that is more precise than the individual study results (Schlesselman and Collins, 2003). Thus, the systematic review involves explicit, transparent methods which are clearly stated, and reproducible by others. Whether a systematic review of randomized controlled trials adheres to the guidelines can easily be evaluated by means of a widely used checklist (the QUORUM statement) (Moher et al., 1999). The strengths of the systematic review include the narrow focus of the question, the comprehensive search for evidence, the criterion-based selection of relevant evidence, the rigorous appraisal of validity, the objective or quantitative summary, and the evidence-based inferences (Cook et al., 1997).
For some review topics, however, the strengths of the systematic review may turn into weaknesses. The primary problem is that the narrow focus and prescribed methods of the systematic review do not allow for comprehensive coverage. For example, the historical review is an irreplaceable means of tracing the development of a scientific principle or clinical concept, but the narrative thread could be lost in the strict rules of systematic review. As other examples, it would be burdensome to apply systematic methods to a survey on aneuploidy and fertility in the aging female or to an assessment of mouse knockout models and polycystic ovarian phenotype. Such topics would require the wider scope of a traditional narrative review, in which less explicit methods are the trade-off for broader coverage.
The majority of review articles are narrative rather than systematic. Narrative reviews generally are comprehensive and cover a wide range of issues within a given topic, but they do not necessarily state or follow rules about the search for evidence. Also, typical narrative reviews do not reveal how the decisions were made about relevance of studies and the validity of the included studies. Of course, the results of the search, selection and assessment procedures must meet the referees’ and editors’ sense of propriety, but readers may not be privy to the methods and thus could not make judgments about the authors choices.
Neither the systematic reviews with their narrow scope nor the narrative reviews with their individuality can satisfy the range of topics for review. Currently, progress in reproductive medicine depends primarily on knowledge of developments in molecular biology, genetics and pharmacology. Background knowledge, evolving concepts and controversy require the flexibility of a narrative review with broad coverage and situational choices about the inclusion of evidence. In contrast, the rigour of a systematic review is needed for effectiveness of diagnostic and treatment interventions and for the outcomes of natural and therapeutic exposures, including adverse events and costs. The choice is more open for many other scientific and clinical topics.
Recognizing that there is a need for both systematic and narrative reviews, could one review type learn from the other? Because readers value transparency and reproducibility, some narrative reviews could gain by drawing from the rigour of systematic reviews. Authors could arrange the subject matter in a series of objective questions, each section based on specified procedures for search, relevance and validity and tied to other sections by appropriate descriptive links. One of the many types of statistical summarization would be helpful to readers. Inferences would adhere to the cited evidence and abstain from opinion. Systematic reviews, on the other hand, could adopt some of the strengths of the narrative review without compromising validity. Their formulaic nature can be boring to read, but this could be countered by non-technical idiomatic language, novel approaches to graphics, and new ways to deal with the baggage of massive tables. Also the excessive concentration in systematic reviews on odds ratios and relative risks is anachronistic, now that absolute differences and numbers needed to treat are the preferred measures of treatment effects (Sackett and Cook, 1994). The procedures for calculating summary absolute effects and their heterogeneity are similar to those for relative effects (Greenland, 1987; Deeks et al., 2001).
Review journals such as Human Reproduction Update have high impact factors because readers need and appreciate comprehensive, relevant, valid summaries that clearly synthesize scientific and clinical evidence. While systematic reviews are more appropriate for focused topics and traditional narrative reviews are better suited to comprehensive topics, either approach can be adapted to clinical or scientific subjects. An infusion of systematic review methods would strengthen narrative reviews and in turn systematic reviews could benefit from the presentation strengths of narrative reviews. The goal is to ensure that the methods of all reviews should be explicit, transparent, clearly stated and reproducible by interested readers.
1Department of Obstetrics and Gynecology, McMaster University, Hamilton, Ontario, 2Department of Obstetrics and Gynaecology, Dalhousie University, Halifax, Nova Scotia, Canada and 3Department of Reproductive Medicine, University Medical Center, Utrecht, The Netherlands