Keywords

It is the habitual carriage of the umbrella that is the stamp of Respectability. The umbrella has become the acknowledged index of social position…. Crusoe was rather a moralist than a pietist, and his leaf-umbrella is as fine an example of the civilized mind striving to express itself under adverse circumstances as we have ever met with. Robert Louis Stevenson (1851–1894)2[1]

What is evidence? Evidence can be defined as the body of facts and information available on a specific belief [2, 3]. And what is synthesis? Synthesis can be regarded as the combination of different entities to form a coherent system.

Not unexpectedly, the pursue of accurate, precise, and efficient evidence synthesis remains rather challenging. Thanks to the increased participation of multiple investigators and stakeholders, sources of evidence in clinical medicine as well as in other fields of human endeavor will continue to increase, possibly exponentially [4]. Thus, the only sensible means to navigate such information overload are flexible yet powerful tools for evidence synthesis [5, 6].

Evidence is also hierarchical (Table 1.1). From preclinical studies to primary studies (e.g., randomized controlled trials) and secondary studies (e.g., systematic reviews, pairwise meta-analyses, and network meta-analyses), there is a continuum of different study designs, yielding altogether different results, in terms of internal and external validity [3, 15]. Indeed, major developments have occurred in the field of secondary research, and in particular the introduction and now rather common application of network meta-analytic techniques have enabled powerful, robust, and elegant synthesis of apparently incoherent sets of evidence [16]. Further refinements of these approaches are expected, including multivariate meta-analytic studies, capable of providing insights on the comparative safety and efficacy of different interventions on different domains of a given condition or even on different conditions as well.

Table 1.1 Galaxy of research designs, distinguishing three main levels of research, with corresponding study designs and features [2, 3, 7–14]
Full size table

Nonetheless, we cannot consider the exponential accrual of secondary research studies (e.g., systematic reviews and meta-analyses) as solely positive. It is already very difficult to remain up to date given that so many reviews are being published on a daily basis (for instance, on average at least four network meta-analyses were being published daily during the months leading to the completion of this book in the summer of 2015). Moreover, it is not uncommon to find different meta-analyses focusing on similar topics and providing potentially different results. Finally, no systematic review per se is usually capable of providing a comprehensive yet succinct perspective on complex conditions or problems.

The idea of looking at reviews as objects of research rather than solely as a scholarly product is not new and was pioneered in the 1980 and 1990s by several leaders in evidence synthesis, such as Andrew Oxman and Gordon Guyatt, among many others [2, 17]. The success of the Cochrane Collaboration and its leadership worldwide [7, 18, 19], as well as the commitment of other leading institutions such as the Joanna Briggs Institute [8], have further supported the development of a new set of tools for evidence synthesis, operating at a higher level than systematic reviews and meta-analyses, which editors, reviewers, and readers already seem to enjoy quite remarkably [7, 8, 20, 21]. Moreover, the flexibility of this type of research design is substantial, as animal experiments and observational studies can also be included and findings may be combined formally with multivariate modeling [22–24].

The concept of this book stemmed from the successful collaborative effort we have conducted in 2013–2014 in producing the first textbook solely dedicated to network meta-analysis [3] and to the interest we have had for more than 15 years in looking at systematic reviews and meta-analyses as uniquely elegant and interesting tools, worthy of study, comparison, and synthesis. Accordingly, we have planned a comprehensive textbook, the first uniquely dedicated to umbrella reviews, overviews of reviews, and meta-epidemiologic studies. As our common ultimate goal remains evidence synthesis, this book should best be appraised together with our own opus and similar ones on mixed treatment comparisons, as there is a substantial continuum and overlap between these apparently different research designs [3, 9, 22].

First and foremost, why should we use three different identifying terms in a book which focuses mainly on overviews of systematic reviews [10, 25, 26]? Actually, umbrella reviews, overviews of reviews, and meta-epidemiologic studies share much in common, but cannot be considered perfect synonyms (Fig. 1.1). Indeed, they are different types of tertiary research, i.e., research mainly using as study objects systematic reviews.Footnote 1 Yet, umbrella reviews can be operatively considered exercises in evidence synthesis focusing on a specific clinical topic or condition, and including mainly systematic reviews, but with the possible inclusion of primary studies outside any prior meta-analysis [23, 28]. Overviews of systematic reviews also focus explicitly on a clinical topic or condition, but should not typically include primary studies or other non-review studies outside the realms of prior systematic reviews [29]. Finally, meta-epidemiologic studies usually disregard the goal of providing practical guidance on a specific clinical condition or topic, but usually include secondary research studies as well as primary research ones [30]. At one theoretical extreme, a meta-epidemiologic study could, for instance, include only editorials and thus disregard altogether primary or secondary research studies. Thus, while the goals or the analysis sets may differ substantially, these three types of studies typically do share more than what they do not, especially in terms of scope and methods. More pragmatically, the premises, the tools used, the skills required, and the final products are similar enough for those interested in critically reading or proficiently conducting them to justify, in our humble opinion, a common playground for their scholarly presentation [9, 31].

Fig. 1.1
figure 1

Venn diagram showcasing the overlap between umbrella reviews, overviews of reviews, and meta-epidemiologic studies. For instance, umbrella reviews and meta-epidemiologic studies may both include primary studies (e.g., randomized controlled trials) not included in any systematic review, whereas umbrella reviews and overviews of reviews typically focus on a specific clinical topic, at odds with meta-epidemiologic studies

Full size image

More explicitly, what is our purpose with the compilation of this multiauthored textbook? This book aims at providing readers a practical opportunity for comprehensive, effective, and efficient evidence synthesis, through an explicit structure divided in four main sections. The first one highlights in different chapters the peculiarities of umbrella reviews, overviews of reviews, and meta-epidemiologic studies and their rightful place in the modern hierarchy of evidence. The second section contains chapters which guide the reader through the process of designing, registering, and conducting with the utmost validity and transparency an umbrella review, an overview of reviews, or a meta-epidemiologic study [11, 32]. In particular, we provide explicit details on searching, abstracting, and appraising evidence and then perform statistical analysis and appraisal of homogeneity, small study effects, moderators and confounders, as well as state-of-the-art reporting. In the third section the reader will find several authoritative case studies of tertiary research, which highlight the strengths as well as weaknesses of this type of research endeavor. Finally, the fourth and final section suggests how to move from the process of synthesizing evidence to actually acting upon it and which future areas of research and development for this field can be envisioned today. Indeed, tertiary research simply represents one of the steps in the life cycle of evidence, with a persistent continuum between the different levels of research, each informing on the following as well as the previous ones (Fig. 1.2) [5]. Accordingly, only in putting umbrella reviews in the larger context of evidence accrual can we righteously use them.

Fig. 1.2
figure 2

The evidence mill, highlighting how unmet needs and novel hypotheses fuel the conduct of preclinical studies and primary clinical research explicitly guiding decision-making. Such research products are then the object of systematic reviews and umbrella reviews, eventually informing on needs and hypotheses, as well as influencing further primary studies and decision-making. Meta-epidemiologic studies simultaneously offer an alternative way to appraise the complex relationships between these types of research designs and the potential weaknesses in the evidence base, thus also, albeit indirectly, guiding decision-making

Full size image

Whereas we do not wish nor need to defend reviews in general in this introductory chapter of the book [33], we would like to make the case that umbrella reviews, overviews of reviews, and meta-epidemiologic studies do close the circle of evidence, potentially reconciling all sources of evidence, even those of lower quality or focusing on less important issues or interventions [16, 34]. Indeed, a historical critique of systematic reviews and meta-analyses is that they can easily end up mixing apples and oranges, especially if pooling heterogeneous trials [2]. Even if we concede this, looking comparatively at apples and oranges will often tell us a great deal about fruit in general. Umbrella reviews, overviews of reviews, and meta-epidemiologic studies enable us to make another step in generalization, such that we could now, metaphorically, focus on food in general, rather than on fruit only, in a strenuous yet constructive effort against reductionism.

On the other hand, on a cautionary metaphorical note, an umbrella is a useful tool, but only if it rains. In addition, it may help staying dry, but cannot save from drowning in case of a flood. Accordingly, an umbrella review, an overview of reviews, or a meta-epidemiologic study including few or faulty primary or secondary studies will most likely have a hard time providing credible and useful conclusions. In addition, while our work aimed to be comprehensive, we have not focused on other types of review, such as rapid reviews or scoping reviews, which are well and poignantly discussed elsewhere (Table 1.1) [12–14, 35, 36]. Moreover, we recommend our readers to also diligently study the Cochrane Handbook for Systematic Reviews of Interventions and the Joanna Briggs Institute Reviewers’ Manual, which both provide very useful and sound guidance on how to best conduct an overview of reviews and an umbrella review [7, 8]. Other very important resources, albeit mainly focusing on secondary level research, are the Standards for Systematic Reviews issued by the US Institute of Medicine (IOM), the Methods Guide for Effectiveness and Comparative Effectiveness Reviews issued by the US Agency for Healthcare Research and Quality (AHRQ), and the Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Healthcare issued by the UK Centre for Reviews and Dissemination (CRD) [37–39].

This being a multiauthored opus with some overlap between chapters, we can surely expect some apparent discrepancies in the way the contributors have set the boundaries of umbrella reviews, overviews of reviews, and meta-epidemiologic studies or have provided specific recommendations for best practices. Frankly, we are more than happy with that, as no single recipe or formula can be considered correct per se, and the best service we can offer the reader is to help him or her navigate the complexity of evidence synthesis with tertiary level studies, but without overlooking the nuances and the constructive debates that still persist between experts. Given the novelty of the topic and our enthusiasm in leading this authoritative group of international experts, errors, inaccuracies, and typos are also unfortunately likely. We will be more than happy to receive any type of feedback in order to improve the future editions of the book.

Finally, it is paradoxically our hope that this book will have become obsolete in a few years. This would mean in fact that other and better books on the topic have become available or that this scholarly field has progressed so remarkably to challenge most of what is available here. In the meanwhile, this being the only textbook explicitly dedicated to the fascinating topic of umbrella reviews, overview of reviews, and meta-epidemiologic studies, we humbly recommend its critical albeit constructive perusal.