Introduction

The field of Medicine evolved enormously during the twentieth century. To optimise and enhance decision-making for medical professionals, there is emphasis on evidence-based medicine. Although medical professionals are always looking for the latest research, they can feel overwhelmed with the amount of new publications that come out each year. Systematic review and meta-analysis provide qualitative and quantitative synthesis of the evidence on an aspect of a particular topic under consideration, hence making it easier for medical professionals to make appropriate decisions for their patients without being misdirected by any inappropriately designed or biased study.

According to Cochrane, systematic review summarises the results of all available carefully designed healthcare studies and provides a high level of evidence on the effectiveness of healthcare interventions. Meta-analysis is a statistical tool for estimating the mean and variance of underlying population effects from a collection of empirical studies [1]. For most medical professionals, systematic review and meta-analysis look like a creation from the special gizmos of statisticians, and they would not even think about conducting one. On the other hand, others think it is a very straightforward task that merely involves analysing work of other researchers; however, once they have started one, they soon realise the magnitude of the task they have committed to. Many systematic reviews are abandoned as unfinished projects. The aim of this paper is to simplify the process for medical professionals. It will require you to get help and guidance from other professionals including librarians and statisticians.

Figure 1 shows the hierarchy of evidence [2] and gives an idea of the value of meta-analysis in the research methodology. Systematic review and meta-analysis are the highest level of evidence possible.

Fig. 1
figure 1

Hierarchy of evidence showing the ranking in research methodology

Full size image

Steps in conducting a systematic review and meta-analysis

Research question

The first step in a systematic review and meta-analysis is to formulate a research question. Although it may appear simple, it is the most vital element in the whole project. Data should summarise the available literature on the topic. This gives you an opportunity to gather the research available and conduct a qualitative analysis and perform statistical tests and finally come up with a quantitative result about the question under consideration.

Development of protocol

The PICOS structure has been developed for formulating the search question. PICOS model is a search structuring tool, which helps you in making a purposeful and useful search possible. In this PICOS model, P stands for population under consideration, I for intervention, C for comparison group, O for outcome and S for study design. It also includes making inclusion and exclusion criteria. This will help in minimising bias in the systematic review and meta-analysis. You should decide to use one of the standardised reporting systems. The most commonly used is PRISMA [3] protocol.

Registration of the systematic review and meta-analysis

Always be smart: do not let anyone duplicate your study. PROSPERO [4] is an international prospective register of systematic reviews. This register makes sure that systematic review topics are not duplicated. So, once you have finalised your topic, the next step should be to register it. That will identify any similar systematic review and will avoid you duplicating a topic which is an ongoing project for another researcher. When you are writing, or reporting your systematic review and meta-analysis, you need to mention the PROSPERO registration number.

Inclusion and exclusion criteria

As discussed earlier, you must develop a protocol for your systematic review and meta-analysis. This protocol will define the outcome of interest. At this point, you should define the inclusion and exclusion criteria. This will include any restriction based on geography, language, age, study design (RCT’s only or other study designs as well), only human studies and any other criteria you will apply. Clarity about the inclusion and exclusion criteria will help minimise bias.

Search strategy

This step is the backbone of any systematic review and meta-analysis. The main aim of all this exercise is to have an exhaustive and reproducible search. You need to make a balance between sensitivity and specificity. A narrow very specific search may not include important articles on topic, which might not have been defined correctly in the mesh with their keywords. On the other hand, if your search is too sensitive, you might have thousands of articles in your hand at the end of search. And the shear burden of going through them might put you off the meta-analysis. At this point, do not hesitate to get help from an experienced librarian. You have huge number of databases; PubMed, Ovid Medline, EMBASE, Web of Science, SCOPUS and Cochrane Library are only few to name. Your librarian can help you in developing search strategy with the help of keywords (“MeSH” terms) related to each component of PICOS. On the other hand, if that is not the case in the database you are using, all the alternative words should be included in the search to avoid missing important articles. Although most part of the search is done on electronic databases, still you need to do hand search through the bibliography of the relevant articles to make sure you include all the relevant articles. The search strategy for at least one of the databases should be included at the end of the meta-analysis as an appendix to maintain transparency and reproducibility. Figure 2 shows a search strategy for a meta-analysis in Cochrane library [5]. Patients with venous ulcers given patient education were compared with patients without that and compliance was analysed. Search strategies are added to the appendices of the articles to make them transparent and reproducible.

Fig. 2
figure 2

Search strategy for a Cochrane study explaining all the strategy used in searching of databases

Full size image

Screening

Once the titles and abstracts of all articles are retrieved by the search strategy, the screening process starts. It should be done by two researchers to minimise bias. Screening titles and abstracts of all articles should be carried out and all irrelevant articles should be removed. If there is any disagreement about any study, the decision should be made after discussion and with consensus between two researchers or by a third researcher. The full text of the articles selected after the initial screening be retrieved. A librarian’s assistance will be helpful in using the library’s document supply service.

Final selection

At this stage, the full text article of the included papers must be read carefully to see if they meet your inclusion criteria. The reason for exclusion of any article should be explained. All the articles meeting inclusion criteria should be taken for qualitative analysis (systematic review) and later if providing enough information quantitative analysis (meta-analysis).

Data extraction

Data extraction is a vital part of meta-analysis. A data extraction form should be designed and agreed on by all the authors. Data extraction must be performed independently by two researchers. The first element of the data extraction form includes information about the paper (journal of publication, year of publication, country of research, etc.). The latter part should include information regarding the outcomes of interest.

Quality assessment

All the studies that are included in the meta-analysis should be assessed for the quality. There are multiple different tools available that can help researchers in forming objective assessment of the study. The Newcastle-Ottawa Scale, Jadad Scale and PEDro scale are a few of the most established scoring systems. They aim to standardise quality assessment of studies.

Analysing the data

Analysing the data can be a difficult experience for many medical professionals. It involves either using Hunter and Schmidt [6] or Hedges and colleagues [7] using random effect or fixed effect models. This involves a lot of statistical work and use of some advance mathematical formulas, which is usually beyond the scope of a medical professional. In this era of computer technology, things have been simplified. There are multiple software packages available that can help in extracting results. RevMan, Meta-Analyst, Mix, Stata/Win BUGS and Comprehensive Meta-Analysis are a few of them [8]. There are cost differences between these packages, so the decision on which to use depends on available funding. Some of them like RevMan and Open Meta-Analyst are free for use. If you want to use them, you might need tutorials or a short course.

You also need to decide what effect model you are going to use. There are two available effect models: the fixed effect model and the random effect model. If sample size is large and there is very small or negligible inter-study heterogeneity, you can use fixed model effect. Otherwise, you should use random effect size.

Forest plots

A forest plot is a method of graphically representing the effects of an intervention. The analysing software can produce a forest plot for you with a few clicks at the end of the analysis. Historically, horizontal lines and boxes were made for individual studies and for the overall effect in the meta-analyses [9]. But the problem with this way of representing the result is that the smaller studies with larger confidence interval were depicted as big box on the result and attract more attention, while larger studies with smaller confidence intervals have smaller boxes. This problem was solved by changing to the forest plot. The forest plot concept was proposed by Stephan Evans at Royal Statistical Society Medical Section 1983 Meeting in London. It was based on a modified box plot idea [10].

Interpretation of the forest plot

Once the data is analysed, you need to interpret and know what the important features in the results are. This will direct the presentation of the results in a more attractive and interesting manner. Firstly, does the result favour the hypothesis or not. It can be measured either as an odds ratio or risk ratio. The odds ratio describes the odds of benefitting from the procedure or intervention in question, and the risk ratio provides relative risk of failure.

These ratios can be only used if the data is dichotomous. If the data is continuous, then you need to use mean difference in mean of the values. Whether your result favours the hypothesis or not can be easily seen by where the “diamond” plots in your forest plot are. The peak of your diamond represents the cumulative effect size, while the sides which represent the 95% confidence interval are also shown. If your diamond is towards favour hypothesis but it crosses the midline, it becomes equivocal and non-conclusive. As can be seen in Fig. 3, the diamond crosses the midline, so the result is not conclusive. Although the peak of the diamond is on right side that favours CNV [11], the 95% confidence interval crosses the midline, which indicates that the CNV is not always an effective intervention.

Fig. 3
figure 3

Forest plot shows that studies favour CNV over PPH but as the diamond crosses the midline, result is inconclusive

Full size image

If the entire diamond is on the favoured side, it means that the 95% confidence interval is fully to the right of the line of no effect on that side, and only then is there a positive result. It can be seen in Fig. 4 that the whole diamond is on the right side which favours pCR [12] and all of 95% confidence interval is on the right of the line of no effect.

Fig. 4
figure 4

Forest plot on the right side clearly shows that results from studies favour pCR over no pCR

Full size image

In another example in Fig. 5, you can see that the complete diamond along with 95% confidence interval is to the left of the line favouring radial artery over ulnar artery [13].

Fig. 5
figure 5

In this forest plot, the diamond of result is on the left side favouring radial artery over femoral artery

Full size image

The next value you need to know is heterogeneity. Heterogeneity is defined as characteristic of being non-uniform and dissimilar. In a meta-analysis, it shows dissimilarity among the studies involved in the meta-analysis. I 2 is the value which determines heterogeneity among the studies included in the meta-analysis. I 2 is defined as the percentage of variation across studies that is due to heterogeneity rather than chance [14, 15]. I 2 is an intuitive and simple expression of the inconsistency of the study results.

Funnel plot

A funnel plot is a simple scatter plot of the intervention effect estimates from individual studies against some measure of each study’s size or precision [16]. It is a measure to see the degree of bias in the studies included in the meta-analysis. If there is minimal bias and studies are quite similar, the scatter plot should look like an inverted funnel. As can be seen in Fig. 6, the studies are evenly scattered, forming a shape of an inverted funnel. This indicates that there is a minimal bias in the studies involved in this meta-analysis [17].

Fig. 6
figure 6

Evenly spaced and scattered studies in funnel plot showing minimum bias in the result of meta-analysis

Full size image

The larger-sized studies are placed towards the top of the funnel plot, while smaller studies with minimal effect are placed towards the base. If there is bias because of smaller studies showing no statistically significant effects, then such publication bias will lead to an asymmetrical appearance of the funnel plot. As can be seen in Fig. 7, there is asymmetrical plotting of the studies in the funnel plot, indicating bias [18].

Fig. 7
figure 7

Asymmetrical plotting of studies in funnel plot indicating bias in result of meta-analysis

Full size image

Additional statistical analysis

Additional statistical analysis of the data pooled from all the studies can be done using SPSS. If the data available is categorical (nominal), the chi-square test is the preferred test but sometimes the Fisher Exact must be used if assumption is violated. Alternatively, other test of significance and independence like independent sample t test, paired sample t test and etc. can be done where appropriate.

Reference management software

Other useful software that can be very beneficial for your work is reference management software. There are multiple options available including EndNote, Mendeley, ReadCube, Reference Manager and Bookends. They help you in managing the references in an effective way. And, there is a requirement of different referencing style by the journals. These allow you to change between styles seamlessly in case you choose a different journal for publication.

Write it up

Once you have done all the above-mentioned analysis, act quickly and write it up. Provide information about the search strategy used and include it in an appendix. Inclusion and exclusion criteria must be very clearly reported. Use tables and graphical representations to show your results. Your result should include the inclusion and 95% confidence interval and at the end, a concise but comprehensive conclusion should be made based on the results. Use the PRISMA reporting checklist as a guideline in writing it up.