Introduction

Globally, the increasing occurrence of climate change and its impact on nature and human socioeconomic routines are well documented by many scholars. Collins et al. (2019) outlined the various effects of climate change. In a period of 10 years from 2006 to 2015, the Greenland ice sheet lost its ice mass at an average rate of 278 ± 11 Gt yr−1 (equivalent to 0.77 ± 0.03 mm yr−1 of global sea level rise). On the other hand, the global mean sea level is expected to increase between 0.43 and 0.84 m by 2100. Increasingly frequent extreme climate events have also been recorded across the globe. Peru, for example, has recorded an increased temperature of up to 10 °C in the shallow ocean in the northern region, whereas Bangladesh has reported extreme rainfall events for 6 days during the pre-monsoon season. Similarly, the Persian Gulf recorded severe warming with an increase of 5.5 °C in the reef temperature, resulting in thermal stress (Rimi et al. 2018; Burt et al. 2019; Collins et al. 2019).

Climate change does not just affect the environment. Its impact brings great suffering to humans as well. Those who rely heavily on a stable natural environment are especially susceptible to these changes. For example, extreme weather will obstruct the fishermen from catching fish whilst farmers will suffer from destroyed crops or reduced productivity. To overcome these negative impacts, stronger adaptation strategies are needed against climate change. Adaptation to climate change can be understood as the efforts related to changes made in processes, practices, and structures to respond to potential damages that arise from or to take advantages of the opportunities associated with climate change (Burton et al. 2018).

In recent years, there has been an increase in the number of studies related to climate change adaptation as more scholars are starting to realise its importance. Shaffril et al. (2019), Graziano et al. (2018), and Shaffril et al. (2017) investigated the adaptation of fishermen towards climate change, whilst Thinda et al. (2020), Marie et al. (2020), and Ojo and Baiyegunhi (2020) focused on farmers’ adaptation. Furthermore, Fayazi et al. (2020), Makondo and Thomas (2018), and Audefroy and Sánchez (2017) evaluated the response of the indigenous people towards the formidable impact of climate change. Mabe and Asase (2020) studied the response of aquaculture entrepreneurs towards climate risks. Apart from that, studies by Lawler and Patel (2012) examined how children are adapting towards climate change whilst Filho et al. (2019) looked at the adoption strategies taken by urban citizens in the face of climate change. Due to the sheer number of studies related to climate change adaptation, it is necessary to review all these existing knowledge more systematically. One of the methods is via a systematic literature review (SLR). SLR is defined by Higgins et al. (2011) as a comprehensive and organised way of reviewing existing literature in which the method of performing it must be transparent and replicable by others.

Although the traditional method of the literature review is still practised nowadays, it is vital to shift towards performing SLR as it will cause fewer problems to the researchers. As stated by Snyder (2019) and Littlewood et al. (2012), a traditional literature review is associated with transparency issues and often lacks rigorous methodology. Furthermore, the process of quality appraisal is skipped in traditional literature reviews and this may consequently result in significant bias in the review.

Despite its usefulness and importance, Berrang-Ford et al. (2015) claimed that scholars from non-health and medical-related fields are still unclear about SLR, particularly on its methodology. Berrang-Ford et al. (2015) attributed this issue to the fact that SLR has only been heavily used and applied in studies related to health and medical issues, whether intentionally or unintentionally. Thus, its use in other fields of studies is limited. SLR is beneficial in the research field that deals with complex research questions and involves a wide range of conceptual and epistemological approaches as well as diverse information sources. Recently, some non-medical researchers have started to develop basic SLR guidelines for their respective field of studies, such as Xiao and Watson (2019) for education planning, Kraus et al. (2020) for entrepreneurship, Durach et al. (2017) for management, Kitchenham and Charters (2007) for software engineering, and Petticrew and Roberts (2006) for social science. However, to date, none of them specifically focused on developing a methodology guideline for studies related to climate change adaptation. In response to this, this article aimed to outline a clear guideline associated to the basic methodology in developing SLR related to climate change adaptation related studies, which includes guided by review protocol/publication standard/established guidelines, followed by formulation of review questions, systematic searching strategies, appraisal of quality, data extraction and analysis and lastly data demonstration. 

Methodology

To develop the guidelines in this study, articles or documents were searched for based on the selected keywords such as systematic literature review, systematic review, methodology, review protocol, publication standard, reporting standard, established guidelines, systematic searching, searching techniques, advance searching techniques, manual searching techniques, selection criteria, inclusion criteria, exclusion criteria, appraisal of quality, quality control, data extraction, data selection, data analysis, data synthesis, data demonstration and data reporting. The keywords were first searched separately in the databases. Whenever appropriate, these keywords were also combined using the Boolean operators (AND, OR), phrase searching, truncation, wild card, and field code functions. Two main searching techniques were used, namely advanced searching in the selected database (by formulating full search string) or by manual searching (handpicking and snowballing).

Durach et al. (2017) stated that it is important to use more databases to avoid retrieval bias in the searching process. In this study, three main databases were involved in the searching process, namely Scopus, Web of Science, and Science Direct. These three databases were reported by Gusenbauer and Haddaway as highly suitable to be the leading databases in systematic searching due to their wider coverage, advanced ability in the search query, and reproducibility. Younger (2010) emphasised on the importance of supporting databases in systematic searching to complement any deficiencies of the leading databases. Thus, the researchers included two supporting databases in this study, namely Google Scholar and Dimensions.ai.

At the first stage of the searching process, the authors managed to retrieve a total of 87 potential articles for developing the guidelines. A total of 70 articles retrieved from database searching whilst the other 17 articles/documents were selected using manual techniques. In the second stage, the researchers then determined several inclusion criteria. Firstly, the selected sources must be focused on methodology related to SLR which denotes that the selected sources must be related to at least one of the six methodological steps introduced in the present guidelines. The researchers also considered any articles or documents that may not be directly discussing SRL methodology but still indirectly related to the guidelines. Secondly, the researchers included only articles journals, books, and guidelines whilst sources in a form of conference proceeding and thesis were excluded. Based on these inclusion criteria, a total of 32 articles have been excluded whilst the remaining 55 articles/documents were ready for eligibility process. The third stage—eligibility—is second process of screening, whereby the authors re-examined and re-checked remaining articles to ensure that the remaining articles/documents are in line with the inclusion criteria. The process was done by reading first the title and the abstract and if needed, the researchers read the content of the selected sources. Based on this practice, a total of 5 articles/documents were excluded as they offer merely empirical data rather and not offering any point from methodological perspectives. In total, 50 articles/documents were selected in which several of them are referred to in different sections of the methodology (Fig. 1 and Table 1).

Fig. 1
figure 1

Process of selecting the articles/documents for developing the guidelines

Full size image
Table 1 Selected articles and documents
Full size table

The suggested guideline—the GuFSyADD guidelines

The GuFSyADD guidelines were introduced in this article as a guide to performing SLR for the studies related to climate change adaptation. The guidelines involved six steps, starting with (1) guided by review of protocol/publication standard/established guidelines/related published articles, (2) formulation of review questions, (3) systematic searching strategies, (4) appraisal of quality, (5) data extraction and analysis, and (6) data demonstration (Fig. 2).

Fig. 2
figure 2

The GuFSyADD guidelines

Full size image

Guided by review of protocol/reporting standard, established guidelines/related published articles

Any SLR must be guided either by developing a review of protocol, or referring to a publication standard, established guidelines, or related published CC-SLR articles. This guidance is important to assist researchers in planning things that they want to include in their review. Furthermore, the guidelines will reduce bias as well as ensure transparency and quality assurance of systematic reviews (Haddaway and Macura 2018). One of the best guides for researchers in CC-SLR is for them to develop and validate a review protocol or by referring to a publication standard. CC-SLR researchers can use several established review protocols and publication standards such as Cochrane, Campbell, Joanna Briggs, PRISMA (Preferred Reporting Items for Systematic Reviews), or RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards). However, it is important to note that these review protocols and publication standards are more tailored for SLR related to randomised and non-randomised control trials (observational and diagnostic) or studies assessing the benefits and harms of interventions. Therefore, CC-SLR researchers need to make some changes in these standards in order to adapt them for the use of their SLR. Such practice, although allowed, shows that the review protocols may not be a good fit for CC-SLR.

To overcome this issue, Haddaway et al. (2017) and Haddaway and Macura (2018) developed a new reporting standard specially tailored for SLR known as ROSES (Reporting RepOrting standards for Systematic Evidence Syntheses). ROSES is designed specifically for the use of systematic reviews and maps in the field of environmental management and conservation. For researchers conducting CC-SLR, they can choose either the ROSES systematic review protocol or ROSES systematic review report. There is a slight difference between these two sources. The review protocol focuses on providing descriptive guidance to report the planned methods for SLR, whereas the review reports include descriptive guidance for the reporting of both methods and results (Haddaway et al. 2017; Haddaway and Macura 2018).

Furthermore, the protocol consists of 15 sections and 38 topics whilst the report contains 18 sections and 60 topics that need to be considered by CC-SLR researchers. By reading and understanding these suggested sections and topics, CC-SLR researchers will be better equipped to provide the rationales for their review question and proposed methods. At the same time, the guideline can help to minimise bias as well as strengthen how the different types of studies will be positioned, evaluated, and reported. Similar to other review protocols and publication standards, ROSES also recommends the ways to perform a quality appraisal of the studies, apart from the searching and screening strategies and the data synthesis and presentation for SLR.

Apart from the review protocol or publication standard, potential CC-SLR researchers can also refer to established guidelines or other published SLR articles related to environmental science to guide them in their SLR. Compared to review protocol or publication standard, established guidelines or published SLR articles might offer simpler instructions that can be easily understood or provide creative ideas to the potential CC-SLR researchers. There were several established guidelines for SLR related to environmental sciences, including Mengist et al. (2020), Pullin and Stewart (2017), and Centre for Evidence Based Conservation (2013). These guidelines provide vital information regarding the practices of SLR and the available options that should be considered by the researchers. Additionally, the published SLR articles by Okoli (2015) and Xiao and Watson (2019), although not specifically tailored to environmental science, are still good references for CC-SLR researchers. As SLR emphasises the concept of replication, any published SLR articles must be able to be replicated by future researchers. Therefore, it is a good practice for the researchers to read several published SLR articles and compare the advantages and weaknesses of these articles before deciding on the best way to replicate them in their SLR. Several recently published CC-SLR articles can be good guidance such as the one by Menghistu et al. (2020), Owen (2020), Shaffril et al. (2020), and Islam et al. (2020).

Formulation of review questions

The researcher must have at least one review question in their SLR. A research question is needed to guide the entire SLR process, especially in terms of the systematic searching strategies, data extraction, data analysis, and presentation. Within the CC-SLR context, researchers should consider the suggestion by Cronin et al. (2008) and Burgers et al. (2019) to formulate a specific research question in their SLR. Understandably, a specific research question will facilitate searching efforts, data extraction, and analysis. However, CC-SLR researchers must strike a balance when formulating the research question. Whilst having a specific research question is beneficial for any SLR, review questions that are too specific must be avoided as it might result in too few articles to be reviewed and subsequently preventing the development of SLR (Petticrew and Roberts 2006). An example of a specific research question that is nice and a good fit for CC-SLR would be “What are the climate change adaptation strategies among Asian fishermen?” In contrast, research questions such as “What are the climate change adaptation strategies among Asian’s fishermen aged between 25–35 years old” would be too specific and might complicate the SLR searching process and possibly lead to a very limited number of articles for the SLR.

Given the intricacy in developing a research question, it is highly encouraged to use mnemonics or also known as the Research Question Development Tool (RQDT) for CC-SLR. This tool helps the researchers to determine the research scope and to ease the process of formulating the review question. Furthermore, CC-SLR researchers can consider the recommended framework of PICOC (Population, Intervention, Comparison, Outcome, and Context) by Mengist et al. (2020) to assist them in identifying the research scope so that they can formulate the appropriate review questions (Table 2).

Table 2 PICOC for developing the research questions
Full size table

Modified and adapted from Mengist et al. (2020) and Booth et al. (2016)

Examples of research questions related to CC-SLR (developed based on PICOC framework)

  1. 1.

    Which community groups are affected by climate change?

  2. 2.

    What are the adaptation strategies taken by the community to face the impact of climate change?

  3. 3.

    Are there any differences in their adaptations and what are the strength and weaknesses of these adaptation practices?

  4. 4.

    What type of research design was used in previous studies to identify community adaptation towards the impact of climate change?

  5. 5.

    What are the strength and weaknesses of these research designs?

  6. 6.

    Which areas of climate change are most studied by scholars?

  7. 7.

    What are the knowledge gaps in terms of climate change adaptation that future scholars should be aware of?

In addition to PICOC, there is an abundance of mnemonics or RQDT to be chosen, such as PICO (population, intervention, control, and outcomes), PICo (Population, Interest Context), PICOS (population, intervention, control, outcomes, and study design), and SPIDER (sample, the phenomenon of interest, design, evaluation, and research type). Each of these RQDTs is designed to fit certain research design and purposes. Thus, the researchers should select the RQDT that is best suited to their SLR.

Systematic searching strategies

Identification

Three main things that are emphasised in this step, namely to enrich the keywords used, to create a diverse searching technique, and to maximise the number of databases (James et al. 2016). Suitable keywords for SLR can be developed based on the review questions. For example, if the research question developed is “What are the climate change adaptation strategies among fishermen?”, then the potential keywords that can be abstracted from this review questions include climate change, adaptation, and fishermen. However, to maximise the number of related articles that can be retrieved for the SLR, the researchers should not merely rely on the main keywords. Instead, they need to enrich the list of keywords by searching with their synonyms or other related terms and variation. This can be done by referring to sources such as thesaurus online, keywords used in past studies, keywords suggested by databases (e.g. Scopus, Web of Science), or expert opinions (Shaffril et al. 2020). Table 3 demonstrates the results of identification based on the three selected keywords, namely climate change, adaptation, and fishermen.

Table 3 Results of identification
Full size table

After producing the keywords, the researchers can now search for related articles. CC-SLR researchers should apply a diverse range of searching techniques. Notably, there are two main techniques, either by using the manual technique or by advanced searching technique. Among the manual searching techniques that can be applied by CC-SLR author are handpicking or searching at electronic databases, scanning references lists of relevant studies, searching trials registries, contacting experts or manufacturers, searching relevant internet sources, citation searching, sending queries to researchers, and using a project website to canvas for more studies (Cooper et al. 2018).

In addition, the CC-SLR researchers can also increase the accuracy of the search by applying advanced searching techniques such as Boolean operators, phrase searching, wild card, truncation, and field code functions in several databases. These functions allow the CC-SLR researchers to combine all potential keywords in one search string with specific symbols and coding as demonstrated by Wang et al. (2018)

((“climat* chang*”OR“global warm*”OR“environment*chang*”OR“global chang*”OR“climat* variability and change”OR“greenhouse gas”OR“greenhouse effect*”OR“carbon emission*”) AND (adaptation OR adaptation OR adaptability))

or one developed by Shaffril et al. (2020) in their article

(("Climat* chang*" OR "Climat* risk*" OR "climat* variabilit*" OR "climat* extrem*" OR "climat* uncertaint*" OR "global warming*" OR "temperature ris*" OR "sea level ris*" OR "el-nino" OR "la-nina") AND ("Adapt* abilit*" OR "adapt* strateg*" OR "adapt* capacit*" OR "adapt* capabilit*" OR "adapt* strength*" OR "adapt* potential*" OR "adopt* abilit*" OR "adopt* capacity*" OR "adopt* capabilit*" OR "Adopt* potential*" OR "adopt* strategy*") AND ("indigenous people*" OR "indigenous communit*" OR "indigenous group*" OR "native* people*" OR "primitive* people*" OR "primitive* communit*" OR "primitive* group*" OR "aboriginal* people*" OR "aboriginal* communit*"))

In general, not all databases support these unique functions. For example, users of Scopus and Web of Science can enjoy complete searching functions as stated in Table 4. However, some searching functions such as wildcard and/or truncation are not available on Google Scholars, Science Direct, and DOAJ (Directory Open Access Journals).

Table 4 Selected functions for advance searching in database
Full size table

Whilst it is commonly known that CC-SLR researchers should use more than one database in their searching process to avoid retrieval bias (Durach et al. 2017), the exact number of databases that should be used remains a subjective matter. Whilst some consider two databases to be adequate for their searching, others might think that two databases are too limited. As a solution, Levy and Ellis (2006) suggested that researchers should stop searching when the resulted articles from the databases start to show duplication.

At present, there are many databases that researchers can choose from to perform their searching efforts. Gusenbauer and Haddaway (2019) suggested the researchers to select the databases that are most closely related to their objectives. For CC-SLR researchers, they can consider searching for studies in leading journals with scopes related to climate change. Some of the examples of journals include Global Environmental Change-Human Policy Dimensions, Environmental Innovation and Societal Transition, Wiley Interdisciplinary Reviews-Climate Change, Journal of Environment Development, and Environmental Science and Pollution Research. Apart from that, indexing databases such as Journal Citation Reports (via Social Science Citation Index) or Scimago can also assist the researchers in finding more journals related to climate change adaptation. Among the indexing databases that CC-SLR researchers can use for the searching purpose are Scopus and Web of Science. Scopus has more than 2501 journals related to environmental science, covering diverse subject areas such as management, monitoring, policy and law, nature and landscape conservation, and global and planetary change. Web of Science, on the other hand, offers more than 1500 journals that encompass diverse environment-related subject areas such as environmental science, environment/ecology, environmental, engineering and energy, environmental, medicine and public health, environmental studies, geography, and development.

A study by Gusenbauer and Haddaway (2019) is a good reference for CC-SLR researchers in determining the best database for their search. They shortlisted 14 databases that can be used as the primary databases in SLR, namely ACM Digital Library, BASE, ClinicalTri-als.gov, Cochrane Library, EbscoHost (tested for ERIC, Medline, EconLit, CINHAL Plus, SportsDis-cus), OVID (tested for Embase, Embase Classic, PsychINFO), ProQuest (tested for Nursing & Allied Health Database, Public Health Database), PubMed, ScienceDirect, Scopus, TRID, Virtual Health Library, Web of Science (tested for Web of Science Core Collection, Medline), and Wiley Online Library. Additionally, Google Scholars is another popular source for articles/document searching that can be considered in CC-SLR. The advantages of Google Scholar included its ability in identifying highly cited documents (Martin-martin et al. 2017), more free articles (Chen et al. 2014), and documents published in diverse languages (Fagan 2017). The strength of Google Scholar in terms of a higher quantity of studies was also confirmed by Orduna-Malea et al. (2015). Nevertheless, Haddaway et al. (2015) mentioned their reservation towards the lack of quality and technical deficiencies of Google Scholar and urged the researchers to consider it as a supporting database rather than a leading database.

Screening

Next, screening is the process in which the CC-SLR researchers either include or exclude the identified documents from their review based on the predetermined criteria. The criteria that CC-SLR researchers should choose for their review are very subjective. Nevertheless, some scholars have attempted to provide guidance for researchers to select the best criteria for their SLR. Generally speaking, different situations will stimulate the need to select different criteria for SLR. Kitchenham and Charters (2007) recommended researchers to select any criteria that will be helpful in answering their review questions, whereas Okoli (2015) claimed that any criteria are acceptable as long as the researchers can justify the selection.

Despite that, some scholars insisted on specific criteria that must be taken into account in the conduct of CC-SLR. Mengist et al. (2020), for example, emphasised publication type. The study claimed that review paper, grey literature, extended abstracts, presentations, keynotes, and book chapters should be excluded in the review. Instead, they suggested the researchers to focus only on primary sources, i.e. article journals with empirical data. In addition, Mengist et al. (2020) also urged the researchers to consider the language and timeline of the publication as inclusion criteria. With regard to timeline publication, there is mixed opinions among researchers on the most optimum timeline of publication. Is the most appropriate timeline 5 years? Ten years? Or even 20 years? Whilst there is no exact answer to this, Kraus et al. (2020) reminded the researchers to consider the maturity of the study. If the study has matured enough, it means that a good number of related studies have been conducted. Thus, the duration of the timeline may be shorter compared to studies that are less matured (less related studies have been conducted). As for language, the Cochrane systematic review encouraged the researchers to consider all types of languages in their review to avoid bias. However, Linares-Espinós et al. (2018) showed a contradicting view. For them, selecting the studies published in a language not understood by the researchers can potentially create problems in terms of confusion, poor understanding, and the need for additional cost and time.

Last but not least, the latest publications in the field of CC-SLR can also provide ideas to researchers on the selection criteria. A study by Shaffril et al. (2020) chosen criteria such as language, timeline publication, publication type, and regions whilst Islam et al. (2020) selected slightly different criteria such as timeline publication, type of indexing, type of keywords used, and focus of study’s contents. By comparison, Galappaththi et al. (2020) relied on criteria such as language, publication type, people involved, type of adaptation responses, activities and actions related to climate change adaptation, focus, timeline, type of industry, and type of climate change. Carman and Zint (2020), on the other hand, selected criteria such as language, peer-review article, content (selected climate change studies must focus on adaptation), and focus of study (household, individual, or community level).

Eligibility

Eligibility is the second process of screening. It aims to ensure that all articles retrieved during the screening process fulfil the criteria determined (Liberati et al. 2009). To do it, the researchers do not need to read the entire article. Instead, they can just scan through the title and abstract to determine if the article is suitable to be included in the review (Bilotta et al. 2014). However, if this step is unable to provide a clear idea to the researchers on the suitability of the articles, then the content of the articles needs to be examined.

Appraisal of quality

Following that, the eligible articles must be appraised to ensure quality. The articles must be bias-free and any articles with poor methodology must be excluded (Littlewood et al. 2012). Under the section of quality appraisal, there are three main things to be considered, namely (1) who should appraise the quality? (2) criteria to determine the quality, and (3) methods to determine the quality.

Who should appraise the quality?

It can be done either by the experts or the researchers themselves. However, to reduce bias, the number of reviewers should ideally be more than one (Gomersall et al. 2015). If the researchers themselves are appraising the quality, it can be done in two ways. The reviewers can appraise the selected papers together before splitting the task to make sure that they are on the same page and coding the papers in a similar manner. The other way is for the reviewers to appraise each study independently (Gomersall et al. 2015).

Criteria to determine its quality

Several sources contain the criteria to determine the quality of the articles. Mengist et al. (2020) in his paper on environmental-related systematic review listed four criteria to appraise the quality of the articles:

  1. 1.

    Are the inclusion and exclusion criteria for the review appropriate well-described?

  2. 2.

    Is the literature search likely to have covered all relevant studies on the topic?

  3. 3.

    Did the selected publication use blind reviewers to assess the quality/validity of the study?

  4. 4.

    Was the type of environmental science mentioned in the publication described adequately?

In addition to Mengist et al. (2020), there are other assessment tools, scales, and checklists that can be used by the researchers. Seehra et al. (2016) compared more than 50 tools that are used in assessing the quality of articles, such as the Newcastle Ottawa Scale, AXIS, Mixed Method Appraisal Tool (MMAT) and Critical Skill Appraisal Programme (CASP). These tools vary in terms of their functions and objective. For example, AXIS is designed to assess the quality of quantitative articles (cross-sectional survey), CASP is for qualitative research design, whilst MMAT is developed for mixed method research design. CC-SLR researchers should select the most appropriate tools for their review. Furthermore, the researchers can also rely on the best evidence synthesis technique whereby selected articles are appraised by the systematic review team based upon the predefined guidelines before deciding if the studies are scientifically admissible or not (Littlewood et al. 2012). Lastly, according to Petticrew and Roberts (2006), researchers can also examine the methodological aspects of studies individually.

Methods to determine the quality

The quality of the selected articles can be determined in two ways, either quantitatively or qualitatively. For the quantitative quality appraisal, CC-SLR researchers can opt for either Cohen’s Kappa analysis (two reviewers) or Fleiss Kappa analysis (no limit on the number of reviewers). Cohen’s Kappa is a statistic that is used to measure the inter-rater and intra-rater reliabilities for categorical items. On the other hand, Fleiss’ Kappa analysis aims to measure the reliability of agreement between a fixed number of raters when categorical ratings are assigned. These two analyses will produce a value between 0.1 and 1.0. A value of 0.75 or more is deemed as excellent reliability, whereas a value between 0.40 and 0.75 is considered as fair to a good agreement. Any value below 0.40 indicates poor reliability (Fleiss 1981). To ensure the quality of the review, researchers are encouraged to include only articles ranked as excellent and fair to good. Articles ranked as poor should be excluded.

On the other hand, Kitchenham and Charters (2007) have listed the criteria for quality assessment. Each criterion can be scored on a scale of 1 as Yes, 0.5 as Partly, and 0 as No. Commonly, researchers should only include articles that meet at least 50% of the quality. For example, suppose an author is assessing the quality of an article based on the AXIS quality assessment tool that consists of 20 quality criteria. In that case, the researchers will only include articles with a score of higher than 10.0 (50% of 20.0, the full marks). However, some researchers such as Alsolai and Roper (2020) have included articles that only met 30% of the determined quality.

Apart from that, the assessment of the study quality can also be performed qualitatively. Qualitative assessment is performed narratively by differentiating clearly between studies of higher and lower quality. In the qualitative appraisal of quality, Petticrew and Roberts (2006) suggested the researchers to weigh the quality based on three ranks, namely high, intermediate, and low. The CC-SLR researchers also suggested the inclusion of articles that ranked as high and intermediate and excluding articles ranked as low quality.

The researchers should understand that no articles are perfect. Furthermore, there is a possibility that the same article might produce different qualities when assessed with different quality assessment tools. The researchers must bear in mind that the quality assessment procedure is not done to achieve perfect articles. If the researchers set out with this intention, they might not find any suitable articles to review. In short, they must be aware that the quality assessment is not aimed at searching for the most perfect articles but rather to assist them in selecting the articles that fit the purposes of the review.

Data extraction and analysis

After finalising on what articles to be reviewed, the CC-SLR researchers should now turn their attention to the extraction of the relevant data from the selected articles. These data will be used in the analysis. The review questions should guide the extraction and analysis process.

To ease the analysis process, Okoli (2015) suggested to place the extracted data systematically in a table. According to Okoli (2015), there are four types of data analysis. The first type is known as meta-analysis or the quantitative synthesis of quantitative data. This type of review focuses on the numerical data of previous studies, and each study is considered as a case with certain independent variables that are hypothesised to have an effect or non-effect on a given dependent variable. Although it is possible to develop meta-analysis on climate change adaptation-related studies, it would be challenging for CC-SLR researchers as most of the studies focus only on the quantitative aspects rather than qualitative or mixed research designs.

Okoli (2015) highlighted the three types of qualitative analysis. The first one is known as a qualitative analysis of qualitative studies in which the researchers will only review previously published qualitative studies. Secondly, there is also a qualitative synthesis of quantitative studies that focuses on previous studies with quantitative data. Lastly, the third type of analysis is the qualitative  analysis of mixed research designs (a combination of qualitative and quantitative studies). It remains debatable if it is possible to have a mixed research design in an SLR. Sandelowski et al. (2006) and Mays et al. (2005) stressed that methodologically, it is impossible to have diverse research designs in a review. On the contrary, Dixon-Woods et al. (2005) and Whittemore and Knafl (2005) explained that it is important to have mixed research design in a review because the best way to understand an issue is by looking at it from a diverse perspective. There are three types of mixed method review, namely segregated design, integrated design, and contingent design.

To analyse the data, a number of analysis techniques can be considered by CC-SLR researchers, including thematic analysis, narrative summary, meta-study, Miles and Huberman’s cross-case techniques, realist synthesis, content analysis, case survey, and qualitative comparative analysis method. The details of these techniques are presented in Table 5.

Table 5 Type of synthesis techniques
Full size table

Data demonstration

To allow future researchers to replicate the SLR methodology in their studies, CC-SLR researchers need to report their systematic reviews as clear as possible. The ROSES reporting standard is important as it guides researchers in CC-SLR to follow rigorous methodological steps that permit verification and replication. This allows for the clarification of key steps and acts as a critical appraisal process in SLR (Haddaway and Macura 2018). ROSES also provides a list of reporting standards for systematic review reports including the descriptive guidance on how to report the methods and results of the review. Additionally, CC-SLR researchers can also consider using PRISMA or RAMESES publication standards whenever appropriate.

Furthermore, if the researchers developed a full search string on databases, it should be outlined in a table. Whenever appropriate, the search string should include coding and symbols to ensure that comprehensive document searching can be performed by other researchers (Peters et al. 2015). To better illustrate the steps, a flow diagram can be included in CC-SLR. The flow diagram includes the steps from searching, screening, coding/meta-data extraction, data extraction, and critical appraisal, to analyse in CC-SLR (Haddaway and Macura 2018). The researchers can either develop their flow diagram or refer to established flow diagrams such as ROSES (Haddaway and Macura 2018), PRISMA (Moher et al. 2009), or one suggested by Shaffril et al. (2019).

To present the findings systematically, the analysis result of the CC-SLR should be arranged in a table. All the analyses should be outlined descriptively in the results section and explained in depth in the discussion sections. Such an explanation will allow readers and other like-minded researchers to understand the current pattern of studies related to climate change adaptation. By identifying the existing pattern, the direction of future studies can be determined (Petticrew and Roberts 2006).

Conclusion

For CC-SLR researchers, one of the ways to overcome issues related to transparency, low level of rigour in the methodology, and lack of quality appraisal of the conventional literature review is by practising SLR. SLR emphasises on comprehensive and organised methodology to enhance transparency and to facilitate replication. CC-SLR is growing in importance as it allows researchers to understand the current pattern of climate change adaptation and to plan for future research. Several guidance in a form of review protocol and publication/reporting standard are available for interested CC-SLR researchers; however, it is important to note that these review protocols and publication standards are more tailored for SLR related to randomised and non-randomised control trials (observational and diagnostic) or studies assessing the benefits and harms of interventions. Berrang-Ford et al. (2015) further added that scholars from non-health and non-medical-related fields are still unclear about SLR methodology and it might be caused by the fact that SLR has been heavily practiced and applied from health and medical related perspectives, whether intentionally or unintentionally. Thus, its use in other fields of studies is limited. To overcome this limitation, the present article comes with a detailed set of methodological standards that have been tailored to the field. By increasing methodological options and providing rich instructional information, the present guidance attempted to demonstrate the necessary level of rigor in a CC-SLR and assist researcher to have a standard methodological procedure in their review. This article offers vital guidance on CC-SLR for researchers by introducing and describing the main process involved in the GuFSyADD guidelines. The guidelines inform several important points to be considered before developing CC-SLR: first, interested researchers need to start their SLR by developing the review protocol or guided by publication standard, established guidelines, published SLR articles, and then, they need to formulate their review questions. Next, researchers need to run systematic searching process to ensure a rigorous searching efforts, followed by controlling the review quality by appraising the selected articles/documents and the final process require researchers to extract and analyse the extracted data, either qualitatively or quantitatively. The way CC-SLR researchers reporting their systematic review should be clear as it will allow the future researchers to replicate the SLR methodology in their studies. The GuFSyADD guidelines will benefit not only the CC-SLR community but also those who wish to develop a systematic literature review but do not have the methodological basis to guide them to do so.