Keywords

Information is gathered at different stages of the integrative review (IR) process, including background information for formulating the question, and to support analysis and discussion. This chapter will focus on searching for, and identifying experimental; nonexperimental; and theoretical literature, which will be screened for inclusion or exclusion in the IR sample.

The IR process should include a clearly documented and comprehensive literature search, defining in detail all databases, search terms, limiters, eligibility (inclusion/exclusion), and criteria used, and describing any additional search methods (Cooper 1982; Whittemore and Knafl 2005). The search process for the systematic review has formal search guidelines and standards, which are clearly articulated; reviewed; and evaluated. However, the IR search process has been inconsistently described through articles and book chapters (Torraco 2005, 2016; Russell 2005; Soares et al. 2014; Hopia et al. 2016). The IR search process reflects the nature of the purpose and/or review questions addressed, which tends to be broader than the systematic review question, and draws its strategies from writings on knowledge synthesis and literature review, including systematic review.

3.1 Librarian Support

Reviewing published IRs reveals how librarians can support the search process (Coyne et al. 2018; Harstäde et al. 2018; Tobiano et al. 2015), and how librarians can play a valuable role (Middlebrooks et al. 2016). Librarians have experience building searches, identifying documentation tools, and organizing results. Working with a librarian helps to decrease the risk of bias by supporting a thorough and comprehensive search process (Evans 2007; Cooper et al. 2018). Bias is defined as any tendency that prevents unprejudiced consideration of a question (Pannucci and Wilkins 2010). There are different types of bias that can occur at different stages of the review process, which will be discussed in later sections of the book.

Many steps in the search process may be familiar to students and reviewers, but because the IR process requires documentation of each step, several steps benefit from more than one perspective. Working with a librarian can support rigor in a review, especially for a reviewer who is conducting the review alone. Although the process for developing the IR review question was addressed earlier, it should be remembered that preliminary searching may suggest revisions in the review question(s), and in that sense, the relationship between the review question(s) and the search strategy is a dynamic one. It is not unusual for sample searches to further define the question(s) and the eligibility (inclusion/exclusion) criteria. Engagement with search results may impact how the question(s) is finally articulated.

3.2 Search Organization and Reporting Strategies

A librarian can be helpful with recommending tools for literature organization. The reviewer is advised to consider how to document information described below before beginning the search process. Many of these topics will be discussed in more depth later in the chapter.

  • Database selection benefits from discussion with a librarian. Depending on the nature of the IR review question, there may be relevant databases outside the normal scope of the reviewer’s experience that should be considered (more information is found in Sect. 3.3.1). The names of selected databases, and the reasons why they were selected, can be saved in an Excel spreadsheet, a Word document, or other software application.

  • Database searches should be saved, so search methods can be thoroughly described in the review, and potentially replicated. Searches can be saved in EBSCO, Ovid, PubMed, and many other databases, and while processes may be similar, they also may vary considerably. Tutorials (usually in video, PDF, or PowerPoint format) are helpful, and can be found through a Google search, such as “saving searches in CINAHL.”

  • Search results should be saved in an organized manner, so they can be screened for inclusion in the final review. For this purpose, citation management software (sometimes called bibliographic software applications) is helpful. Some require either an individual or institutional subscription; others are available at no cost.

  • Search methods and results for gray literature (i.e., literature not controlled by commercial publishers, such as conference proceedings, clinical trials, dissertations/theses) can also be saved in citation management software. Although databases can automatically export to citation management software, many “gray” resources can be entered manually. Gray literature search methods will be discussed in more detail later in this chapter.

  • Screening literature requires documentation that can be accomplished using citation management software, review software, or an Excel spreadsheet matrix.

  • The IR search process will be reported in detail. This is done using the following:

    • A narrative description of all information sources, including databases, that were used; limiters used to narrow search results, such as year of publication, language, and publication status; and search terms used.

    • A search diagram format that depicts the flow of information through different phases of the review. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is an example of a reporting model that provides both a checklist and a search flow diagram that can be adapted for IR use (Moher et al. 2009). The PRISMA Flow Diagram (2015) will be discussed in later sections of this chapter.

3.3 Searching Considerations to Increase Rigor

Although recognized guidelines exist for conducting and reporting (PRISMA Flow Diagram 2015) systematic reviews and scoping reviews, IR searching guidelines have not been similarly formalized. Researchers who have studied IR methodology recommend that an IR includes a comprehensive search, one that captures as much literature pertaining to the topic as possible (Evans 2007; Whittemore 2007) and includes multiple strategies (Whittemore and Knafl 2005), such as searching more than one electronic databases to find peer-reviewed articles; searching the gray literature to find unpublished research and theoretical literature, which may not be included in electronic databases; and handsearching relevant journals and reference lists. Such a broad-based approach will help the reviewer to minimize bias and retrieve as much relevant literature as possible.

Because no single database indexes all relevant literature, searching only one database would result in the inclusion of a limited representation of studies/results. In other words, an IR search should go beyond CINAHL, and should utilize more than one or two search terms or phrases. Searching multiple databases contributes to a more comprehensive and rigorous review (Whittemore 2007; Higgins et al. 2019; Ganong 1987; Cooper 1998). Working with a librarian to select databases helps to identify useful sources that may be unfamiliar to the reviewer, another method to make the review more comprehensive and avoid bias.

3.3.1 Choosing Databases

Selecting databases for an IR can be a daunting task. The reviewer should be willing to move beyond familiar databases and consider new ones, documenting which databases were chosen and why. Fowler identified questions the reviewer should consider (Fowler 2017) when selecting databases for their review:

  • Are the topics in the research (review) question covered in the database? CINAHL includes nursing and allied health publications; however, databases that include literature of other disciplines are suggested.

  • What types of sources are indexed? IR can include research (experimental or nonexperimental) and theoretical literature. Are those included in the chosen database?

  • Is this the only and/or best platform for this database? For example, Medline can be searched on the Ovid platform, which requires a subscription, and the PubMed platform, which is freely available. Which works best for the reviewer? Considerations include the reviewer’s familiarity with the search interface, whether a platform links to available full-text literature, and whether the platform supports export to a citation management software. Definitions of platform and search interface appear in the next section. Reviewing the “About” section of a database’s website can answer some of the questions above, as can consultation with a librarian.

Literature identified during the question formulation stage, or during preliminary searches, can help identify relevant databases. From an article citation found in a preliminary search, one can discover other databases in which the journal was indexed. For example, if a useful article is found in the Journal of Pain and Symptom Management, published by Elsevier, a visit to that journal’s website reveals that it is indexed in multiple databases, including CINAHL, EMBASE, PubMed, Scopus, PyschInfo, and others, all of which may be considered for the IR. Exploring the “Methods” sections of published IRs on similar topics will also suggest possible databases.

3.3.2 Terminology

Database terminology can be confusing, especially because the words used may have different meanings in different disciplines. For the purposes of this chapter, the terms platform, database, interface, and search engine will be used as described below:

  • A platform refers to the software used by a specific database provider. The platform does not always have the same name as the database. CINAHL is delivered on the EBSCO platform, while Medline is available on Ovid and ProQuest platforms.

  • A database refers to an electronic, searchable collection of published materials, including some combination of journal articles, book chapters, reports, dissertations, and conference proceedings. Each database indexes a different set of resources and discloses information about those resources (titles, dates, publisher, etc.). Most databases are available through college, university, or hospital institutional subscriptions.

  • A search interface refers to the search page and features that allow a user to search a database. Most search interfaces include basic and advanced search fields and a variety of limiters. Database search interfaces enable saving of search histories, which are needed to document the IR search process.

  • The term search engine is used here to describe systems like Google, Google Scholar, Bing, and Yahoo, which enable users to search the World Wide Web. Unlike databases, search engines do not disclose exactly where and how they find their results; therefore, the search engine search process cannot be documented in the same way that the database search process can. The term search engine can also refer to resources beyond the popular examples shown here. Many libraries refer to their integrated search platform as a search engine, because it searches across multiple databases and includes the library catalog.

3.3.3 Nursing, Allied Health, and Medical Databases

Table 3.1 includes databases of literature in nursing, allied health, and medicine. Literature in these databases includes articles from scholarly journals published by multiple publishers, conference proceedings, and reports. All databases on this table, with the exception of PubMed, require access through subscription, and are most often available through a hospital; college; or university library. Depending upon the reviewer’s institution, some databases may be available, but not all. For instance, CINAHL is the widely recognized and available through many hospitals, nursing schools, and some professional organizations including CINAHL access in its membership fee. Engaging a librarian in a discussion of which databases are available will support a decision about which should be included (Table 3.1).

Table 3.1 Databases for Nursing and Medical Literature
Full size table

3.3.4 Interdisciplinary Databases

A reviewer may consider a question or questions from the perspective of economics, education, psychology, and sociology, among other disciplines. The librarian can be helpful in deciding whether databases in related disciplines should be included in the search strategy. As with nursing and medical databases, questions of availability must be considered. Table 3.2 illustrates examples of databases in related disciplines; this is not an exhaustive list, but it includes some major databases to consider. Like the databases in Table 3.1, databases in Table 3.2 may appear on more than one platform and will be accessible through institutional subscription, or possibly membership in a professional organization (Table 3.2).

Table 3.2 Examples of databases in related disciplines
Full size table

3.4 Searching Systematically

Systematic database searching includes the methods described subsequently. Natural and controlled language, Boolean operators, and advanced search techniques are features of most databases, and when used together, they help to build an effective search.

3.4.1 Natural and Controlled Language

Keywords, or natural language search terms, can be identified by brainstorming about the review question and by reviewing article examples and search results found in preliminary searches. Searching with natural language terms will capture any results that include the term—whether the literature is “about” the topic or simply mentions it. Keyword searches usually return the greatest number of results, but not necessarily highest in relevance. Many databases use keyword or natural language searches as their default option.

Controlled vocabulary refers to a standardized predefined set of terms used by a database to describe and categorize articles or sources of information based on content. Subject terms may not be consistent across databases. Controlled vocabulary may also be called controlled language subject headings or thesaurus terms. MeSH (Medical Subject Headings), developed by the National Library of Medicine and assigned to articles in Medline, PubMed, CINAHL, Cochrane, and other databases, is an example of a controlled language. Most but not all databases use controlled language systems or thesaurus.

Reviewers are advised to begin with the familiar natural language, or keywords; find the most relevant search results, and within those, identify the controlled language, or subject terms, associated with those results. Controlled language searches may return fewer, but more relevant, results than natural language searches.

Both natural language and controlled language searches are important to include in the search strategy, because each method may yield different results. Figure 3.1 shows how a natural language example translates to controlled language terms in three different databases.

Fig. 3.1
figure 1

From natural to controlled language (From: What to know about search queries EBSCO, ca. 2017)

Full size image

3.4.2 Combining Search Terms Using Boolean Logic

When natural and controlled language search terms have been identified through reading results from preliminary database searches, a revised search strategy can be formed. Whether using controlled or natural language, search terms are usually combined using the Boolean operators AND, OR, and NOT.

The operator OR will expand results and is usually used to combine similar terms. Often when typing a search term into a search box, a search string will appear using the OR operator, such as teamwork OR collaboration OR mutual support. This search will look for articles that include any of these search terms, or phrases. A library adage says OR gets you more.

The operator AND will identify articles which includes ALL of the search terms joined by AND: (teamwork OR collaboration OR mutual support) AND (nursing students OR nursing education) will only return articles that include references to search terms from both the first group and the second group. Using AND reduces the number of results.

When using OR and AND, it may be helpful to consider that applying these operators to a search yields counterintuitive results. In daily conversation, using or usually implies a limitation: “I will have pie or ice cream” means choosing only one. However, in Boolean searching, looking for a literature on pie OR ice cream will return literature on pie alone, ice cream alone, or on both concepts. Likewise, in conversation, using and implies addition: “I will have pie and ice cream” means choosing both. In Boolean searching, searching for pie AND ice cream will only return literature that address both pie and ice cream. Any literature on just pie, or just ice cream, will not appear.

The operator NOT excludes concepts. The search (teamwork OR collaboration OR mutual support) AND (nursing students OR nursing education) NOT (hospital OR acute setting) will return literature that includes teamwork and nursing students, but will not include references to hospitals. Such results may instead include literature about teamwork and nursing students in the community, or in the classroom. This is an approach to narrowing results by exclusion of search terms, rather than inclusion.

Boolean operators should be capitalized. Although some databases, including CINAHL, do not strictly require capitalized operators, there is a risk that using lower case for Boolean operators will result in those terms being searched as text, which will confuse search results. It is wiser to err on the side of caution (Cooper 1998).

3.4.3 Advanced Search Techniques

Advanced search techniques refine and focus search results and can be used to address exclusion and inclusion criteria. Each database interface provides a framework determining how limiters and search queries are represented, and while there are some similarities across databases, there are also variations. Understanding how each database uses advanced search techniques can support the return of relevant results. Some common techniques are shown subsequently:

  • Common limiters found in many databases include time frame, language, publication type, age of study participants, and geographic location. The use of such limiters should align with the inclusion and exclusion criteria identified at question formulation stage, and if new limiters for criteria are determined during the search design process, they are documented and described in the review itself. An example can be seen in the Toronto and LaRocco (2019) IR, which identified a time period of 1994 through 2017, and justify their reason for beginning with 1994: because in that year, the first instance of a professional nursing organization publishing a statement on family presence appeared (Toronto and LaRocco 2019). This date was identified during preliminary searching.

  • The truncation symbol replaces the ending, or final letters, of a search term, to capture all forms of that word. A common truncation symbol is the asterisk (∗), used by CINAHL, Embase, Medline, and PubMed. For example, the search term cardio ∗ will return instances of cardiology or cardiovascular and cardiopulmonary.

  • Parentheses are used to group terms in a particular order, in the same way that parentheses in algebraic equations determine the order of operation. Boolean operators within parentheses indicate what combinations of terms to search and in what order. Search terms within parentheses are treated as a unit, and search terms without parentheses are searched from left to right. A search for [(prostate cancer OR prostatic neoplasms) AND (screening OR assessment)] will find articles about prostate cancer OR prostatic neoplasms (one unit) that also reference screening OR assessment (another unit). A search for the same terms without parentheses will not group terms as units, but combine them as directed by the Boolean operator, one term at a time.

  • Quotation marks can be used around a phrase or concept of two or more words. Doing this instructs the database to search for the entire concept, rather than searching the terms separately. Searching the phrase “unpleasant symptoms” tells the database to search for exactly that phrase. Searching unpleasant symptoms without quotation marks will search for both terms, together or apart.

  • The wildcard symbol replaces an unknown character, usually within a search term. A common wildcard symbol is the question mark (?), used in CINAHL, Embase, and Medline. For example, the search term wom?n will return instances of the terms woman or women; and midwi?e will return midwife and midwive.

  • Proximity searching identifies search terms that appear near one another, but not necessarily next to each other. Searching for the search phrase family presence may return only results in that order of appearance. To find the literature where the concept of family and the concept of presence appear near one another (e.g., family members who are present in the hospital), proximity searches are useful and more precise than simply searching (famil∗ AND presen∗), which may produce results where those terms are not in relationship with one another. Proximity searches use designated letters combined with numbers to indicate the location of the terms. Databases use different proximity operators, which can usually be identified in the database help section.

3.5 Defining the Search Strategy

IRs are characteristically broad in nature. A review may explore how a theory has been applied to research or look for studies on the attitudes of a specific population toward a specific treatment, searching for literature on a concept or phenomenon rather than on the effectiveness of a clinical intervention. Using the review question(s) as a guide, key concepts are identified, related search terms are selected, and limiters identified, to focus the strategy. These steps require actively working within the selected databases. Westlake states: “The refinement of the question and the review of the literature is an iterative process that is recursive in nature, with the desired end point a fully refined … question with a matching review of the literature” (Westlake 2012, p. 245). As with all stages of the review, it is important to document preliminary searches, especially keeping track of any revisions made to the research question itself, or eligibility criteria.

Questions to consider in the early phase of searching: When does the topic or concept first appear in the database? Does the database include specific subject headings to support focused searching, or related terms to consider? Do preliminary findings suggest that the review question should be phrased differently or that the inclusion/exclusion criteria need reconsideration? The process of testing the search in this way supports further development of the search strategy.

Reviewers are encouraged to seek a balance between comprehensiveness (search recall or sensitivity) and relevance (search precision) (Levay and Craven 2018; National Institute for Health and Care Excellence 2014). The term recall is associated with how well a search captures all relevant literature, and the term precision is associated with how well a search avoids irrelevant results. The goal is to find as many relevant studies as possible, without being overwhelmed by results that are not useful. Clarifying concepts and search terms, and using limiters, can help.

3.5.1 Choosing Search Terms: Identifying Concepts

Concepts should be clearly defined in order to identify effective search terms (Evans 2007). Because many IRs deal with complex concepts or phenomena (such as dignity-conserving care actions in palliative care (Harstäde et al. 2018)), exploratory searches may be needed to uncover all relevant terms, after which synonyms for each concept can be identified. Below are two examples of integrative review questions and possible search strategies.

3.5.1.1 Identifying Concepts and Search Terms: Example 1

“What coping strategies are reported by family members of critically ill hospitalized patients?” (Rückholdt et al. 2019)

Concepts: Coping, family members, critically ill patients, hospital

Related terms and synonyms: coping, managing, tolerating; family members or family; critically ill patients, intensive care unit patients, critical care patient; hospital, acute setting, inpatient.

These terms can then be combined using some of the search methods mentioned above. Applying Boolean operators and truncation (∗) to these search terms would result in a search string such as this:

(cop∗ OR manag∗ OR tolerat∗) AND (family member∗ OR family OR famil∗) AND (critically ill patient∗ OR intensive care unit patient∗ OR critical care patient∗) AND (hospital∗OR acute setting OR inpatient)

The above search string includes natural language terms—those that are identified by the reviewer, through brainstorming, and by reading about the subject. This search would yield results from which controlled language (or subject or thesaurus) terms could be identified. In CINAHL, some of the major subject headings associated with these search results include the following:

Critically ill patients; critical care; family coping; family role; caregivers—psychosocial factors; and stress management

Searching by major subject headings may return many of the same results as searching by natural language terms; nevertheless, a comprehensive search will include both methods. Any duplicate results can later be identified and removed.

3.5.1.2 Identifying Concepts and Search Terms: Example 2

“What are the transition-to-practice experiences of internationally educated nurses working in the United States?” (Ghazal et al. 2019, p. 5)

Concepts: Transition-to-practice experiences; internationally trained students; USA

Related terms and synonyms: transition to practice, internationally educated nurses, migrant nurses, foreign nurses, practice experience, United States, USA

A search string with Boolean operators and truncation (∗) could look like this:

(internationally educated nurs∗ OR migrant nurs∗ OR foreign nurs∗) AND (transition to practice OR practice experience OR practice) AND (United States OR USA)

CINAHL subject headings associated with search results include the following:

Foreign nurses, international nursing, transitional programs, practice patterns, workforce—United States

The examples described above illustrate possible search strategies and are not meant to suggest this is the only approach. Working with a librarian on this phase can support the reviewer’s understanding of the search development process, which can seem ambiguous or not intuitive to the novice reviewer.

3.5.2 Document the Search Process

All search history (searches, search terms, results from those searches, and article citations) should be saved—even search results that may be excluded later. This information will be combined into a final reporting format, such as the PRISMA Flow Diagram (2015) or other type of search flow diagram.

Most colleges and universities provide subscription access to one or more citation managers, such as EndNote or RefWorks, so the reviewer will need to determine which software is available. Open-access citation management software like Zotero and Mendeley is freely available on the web and requires no institutional affiliation. When conducting the review with other authors, it is important to select a citation management software, which allows reviewers to share saved citations.

Articles saved to the reviewer’s database user account can be exported to citation management software for the removal of duplicate articles from combined search results. Removing duplicates prior to screening for eligibility avoids unintended double counting of data, which can result in biased or incorrect review results (Tramer et al. 1997). The number of duplicates removed will also be documented in the final reporting format.

Citation management software also offers integration with word processing software, facilitating creation of in-text citations from the reference list, and changing reference styles as needed suit publishing requirements. Tutorials describing exporting articles from databases, removal of duplicates, in-text citation support, and reference formatting can be found by a Google search of these subjects combined with the specific citation management software.

3.5.3 When Is the Database Search Process Complete?

Jewell, Fowler, and Foster include the following points made by Cooper and Valentine (Jewell et al. 2017):

  • All databases likely to contain the highest number of citations have been searched.

  • The search strategy has been modified by adding terms based on citations highly relevant to the topic.

  • New searches return no new, unique, and relevant results.

  • Author searches on the most prolific authors of the topic show no new citations.

Questions regarding the quality of the searches undertaken are raised by McGowan et al. McGowan et al. (2016, p. 42):

  • The search concepts are clear, not too narrow or too broad.

  • All spelling variants and synonyms have been searched, including abbreviations.

  • Appropriate subject headings, or controlled language terms, have been identified and searched.

  • Search limiters, filters, and Boolean operators have been used correctly.

3.6 Screening for Study Selection

Database search results, whether saved in citation management software or excel matrix, are next reviewed to determine which studies will be included in the review sample. The eligibility or inclusion/exclusion criteria guide the screening process. First, article titles are examined, and any duplicate or clearly irrelevant titles are removed. Next, reviewers look at the abstracts of any studies where there is any doubt of the relevance from the title—or, where it is impossible to judge relevance from the title alone. Citations with potentially relevant abstracts are identified as candidates for full-text screening. Citations determined to be irrelevant are excluded at this phase, and the number of citations excluded is documented. Once all potentially relevant studies have been identified, full-text articles for these citations are obtained and stored for further screening. Further literature may be excluded when the full text are screened, and these too should be documented (Whittemore and Knafl 2005). Table 3.3 illustrates how several published IRs applied inclusion and exclusion criteria to sample studies (Middlebrooks et al. 2016; Vivieros et al. 2019; Blakeman 2019; Settecase-Wu and Whetsel 2018) (Table 3.3).

Table 3.3 Examples of documented inclusion and exclusion criteria
Full size table

The process of searching databases, and screening results, is documented in the final review in narrative form and through a search flow diagram. The IR search flow diagram will be constructed after other search techniques and the gray literature have been investigated for additional studies. The flow diagram will include details of databases searched, records identified through database searching, number of records after duplicates removed, number of records screened, number of full-text records excluded (with reasons), and number of studies included in the final sample, as illustrated in the PRISMA Flow Diagram (2015). The next section addresses search techniques such as citation searching and non-database sources, sometimes referred to as gray literature.

3.7 Beyond Database Searching

The first part of the search focuses on developing well-crafted searches in databases engineered for discovery, and usually most of the records are peer-reviewed articles. The second part of the search involves processes to ensure that all relevant literature has been found by seeking different report formats, using resources other than databases, and utilizing other approaches for locating studies. Different report formats could be conference proceedings, dissertations, white papers, clinical trial registries, and more. Some of these formats are grouped together with the term gray literature, a term used for any reports that are not published in peer-reviewed journals or controlled by commercial publishers. These reports could be produced by a wide range of organizations—business or industry, government entities, academic institutions, or nonprofits—where publishing is not the primary activity. As these organizations are not focused on publishing, reports can be difficult to locate and require different search approaches. Added to that, it has been suggested that quality may be variable due to lack of peer review.

A good question to ask at this point would be—why is this necessary? The most important reason to consider searching beyond databases is minimize publication bias. This type of bias is defined as a situation that leads to a report not being published due to the nature of its results (Russell 2005). It is important to collect relevant literature to answer the IR research review question no matter the direction of the results in order to have the most comprehensive synthesis.

When designing a search in a database, the reviewer is dependent on the citation record to have certain concepts. However, reviewers do not always write abstracts including all the relevant concepts, and relevant thesaurus terms may not be added to the record. Some databases do not index all parts of a journal, leading to literature not being indexed in databases. Therefore, despite a reviewers’ best efforts, relevant literature can be missed.

3.7.1 Gray Literature

Reports other than peer-reviewed literature can be difficult to find and can require knowledge about the field being studied to seek the most relevant sources to the proposed topic. This section will give a description of each type and provide examples of known sources.

3.7.2 Conference Proceedings

Many conferences create proceedings of posters, papers, and other events. Some conferences produce full reports for each event, while others provide abstracts only. Proceedings can be found by going directly to the conference website, seeking the printed proceedings, and searching a database that focuses on conference proceedings or a database that focuses on proceedings. The first step in searching proceedings would be to list relevant conferences to the review topic. From this list, determine which ones are available online or in print, or indexed in a database. More conferences are archiving proceedings online than ever before; however, not all conferences provide free access to their proceedings. Searching proceedings can mean browsing tables of contents online when/if other search options are not available. There are also conference databases that can quickly search across thousands of proceedings including databases that are interdisciplinary (Conference Proceedings Citation Index, COS Conference Papers Index, or Proceedings First), or databases that focus on one discipline like Northern Light from OVID, which covers biomedical conferences. Check with the library for availability as most of the databases are subscription based. Finally, some databases (CINAHL or ERIC) index conferences, but may not index the individual conference presentations. Once a conference report is found, it is important to check to see if it has been published, taking note that the title or first author might have been changed.

3.7.3 Dissertations/Theses

There are many benefits of including dissertations and theses for a comprehensive review. First, newer topics are often found in dissertations earlier than peer-reviewed publications. Second, dissertations are longer, providing more in-depth information about the literature, and tend to have more citations that could provide more information for the review. Finally, while the entire dissertation may be hundreds of pages long, the chapters on the study conducted may be shorter. The main difficulty in searching for dissertations is there is not one main place to search, but many sources. While Cumulative Index of Nursing and Allied Health Literature CINAHL (EBSCO) have index for some nursing dissertations, it will not index all, and many topics to be considered in IRs will cover a wide variety of fields (Table 3.4).

Table 3.4 Sources of dissertations
Full size table

3.8 Additional Methods of Searching

After determining which types of literature to seek, it is time to select ways of searching for potential literature missed in the database searches.

3.8.1 Handsearching

Handsearching is one way to search for literature. In the past, searchers would physically gather 5–10 years of specific journal issues and flip through the pages looking for relevant literature. Today, this can be done electronically by searching table of contents of journals online. Other resources could also be searched through this process—conference proceedings, websites of relevant organizations, and more.

3.8.2 Citation/Related Article Searching

Reference searching also known as ancestry searching is looking at the reference lists of relevant resources—whether those are included in the review, related reviews, or important background articles to the topic. In addition, consider citation searching, looking for resources that have cited identified relevant resources. Both of these can be done most efficiently using one of the citation-tracking databases—Web of Science, Scopus, or Google Scholar. Out of these options, Scopus covers the most amount of journal titles while offering easy ways to collect and export citations (Mongeon and Paul-Hus 2016). These resources also offer related article searching, linking to other citations by the authors. Other resources also offer this feature, such as PubMed. It is important to use these features strategically, limiting how far away from the article these options are followed. Determine the process and time set aside for this type of search to limit the reach.

3.8.3 Subject Experts

Another way of finding literature not yet discovered through aforementioned methods is to query subject experts and professional organizations. To start, create a description of the review and the literature to be included. Select potential authors, professional organizations, or other places based on the review search to date. Subject experts could be those who wrote the literature already identified to be included or those who have presented at conferences. Consider listing national and international organizations, similar to those considered when seeking conference papers, then determine if these organizations have a forum or listserv. Finally, post to the listserv forum or send the request to individual authors. Not only does this have the potential to yield literature that have been missed, it can also provide a networking opportunity.

3.8.4 Overall Gray Literature Resources

There are a few databases that can be searched for gray literature. Google Scholar is usually the first resource that comes to mind when thinking of this type of resource. Although Google Scholar does cover gray literature, Google can be more inclusive and, when searched strategically, provide a useful set of results (Bonato 2018). When searching Google, it is important to set parameters and a time limit as it could easily retrieve hundreds of thousands of results. Table 3.5 shows examples of a Google search with and without limiters. It is good to set a limit of how many results that will be reviewed, 100–200 to browse through maybe a reasonable number (Table 3.5).

Table 3.5 Google example
Full size table

3.9 Reporting the Search Strategy

Throughout the review process, it is important to carefully document the search conducted in each resource for reporting and updating needs. The most important thing to remember about the search report is that it needs to be presented in a way that each search could be reproducible or can at least be properly evaluated. This will require searches to be provided exactly as they were entered into the database. While saving searches within vendor sites (such as Ovid or Ebsco) is the most efficient way to update searches at a later date, copying and pasting the actual search into a word document is also recommended. Searches should also be described narratively to provide additional details such as limiters that were applied or reasons behind decisions (Table 3.6).

Table 3.6 Examples of reporting styles
Full size table

In addition to providing the database searches, all other search methods need to be clearly described. Any limits applied, dates covered, and dates of the search are important to document so that the scope of the search is clearly described (Table 3.7).

Table 3.7 Documenting the searches/what to report
Full size table

3.9.1 Managing the Collected Data

Reviewers will need to describe the entire search process to the reader. After conducting a comprehensive search, all citations will be retrieved and labeled. This requires project management planning from the beginning of the review by developing protocols for each step and selecting tools to make the process more efficient. The first software to consider is citation manager to use in collecting and de-duplicating results. Next is a tool for sorting the records. While this can be done in the citation manager, other tools offer options to have more than one screener and ease in labeling articles included or excluded. Lastly, there is software designed to manage all of the review processes. Characteristics to consider are how well does the software interface with other software (such as spreadsheets), the time it takes to learn, and the cost. Potential tools to consider are listed in Table 3.8.

Table 3.8 Types and examples of software to consider
Full size table

3.9.2 Screening, Selecting, and Sorting

When sorting the collected citations, there are three processes: screening by relevance, then selecting by full text, and finally sorting into studies. While these steps seem easy, this process can be challenging and time-consuming, and there are pitfalls to avoid. Each of these processes should be piloted before completing. In addition, it is important to document the process for reporting in the methods and results section of the review. The PRISMA flowchart (Pannucci and Wilkins 2010) is one way to display the flow of the records through the review process. It includes the number of records collected, screened, and selected. The reporting needs of each process are described in the sections further.

3.9.2.1 Screening for Relevancy

The first look at the collected citations is done by considering titles and/or abstracts. In this case, the process is to discard what is not relevant, that is, looking for reasons to kick out a citation, not looking to see that it matches all of the eligibility criteria. Abstracts are usually too brief or vague to find all of the required criteria, yet hopefully enough to see what is not relevant.

3.9.2.2 Selection

When selecting by full text, the report must have all of the required criteria. Have a plan in place for where the full text (usually pdf files) will be stored. It is also useful to have a naming structure when saving the files, such as first author followed as part of the title: Smith_PatientsWithDiabeticNeuropathy. While it is good to try and retrieve all of the full-text reports of potential literature, there may be some that cannot be found.

3.9.2.3 Sorting

The last part of this process is to sort the reports into studies. Most of the time one article is reporting on just one research project or study, while there are times when one study is described in more than one report, such as a dissertation and a journal article. Instead of counting those as two studies, it would be counted as one study, using both of the reports to collect data about the project. Sometimes one report is multiple studies, which should be treated separately.

3.9.3 Reporting Results of Screening and Selection

When reporting the results of screening and selection, this should be done both narratively and visually. There are different tools to help produce the PRISMA flowchart. First, PRISMA provides a template on its website (PRISMA Flow Diagram 2015) as a MS Word document or a pdf. One thing to note is that the PRISMA flowchart does not have to be exactly as shown in the template. The idea is to provide the information about the flow and the processes and the numbers, not that it must match the template exactly. Another option is to use a website like the PRISMA flowchart generator (Toronto Health Economics and Technology Assessment Collaborative n.d.) to create a flowchart which can be downloaded in a variety of formats including pdf, gif, and more. Lastly, Microsoft PowerPoint or other software helps in creating flowcharts or diagrams.

3.10 Conclusion

Many consider the search for an IR to be one phase of the review, but it is best to view it as an iterative process. It will require planning and can be greatly enhanced by collaborating with a librarian. There are standards to minimize publication bias and best practices to ensure efficiency. By selecting appropriate resources and software and setting realistic timelines, the search process will be less daunting and provide a comprehensive set of resources on which to base the review.