Abstract
Background
Well-defined, systematic, and transparent processes to identify health research gaps, needs, and priorities are vital to ensuring that available funds target areas with the greatest potential for impact.
Objective
The purpose of this review is to characterize methods conducted or supported by research funding organizations to identify health research gaps, needs, or priorities.
Method
We searched MEDLINE, PsycINFO, and the Web of Science up to September 2019. Eligible studies reported on methods to identify health research gaps, needs, and priorities that had been conducted or supported by research funding organizations. Using a published protocol, we extracted data on the method, criteria, involvement of stakeholders, evaluations, and whether the method had been replicated (i.e., used in other studies).
Results
Among 10,832 citations, 167 studies were eligible for full data extraction. More than half of the studies employed methods to identify both needs and priorities, whereas about a quarter of studies focused singularly on identifying gaps (7%), needs (6%), or priorities (14%) only. The most frequently used methods were the convening of workshops or meetings (37%), quantitative methods (32%), and the James Lind Alliance approach, a multi-stakeholder research needs and priority setting process (28%). The most widely applied criteria were importance to stakeholders (72%), potential value (29%), and feasibility (18%). Stakeholder involvement was most prominent among clinicians (69%), researchers (66%), and patients and the public (59%). Stakeholders were identified through stakeholder organizations (51%) and purposive (26%) and convenience sampling (11%). Only 4% of studies evaluated the effectiveness of the methods and 37% employed methods that were reproducible and used in other studies.
Discussion
To ensure optimal targeting of funds to meet the greatest areas of need and maximize outcomes, a much more robust evidence base is needed to ascertain the effectiveness of methods used to identify research gaps, needs, and priorities.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
BACKGROUND
Well-defined, systematic, and transparent methods to identify health research gaps, needs, and priorities are vital to ensuring that available funds target areas with the greatest potential for impact.1,2 As defined in the literature,3,4 research gaps are defined as areas or topics in which the ability to draw a conclusion for a given question is prevented by insufficient evidence. Research gaps are not necessarily synonymous with research needs, which are those knowledge gaps that significantly inhibit the decision-making ability of key stakeholders, who are end users of research, such as patients, clinicians, and policy makers. The selection of research priorities is often necessary when all identified research gaps or needs cannot be pursued because of resource constraints. Methods to identify health research gaps, needs, and priorities (from herein referred to as gaps, needs, priorities) can be multi-varied and there does not appear to be general consensus on best practices.3,5
Several published reviews highlight the diverse methods that have been used to identify gaps and priorities. In a review of methods used to identify gaps from systematic reviews, Robinson et al. noted the wide range of organizing principles that were employed in published literature between 2001 and 2009 (e.g., care pathway, decision tree, and patient, intervention, comparison, outcome framework,).6 In a more recent review spanning 2007 to 2017, Nyanchoka et al. found that the vast majority of studies with a primary focus on the identification of gaps (83%) relied solely on knowledge synthesis methods (e.g., systematic review, scoping review, evidence mapping, literature review). A much smaller proportion (9%) relied exclusively on primary research methods (i.e., quantitative survey, qualitative study).7
With respect to research priorities, in a review limited to a PubMed database search covering the period from 2001 to 2014, Yoshida documented a wide range of methods to identify priorities including the use of not only knowledge synthesis (i.e., literature reviews) and primary research methods (i.e., surveys) but also multi-stage, structured methods such as Delphi, Child Health and Nutrition Research Initiative (CHNRI), James Lind Alliance Priority Setting Partnership (JLA PSP), and Essential National Health Research (ENHR).2 The CHNRI method, originally developed for the purpose of setting global child health research priorities, typically employs researchers and experts to specify a long list of research questions, the criteria that will be used to prioritize research questions, and the technical scoring of research questions using the defined criteria.8 During the latter stages, non-expert stakeholders’ input are incorporated by using their ratings of the importance of selected criteria to weight the technical scores. The ENHR method, initially designed for health research priority setting at the national level, involves researchers, decision-makers, health service providers, and communities throughout the entire process of identifying and prioritizing research topics.9 The JLA PSP method convenes patients, carers, and clinicians to equally and jointly identify questions about healthcare that cannot be answered by existing evidence that are important to all groups (i.e., research needs).10 The identified research needs are then prioritized by the groups resulting in a final list (often a top 10) of research priorities. Non-clinical researchers are excluded from voting on research needs or priorities but can be involved in other processes (e.g., knowledge synthesis). CHNRI, ENHR, and JLA PSP usually employ a mix of knowledge synthesis and primary research methods to first identify a set of gaps or needs that are then prioritized. Thus, even though CHNRI, ENHR, and JLA PSP have been referred to as priority setting methods, they actually consist of a gaps or needs identification stage that feeds into a research prioritization stage.
Nyanchoka et al.’s review found that the majority of studies focused on the identification of gaps alone (65%), whereas the remaining studies focused either on research priorities alone (17%) or on both gaps and priorities (19%).7 In an update to Robinson et al.’s review,6 Carey et al. reviewed the literature between 2010 and 2011 and observed that the studies conducted during this latter period of time focused more on research priorities than gaps and had increased stakeholder involvement, and that none had evaluated the reproducibility of the methods.11
The increasing development and diversity of formal processes and methods to identify gaps and priorities are indicative of a developing field.2,12 To facilitate more standardized and systematic processes, other important areas warrant further investigation. Prior reviews did not distinguish between the identification of gaps versus research needs. The Agency for Healthcare Research and Quality Evidence-based Practice Center (AHRQ EPC) Program issued a series of method papers related to establishing research needs as part of comparative effectiveness research.13,14,15 The AHRQ EPC Program defined research needs as “evidence gaps” identified within systematic reviews that are prioritized by stakeholders according to their potential impact on practice or care.16 Furthermore, Nyanchoka et al. relied on author designations to classify studies as focusing on gaps versus research priorities and noted that definitions of gaps varied across studies, highlighting the need to apply consistent taxonomy when categorizing studies in reviews.7 Given the rise in the use of stakeholders in both gaps and prioritization exercises, a greater understanding of the range of practices involving stakeholders is also needed. This includes the roles and responsibilities of stakeholders (e.g., consultants versus final decision-makers), the composition of stakeholders (e.g., non-research clinicians, patients, caregivers, policymakers), and the methods used to recruit stakeholders. The lack of consensus of best practices also highlights the importance of learning the extent to which evaluations to determine the effectiveness of gaps, needs, and prioritization exercises have been conducted, and if so, what were the resultant outcomes.
To better inform efforts and organizations that fund health research, we conducted a scoping review of methods used to identify gaps, needs, and priorities that were linked to potential or actual health research funding decision-making. Hence, this scoping review was limited to studies in which the identification of health research gaps, needs, or priorities was supported or conducted by funding organizations to address the following questions1: What are the characteristics of methods to identify health research gaps, needs, and priorities? and2 To what extent have evaluations of the impact of these methods been conducted? Given that scoping reviews may be executed to characterize the ways an area of research has been conducted,17,18 this approach is appropriate for the broad nature of this study’s aims.
METHODS
Protocol and Registration
We employed methods that conform to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews.19 See Appendix A in the Supplementary Information. The scoping review protocol is registered with the Open Science Framework (https://osf.io/5zjqx/).
Eligibility Criteria
Studies published in English that described methods to identify health research gaps, needs, or priorities that were supported or conducted by funding organizations were eligible for inclusion. We excluded studies that reported only the results of the exercise (e.g., list of priorities) absent of information on the methods used. We also excluded studies involving evidence synthesis (e.g., literature or systematic reviews) that were solely descriptive and did not employ an explicit method to identify research gaps, needs, or priorities.
Information Sources and Search Strategy
We searched the following electronic databases: MEDLINE, PsycINFO, and Web of Science. Our database search also included an update of the Nyanchoka et al. scoping review, which entailed executing their database searches for the time period following 2017 (the study’s search end date).7 Nyanchoka et al. did not include database searches for research needs. The electronic database search and scoping review update were completed in August and September 2019, respectively. The search strategy employed for each of the databases is presented in Appendix B in the Supplementary Information.
Selection of Sources of Evidence and Data Charting Process
Two reviewers screened titles and abstracts and full-text publications. Citations that one or both reviewers considered potentially eligible were retrieved for full-text review. Relevant background articles and scoping and systematic reviews were reference mined to screen for eligible studies. Full-text publications were screened against detailed inclusion and exclusion criteria. Data was extracted by one reviewer and checked by a second reviewer. Discrepancies were resolved through discussion by the review team.
Information on study characteristics were extracted from each article including the aims of the exercise (i.e., gaps, needs, priorities, or a combination) and health condition (i.e., physical or psychological). Based on definitions in the literature,3,4,5 the aims of the exercise were coded according to the activities that were conducted, which may not have always corresponded with the study authors’ labeling of the exercises. For instance, the JLA PSP method is often described as a priority exercise but we categorized it as a needs and priority exercise. Priority exercises can be preceded by exercises to identify gaps or needs, which then feed into the priority exercise such as in JLA PSP; however, standalone priority exercises can also be conducted (e.g., stakeholders prioritize an existing list of emerging diseases).
For each type of exercise, information on the methods were recorded. An initial list of methods was created based on previous reviews.9,12,20 During the data extraction process, any methods not included in the initial list were subsequently added. If more than one exercise was reported within an article (e.g., gaps and priorities), information was extracted for each exercise separately. Reviewers extracted the following information: methods employed (e.g., qualitative, quantitative), criteria used (e.g., disease burden, importance to stakeholders), stakeholder involvement (e.g., stakeholder composition, method for identifying stakeholders), and whether an evaluation was conducted on the effectiveness of the exercise (see Appendix C in the Supplementary Information for full data extraction form).
Synthesis of results entailed quantitative descriptives of study characteristics (e.g., proportion of studies by aims of exercise) and characteristics of methods employed across all studies and by each type of study (e.g., gaps, needs, priorities).
RESULTS
The electronic database search yielded a total of 10,548 titles. Another 284 articles were identified after searching the reference lists of full-text publications, including three systematic reviews21,22,23 and one scoping review24 that had met eligibility criteria. Moreover, a total of 99 publications designated as relevant background articles were also reference mined to screen for eligible studies. We conducted full-text screening for 2524 articles, which resulted in 2344 exclusions (440 studies were designated as background articles). A total of 167 exercises related to the identification of gaps, needs, or priorities that were supported or conducted by a research funding organization were described across 180 publications and underwent full data extraction. See Figure 1 for the flow diagram of our search strategy and reasons for exclusion.
Characteristics of Sources of Evidence
Among the published exercises, the majority of studies (152/167) conducted gaps, need, or prioritization exercises related to physical health, whereas only a small fraction of studies focused on psychological health (12/167) (see Appendix D in the Supplementary Information).
Methods for Identifying Gaps, Needs, and Priorities
As seen in Table 1, only about a quarter of studies involved a singular type of exercise with 7% focused on the identification of gaps only (i.e., areas with insufficient information to draw a conclusion for a given question), 6% on needs only (i.e., knowledge gaps that inhibit the decision-making of key stakeholders), and 14% priorities only (i.e., ranked gaps or needs often because of resource constraints). Studies more commonly conducted a combination of multiple types of exercises with more than half focused on the identification of both research needs and priorities, 14% on gaps and priorities, 3% gaps, needs, and priorities, and 3% gaps and needs.
Across the 167 studies, the three most frequently used methods were the convening of workshops/meetings/conferences (37%), quantitative methods (32%), and the JLA PSP approach (28%). This was followed by methods involving literature reviews (17%), qualitative methods (17%), consensus methods (13%), and reviews of source materials (15%). Other methods included the CHNRI process (7%), reviews of in-progress data (7%), consultation with (non-researcher) stakeholders (4%), applying a framework tool (4%), ENHR (1%), systematic reviews (1%), and evidence mapping (1%).
The criterion most widely applied across the 167 studies was the importance to stakeholders (72%) (see Table 2). Almost one-third (29%) considered the potential value and 18% feasibility as criteria. Burden of disease (9%), addressing inequities (8%), costs (6%), alignment with organization’s mission (3%), and patient centeredness (2%) were adopted as criteria to a lesser extent.
About two-thirds of the studies included researchers (66%) and clinicians (69%) as stakeholders (see Appendix E in the Supplementary Information). Patients and the public were involved in 59% of the studies. A smaller proportion included policy makers (20%), funders (13%), product makers (8%), payers (5%), and purchasers (2%) as stakeholders. Nearly half of the studies (51%) relied on stakeholder organizations to identify stakeholders (see Appendix F in the Supplementary Information). A quarter of studies (26%) used purposive sampling and some convenience sampling (11%). Few (9%) used snowball sampling to identify stakeholders. Only a minor fraction of studies, seven of the 167 (4%), reported some type of effectiveness evaluation.25,26,27,28,29,30,31
DISCUSSION
Our scoping review revealed that approaches to identifying gaps, needs, and priorities are less likely to occur as discrete processes and more often involve a combination of exercises. Approaches encompassing multiple exercises (e.g., gaps and needs) were far more prevalent than singular standalone exercises (e.g., gaps only) (73% vs. 27%). Findings underscore the varying importance placed on gaps, needs, and priorities, which reflect key principles of the Value of Information approach (i.e., not all gaps are important, addressing gaps do not necessarily address needs nor does addressing needs necessarily address priorities).32
Findings differ from Nyanchoka et al.’s review in which studies involving the identification of gaps only outnumbered studies involving both gaps and priorities.7 However, Nyanchoka et al. relied on author definitions to categorize exercises, whereas our study made designations based on our review of the activities described in the article and applied definitions drawn from the literature.3,4 Lack of consensus on definitions of gaps and priority setting has been noted in the literature.33,34 To the authors’ knowledge, no prior scoping review has focused on methods related to the identification of “research needs.” Findings underscore the need to develop and apply more consistent taxonomy to this growing field of research.
More than 40% of studies employed methods with a structured protocol including JLA PSP, ENHR, CHRNI, World Café, and the Dialogue model.10,35,36,37,38,39,40 The World Café and Dialogue models particularly value the experiential perspectives of stakeholders. The World Café centers on creating a special environment, often modeled after a café, in which rounds of multi-stakeholder, small group, conversations are facilitated and prefaced with questions designed for the specific purpose of the session. Insights and results are reported and shared back to the entire group with no expectation to achieve consensus, but rather diverse perspectives are encouraged.36 The Dialogue model is a multi-stakeholder, participatory, priority setting method involving the following phases: exploratory (informal discussions), consultation (separate stakeholder consultations), prioritization (stakeholder ratings), and integration (dialog between stakeholders).39 Findings may indicate a trend away from non-replicable methods to approaches that afford greater transparency and reproducibility.41 For instance, of the 17 studies published between 2000 and 2009, none had employed CHNRI and 6% used JLA PSP compared to the 141 studies between 2010 and 2019 in which 8% applied CHNRI and 32% JLA PSP. However, notable variations in implementing CHNRI and JLA PSP have been observed.41,42,43 Though these protocols help to ensure a more standardized process, which is essential when testing the effectiveness of methods, such evaluations are infrequent but necessary to establish the usefulness of replicable methods.
Convening workshops, meetings, or conferences was the method used by the greatest proportion of studies (37%). The operationalization of even this singular method varied widely in duration (e.g., single vs. multi-day conferences), format (e.g., expert panel presentations, breakout discussion groups), processes (e.g., use of formal/informal consensus methods), and composition of stakeholders. The operationalization of other methods (e.g., quantitative, qualitative) also exhibited great diversity.
The use of explicit criteria to determine gaps, needs, or priorities is a key component of certain structured protocols40,44 and frameworks.9,45 In our scoping review, the criterion applied most frequently across studies (71%) was “importance to stakeholders” followed by potential value (31%) and feasibility (18%). Stakeholder values are being incorporated into the identification of gaps, needs, and exercises across a significant proportion of studies, but how this is operationalized varies widely across studies. For instance, the CHNRI typically employs multiple criteria that are scored by technical experts and these scores are then weighted based on stakeholder ratings of their relative importance. Other studies totaled scores across multiple criteria, whereas JLA PSP asks multiple stakeholders to rank the top ten priorities. The importance of involving stakeholders, especially patients and the public, in priority setting is increasingly viewed as vital to ensuring the needs of end users are met,46,47 particularly in light of evidence demonstrating mismatches between the research interests of patients and researchers and clinicians.48,49,50 In our review, clinicians (69%) and researchers (66%) were the most widely represented stakeholder groups across studies. Patients and the public (e.g., caregivers) were included as stakeholders in 59% of the studies. Only a small fraction of studies involved exercises in which stakeholders were limited to researchers only. Patients and the public were involved as stakeholders in 12% of studies published between 2000 and 2009 compared to 60% of studies between 2010 and 2019. Findings may reflect a trend away from researchers traditionally serving as one of the sole drivers of determining which research topics should be pursued.
More than half of the studies reported relying on stakeholder organizations to identify participants. Partnering with stakeholder organizations has been noted as one of the primary methods for identifying stakeholders for priority setting exercises.34 Purposive sampling was the next most frequently used stakeholder identification method. In contrast, convenience sampling (e.g., recommendations by study team) and snowball sampling (e.g., identified stakeholders refer other stakeholders who then refer additional stakeholders) were not as frequently employed, but were documented as common methods in a prior review conducted almost a decade ago.14 The greater use of stakeholder organizations than convenience or snowball sampling may be partly due to the more recent proliferation of published studies using structured protocols like JLA PSP, which rely heavily on partnerships with stakeholder organizations. Though methods such as snowball sampling may introduce more bias than random sampling,14 there are no established best practices for stakeholder identification methods.51 Nearly a quarter of studies provided either unclear or no information on stakeholder identification methods, which has been documented as a barrier to comparing across studies and assessing the validity of research priorities.34
Determining the effectiveness of gaps, needs, and priority exercises is challenging given that outcome evaluations are rarely conducted. Only seven studies reported conducting an evaluation.25,26,27,28,29,30,31 Evaluations varied with respect to their focus on process- (e.g., balanced stakeholder representation, stakeholder satisfaction) versus outcome-related impact (e.g., prioritized topics funded, knowledge production, benefits to health). There is no consensus on what constitutes optimal outcomes, which has been found to vary by discipline.52
More than 90% of studies involved exercises related to physical health in contrast to a minor portfolio of work being dedicated to psychological health, which may be an indication of the low priority placed on psychological health policy research. Understanding whether funding decisions for physical versus psychological health research are similarly or differentially governed by more systematic, formal processes may be important to the extent that this affects the effective targeting of funds.
Limitations
By limiting studies to those supported or conducted by funding organizations, we may have excluded global, national, or local priority setting exercises. In addition, our scoping review categorized approaches according to the actual exercises conducted and definitions provided in the scientific literature rather than relying on the terminology employed by studies. This resulted in instances in which the category assigned to an exercise within our scoping review could diverge from the category employed by the study authors. Lastly, this study’s findings are subject to limitations often characteristic of scoping reviews such as publication bias, language bias, lack of quality assessment, and search, inclusion, and extraction biases.53
Conclusions
The diversity and growing establishment of formal processes and methods to identify health research gaps, needs, and priorities are characteristic of a developing field. Even with the emergence of more structured and systematic approaches, the inconsistent categorization and definition of gaps, needs, and priorities inhibit efforts to evaluate the effectiveness of varied methods and processes, such efforts are rare and sorely needed to build an evidence base to guide best practices. The immense variation occurring within structured protocols, across different combinations of disparate methods, and even within singular methods, further emphasizes the importance of using clearly defined approaches, which are essential to conducting investigations of the effectiveness of these varied approaches. The recent development of reporting guidelines for priority setting for health research may facilitate more consistent and clear documentation of processes and methods, which includes the many facets of involving stakeholders.34 To ensure optimal targeting of funds to meet the greatest areas of need and maximize outcomes, a much more robust evidence base is needed to ascertain the effectiveness of methods used to identify research gaps, needs, and priorities.
References
Chalkidou K, Whicher D, Kary W, Tunis SR. Comparative Effectiveness Research Priorities: Identifying Critical Gaps in Evidence for Clinical and Health Policy Decision Making. International journal of technology assessment in health care. 2009;25(3):241-8. doi:https://doi.org/10.1017/S0266462309990225
Yoshida S. Approaches, Tools and Methods Used for Setting Priorities in Health Research in the 21(st) Century. Journal of global health. 2016;6(1):010507-010507. doi:https://doi.org/10.7189/jogh.06.010507
Robinson KA, Saldanha IJ, McKoy NA. Development of a Framework to Identify Research Gaps from Systematic Reviews. Journal of clinical epidemiology. 2011;64(12):1325-30. [Comment in: J Clin Epidemiol. 2013 May;66(5):522-3; [https://www.ncbi.nlm.nih.gov/pubmed/23265604]]. doi:https://doi.org/10.1016/j.jclinepi.2011.06.009
Saldanha IJ, Wilson LM, Bennett WL, Nicholson WK, Robinson KA. Development and Pilot Test of a Process to Identify Research Needs from a Systematic Review. J Clin Epidemiol. May 2013;66(5):538-45. doi:https://doi.org/10.1016/j.jclinepi.2012.07.009
Viergever RF, Terry R, Matsoso M. Health research prioritization at WHO: an overview of methodology and high level analysis of WHO led health research priority setting exercises. Geneva: World Health Organization. 2010;
Robinson KA, Akinyede O, Dutta T, et al. Framework for Determining Research Gaps During Systematic Review: Evaluation. 2013. AHRQ Methods for Effective Health Care.
Nyanchoka L, Tudur-Smith C, Thu VN, Iversen V, Tricco AC, Porcher R. A Scoping Review describes Methods Used to Identify, Prioritize and Display Gaps in Health Research. J Clin Epidemiol. Jan 30 2019;doi:https://doi.org/10.1016/j.jclinepi.2019.01.005
Rudan I, Gibson JL, Ameratunga S, et al. Setting priorities in global child health research investments: guidelines for implementation of CHNRI method. Croatian medical journal. 2008;49(6):720-33.
Viergever RF, Olifson S, Ghaffar A, Terry RF. A Checklist for Health Research Priority Setting: Nine Common Themes of Good Practice. Health Res Policy Syst. Dec 15 2010;8:36. doi:https://doi.org/10.1186/1478-4505-8-36
James Lind Alliance. The James Lind Alliance Guidebook. March 2020. http://www.jla.nihr.ac.uk/jla-guidebook/downloads/JLA-Guidebook-V9-download-March-2020.pdf
Carey TY, A.; Beadles, C.; Wines, R. Prioritizing Future Research through Examination of Research Gaps in Systematic Reviews. 2012.
Carey T, Yon A, Beadles C, Wines R. Prioritizing future research through examination of research gaps in systematic reviews. Prepared for the Patient-Centered Outcomes Research Institute. 2012;
Carey TS, Sanders GD, Viswanathan M, Trikalinos TA, Kato E, Chang S. Framework for Considering Study Designs for Future Research Needs. Methods Future Research Needs Paper No. 8 (Prepared by the RTI–UNC Evidence-based Practice Center under Contract No. 290-2007- 10056-I.) AHRQ Publication No. 12-EHC048-EF. Rockville, MD: Agency for Healthcare Research and Quality. March 2012. Framework for Considering Study Designs for Future Research Needs. https://www.ncbi.nlm.nih.gov/pubmed/22624168
O’Haire C, McPheeters M, Nakamoto E, et al. Methods for Engaging Stakeholders To Identify and Prioritize Future Research Needs. Methods Future Research Needs Report No. 4. (Prepared by the Oregon Evidence-based Practice Center and the Vanderbilt Evidence-based Practice Center under Contract No. 290-2007-10057-I.) AHRQ Publication No. 11-EHC044-EF. Rockville, MD: Agency for Healthcare Research and Quality. Vol. Methods Future Research Needs Reports. 2011. June 2011. https://www.ncbi.nlm.nih.gov/books/NBK62571/
Trikalinos T, Dahabreh I, Lee J, Moorthy D. Methods Research on Future Research Needs: Defining an Optimal Format for Presenting Research Needs. Methods Future Research Needs Report No. 3. (Prepared by the Tufts Evidence-based Practice Center under Contract No. 290-2007-10057-I.) AHRQ Publication No. 11-EHC027-EF. Rockville, MD: Agency for Healthcare Research and Quality. June 2011. www.effectivehealthcare.ahrq.gov/reports/final.cfm.
Kane RL, Guise JM, Hartman K, Rothenberg B, Trikalinos T, Wilt T. Presentation of Future Research Needs. 2012. AHRQ Methods for Effective Health Care.
Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic Review or Scoping Review? Guidance for Authors When Choosing Between a Systematic or Scoping Review Approach. journal article. BMC Medical Research Methodology. November 19 2018;18(1):143. doi:https://doi.org/10.1186/s12874-018-0611-x
Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for Conducting Systematic Scoping Reviews. Int J Evid Based Healthc. Sep 2015;13(3):141-6. doi:https://doi.org/10.1097/xeb.0000000000000050
Tricco AC, Lillie E, Zarin W, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. The PRISMA-ScR Statement. Annals of Internal Medicine. 2018;169(7):467-473. doi:https://doi.org/10.7326/m18-0850
Tomlinson M, Chopra M, Hoosain N, Rudan I. A review of selected research priority setting processes at national level in low and middle income countries: towards fair and legitimate priority setting. Health Research Policy and Systems. 2011;9(1):19.
Rylance J, Pai M, Lienhardt C, Garner P. Priorities for tuberculosis research: a systematic review. Lancet Infect Dis. 2010;10(12):886-892.
Tong A, Chando S, Crowe S, et al. Research priority setting in kidney disease: a systematic review. American journal of kidney diseases: the official journal of the National Kidney Foundation. 2015;65(5):674-83. [Comment in: Am J Kidney Dis. 2015 May;65(5):641-3; [https://www.ncbi.nlm.nih.gov/pubmed/25919496]]. doi:https://doi.org/10.1053/j.ajkd.2014.11.011
Tong A, Sautenet B, Chapman JR, et al. Research priority setting in organ transplantation: a systematic review. Transplant international: official journal of the European Society for Organ Transplantation. 2017;30(4):327-343. doi:https://doi.org/10.1111/tri.12924
Bourne AM, Johnston RV, Cyril S, et al. Scoping Review of Priority Setting of Research Topics for Musculoskeletal Conditions Review. BMJ Open. 2018;8(12):9 e023962. doi:https://doi.org/10.1136/bmjopen-2018-023962
Bennett WL, Nicholson WK, Saldanha IJ, Wilson LM, McKoy NA, Robinson KA. Future Research Needs for the Management of Gestational Diabetes. 2010. AHRQ Future Research Needs Papers.
Buckley BS, Grant AM, Glazener CMA. Case study: a patient-clinician collaboration that identified and prioritized evidence gaps and stimulated research development. Journal of clinical epidemiology. 2013;66(5):483-9. doi:https://doi.org/10.1016/j.jclinepi.2011.03.016
Elberse JE, Pittens CACM, de Cock Buning T, Broerse JEW. Patient Involvement in a Scientific Advisory Process: Setting the Research Agenda for Medical Products. Health policy (Amsterdam, Netherlands). 2012;107(2-3):231-42. doi:https://doi.org/10.1016/j.healthpol.2012.05.014
Fun WH, Sararaks S, Tan EH, et al. Research funding impact and priority setting - advancing universal access and quality healthcare research in Malaysia. BMC Health Services Research. Apr 24 2019;19(1):248.
Husereau D, Boucher M, Noorani H. Priority Setting for Health Technology Assessment at CADTH. Article. International Journal of Technology Assessment in Health Care. 2010;26(3):341-347. doi:https://doi.org/10.1017/s0266462310000383
Mador RL, Kornas K, Simard A, Haroun V. Using the Nine Common Themes of Good Practice Checklist as a Tool for Evaluating the Research Priority Setting Process of a Provincial Research and Program Evaluation Program. Health research policy and systems. 2016;14:22. doi:https://doi.org/10.1186/s12961-016-0092-5
Sanders GD, Powers B, Crowley M, et al. Future Research Needs for Angiotensin Converting Enzyme Inhibitors or Angiotensin II Receptor Blockers Added to Standard Medical Therapy for Treating Stable Ischemic Heart Disease: Identification of Future Research Needs from Comparative Effectiveness Review No. 18. Future Research Needs for Angiotensin Converting Enzyme Inhibitors or Angiotensin II Receptor Blockers Added to Standard Medical Therapy for Treating Stable Ischemic Heart Disease: Identification of Future Research Needs from Comparative Effectiveness Review No 18. 2010. AHRQ Future Research Needs Papers.
Minelli C, Baio G. Value of Information: a Tool to Improve Research Prioritization and Reduce Waste. PLos Med. 2015;12(9):e1001882-e1001882. doi:https://doi.org/10.1371/journal.pmed.1001882
Bryant J, Sanson-Fisher R, Walsh J, Stewart J. Health Research Priority Setting in Selected High Income Countries: a Narrative Review of Methods Used and Recommendations for Future Practice. Cost Eff Resour Alloc. 2014;12:23. doi:https://doi.org/10.1186/1478-7547-12-23
Tong A, Synnot A, Crowe S, et al. Reporting Guideline for Priority Setting of Health Research (REPRISE). BMC Med Res Methodol. Dec 28 2019;19(1):243. doi:https://doi.org/10.1186/s12874-019-0889-3
Uneke CJ, Ezeoha AE, Ndukwe CD, Oyibo PG, Onwe F, Aulakh BK. Research Priority Setting for Health Policy and Health Systems Strengthening in Nigeria: the Policymakers and Stakeholders Perspective and Involvement. The Pan African medical journal. 2013;16:10. doi:https://doi.org/10.11604/pamj.2013.16.10.2318
MacFarlane A, Galvin R, O’Sullivan M, et al. Participatory Methods for Research Prioritization in Primary Care: an Analysis of the World Cafe Approach in Ireland and the USA. Family practice. 2017;34(3):278-284. doi:https://doi.org/10.1093/fampra/cmw104
Nelson ML, McKellar KA, Munce S, et al. Addressing the Evidence Gap in Stroke Rehabilitation for Complex Patients: a Preliminary Research Agenda. Archives of physical medicine and rehabilitation. 2018;99(6):1232-1241. doi:https://doi.org/10.1016/j.apmr.2017.08.488
Restall GJ, Carnochan TN, Roger KS, Sullivan TM, Etcheverry EJ, Roddy P. Collaborative Priority Setting for Human Immunodeficiency Virus Rehabilitation Research: a Case Report. Canadian journal of occupational therapy Revue canadienne d’ergotherapie. 2016;83(1):7-13. doi:https://doi.org/10.1177/0008417415577423
Abma TA, Broerse JEW. Patient Participation as Dialogue: Setting Research Agendas. Health expectations: an international journal of public participation in health care and health policy. 2010;13(2):160-73. doi:https://doi.org/10.1111/j.1369-7625.2009.00549.x
Rudan I. Setting Health Research Priorities Using the CHNRI Method: IV. Key conceptual advances. J Glob Health. 2016;6(1):010501. doi:https://doi.org/10.7189/jogh-06-010501
Yoshida S, Wazny K, Cousens S, Chan KY. Setting Health Research Priorities Using the CHNRI Method: III Involving Stakeholders. J Glob Health. Jun 2016;6(1):010303. doi:https://doi.org/10.7189/jogh.06.010303
Rudan I, Yoshida S, Chan KY, et al. Setting Health Research Priorities Using the CHNRI Method: VII. A Review of the First 50 Applications of the CHNRI Method. J Glob Health. 2017;7(1):011004. doi:https://doi.org/10.7189/jogh.07.011004
Nygaard A, Halvorsrud L, Linnerud S, Grov EK, Bergland A. The James Lind Alliance Process Approach: Scoping Review. BMJ Open. 2019;9(8):e027473. doi:https://doi.org/10.1136/bmjopen-2018-027473
Okello D, Chongtrakul P, The COHRED Working Group on Priority Setting. A Manual for Research Priority Setting using the ENHR Strategy. Reference mining. Accessed July 24, 2019. http://www.cohred.org/downloads/578.pdf
Fadlallah R, El-Harakeh A, Bou-Karroum L, et al. A Common Framework of Steps and Criteria for Prioritizing Topics for Evidence Syntheses: a Systematic Review. J Clin Epidemiol. 2020;120:67-85. doi:https://doi.org/10.1016/j.jclinepi.2019.12.009
Natale CV, Gross D. The ROI of engaged patients. Healthc Financ Manage. 2013;67(8):90-7.
Clavisi O, Bragge P, Tavender E, Turner T, Gruen RL. Effective Stakeholder Participation in Setting Research Priorities Using a Global Evidence Mapping Approach. Journal of clinical epidemiology. 2013;66(5):496-502.e2. doi:https://doi.org/10.1016/j.jclinepi.2012.04.002
Tallon D, Chard J, Dieppe P. Relation Between Agendas of the Research Community and the Research Consumer. Lancet. 2000;355(9220):2037-40. doi:https://doi.org/10.1016/S0140-6736(00)02351-5
Crowe S, Fenton M, Hall M, Cowan K, Chalmers I. Patients’, Clinicians’ and the Research Communities’ Priorities for Treatment Research: There is an Important Mismatch. Res Involv Engagem. 2015;1:2. doi:https://doi.org/10.1186/s40900-015-0003-x
Kapiriri L, Tomlinson M, Gibson J, et al. Setting priorities in global child health research investments: addressing values of stakeholders. Croatian medical journal. 2007;48(5):618-627.
Guise J-M, O’Haire C, McPheeters M, et al. A Practice-Based Tool for Engaging Stakeholders in Future Research: a Synthesis of Current Practices. Journal of clinical epidemiology. 2013;66(6):666-74. doi:https://doi.org/10.1016/j.jclinepi.2012.12.010
Sibbald SL, Singer PA, Upshur R, Martin DK. Priority Setting: What Constitutes Success? A Conceptual Framework for Successful Priority Setting. Article. Bmc Health Services Research. 2009;9:12. 43. doi:https://doi.org/10.1186/1472-6963-9-43
Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A Scoping Review of Scoping Reviews: Advancing the Approach and Enhancing the Consistency. Research synthesis methods. 2014;5(4):371-385. doi:https://doi.org/10.1002/jrsm.1123
Acknowledgements
This scoping review is part of research that was sponsored by Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury (now Psychological Health Center of Excellence).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
ESM 1
(PDF 1205 kb)
Rights and permissions
About this article
Cite this article
Wong, E.C., Maher, A.R., Motala, A. et al. Methods for Identifying Health Research Gaps, Needs, and Priorities: a Scoping Review. J GEN INTERN MED 37, 198–205 (2022). https://doi.org/10.1007/s11606-021-07064-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-021-07064-1