Abstract
Background
Key performance indicators (KPIs) are quantifiable measures used to monitor the quality of health services. Implementation guidelines for clinical pharmacy services (CPS) do not specify KPIs.
Aim
To assess the quality of the studies that have developed KPIs for CPS in inpatient hospital settings.
Method
A systematic review was conducted by searching in Web of Science, Scopus, and PubMed, supplemented with citation analyses and grey literature searches, to retrieve studies addressing the development of KPIs in CPS for hospital inpatients. Exclusions comprised drug- or disease-specific studies and those not written in English, French, Portuguese, or Spanish. The Appraisal of Indicators through Research and Evaluation (AIRE) instrument assessed methodological quality. Domain scores and an overall score were calculated using an equal-weight principle. KPIs were classified into structure, process, and outcome categories. The protocol is available at https://doi.org/10.17605/OSF.IO/KS2G3.
Results
We included thirteen studies that collectively developed 225 KPIs. Merely five studies scored over 50% on the AIRE instrument, with domains #3 (scientific evidence) and #4 (formulation and usage) displaying low scores. Among the KPIs, 8.4% were classified as structure, 85.8% as process, and 5.8% as outcome indicators. The overall methodological quality did not exhibit a clear association with a major focus on outcomes. None of the studies provided benchmarking reference values.
Conclusion
The KPIs formulated for evaluating CPS in hospital settings primarily comprised process measures, predominantly suggested by pharmacists, with inadequate evidence support, lacked piloting or validation, and consequently, were devoid of benchmarking reference values.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Impact statements
-
This systematic review evaluated the quality of key performance indicators for clinical pharmacy through the Appraisal of Indicators through Research and Evaluation (AIRE) instrument.
-
Key performance indicators for clinical pharmacy were developed with insufficient evidence support and lacked piloting or validation.
-
Patients and other healthcare professionals were not included in the consensus panels that developed clinical pharmacy KPIs.
-
Clinical pharmacy KPIs predominantly focus on measuring processes rather than patients' health outcomes.
-
There are no benchmarking reference values available for clinical pharmacy KPIs.
Introduction
Health care quality is defined as the extent to which health services, for individuals and populations, enhance the likelihood of desired health outcomes and align with current professional knowledge [1]. Clinical pharmacy, integral to the healthcare system, is defined by the American College of Clinical Pharmacy (ACCP) as the "area of pharmacy concerned with the science and practice of rational medication use"[2]. While implementation guidelines for clinical pharmacy services (CPS) strongly emphasise enhancing the rational and secure use of drugs, they do not specify indicators for assessing their quality [3,4,5,6].
Differing from quality indicators [7], key performance indicators (KPIs) are quantifiable measures reflecting critical success factors of an organization and are instrumental in monitoring continuous improvement in the quality of health service delivery [8]. Various procedures have been devised for KPI development, ranging from mathematical formulation [9] to the appraisal of linguistic summaries [10]. However, consensus methods involving relevant stakeholders, such as the Delphi technique, are the most employed processes for KPI development [11,12,13,14,15].
While pharmacists collaborate as part of a care team, this collaboration should not hinder the pharmacy profession from quantifying its direct impact on patient care [16]. Clinical pharmacy KPIs aim to encapsulate the values of CPS and demonstrate their impact on patient outcomes to stakeholders, administrators, and multidisciplinary teams. However, as of now, there are no standardized clinical pharmacy KPIs at the national or international level.
Aim
The aim of this systematic review was to assess the quality of the studies that developed KPIs for CPS in inpatient hospital settings.
Method
This systematic review adhered to the Cochrane methodological recommendations [17] and followed the reporting guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [18]. The review protocol was published a priori on the Open Science Framework (OSF) and can be accessed at DOI: https://doi.org/10.17605/OSF.IO/KS2G3.
Literature selection
In July 2023, a systematic search process was undertaken using three bibliographic sources: Web of Science, Scopus, and PubMed. Studies addressing the development, implementation, and benchmarking of KPIs in CPS for hospital inpatients were included. The search strategies are outlined in Supplementary File 1. Additionally, a grey literature search was conducted on Google Scholar to identify studies not indexed in bibliographic databases. Furthermore, a bidirectional systematic citation analysis (backward and forward) of the included studies was performed. The backward retrieval strategy involved searching the references of all included studies, while the forward process entailed searching for articles citing the included studies using the Web of Science. All records were imported into an EndNote database (Clarivate, London, UK), and duplicate records were removed.
Two reviewers independently performed the eligibility phase in two steps. First they screened potentially relevant titles and abstracts. An inter-rater agreement analysis using the prevalence-adjusted and bias-adjusted kappa (PABAK) [19] was conducted during the screening phase, considering Kappa values over 0.7 as acceptable. Discrepancies were resolved through a consensus meeting. Secondly, full-text papers not excluded during the screening phase underwent further evaluation, and studies were excluded based on the following criteria: (1) studies lacking detailed reporting of clinical pharmacy KPIs; (2) studies solely addressing KPIs related to drug- or disease-specific conditions (e.g., infections, cancer, etc.); (3) studies written in languages other than English, French, Portuguese, or Spanish; (4) articles reporting studies conducted outside the hospital inpatient environment; and (5) studies published as editorials, letters to the editor, commentaries, or narrative case reports. Multiple articles reporting the results of the same study were consolidated.
Data extraction and quality appraisal
Articles were appraised by one researcher to extract pertinent information, including: 1) author and year of publication; 2) country where the study was conducted; 3) objectives; 4) methods employed for KPI development; 5) composition of the team responsible for KPI evaluation; 6) number of KPIs directly related to CPS; and 7) benchmarking. We decided not contacting the researchers from the different KPI developing teams and relying only in the information they had provided in their published articles.
The Appraisal of Indicators through Research and Evaluation (AIRE) instrument [20] was employed to assess the methodological quality of the studies included in this review. AIRE, a validated instrument designed for indicator quality assessment [21], was initially derived from the AGREE (Appraisal of Guidelines Through Research and Evaluation) instrument [22]. Utilized in several systematic reviews on indicator quality [23,24,25,26], the AIRE instrument comprises 20 items across four domains: "Purpose, relevance, and organizational context," "Stakeholder involvement," "Scientific evidence," and "Additional evidence, formulation, and usage." Each item presents a statement about the quality of the indicator and is scored on a 4-point scale (1 ‘totally disagree or no information provided’ to 4 ‘strongly agree’). Two authors independently applied the AIRE instrument to each included study. Subsequently, a standardized domain score was calculated per AIRE guidelines using the formula: (total score − minimum possible score)/(maximum score − minimum possible score) × 100%. Likewise, an overall standardized score was computed using the same method across all instrument items. In both cases, a higher standardized score indicates better methodological quality (range 0–100%). Studies were classified with high methodological quality in each domain if they scored ≥ 50%, corresponding to an overall “agree” or “strongly agree” [27].
KPI categorization
The KPIs proposed in the included studies were extracted and categorized according to Donabedian’s structure, process, or outcome (SPO) paradigm [28]. Structure measures encompass various professional and organizational resources associated with care provision. Process measures pertain to the actions clinicians do to and for patients to enhance or maintain health. Outcome measures signify changes in physical status resulting from care processes, such as morbidity, mortality, and improved quality of life. Furthermore, outcome KPIs were sub-categorized based on Kozma et al.'s economic, clinical, and humanistic outcomes (ECHO) model [29]. Economic outcomes include direct, indirect, and intangible costs; clinical outcomes are defined as disease or treatment-related results; and humanistic outcomes encompass the effects of illness or treatment on a patient's quality of life and functional status. Three authors independently conducted both categorizations, and any disagreements were resolved through discussion between the authors.
Results
The initial search identified 531 different records, excluding duplicates. During the screening phase, 488 records were deemed irrelevant, leaving 43 for full-text review. The inter-rater agreement between independent screeners was considered sufficient (PABAK = 0.816). Following the full-text assessment, ten articles corresponding to nine studies met the inclusion criteria and were considered for analysis. Additionally, three other articles were retrieved from citation analyses (i.e., backward and forward searches), and one was obtained from the grey literature search. Ultimately, 14 articles, corresponding to 13 studies, were included in this review (Fig. 1) [30,31,32,33,34,35,36,37,38,39,40,41,42,43].
Study characteristics
Table 1 provides an overview of the studies included in this review. The articles, published between 2010 and 2023, are distributed across all continents: six in Europe, two in America, two in Asia, two in Oceania, and one in Africa. Notably, three studies focused on specific CPS, as declared by the authors: Doerper et al. and Aljamal et al. addressed "medication reconciliation," while King et al. concentrated on "transitions of care."
Most studies involved a comprehensive KPI development process, which included an appraisal of evidence from the literature, followed by an expert panel consensus. Delphi (regular or modified) was the most prevalent method for KPI development, with eight studies utilizing this approach. One study combined nominal and focus group techniques, while two studies adapted consensus indicators previously published by the American College of Clinical Pharmacy (AACP) and the American Society of Health-System Pharmacy (ASHP). Additionally, one study employed an online survey with pharmacists, and another study did not provide information regarding the KPI development approach.
Pharmacists were the predominant stakeholders in the Key Performance Indicator (KPI) development teams, represented in 12 (92%) studies. Additionally, four studies included a minority of other professionals such as physicians, nurses, and risk/quality assurance managers. One study did not provide information about the composition of stakeholders. Notably, none of the studies reported the participation of patients, payers, or policymakers.
Several studies emphasized the significance of standardizing and benchmarking KPIs. The reasons supporting this recommendation included: (1) enabling meaningful comparisons of CPS within and between organizations; (2) facilitating performance management of hospital pharmacists; and (3) enhancing transparency about the quality of hospital pharmacy services. Despite the acknowledgement of benchmark importance, none of the studies provided any.
Critical appraisal
Table 2 presents the scores for the methodological quality of the included studies using the AIRE instrument. Notably, there is a wide variation in the information and level of detail describing the methodological characteristics of the studies developing KPIs. In domain #1 (Purpose, relevance, and organizational context), 11 studies (85%) scored above the 50% threshold, with an overall mean score of 64%. The item "the quality domain the indicator addresses is described in detail," representing the care quality aspect of the indicator (e.g., patient safety, care effectiveness, timeliness, equality, patient-centeredness, etc.), received the lowest score among the studies. In domain #2 (Stakeholder involvement), 10 studies (77%) demonstrated good quality, with an overall mean score of 54%. Two items in this domain scored particularly low: stakeholders' involvement and formal endorsement of the indicator set. Domains #3 (Scientific evidence) and #4 (Additional evidence, formulation, and usage) received the lowest scores, with overall means of 39% and 35%, respectively. Only five studies (38%) and three studies (23%) exceeded the 50% threshold in these domains. In domain #3, the item "supporting evidence has been critically appraised" scored the lowest, while in domain #4, the lowest scores were related to risk adjustment, accuracy, and consistency of the measures, as well as the piloting of the indicators.
Overall, two studies achieved high methodological quality in all four AIRE domains, while one study scored below the 50% threshold across all four AIRE domains. In the overall standardized score, encompassing all items from the four domains, only five studies scored higher than the 50% threshold, whereas three studies scored around 35%.
KPI description and categorization
In total, 225 KPIs were extracted from the 13 studies. A complete list of these 225 KPI is available at Supplementary File 2. The reported number of KPIs in each study ranged from 6 to 41 (median = 16). Following Donabedian’s SPO paradigm [44], 19 KPIs (8.4%) were classified as structure, 193 (85.8%) as process, and 13 (5.8%) as outcome indicators. Among the outcome KPIs, 4 (1.8%) were economic, 7 (3.1%) were clinical, and 2 (0.9%) pertained to the humanistic dimensions of the ECHO model. Only one study developed KPIs spanning all three SPO categories. Merely three studies, contributing to 49 KPIs, described them using Donabedian’s SPO framework. Among these 49 KPIs, authors' classifications aligned with ours in 41 instances, with the remaining 8 originally classified as outcome KPIs by the authors but as process KPIs by our team. Notably, none of the studies utilized the ECHO model to categorize the KPIs. A comprehensive description of the KPIs is provided in Supplementary File 3.
Structure KPIs concentrated on the organizational aspects of the service and included supportive activities (9 KPIs in 3 studies), availability of protocol/policy (5 KPIs in 2 studies), technical and human resources (4 KPIs in 4 studies), and professional qualification (1 KPI in 1 study).
Process KPIs could be categorized into ten services: medication reconciliation (75 KPIs in 12 studies), medication review (46 KPIs in 10 studies), pharmacist advice (25 KPIs in 8 studies), care team involvement (16 KPIs in 5 studies), patient education (14 KPIs in 7 studies), care plan implementation (7 KPIs in 4 studies), dispensing (4 KPIs in 4 studies), outpatient services (2 KPIs in 2 studies), student mentoring (2 KPIs in 2 studies), and care team members’ satisfaction (2 KPIs in 1 study).
Among the 7 clinical outcomes KPIs, the focus was on adverse drug reactions (3 KPIs in 2 studies), patient readmission (3 KPIs in 1 study), and length of stay (1 KPI in 1 study). The 4 economic outcomes KPIs included cost savings (3 KPIs in 2 studies) and cost of therapy (1 KPI in 1 study). Additionally, the 2 humanistic outcomes KPIs aimed at patient satisfaction (2 KPIs in 2 studies).
A higher prevalence of outcome KPIs in a study was not associated with the overall methodological quality assessed by the AIRE. Among the five studies scoring above the 50% threshold, only two developed any outcome KPI. Notably, the study that produced KPIs from all three SPO categories scored below the 50% threshold.
Discussion
Statement of key findings
A total of 225 clinical pharmacy KPIs for hospital inpatient settings were identified in 13 studies conducted across all continents. The methodological quality and applicability of the KPI sets exhibited considerable variability, with a notable weakness in the supporting scientific evidence and a lack of piloting and post-development evaluation. Less than 6% of the KPIs developed in these studies were classified as outcome KPIs.
Strengths and weaknesses
To our knowledge, this is the first systematic review focusing on KPI development studies in hospital settings. Other reviews have been conducted in different contexts [45] or have focused on specific subjects [46, 47]. Additionally, this study marks the first application of the AIRE instrument to evaluate KPIs in this setting. While other studies have applied AIRE to KPI development studies in specific conditions or activities [22, 46], the emphasis here is on generalist KPIs that are adaptable for use in different countries, facilitating international comparisons. Similarly to what happens in other assessment metrics, the debate between generalist or specific KPIs still persists [48, 49]. We preferred considering generalist KPI because they can be used in economic evaluations, allowing comparisons between CPS and any other health care technology. Although clinical pharmacy is not exclusively performed in hospital environments [50], the specificities of the hospital setting might justify the development of hospital centred KPIs to evaluate the quality of CPS.
This study has limitations. Firstly, our eligibility criteria excluded disease- or drug-specific studies, potentially limiting the variety of KPIs. Nevertheless, the generalist characteristic of the included studies can be considered a strength. Like any evidence-gathering exercise, some relevant literature may not have been identified. To mitigate this potential limitation, we conducted searches in the three most widely used bibliographic databases, performed a thorough citation analysis both backward and forward, and conducted an extensive grey literature search. And, as with any methodological quality appraisal assessment, the results of the AIRE are influenced by the comprehensiveness of the studies' reporting. Poor reporting can lead to underestimating the methodological quality of any study. Additionally, AIRE is an instrument with limited methodological guidance available, leaving room for users' interpretation [22]. Despite the general belief that AIRE domains should not be aggregated into an overall score, as no confirmatory factorial analysis evaluated its fit to a pre-established structure, we opted to create an overall score attributing equal weight to each domain. While this approach could be subject to debate, we supported it based on the same rationale that AIRE attributes equal weight to items in each domain.
Interpretation
In general, the quality of the studies that developed KPIs for CPS in hospital setting was low, with an AIRE score of around 45%. Two items were poorly scored: evidence supporting the KPIs and the lack of piloting and post-development evaluation. Our findings align with previous analyses in areas such as school feeding programs [23], long-term care [26], or dental care [24]. Higher scores in domain #1 (Purpose, relevance, and context) suggest that KPI developers have a clear vision of their objectives. In domain #2 (Stakeholder involvement), the difference between the higher scores of items 2.1 and 2.2 may indicate that KPI developers included highly relevant stakeholders but may have overlooked some essential ones, which is consistent with a review of quality indicators for community care for older people [25]. The poor evidence supporting the developed KPIs may signify a need for more research, not only before KPI development but also during the development process.The low scores in domain #4 are associated with a poor description of the developed KPIs, likely due to inadequate research and evidence support, and the absence of a piloting exercise to validate the new indicators. This observation aligns with previous literature. [25]. In summary, it appears that KPIs are developed by highly motivated and expert steering groups, possibly working in isolation from some stakeholders. The ideas behind these KPIs may not be sufficiently supported by scientific evidence, and there is a lack of adequate validation before publishing the development report[51].
An important step in KPI development is achieving consensus to select indicators. The Delphi technique facilitates the development of quality indicators in areas where the evidence is insufficient or controversial, synthesizing accumulated expert opinion [52, 53]. A key issue to strengthen the consensus and enhance the credibility and acceptability of the agreed KPIs is ensuring the involvement of panellists including all potential stakeholders [54]. The selection of appropriate stakeholders in healthcare innovation has been demonstrated to be complex [55]. In our review, eight studies (61%) adopted the Delphi technique, but only four included professionals other than pharmacists.
Less than 6% of the compiled KPIs were classified as outcome-related KPIs, a finding consistent with other analyses of KPI development studies [46]. The limited focus on outcome KPIs may be associated with the unclear aim of clinical pharmacy towards health outcomes. The abridged definition of clinical pharmacy provided by the ACCP omits any mention of health outcomes. In contrast, their unabridged definition states that clinical pharmacy "optimizes medication therapy and promotes health, wellness, and disease prevention" [2]. While optimizing therapy is part of the care process, the remaining three goals can be considered as health outcomes. The European Society of Clinical Pharmacy's definition mentions the optimization of medicines' utilization "in order to achieve person-centred and public health goals," which are likely health outcomes [56]. Although clinical pharmacy is considered to "embrace the philosophy of pharmaceutical care," leading societies were not as explicit in outcome orientation as pharmaceutical care definitions [57, 58]. The ambiguity between these two concepts remains unresolved [50, 59].
Despite the straightforward interpretation of process indicators [46], there is a pressing need for a greater focus on health outcomes when developing KPIs for CPS. While association between process and outcome is expected, assuming a linear correlation cannot be taken for granted [28]. Salampessy et al. demonstrated that structure and process are frequently not correlated with outcome indicators [60]. The lack of clarity regarding the goals of any clinical activity, including CPS, has been criticized for fostering "gaming of the system" and impeding progress in performance improvement [61]. Process indicators have been described as having limited impact on value and garnering little attention from patients [62]. Measuring performance in healthcare should always entail measuring health outcomes. Like other healthcare areas [63], KPIs for CPS should be outcome oriented. An outcome "denotes the effects of care on the health status of patients and populations" [28]. This implies that some measures frequently described as outcomes are, in fact, process indicators (e.g., unnecessary drug use, medication complexity, medication adherence, drug-related problems) [64, 65]. This incorrect classification was common among the KPI development exercises.
Implementing KPI evaluation incurs costs that should be considered during the KPI development process. While recognizing the need for KPI assessment, clinical pharmacists have expressed concerns about the burden associated with KPI implementation [66]. In our review, the number of KPIs developed in each exercise varied from 6 to 41 (median = 16). Meyer et al. advocate for applying a parsimony principle to focus on measuring what matters, based on end-user needs [67].
Further research
Our study revealed three major deficiencies in the clinical pharmacy KPI development processes: the absence of internationally standardized KPIs; a predominant focus on assessing processes rather than outcomes; and a complete lack of benchmarking reference values. To enhance the accountability of CPS, addressing these three deficiencies is essential. This can be achieved by creating a concise list of internationally accepted KPIs with benchmarking reference values to set minimum practice standards [68]. The use of publicly reported KPIs has been identified as a driver of higher quality levels [69].
In value-based health care systems, the assessment of performance should rely exclusively on outcome measures [62]. Outcome measures serve in cost-effectiveness analyses, demonstrating the cost–benefit ratio of CPS, but also in economic evaluations, showcasing CPS as the optimal option in certain care processes [70]. Selecting the appropriate outcomes to assess the quality of CPS poses a major challenge. The debate surrounding the validity of surrogate outcomes and hard outcomes remains unresolved in any health care field [71]. As Donabedian noted, measuring survival or mortality in non-fatal situations would not be useful [44]. Establishing a core outcome set for use in CPS KPIs should be a significant focus of clinical pharmacy research. These outcomes should be sensitive enough to possess discriminatory power when CPS are implemented and relevant enough to facilitate comparisons with other health care interventions.
Conclusion
Our study reveals that the clinical pharmacy KPIs developed for the evaluation of CPS in hospital settings were predominantly process measures, crafted by expert panels mainly composed of pharmacists, lacking robust scientific evidence support, and were not piloted or validated, thereby lacking benchmarking reference values. Future efforts should focus on creating an internationally standardized core outcome set to effectively measure the quality of CPS and facilitate comparisons of their value with other healthcare activities.
References
Farquhar M. AHRQ Quality Indicators. In: Hughes RG, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Advances in Patient Safety. Rockville (MD): AHRQ 2008.
American College of Clinical Pharmacy. The definition of clinical pharmacy. Pharmacotherapy. 2008;28(6):816–7. https://doi.org/10.1592/phco.28.6.816.
ASHP Statement on Pharmaceutical Care. Am J Health Syst Pharm. 1993;50(8):1720–3. https://doi.org/10.1093/ajhp/50.8.1720.
Royal Pharmaceutical Society. Professional standards for hospital pharmacy services. https://www.rpharms.com/Portals/0/RPS%20document%20library/Open%20access/Professional%20standards/Professional%20standards%20for%20Hospital%20pharmacy/rps-professional-standards-for-hospital-pharmacy.pdf. Accessed 31 Dec 2023.
The European Statements of Hospital Pharmacy. Eur J Hosp Pharm. 2014;21(5):256–8. https://doi.org/10.1136/ejhpharm-2014-000526.
Paudyal V, Okuyan B, Henman MC, et al. Scope, content and quality of clinical pharmacy practice guidelines: a systematic review. Int J Clin Pharm. 2024;46(1):56–69. https://doi.org/10.1007/s11096-023-01658-x.
Fujita K, Halvorsen KH, Sato N, et al. Pharmaceutical Care Network Europe definition of quality indicators for pharmaceutical care: a systematic literature review and international consensus development. Int J Clin Pharm. 2023;46:70-9. https://doi.org/10.1007/s11096-023-01631-8.
Should key performance indicators for clinical services be mandatory? Can J Hosp Pharm. 2011;64(1):55–7. https://doi.org/10.4212/cjhp.v64i1.989
van Schaik MO, Sucu S, Cappon HJ, et al. Mathematically formulated key performance indicators for design and evaluation of treatment trains for resource recovery from urban wastewater. J Environ Manag. 2021;282: 111916. https://doi.org/10.1016/j.jenvman.2020.111916.
Gilsing R, Wilbik A, Grefen P, et al. Defining business model key performance indicators using intentional linguistic summaries. Softw Syst Model. 2021;20(4):965–96. https://doi.org/10.1007/s10270-021-00894-x.
Morar PS, Hollingshead J, Bemelman W, et al. Establishing Key Performance Indicators [KPIs] and their importance for the surgical management of inflammatory bowel disease-results from a pan-European, Delphi consensus study. J Crohns Colitis. 2017;11(11):1362–8. https://doi.org/10.1093/ecco-jcc/jjx099.
Shawahna R. Development of key performance indicators to capture in measuring the impact of pharmacists in caring for patients with epilepsy in primary healthcare: a Delphi consensual study. Epilepsy Behav. 2019;98(Pt A):129–38. https://doi.org/10.1016/j.yebeh.2019.07.034.
Mellett C, O’Donovan A, Hayes C. The development of outcome key performance indicators for systemic anti-cancer therapy using a modified Delphi method. Eur J Cancer Care Engl. 2020;29(4): e13240. https://doi.org/10.1111/ecc.13240.
Salgado TM, Reynolds TN, Frankart LM, et al. A key performance indicators redefinition initiative at a school of pharmacy using a modified Delphi consensus technique. Pharm Pract. 2020;18(4):2120. https://doi.org/10.18549/PharmPract.2020.4.2120.
Sato N, Fujita K, Kushida K, et al. Development and consensus testing of quality indicators for geriatric pharmacotherapy in primary care using a modified Delphi study. Int J Clin Pharm. 2022;44(2):517–38. https://doi.org/10.1007/s11096-022-01375-x.
Cassat S, Massey L, Buckingham S, et al. Development of health-system inpatient pharmacy clinical metrics. Am J Health Syst Pharm. 2019;76(23):1958–64. https://doi.org/10.1093/ajhp/zxz225.
Cochrane Handbook for Systematic Reviews of Interventions. https://training.cochrane.org/handbook/current. Accessed 31 Dec 2023.
Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372: n71. https://doi.org/10.1136/bmj.n71.
Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–9. https://doi.org/10.1016/0895-4356(93)90018-v.
de Koning J, Smulders A, Klazinga N. Appraisal of Indicators through Research and Evaluation (AIRE). Amsterdam: Academic Medical Center-University of Amsterdam; 2007.
de Koning J. Development and validation of a measurement instrument for appraising indicator quality: appraisal of indicators through research and evaluation (AIRE) instrument. https://www.egms.de/static/en/meetings/gmds2007/07gmds798.shtml. Accessed 31 Dec 2023.
De Roo ML, Leemans K, Claessen SJ, et al. Quality indicators for palliative care: update of a systematic review. J Pain Symptom Manag. 2013;46(4):556–72. https://doi.org/10.1016/j.jpainsymman.2012.09.013.
Bicalho D, Santos TSS, Slater B, et al. Evaluation of quality indicators for management of the National School Feeding Program in Brazil: a systematic review. Cien Saude Colet. 2021;26(8):3099–110. https://doi.org/10.1590/1413-81232021268.03802020.
Righolt AJ, Sidorenkov G, Faggion CM Jr, et al. Quality measures for dental care: a systematic review. Community Dent Oral Epidemiol. 2019;47(1):12–23. https://doi.org/10.1111/cdoe.12429.
Joling KJ, van Eenoo L, Vetrano DL, et al. Quality indicators for community care for older people: a systematic review. PLoS ONE. 2018;13(1): e0190298. https://doi.org/10.1371/journal.pone.0190298.
Osinska M, Favez L, Zuniga F. Evidence for publicly reported quality indicators in residential long-term care: a systematic review. BMC Health Serv Res. 2022;22(1):1408. https://doi.org/10.1186/s12913-022-08804-7.
Strudwick K, Nelson M, Martin-Khan M, et al. Quality indicators for musculoskeletal injury management in the emergency department: a systematic review. Acad Emerg Med. 2015;22(2):127–41. https://doi.org/10.1111/acem.12591.
Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743–8. https://doi.org/10.1001/jama.260.12.1743.
Kozma CM, Reeder CE, Schulz RM. Economic, clinical, and humanistic outcomes: a planning model for pharmacoeconomic research. Clin Ther. 1993;15(6):1121–32.
Ng J, Harrison J. Key performance indicators for clinical pharmacy services in New Zealand public hospitals: stakeholder perspectives. J Pharm Health Ser Res. 2010;1(2):75–84. https://doi.org/10.1111/j.1759-8893.2010.00001.x.
Doerper S, Morice S, Piney D, et al. La conciliation des traitements médicamenteux: logigramme d’une démarche efficiente pour prévenir ou intercepter les erreurs médicamenteuses à l’admission du patient hospitalisé. Pharm Hosp Clin. 2013;48(3):153–60. https://doi.org/10.1016/j.phclin.2013.03.006.
Fernandes O, Gorman SK, Slavik RS, et al. Development of clinical pharmacy key performance indicators for hospital pharmacists using a modified Delphi approach. Ann Pharmacother. 2015;49(6):656–69. https://doi.org/10.1177/1060028015577445.
Fernandes O, Gorman SK, Slavik RS, et al. Development of clinical pharmacy key performance indicators for hospital pharmacists using a modified Delphi approach. Can J Hosp Pharm. 2016;69(1):55.
Aljamal MS, Ashcroft D, Tully MP. Development of indicators to assess the quality of medicines reconciliation at hospital admission: an e-Delphi study. Int J Pharm Pract. 2016;24(3):209–16. https://doi.org/10.1111/ijpp.12234.
Lloyd GF, Singh S, Barclay P, et al. Hospital pharmacists’ perspectives on the role of key performance indicators in Australian pharmacy practice. J Pharm Pract Res. 2016;47(2):87–95. https://doi.org/10.1002/jppr.1156.
Cillis M, Spinewine A, Krug B, et al. Development of a tool for benchmarking of clinical pharmacy activities. Int J Clin Pharm. 2018;40(6):1462–73. https://doi.org/10.1007/s11096-018-0725-6.
Krzyzaniak N, Pawlowska I, Bajorek B. Quality pharmacy services and key performance indicators in Polish NICUs: a Delphi approach. Int J Clin Pharm. 2018;40(3):533–42. https://doi.org/10.1007/s11096-018-0623-y.
Shawahna R. Development of key performance indicators for capturing impact of pharmaceutical care in palestinian integrative healthcare facilities: a Delphi consensus study. Evid Based Complement Alternat Med. 2020;2020:7527543. https://doi.org/10.1155/2020/7527543.
Al-Jazairi AS, Alnakhli AO. Quantifying clinical pharmacist activities in a tertiary care hospital using key performance indicators. Hosp Pharm. 2021;56(4):321–7. https://doi.org/10.1177/0018578719897074.
King PK, Burkhardt C, Rafferty A, et al. Quality measures of clinical pharmacy services during transitions of care. J Am Coll Clin Pharm. 2021;4(7):883–907. https://doi.org/10.1002/jac5.1479.
Lopes H, Lopes AR, Farinha H, et al. Defining clinical pharmacy and support activities indicators for hospital practice using a combined nominal and focus group technique. Int J Clin Pharm. 2021;43(6):1660–82. https://doi.org/10.1007/s11096-021-01298-z.
Ruiz Ramos J, Calderon Hernanz B, Castellanos Clemente Y, et al. Pharmacist care in hospital emergency departments: a consensus paper from the Spanish hospital pharmacy and emergency medicine associations. Emergencias. 2023;35(3):205–17.
Anene-Okeke C. Clinical pharmacy Key Performance Indicators (cpKPI) for hospital pharmacists in Nigeria. J Basic Clin Pharm. 2022;13(4):180–5. https://doi.org/10.37532/0976-0113.13(4).180.
Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3 Suppl):166–206. https://doi.org/10.2307/3348969.
Lima TM, Aguiar PM, Storpirtis S. Evaluation of quality indicator instruments for pharmaceutical care services: a systematic review and psychometric properties analysis. Res Social Adm Pharm. 2018;14(5):405–12. https://doi.org/10.1016/j.sapharm.2017.05.011.
O’Riordan F, Shiely F, Byrne S, et al. Quality indicators for hospital antimicrobial stewardship programmes: a systematic review. J Antimicrob Chemother. 2021;76(6):1406–19. https://doi.org/10.1093/jac/dkab034.
Aung AK, Walker S, Khu YL, et al. Adverse drug reaction management in hospital settings: review on practice variations, quality indicators and education focus. Eur J Clin Pharmacol. 2022;78(5):781–91. https://doi.org/10.1007/s00228-022-03287-1.
Nazar H, Nazar Z, Yeung A, et al. Consensus methodology to determine minor ailments appropriate to be directed for management within community pharmacy. Res Soc Adm Pharm. 2018;14(11):1027–42. https://doi.org/10.1016/j.sapharm.2018.01.001.
Lemanska A, Poole K, Manders R, et al. Patient activation and patient-reported outcomes of men from a community pharmacy lifestyle intervention after prostate cancer treatment. Support Care Cancer. 2022;30(1):347–58. https://doi.org/10.1007/s00520-021-06404-5.
Dreischulte T, Fernandez-Llimos F. Current perceptions of the term clinical pharmacy and its relationship to pharmaceutical care: a survey of members of the European Society of Clinical Pharmacy. Int J Clin Pharm. 2016;38(6):1445–56. https://doi.org/10.1007/s11096-016-0385-3.
Greenwood D, Tully MP, Martin S, et al. Development of the Manchester framework for the evaluation of emergency department pharmacy services. Int J Clin Pharm. 2022;44(4):930–8. https://doi.org/10.1007/s11096-022-01403-w.
Boulkedid R, Abdoul H, Loustau M, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE. 2011;6(6): e20476. https://doi.org/10.1371/journal.pone.0020476.
Canning ML, Barras M, McDougall R, et al. Defining quality indicators, pharmaceutical care bundles and outcomes of clinical pharmacy service delivery using a Delphi consensus approach. Int J Clin Pharm. 2024. https://doi.org/10.1007/s11096-023-01681-y.
Freitas A, Santana P, Oliveira MD, et al. Indicators for evaluating European population health: a Delphi selection process. BMC Public Health. 2018;18(1):557. https://doi.org/10.1186/s12889-018-5463-0.
Franco-Trigo L, Fernandez-Llimos F, Martinez-Martinez F, et al. Stakeholder analysis in health innovation planning processes: a systematic scoping review. Health Policy. 2020;124(10):1083–99. https://doi.org/10.1016/j.healthpol.2020.06.012.
Dreischulte T, van den Bemt B, Steurbaut S, et al. European Society of Clinical Pharmacy definition of the term clinical pharmacy and its relationship to pharmaceutical care: a position paper. Int J Clin Pharm. 2022;44(4):837–42. https://doi.org/10.1007/s11096-022-01422-7.
Hepler CD, Strand LM. Opportunities and responsibilities in pharmaceutical care. Am J Hosp Pharm. 1990;47(3):533–43.
Allemann SS, van Mil JW, Botermann L, et al. Pharmaceutical care: the PCNE definition 2013. Int J Clin Pharm. 2014;36(3):544–55. https://doi.org/10.1007/s11096-014-9933-x.
Ahmed SI, Hasan SS, Hassali MA. Clinical pharmacy and pharmaceutical care: a need to homogenize the concepts. Am J Pharm Educ. 2010;74(10):193g.
Salampessy BH, Portrait FRM, van der Hijden E, et al. On the correlation between outcome indicators and the structure and process indicators used to proxy them in public health care reporting. Eur J Health Econ. 2021;22(8):1239–51. https://doi.org/10.1007/s10198-021-01333-w.
Porter ME, Larsson S, Lee TH. Standardizing patient outcomes measurement. N Engl J Med. 2016;374(6):504–6. https://doi.org/10.1056/NEJMp1511701.
Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–81. https://doi.org/10.1056/NEJMp1011024.
Townsend JS, Moore AR, Mulder TN, et al. What does a performance measurement system tell us about the national comprehensive cancer control program? J Public Health Manag Pract. 2015;21(5):449–58. https://doi.org/10.1097/PHH.0000000000000124.
Kjeldsen LJ, Olesen C, Hansen MK, et al. Clinical Outcomes Used in Clinical Pharmacy Intervention Studies in Secondary Care. Pharmacy. 2017. https://doi.org/10.3390/pharmacy5020028.
Fernandez-Llimos F, Faus MJ. From “drug-related problems” to “negative clinical outcomes.” Am J Health Syst Pharm. 2005;62(22):2348. https://doi.org/10.2146/ajhp050358.
Minard LV, Deal H, Harrison ME, et al. Pharmacists’ perceptions of the barriers and facilitators to the implementation of clinical pharmacy Key Performance Indicators. PLoS ONE. 2016;11(4): e0152903. https://doi.org/10.1371/journal.pone.0152903.
Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf. 2012;21(11):964–8. https://doi.org/10.1136/bmjqs-2012-001081.
Agency for Healthcare Research and Quality. AHRQ quality indicator tools for data analytics https://www.ahrq.gov/data/qualityindicators/index.html. Accessed 30 Dec 2023.
Chassin MR, Loeb JM, Schmaltz SP, et al. Accountability measures–using measurement to promote quality improvement. N Engl J Med. 2010;363(7):683–8. https://doi.org/10.1056/NEJMsb1002320.
Tonin FS, Aznar-Lou I, Pontinha VM, et al. Principles of pharmacoeconomic analysis: the case of pharmacist-led interventions. Pharm Pract. 2021;19(1):2302. https://doi.org/10.18549/PharmPract.2021.1.2302.
Manyara AM, Davies P, Stewart D, et al. Definitions, acceptability, limitations, and guidance in the use and reporting of surrogate end points in trials: a scoping review. J Clin Epidemiol. 2023;160:83–99. https://doi.org/10.1016/j.jclinepi.2023.06.013.
Funding
Open access funding provided by FCT|FCCN (b-on). This study was partially funded by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)—Brazil (ref: DPG No 0010/2023).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
No conflicts of interest existed for this study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Magedanz, L., Silva, H.L., Galato, D. et al. Clinical pharmacy key performance indicators for hospital inpatient setting: a systematic review. Int J Clin Pharm 46, 602–613 (2024). https://doi.org/10.1007/s11096-024-01717-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11096-024-01717-x