Introduction

Infants admitted to neonatal intensive care units (NICUs) are vulnerable to pain related to procedures or disease etiology [1], may be more sensitive to pain than children or adults [2], and are often prescribed sedative and analgesic drugs for pain treatment [2,3,4]. Consequently, evaluation of infant pain and sedation is critical for guiding clinical treatment [5]. Heterogeneity of neonatal populations and potential variations among subpopulations (e.g., gestational age, respiratory status, congenital abnormalities, disease etiology) in clinical presentation and experience of pain and sedation must be considered [6]. Thus, pragmatic and psychometrically sound instruments assessing pain and sedation of heterogenous neonatal populations are critical for clinical and research applications.

Giordano et al. [7] identified 59 instruments for measuring infant pain or sedation. Of these, only two assessed both pain and sedation—the COMFORT scale [8] and the Neonatal Pain, Agitation, and Sedation Scale (N-PASS) [9]. N-PASS has demonstrated low risk for bias, multidimensionality, ease and quickness of use, clinician preferability, and is recommended by the American Academy of Pediatrics [5, 7, 10, 11]. N-PASS was developed by Hummel et al. as a multidimensional instrument rating pain and sedation in five domains: crying, behavior, facial expression, extremity tone, and vital signs [9]. For pain, each criterion is graded on a 0–2 scale. Scores on each criterion are summed for a total pain score of 0–10 with higher scores indicating greater pain. Sedation is scored using the same criterion as pain; however, each criterion is graded on a −2 to 0 scale. Scores on each criterion are summed for a total sedation score of −10 to 0, lower scores indicating a deeper level of sedation.

Although N-PASS is recommended and widely used clinically [5, 7], little is known about the extent and breadth of its research use, reliability, and validity. Reliability is the consistency of results and validity is the accuracy of measuring an intended element [12]. A more comprehensive understanding of N-PASS use, reliability, and validity for diverse neonatal subpopulations is needed to guide clinical practice and inform future research.

Methods

Search strategy

Following the preferred reporting items for systematic reviews and meta-analyses guidelines [13], two investigators (MEM and CJS) performed an electronic database search in PubMed, CINAHL, Cochrane, and Embase for articles published up until February 2020. Search terms included “NPASS”, “N-PASS”, “Neonatal Pain, Agitation and Sedation Scale”, “Neonatal Pain, Agitation, and Sedation Scale”, and “Neonatal Pain Agitation and Sedation Scale”. We hand-searched reference lists of included studies. Inclusion criteria were (1) sample age range of 0–12 months; (2) reports use of N-PASS to measure pain and/or sedation; and (3) full text available in the English language. We excluded dissertations, theses, letters to the editor, clinical guidelines, quality improvement reports, commentaries, conference proceedings, published abstracts, and articles not available in the English language.

Data extraction

Two investigators (MEM and CJS) extracted data from included studies using a standardized procedure. Investigators independently assessed each study for availability of data to extract and potential risk of bias. Disagreements were resolved until arriving at consensus. The quality of clinical data was graded from 1 to 5 according to the Oxford Centre for Evidence-Based Medicine [14], which considers study design. We extracted information pertaining to (1) measurement domain (pain (acute or prolonged) and/or sedation); (2) sample size; (3) sample age range; (4) sample ventilatory status if explicitly stated (e.g., mechanical ventilation and no respiratory support); (5) unique sample characteristics if explicitly stated (e.g., postoperative patients); and (6) reported results of reliability and/or validity. For sample age range, we used the categories for preterm birth, as described by the World Health Organization [15, 16]: extremely preterm (<28 weeks), very preterm (28 to <32 weeks), late preterm (32–37 weeks), term (>37 weeks).

Results

Study selection

Our initial database search yielded 169 articles, of which 83 were duplicates, resulting in 86 unique articles. Thirty-two studies were excluded following title and abstract screening. Full texts were retrieved and reviewed for the remaining 54 articles. No additional articles were identified through hand searching. After applying inclusion and exclusion criteria, 29 articles were included in this study (see Fig. 1). The quality of clinical data as graded using the Oxford Centre for Evidence-Based Medicine guidelines [14] is presented in Table 1.

Fig. 1: PRISMA flow diagram.
figure 1

The PRISMA flow diagram demonstrates our study selection process.

Full size image
Table 1 Articles included in review.
Full size table

N-PASS use

Of the 29 studies included in our review, 19 measured pain, 3 measured sedation, and 7 measured both pain and sedation (see Table 1). More specifically, 17 measured acute pain and 11 measured prolonged pain.

Acute pain

N-PASS was used to measure acute (e.g., procedural) pain in all neonatal gestational age categories as defined by WHO, including extremely preterm (n = 5) [17,18,19,20,21], very preterm (n = 5) [17, 18, 21,22,23], late preterm (n = 5) [10, 11, 17, 21, 24], and term infants (n = 7) [17, 21, 25,26,27,28,29]. N-PASS was also used to measure acute pain in infants age 1–36 months (n = 1) [30]. Although most studies were conducted in the NICU (n = 13) [10, 11, 17,18,19,20,21,22,23,24,25, 27, 28], N-PASS was used in other clinical settings, including postanesthesia care unit (PACU) (n = 2) [30, 31], procedure area (n = 1) [26], and the postpartum mother and baby unit (n = 1) [29].

In addition to variation observed by gestational age group and clinical setting, N-PASS was used to measure acute pain in neonatal subpopulations including mechanically ventilated infants (n = 6) [10, 11, 18,19,20,21], nonmechanically ventilated infants (n = 8) [18, 20, 21, 23, 27,28,29, 32], and postoperative infants (n = 2) [30, 32].

Prolonged pain

Similar to acute pain, prolonged pain was measured using N-PASS for all gestational age groups (extremely preterm (n = 5) [31, 33,34,35,36]; very preterm (n = 6) [22, 31, 33, 36,37,38]; late preterm (n = 6) [10, 31, 33, 36, 39, 40]; and term infants (n = 4) [31, 33, 36, 40]). All studies investigating prolonged pain were conducted in NICU settings only. Neonatal subpopulations assessed for prolonged pain using N-PASS included mechanically ventilated infants (n = 8) [10, 33,34,35,36,37, 39, 40], nonmechanically ventilated infants (n = 2) [34, 36], and postoperative infants (n = 2) [33, 39].

Sedation

Nine of the 29 studies reported use of the N-PASS sedation domain. Sedation was measured in extremely preterm (n = 5) [33,34,35,36, 41], very preterm (n = 4) [33, 36, 41, 42], late preterm (n = 6) [33, 36, 40,41,42,43], and term infants (n = 5) [33, 36, 40,41,42]. Sedation was measured in the NICU (n = 8) [33,34,35,36, 40,41,42,43] and PACU setting (n = 1) [30]. N-PASS sedation domain was measured for mechanically ventilated (n = 8) [33,34,35,36, 40,41,42,43], nonmechanically ventilated (n = 2) [34, 36], and postoperative (n = 3) infants [30, 33, 41].

N-PASS reliability

Six of the 29 studies reported reliability metrics of N-PASS for measuring acute pain (n = 3), sedation (n = 1), or pain and sedation (n = 2) (see Table 2 and Fig. 2). Specifically, one measured acute pain and sedation and one measured prolonged pain and sedation. All six studies reported N-PASS internal consistency reliability (Cronbach’s alpha) for either acute pain, prolonged pain, and/or sedation. Five of the six studies additionally reported inter-rater reliability (Cohen’s kappa or intraclass correlation coefficient (ICC)).

Table 2 Demonstrated validity and reliability for N-PASS.
Full size table
Fig. 2: N-PASS validity and reliability by gestational age and infant status.
figure 2

EPT extremely preterm, VPT very preterm, LPT late preterm, T term.

Full size image

Acute pain

Four studies reported reliability of N-PASS for assessing acute pain with internal consistency reliability ranging from good to excellent [44] (α = 0.837–0.971) [11, 21, 24, 30]. Inter-rater reliability was reported in three studies [11, 21, 24] and found to be excellent [45] (ICC = 0.93–0.99). Across the four studies, reliability estimates for using N-PASS to measure acute pain included representation from all gestational age groups (extremely preterm [21], very preterm [21], late preterm [11, 21, 24], and term infants [21]). Additionally, reliability estimates included mechanically ventilated [11, 21], nonmechanically ventilated [21], and postoperative neonates [30].

Prolonged pain

One study reported reliability of N-PASS for measuring prolonged pain of either mechanically ventilated or postoperative infants in all gestational age categories [33]. Between two raters, internal consistency reliability ranged from acceptable to excellent [44] (α = 0.72–0.82) [33] and inter-rater reliability was excellent [45] (r = 0.97).

Sedation

Three studies reported reliability of the sedation domain of N-PASS [30, 33, 41]. Reliability was demonstrated for postoperative or mechanically ventilated infants in all gestational age categories. Of note, one study’s sample included postoperative infants ranging from 1 to 36 months of age [30]. Internal consistency reliability ranged from good to excellent [44] (α = 0.854–0.923). One of these studies demonstrated excellent [45] inter-rater reliability (r = 0.92) [33] and one demonstrated almost perfect agreement [46] (k = 0.93) [41].

N-PASS validity

Five studies reported the validity (e.g., construct and content) of N-PASS for measuring acute pain (n = 1), prolonged pain (n = 1), both prolonged pain and sedation (n = 1), both acute and prolonged pain (n = 1), or both acute pain and sedation (n = 1) (see Table 2 and Fig. 2).

Acute pain

Three studies reported the validity of N-PASS for measuring acute pain [10, 21, 30]. Across the three studies all gestational ages were represented in the sample, including both ventilated and nonventilated infants. One study included a sample of postoperative infants and toddlers aged 1–36 months [30]. Two of the studies evaluated construct validity by comparing N-PASS to the premature infant pain profile (PIPP) [10, 21]. Spearman-rank coefficients ranged from moderate positive to high positive correlations [46] (ρ = 0.62–0.75) [10, 21], suggesting convergent validity. One study demonstrated construct validity by comparing N-PASS to the Face, Legs, Arms, Cry, Consolability Scale (FLACC) [30]. Correlations between N-PASS and FLACC were high (r = 0.980–0.996), indicating convergent validity.

Prolonged pain

Three studies reported the validity of N-PASS for assessing prolonged pain [10, 33, 36]. Construct validity for prolonged pain was evaluated by comparing N-PASS and PIPP in two studies [10, 33] with samples representing either mechanically ventilated or postoperative infants from all gestational age categories. Spearman-rank coefficients ranged from moderate to high positive correlations [47] (ρ = 0.62–0.83) [10, 33], suggesting convergent validity. One study evaluated content validity in a sample of mechanically and nonmechanically ventilated infants’ representative of all gestational age groups [36]. N-PASS pain domain was compared to a subjective questionnaire that captured expert neonatal nurse’s assessments of prolonged pain and a low positive correlation [46] was found (ρ = 0.37) [36].

Sedation

Two studies evaluated the validity of the sedation subsection of N-PASS [30, 36, 41]. In one study, construct validity was evaluated by comparing the N-PASS sedation domain with the University of Michigan Sedation Scale for postoperative infants aged 1–36 months [30]. The correlation ranged from high positive to very high positive [47] (r = 0.847–0.967), suggesting convergent validity. One study, with a sample inclusive of mechanically ventilated postoperative infants from all gestational age categories, evaluated content validity by comparing N-PASS to expert nurse’s assessments [36]. Hillman et al. [36] correlated N-PASS to the bedside nurse’s report, demonstrating a low positive correlation [47] (ρ = −0.39) [36].

Discussion

Many tools are available to clinicians and researchers for evaluating infant pain and/or sedation [7]; however, it is critically important to focus on reliable, valid, and pragmatic scales. Our systematic review is the first to comprehensively describe N-PASS use in published research, including its reported reliability and validity. We noted that N-PASS was used in a variety of settings (e.g., NICU, PACU, and postpartum unit), gestational age groups (e.g., extremely preterm, very preterm, late preterm, term, and 1–36 months), for mechanically and nonmechanically ventilated infants, and postoperative infants, and that it is valid and reliable for assessing acute pain, prolonged pain, and sedation for many of these patients and settings. Despite its breadth of use, however, very few studies reported the reliability and validity of N-PASS, and data are unavailable for its reliability and validity for prolonged pain and sedation of nonmechanically ventilated infants. Further research is warranted on N-PASS use, reliability, and validity in these clinical situations.

Neonatal populations are exceptionally heterogenous, especially infants cared for in the NICU. Broad implementation and use of pain and sedation instruments, like N-PASS, must consider reliability and validity for a variety of neonatal subpopulations. Although our study identified 29 studies using N-PASS, only 6 reported reliability and 5 reported validity of N-PASS. In the 21 studies not reporting any reliability or validity information, authors often referenced previous studies reporting N-PASS reliability and validity. However, in many cases, the sample in the study was arguably different from the one referenced. To expand our knowledge of its reliability and validity, future studies using N-PASS are encouraged to report its psychometric properties for their study and thoroughly describe their study sample.

A recent review by Giordano et al. [7] described existing pain and sedation instruments for neonatal and pediatric patients and included validity and reliability results. Similar to our study, the authors found N-PASS to be used, valid, and reliable for assessing acute pain, prolonged pain, and sedation in extremely preterm, very preterm, late preterm, and term infants. Giordano et al.’s review included four studies on N-PASS [21, 33, 39, 41]; our study builds on this existing evidence by identifying 29 articles and providing a more comprehensive analysis of validity and reliability for N-PASS, specifically. Our data extraction strategy considered the heterogeneity of NICU populations and classified samples into meaningful categories to more comprehensively describe the extent of N-PASS reliability and validity. This is critical because, for example, validation of N-PASS for measuring pain in a late preterm, mechanically ventilated neonate does not equate to validation of N-PASS for measuring pain in all late preterm neonates or all mechanically ventilated neonates. In addition, our review extends the contributions of Giordano et al.’s review by additionally identifying validity and reliability of N-PASS for assessing pain or sedation in postoperative and mechanically ventilated neonates.

Our study found that, when reported, N-PASS demonstrates good reliability and validity for many different neonatal subpopulations, adding further support for the American Academy of Pediatrics’ recommendation to use N-PASS. Thus, clinicians can confidently use N-PASS to asses neonatal acute pain in mechanically ventilated or nonmechanically ventilated infants, prolonged pain in mechanically ventilated or postoperative infants, and sedation in mechanically ventilated or postoperative infants. The reliability and validity of N-PASS for measuring sedation or prolonged pain in nonmechanically ventilated neonates are unknown; however, we recognize that nonmechanically ventilated neonates are rarely sedated and treatment of prolonged pain is rare (e.g., osteogenesis imperfecta and epidermolysis bullosa). Furthermore, Giordano et al. [41] compared N-PASS sedation subscale assessments with the opinion of expert neonatal nurses and concluded that N-PASS reliably identified over-sedation but did not reliably identify under-sedation. Overall, numerous studies have demonstrated good reliability and validity of N-PASS for many neonatal subpopulations. Future research can address subpopulations not identified in this review (e.g., infants with neurologic abnormalities such as hypoxic–ischemic encephalopathy), investigate reliability to detect over- and under-sedation, and test strategies to effectively implement N-PASS in clinical settings to improve measurement and treatment of neonatal pain and sedation.

It is important to consider the efficacy of behavioral assessment tools, such as N-PASS, when measuring pain and sedation in infants with neurologic abnormalities (e.g., encephalopathy, hemorrhage, and asphyxia). Not only do infants at risk for neurologic injury experience a greater number of painful procedures [48], but they also experience neurologic variances at baseline which present challenges when relying on behavioral assessments (e.g., facial expression) [49] to indicate pain and sedation level. Further research is needed to determine reliability and validity of N-PASS in this infant population. It is clinically critical to recognize the importance of accurate pain and sedation assessment with N-PASS (e.g., increased or decreased opiate prescriptions, pharmacologic intervention changes, and longer or shorter ICU admission) [50]. Following high-quality protocols developed for N-PASS, such as the Vienna Protocol for Neonatal Pain and Sedation [50], can be helpful for ensuring precise pain and sedation assessment and standardizing patient care.

While our review has several strengths, we do recognize potential limitations to our study. Most importantly, a meta-analysis was not feasible due to extreme variances in included studies (e.g., sample and study design). We did not include studies published in languages other than English, therefore cross-cultural validation is warranted for wider generalizability. Additionally, information extracted from included studies were limited by the demographic data reported in each study. Consequently, we were unable to describe N-PASS use, reliability, and validity by other important neonatal characteristics (e.g., primary medical diagnoses, gender, and comorbidities). Despite this, our methods are rigorous and the conclusions that we are able to draw remain strong.

Conclusion

Valid and reliable assessment of neonatal pain and sedation is vital for clinical care, treatment, and research. N-PASS is clinically relevant, pragmatic, recommended for use by the American Academy of Pediatrics, and valid and reliable for many neonatal subpopulations, including mechanically ventilated infants of all gestational age categories for both pain (acute and prolonged) and sedation, postoperative infants of all gestational age categories for prolonged pain and sedation, and nonmechanically ventilated infants of all gestational age categories for acute pain. Despite this, our review also identified important research gaps in N-PASS psychometric testing and reporting and encourages increased consideration of the heterogeneity of neonatal populations.