Introduction

Dysphagia is a common complication of stroke worldwide, with a prevalence of 42%. Patients with dysphagia have a four times higher risk of aspiration pneumonia, a leading cause of death in patients with stroke [1, 2]. Early dysphagia screening reduces the incidence of aspiration pneumonia [3, 4]; it is essential to improve the patient’s nutritional status and reduce the risk of aspiration pneumonia [5,6,7]. The gold standard tests for assessing dysphagia are the videofluoroscopic swallowing study (VFSS) and fiberoptic endoscopic evaluation of swallowing (FEES); however, both these tests require specialized equipment and experts, and VFSS is associated with the risk of radiation exposure and is difficult to perform in patients immediately after a stroke [8,9,10]. In contrast, FEES does not pose a risk of radiation exposure and can be performed at the bedside; however, it has limited ability to assess the oral cavity and esophagus and has been associated with side effects such as choking, vomiting, laryngospasm, and vasovagal syncope [11, 12]. Gugging Swallowing Screen, 3-oz water swallowing test, bedside swallowing assessment, and Toronto Bedside Swallowing Screening Test have been developed and used as screening tests for swallowing to simplify such assessments. However, as previous studies have shown, these tests primarily assess only the extent of aspiration [13,14,15]. Normal swallowing is the result of coordinated movements of several involved organs; the organic and coordinated movement of the muscles of the mouth, pharynx, larynx, and vocal cords prevents penetration or aspiration of the bolus [16]. In this context, the Practical Assessment of Dysphagia (PAD) tool was developed to predict the risk of dysphagia in a quantitative and organ-specific manner [17]. The PAD test can identify the organs causing dysphagia and allow for organ-specific swallowing therapy based on targeting the involved organs. Studies have reported that this type of individualized treatment is highly effective in treating dysphagia [18]. Therefore, the investigators aimed to use the PAD test in combination with the VFSS to screen and diagnose dysphagia in patients with stroke and use the results to guide organ-specific treatment. In view of this future goal, this study was designed to verify the reliability and validity of the PAD test.

Materials and Methods

The PAD Test

The PAD is a dysphagia assessment tool developed in 2015; it consists of a total of 18 assessment items; the total possible score is 100 points, and higher scores reflect better swallowing function (Appendix 1). The 18 items are designed to assess the organs and other components involved in swallowing function, as follows: (1) Cognition; (2) Respiration; (3) Lip; (4) Tongue; (5) Chin; (6) Soft palate; (7) Vocal cord; (8) Swallowing. The assessment results are organized to enable the rapid identification of the swallowing organs that are deficient; this information can be used to target treatment [16]. In addition, when the total score is 85 or higher, the area under the receiver operating characteristic (ROC) curve for the evaluation of aspiration is 0.80, and the sensitivity is 100%, making it a useful screening test [17].

Subjects

This prospective study included patients aged ≥ 18 years who were referred to the rehabilitation department for stroke between July 2020 and March 2023. The exclusion criteria were as follows: (1) subjects who were unable to undergo VFSS or were uncooperative; (2) subjects with other neurological diseases (such as Parkinson’s disease); (3) subjects with a history of oropharyngeal and esophageal surgery, including tracheostomy; (4) subjects with comorbidities involving severe dysarthria or aphasia; and (5) subjects who did not consent to participate in the study. A total of 109 stroke patients were included in the study. The control group for ROC curve analysis included recruited patients without evidence of aspiration on VFSS examination. 41 participants were assigned to the experimental group, and 68 participants were assigned to the control group, with no statistically significant differences observed in age and gender between the two groups. Ethical approval was obtained from the Institutional Review Board of the hospital (2022-06-012-002), and informed consent was obtained from participants included in this study.

Statistical Analysis

Categorical variables were reported as total number (N) and percentage, while continuous variables were presented as mean ± standard deviation. The evaluation protocols for the PAD and VFSS tests were adapted from previous studies [8, 17]. The following parameters were assessed: content validity, internal consistency, intra-rater reliability, inter-rater reliability, construct validity, and concurrent validity. Content validity was analyzed by distributing and returning the PAD’s itemized content validity questionnaire to ten experts who diagnose and treat dysphagia; (1) Configuration; (2) Content suitability; (3) Accessibility; (4) Promptness; and (5) Convenience. The questionnaires were rated on a 5-point scale (1; strongly disagree, 2; slightly disagree, 3; moderate, 4; slightly agree, 5; strongly agree). The content validity index of this study was calculated by referring to the study by Yusoff et al. [19]. Internal consistency was measured using Cronbach’s α to assess the equivalence and agreement between PAD items. A test-retest method was applied for reliability analysis; all participants were assessed using the PAD tool by Tester 1, who was an experienced occupational therapist, and the measurements were repeated by Testers 1 and 2 one week later. Cohen’s kappa coefficient was used to compare the intra-rater reliability between Tester 1’s first and second assessments and the inter-rater reliability between each tester’s PAD scores. Results of the VFSS test were analyzed using the functional dysphagia scale (FDS) and penetration-aspiration scale (PAS). Pearson’s correlation coefficients between PAD scores and the FDS and PAS scores were calculated and used to assess concurrent validity. Construct validity was determined using principal component analysis to identify the factors that constitute dysphagia most prominently. Sensitivity and specificity were determined using ROC curves. All statistical analyses were performed using SAS® Analytics Pro Version 9.4 or higher (SAS Institute Inc., Cary, NC, USA).

Results

Baseline Characteristics

The following baseline characteristics of the subjects were analyzed in this study: age, sex, stroke etiology (infarction or hemorrhage), aspiration status and score of PAD and VFSS (PAS, FDS). Of the 895 patients who underwent VFSS, 175 had a stroke, and 66 did not meet the inclusion criteria (Fig. 1). Therefore, a total of 109 stroke patients (67 men {61.5%}, 42 women {38.5%}; mean age 70.1 ± 14.1 years) were included in the study; 41 (37.6%) exhibited aspiration detected by VFSS (Table 1).

Fig. 1
figure 1

Patient flow chart

Full size image
Table 1 Characteristics of the included patients (n = 109)
Full size table

Content Validity

The five-item questionnaire for content validity (Configuration, Content suitability, Accessibility, Promptness, Convenience) was administered by ten experts. The calculated content validity index of the PAD test was 0.95, achieving a satisfactory level of content validity.

Internal Consistency

The internal consistency reliability test for PAD scores showed a good internal consistency of 0.84 for all patients (n = 109) and a good internal consistency of 0.87–0.89 for the repeat tests (n = 32), independent of the examiner (Fig. 2). The level of clinical significance by alpha coefficient was interpreted as good (0.80–0.89) [20].

Fig. 2
figure 2

Internal consistency of the practical assessment of dysphagia (PAD) test

Full size image

Inter-Rater Reliability

The inter-rater reliability of the two examiners was excellent, with a correlation of 0.99; the individual item correlations ranged from 0.69 to 1.00 (Table 2). The level of clinical significance by levels of kappa was interpreted as good (0.60–0.74) and excellent (0.75–1.00) [20]. Only one item (Strength control of phonation) showed good significance at 0.69, while the others showed excellent significance.

Table 2 Inter-rater reliability of the Practical Assessment of Dysphagia (PAD) test (n = 32)
Full size table

Intra-Rater Reliability

To analyze the stability of the test results, the test-retest reliability of the PAD tests performed by the same examiner twice within one week was assessed. The results showed that the inter-rater reliability was excellent, at 0.98. The test-retest correlation for each item was also excellent, at 0.87-1.00 (Table 3).

Table 3 Intra-rater reliability of the Practical Assessment of Dysphagia (PAD) test (n = 32)
Full size table

Concurrent Validity

The concurrent validity was measured by correlating the PAD scores with the PAS and FDS scores (evaluation of the results from the VFSS, a validated diagnostic test for dysphagia). The PAS showed a moderate to strong negative correlation with PAD (r=-0.62); the FDS showed a moderate negative correlation with PAD (r=-0.50). Both correlations were statistically significant (p < 0.001) (Table 4).

Table 4 Pearson correlation test for the Practical Assessment of Dysphagia (PAD) test and the Video Fluoroscopic Swallowing Study (VFSS)
Full size table

Construct Validity

For evaluating the construct validity, principal component analysis was performed on the PAD data of 109 participants. The 18 items of the PAD tool were clustered into six factors with eigenvalues of 1 or greater. The eigenvalue of the main factor was 6.42, explaining 36% of the total variance (Table 5).

Table 5 Factor loadings from a principal components analysis of the Practical Assessment of Dysphagia (PAD)
Full size table

Sensitivity and Specificity

The sensitivity and specificity were calculated using the ROC curve, and the area under the curve was observed to be 0.833. At PAD test cutoffs of 34, 47, and 72 points, the sensitivity and specificity were as follows: sensitivity: 24.4%, 70.7%, and 100%, respectively; specificity: 100%, 80.9%, and 26.5%, respectively; positive predictive value:100%, 69%, and 45.1%, respectively; and negative predictive value: 68.7%, 82.1%, and 100%, respectively (Fig. 3).

Fig. 3
figure 3

The receiver operating characteristic curve of the practical assessment of dysphagia (PAD) test

Full size image

Discussion

The harmful effects of dysphagia include difficulty in eating and obtaining adequate nutrition, physical or psychological distress, and aspiration pneumonia [21]. Aspiration pneumonia is a life-threatening problem. Pneumonia is a major threat to the health of South Koreans and was the third leading cause of mortality in the country in 2020 [22]. Furthermore, aspiration pneumonia is causing over 57,000 deaths annually in the United States, and has been showing a steady increase since 2009 [23]. . Although aspiration pneumonia due to dysphagia is not the only cause of death from pneumonia, as life expectancy increases and the population ages, mortality from aspiration pneumonia is expected to increase. Therefore, it is important to diagnose dysphagia early and provide treatment tailored to the organs involved in swallowing. This is particularly important in medically underserved areas or countries that do not have access to expensive dysphagia diagnostic tools such as VFSS or FEES; the need for an assessment tool that can replace or complement the above mentioned tools is especially high in such areas.

Dysphagia has been reported to have several causes [24]. If the cause of dysphagia is transient and reversible, causal treatment (such as surgery, botulin toxin injection, or pharmacology) should be considered first [25]. However, if the underlying cause is difficult to treat or chronic, such as dementia, stroke, Parkinson’s disease, or old age, the immediate goal of treatment should be to restore and treat the swallowing function, and not to resolve the cause of the swallowing disorder. Currently, a one-size-fits-all approach is applied to swallowing rehabilitation rather than an individualized, organ-specific approach. In this regard, the value of the PAD test is significant; accordingly, we conducted a validation analysis to demonstrate its utility in patients with stroke.

This study found that the PAD test had high internal consistency and inter- and intra-rater reliability. In addition, the area under the curve was high at 0.833, and PAD scores showed a significant correlation with PAS and FDS scores (which reflect VFSS findings). Therefore, PAD shows promise as an alternative test to the VFSS, the gold standard test for dysphagia. At cut-off values of 34, 47, and 72, the sensitivities of the PAD test were 24.4, 70.7, and 100%, and the specificities were 100, 80.9, and 26.5%, respectively; these values are comparable to those of other previously reported screening tests, and are indicative of the validity and utility of this test [13, 14]. Thus, the PAD test is expected to be useful both as a screening test and as a replacement for the VFSS.

A limitation of this study was the low level of explained variance (36%), which is a characteristic of PAD tests. As explained in a previous report, PAD tests are designed as quantitative and organ-specific tests, so it is difficult to explain the deficiencies and abnormalities of each item using only one factor [17]. Paradoxically, this design may help identify the cause of dysphagia and its organ-specific treatment. Furthermore, the conclusion of construct validity provides us with a new perspective. For example, items 5, 7, 8, 9, and 10, which load most heavily on the first factor, are associated with vocal production and number of chewing cycles, while items 12, 13, and 14 grouped under the second factor are related to the ability or quality of speech. Thus, the results of principal component analysis suggest that through additional research, the current PAD assessment consisting of eighteen items could be further unified and simplified into more cohesive elements. In addition, although the total PAD score showed high inter-rater reliability (ICC = 0.99), the inter-rater reliability of some items showed relatively low correlations compared with other items (Strength control of phonation, ICC = 0.69; Movement of soft palate, ICC = 0.78; and Excursion of thyroid cartilage, ICC = 0.79), although they were interpreted as having good to excellent clinical significance [20]. Unlike other items (3-step obey command, repetition number of sound, distance of blowout of a candle), the above items are based on the tester’s observations and include subjective interpretation. Therefore, an objective evaluation method should be developed based on additional research.

Conclusion

The PAD test exhibits excellent reliability and validity. Unlike other previously described tests, it is quantitative and organ-specific, showing high sensitivity and specificity despite the use of multiple variables in addition to aspiration. If further studies demonstrate the usefulness of the PAD test not only for stroke but also for various other conditions that cause dysphagia, this test could become a widely used tool in the diagnosis and treatment of dysphagia, even in clinical settings that do not include access to equipment such as that required for performing the VFSS.