Abstract
To investigate the validity and reliability of the Practical Assessment of Dysphagia (PAD) test as a quantitative and organ-specific test for stroke patients. In this study, PAD test data from 109 patients with stroke were used. The internal consistency of the PAD was analyzed using Cronbach’s α value. Inter- and intra-rater reliabilities of the PAD were analyzed using Kappa coefficient. Concurrent validity was evaluated based on the correlation between PAD and the videofluoroscopic swallowing study (VFSS). The diagnostic accuracy of the PAD test in patients with stroke was measured using the area under the receiver operating characteristic (ROC) curve. Intra- and inter-rater reliabilities (Intra-class Correlation Coefficient (ICC) = 0.98 and 0.99, respectively) were significant (p < 0.001) for the total PAD score. The functional dysphagia scale (FDS) score and penetration-aspiration score (PAS) correlated significantly with PAD (p < 0.001). The results of the ROC curve analysis with various cut-off points showed that the PAD test had high sensitivity and specificity. The PAD has high reliability and validity. Therefore, it is a useful screening test for dysphagia in patients with stroke.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Dysphagia is a common complication of stroke worldwide, with a prevalence of 42%. Patients with dysphagia have a four times higher risk of aspiration pneumonia, a leading cause of death in patients with stroke [1, 2]. Early dysphagia screening reduces the incidence of aspiration pneumonia [3, 4]; it is essential to improve the patient’s nutritional status and reduce the risk of aspiration pneumonia [5,6,7]. The gold standard tests for assessing dysphagia are the videofluoroscopic swallowing study (VFSS) and fiberoptic endoscopic evaluation of swallowing (FEES); however, both these tests require specialized equipment and experts, and VFSS is associated with the risk of radiation exposure and is difficult to perform in patients immediately after a stroke [8,9,10]. In contrast, FEES does not pose a risk of radiation exposure and can be performed at the bedside; however, it has limited ability to assess the oral cavity and esophagus and has been associated with side effects such as choking, vomiting, laryngospasm, and vasovagal syncope [11, 12]. Gugging Swallowing Screen, 3-oz water swallowing test, bedside swallowing assessment, and Toronto Bedside Swallowing Screening Test have been developed and used as screening tests for swallowing to simplify such assessments. However, as previous studies have shown, these tests primarily assess only the extent of aspiration [13,14,15]. Normal swallowing is the result of coordinated movements of several involved organs; the organic and coordinated movement of the muscles of the mouth, pharynx, larynx, and vocal cords prevents penetration or aspiration of the bolus [16]. In this context, the Practical Assessment of Dysphagia (PAD) tool was developed to predict the risk of dysphagia in a quantitative and organ-specific manner [17]. The PAD test can identify the organs causing dysphagia and allow for organ-specific swallowing therapy based on targeting the involved organs. Studies have reported that this type of individualized treatment is highly effective in treating dysphagia [18]. Therefore, the investigators aimed to use the PAD test in combination with the VFSS to screen and diagnose dysphagia in patients with stroke and use the results to guide organ-specific treatment. In view of this future goal, this study was designed to verify the reliability and validity of the PAD test.
Materials and Methods
The PAD Test
The PAD is a dysphagia assessment tool developed in 2015; it consists of a total of 18 assessment items; the total possible score is 100 points, and higher scores reflect better swallowing function (Appendix 1). The 18 items are designed to assess the organs and other components involved in swallowing function, as follows: (1) Cognition; (2) Respiration; (3) Lip; (4) Tongue; (5) Chin; (6) Soft palate; (7) Vocal cord; (8) Swallowing. The assessment results are organized to enable the rapid identification of the swallowing organs that are deficient; this information can be used to target treatment [16]. In addition, when the total score is 85 or higher, the area under the receiver operating characteristic (ROC) curve for the evaluation of aspiration is 0.80, and the sensitivity is 100%, making it a useful screening test [17].
Subjects
This prospective study included patients aged ≥ 18 years who were referred to the rehabilitation department for stroke between July 2020 and March 2023. The exclusion criteria were as follows: (1) subjects who were unable to undergo VFSS or were uncooperative; (2) subjects with other neurological diseases (such as Parkinson’s disease); (3) subjects with a history of oropharyngeal and esophageal surgery, including tracheostomy; (4) subjects with comorbidities involving severe dysarthria or aphasia; and (5) subjects who did not consent to participate in the study. A total of 109 stroke patients were included in the study. The control group for ROC curve analysis included recruited patients without evidence of aspiration on VFSS examination. 41 participants were assigned to the experimental group, and 68 participants were assigned to the control group, with no statistically significant differences observed in age and gender between the two groups. Ethical approval was obtained from the Institutional Review Board of the hospital (2022-06-012-002), and informed consent was obtained from participants included in this study.
Statistical Analysis
Categorical variables were reported as total number (N) and percentage, while continuous variables were presented as mean ± standard deviation. The evaluation protocols for the PAD and VFSS tests were adapted from previous studies [8, 17]. The following parameters were assessed: content validity, internal consistency, intra-rater reliability, inter-rater reliability, construct validity, and concurrent validity. Content validity was analyzed by distributing and returning the PAD’s itemized content validity questionnaire to ten experts who diagnose and treat dysphagia; (1) Configuration; (2) Content suitability; (3) Accessibility; (4) Promptness; and (5) Convenience. The questionnaires were rated on a 5-point scale (1; strongly disagree, 2; slightly disagree, 3; moderate, 4; slightly agree, 5; strongly agree). The content validity index of this study was calculated by referring to the study by Yusoff et al. [19]. Internal consistency was measured using Cronbach’s α to assess the equivalence and agreement between PAD items. A test-retest method was applied for reliability analysis; all participants were assessed using the PAD tool by Tester 1, who was an experienced occupational therapist, and the measurements were repeated by Testers 1 and 2 one week later. Cohen’s kappa coefficient was used to compare the intra-rater reliability between Tester 1’s first and second assessments and the inter-rater reliability between each tester’s PAD scores. Results of the VFSS test were analyzed using the functional dysphagia scale (FDS) and penetration-aspiration scale (PAS). Pearson’s correlation coefficients between PAD scores and the FDS and PAS scores were calculated and used to assess concurrent validity. Construct validity was determined using principal component analysis to identify the factors that constitute dysphagia most prominently. Sensitivity and specificity were determined using ROC curves. All statistical analyses were performed using SAS® Analytics Pro Version 9.4 or higher (SAS Institute Inc., Cary, NC, USA).
Results
Baseline Characteristics
The following baseline characteristics of the subjects were analyzed in this study: age, sex, stroke etiology (infarction or hemorrhage), aspiration status and score of PAD and VFSS (PAS, FDS). Of the 895 patients who underwent VFSS, 175 had a stroke, and 66 did not meet the inclusion criteria (Fig. 1). Therefore, a total of 109 stroke patients (67 men {61.5%}, 42 women {38.5%}; mean age 70.1 ± 14.1 years) were included in the study; 41 (37.6%) exhibited aspiration detected by VFSS (Table 1).
Content Validity
The five-item questionnaire for content validity (Configuration, Content suitability, Accessibility, Promptness, Convenience) was administered by ten experts. The calculated content validity index of the PAD test was 0.95, achieving a satisfactory level of content validity.
Internal Consistency
The internal consistency reliability test for PAD scores showed a good internal consistency of 0.84 for all patients (n = 109) and a good internal consistency of 0.87–0.89 for the repeat tests (n = 32), independent of the examiner (Fig. 2). The level of clinical significance by alpha coefficient was interpreted as good (0.80–0.89) [20].
Inter-Rater Reliability
The inter-rater reliability of the two examiners was excellent, with a correlation of 0.99; the individual item correlations ranged from 0.69 to 1.00 (Table 2). The level of clinical significance by levels of kappa was interpreted as good (0.60–0.74) and excellent (0.75–1.00) [20]. Only one item (Strength control of phonation) showed good significance at 0.69, while the others showed excellent significance.
Intra-Rater Reliability
To analyze the stability of the test results, the test-retest reliability of the PAD tests performed by the same examiner twice within one week was assessed. The results showed that the inter-rater reliability was excellent, at 0.98. The test-retest correlation for each item was also excellent, at 0.87-1.00 (Table 3).
Concurrent Validity
The concurrent validity was measured by correlating the PAD scores with the PAS and FDS scores (evaluation of the results from the VFSS, a validated diagnostic test for dysphagia). The PAS showed a moderate to strong negative correlation with PAD (r=-0.62); the FDS showed a moderate negative correlation with PAD (r=-0.50). Both correlations were statistically significant (p < 0.001) (Table 4).
Construct Validity
For evaluating the construct validity, principal component analysis was performed on the PAD data of 109 participants. The 18 items of the PAD tool were clustered into six factors with eigenvalues of 1 or greater. The eigenvalue of the main factor was 6.42, explaining 36% of the total variance (Table 5).
Sensitivity and Specificity
The sensitivity and specificity were calculated using the ROC curve, and the area under the curve was observed to be 0.833. At PAD test cutoffs of 34, 47, and 72 points, the sensitivity and specificity were as follows: sensitivity: 24.4%, 70.7%, and 100%, respectively; specificity: 100%, 80.9%, and 26.5%, respectively; positive predictive value:100%, 69%, and 45.1%, respectively; and negative predictive value: 68.7%, 82.1%, and 100%, respectively (Fig. 3).
Discussion
The harmful effects of dysphagia include difficulty in eating and obtaining adequate nutrition, physical or psychological distress, and aspiration pneumonia [21]. Aspiration pneumonia is a life-threatening problem. Pneumonia is a major threat to the health of South Koreans and was the third leading cause of mortality in the country in 2020 [22]. Furthermore, aspiration pneumonia is causing over 57,000 deaths annually in the United States, and has been showing a steady increase since 2009 [23]. . Although aspiration pneumonia due to dysphagia is not the only cause of death from pneumonia, as life expectancy increases and the population ages, mortality from aspiration pneumonia is expected to increase. Therefore, it is important to diagnose dysphagia early and provide treatment tailored to the organs involved in swallowing. This is particularly important in medically underserved areas or countries that do not have access to expensive dysphagia diagnostic tools such as VFSS or FEES; the need for an assessment tool that can replace or complement the above mentioned tools is especially high in such areas.
Dysphagia has been reported to have several causes [24]. If the cause of dysphagia is transient and reversible, causal treatment (such as surgery, botulin toxin injection, or pharmacology) should be considered first [25]. However, if the underlying cause is difficult to treat or chronic, such as dementia, stroke, Parkinson’s disease, or old age, the immediate goal of treatment should be to restore and treat the swallowing function, and not to resolve the cause of the swallowing disorder. Currently, a one-size-fits-all approach is applied to swallowing rehabilitation rather than an individualized, organ-specific approach. In this regard, the value of the PAD test is significant; accordingly, we conducted a validation analysis to demonstrate its utility in patients with stroke.
This study found that the PAD test had high internal consistency and inter- and intra-rater reliability. In addition, the area under the curve was high at 0.833, and PAD scores showed a significant correlation with PAS and FDS scores (which reflect VFSS findings). Therefore, PAD shows promise as an alternative test to the VFSS, the gold standard test for dysphagia. At cut-off values of 34, 47, and 72, the sensitivities of the PAD test were 24.4, 70.7, and 100%, and the specificities were 100, 80.9, and 26.5%, respectively; these values are comparable to those of other previously reported screening tests, and are indicative of the validity and utility of this test [13, 14]. Thus, the PAD test is expected to be useful both as a screening test and as a replacement for the VFSS.
A limitation of this study was the low level of explained variance (36%), which is a characteristic of PAD tests. As explained in a previous report, PAD tests are designed as quantitative and organ-specific tests, so it is difficult to explain the deficiencies and abnormalities of each item using only one factor [17]. Paradoxically, this design may help identify the cause of dysphagia and its organ-specific treatment. Furthermore, the conclusion of construct validity provides us with a new perspective. For example, items 5, 7, 8, 9, and 10, which load most heavily on the first factor, are associated with vocal production and number of chewing cycles, while items 12, 13, and 14 grouped under the second factor are related to the ability or quality of speech. Thus, the results of principal component analysis suggest that through additional research, the current PAD assessment consisting of eighteen items could be further unified and simplified into more cohesive elements. In addition, although the total PAD score showed high inter-rater reliability (ICC = 0.99), the inter-rater reliability of some items showed relatively low correlations compared with other items (Strength control of phonation, ICC = 0.69; Movement of soft palate, ICC = 0.78; and Excursion of thyroid cartilage, ICC = 0.79), although they were interpreted as having good to excellent clinical significance [20]. Unlike other items (3-step obey command, repetition number of sound, distance of blowout of a candle), the above items are based on the tester’s observations and include subjective interpretation. Therefore, an objective evaluation method should be developed based on additional research.
Conclusion
The PAD test exhibits excellent reliability and validity. Unlike other previously described tests, it is quantitative and organ-specific, showing high sensitivity and specificity despite the use of multiple variables in addition to aspiration. If further studies demonstrate the usefulness of the PAD test not only for stroke but also for various other conditions that cause dysphagia, this test could become a widely used tool in the diagnosis and treatment of dysphagia, even in clinical settings that do not include access to equipment such as that required for performing the VFSS.
Data Availability
The datasets generated during the current study are not publicly available since they will contain patient data and the informed consent does not include sharing-data publicly. The datasets are available from the corresponding author on reasonable request.
References
Banda KJ, Chu H, Kang XL, Liu D, Pien LC, Jen HJ, Hsiao SS, Chou KR. Prevalence of dysphagia and risk of pneumonia and mortality in acute stroke patients: a meta-analysis. BMC Geriatr. 2022;13:22:420. https://doi.org/10.1186/s12877-022-02960-5.
Feng MC, Lin YC, Chang YH, Chen CH, Chiang HC, Huang LC, Yang YH, Hung CH. The mortality and the risk of aspiration pneumonia related with dysphagia in stroke patients. J Stroke Cerebrovasc Dis. 2019;28:1381–7. https://doi.org/10.1016/j.jstrokecerebrovasdis.2019.02.011.
Al-Khaled M, Matthis C, Binder A, et al. Dysphagia in patients with acute ischemic stroke: early dysphagia screening may reduce stroke-related pneumonia and improve stroke outcomes. Cerebrovasc Dis. 2016;42:81–9. https://doi.org/10.1159/000445299.
Hinchey JA, Shephard T, Furie K, Smith D, Wang D, Tonn S, Stroke Practice Improvement Network Investigators. Formal dysphagia screening protocols prevent pneumonia. Stroke. 2005;36:1972–6. https://doi.org/10.1161/01.STR.0000177529.86868.8d.
Martino R, Pron G, Diamant N. Screening for oropharyngeal dysphagia in stroke: insufficient evidence for guidelines. Dysphagia. 2000;15:19–30. https://doi.org/10.1007/s004559910006.
Odderson IR, Keaton JC, McKenna BS. Swallow management in patients on an acute stroke pathway: quality is cost effective. Arch Phys Med Rehabil. 1995;76:1130–3. https://doi.org/10.1016/s0003-9993(95)80121-9.
Perry L, Hamilton S, Williams J. Formal dysphagia screening protocols prevent pneumonia. Stroke. 2006;37:765. https://doi.org/10.1161/01.STR.0000204140.01459.dc.
Han TR, Paik NJ, Park JW. Quantifying swallowing function after stroke: a functional dysphagia scale based on videofluoroscopic studies. Arch Phys Med Rehabil. 2001;82:677–82. https://doi.org/10.1053/apmr.2001.21939.
Logemann JA, Pauloski BR, Colangelo L, Lazarus C, Fujiu M, Kahrilas PJ. Effects of a sour bolus on oropharyngeal swallowing measures in patients with neurogenic dysphagia. J Speech Hear Res. 1995;38:556–63. https://doi.org/10.1044/jshr.3803.556.
Song WW, Ye SK, Kim HN, Park JJ, Choi KI, Ryu BJ. Validation of gugging swallowing screen for patients with stroke based on videofluoroscopic swallowing study. Ann Rehabil Med. 2009;33:704–10.
Kim IS, H.T. Evaluation and management of dysphagia. Korean J Stroke. 2006;8:40–8.
Nacci A, Ursino F, La Vela R, Matteucci F, Mallardi V, Fattori B. Fiberoptic endoscopic evaluation of swallowing (FEES): proposal for informed consent. Acta Otorhinolaryngol Ital. 2008;28:206–11.
Edmiaston J, Connor LT, Loehr L, Nassief A. Validation of a dysphagia screening tool in acute stroke patients. Am J Crit Care. 2010;19:357–64. https://doi.org/10.4037/ajcc2009961.
Martino R, Silver F, Teasell R, Bayley M, Nicholson G, Streiner DL, Diamant NE. The Toronto Bedside swallowing screening test (TOR-BSST): development and validation of a dysphagia screening tool for patients with stroke. Stroke. 2009;40:555–61. https://doi.org/10.1161/STROKEAHA.107.510370.
Trapl M, Enderle P, Nowotny M, Teuschl Y, Matz K, Dachenhausen A, Brainin M. Dysphagia bedside screening for acute-stroke patients: the gugging swallowing screen. Stroke. 2007;38:2948–52.
Cook IJ, Dodds WJ, Dantas RO, Kern MK, Massey BT, Shaker R, Hogan WJ. Timing of videofluoroscopic, manometric events, and bolus transit during the oral and pharyngeal phases of swallowing. Dysphagia. 1989;4:8–15. https://doi.org/10.1007/BF02407397.
Lee KM, Kim HJ. Practical assessment of dysphagia in stroke patients. Ann Rehabil Med. 2015;39:1018–27. https://doi.org/10.5535/arm.2015.39.6.1018.
Palmer JB, Drennan JC, Baba M. Evaluation and treatment of swallowing impairments. Am Fam Physician. 2000;61:2453–62.
Yusoff MSB. ABC of content validation and content validity index calculation. Educ Med J. 2019;11:49–54. https://doi.org/10.21315/eimj2019.11.2.6.
Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6:284–90. https://doi.org/10.1037/1040-3590.6.4.284.
Cohen DL, Roffe C, Beavan J, et al. Post-stroke dysphagia: a review and design considerations for future trials. Int J Stroke. 2016;11:399–411. https://doi.org/10.1177/1747493016639057.
Park D, Yun JH, Chun S, Oh BM, Kim HS. The association between antihypertensive drug use and hospitalization for pneumonia in the general population: a case-crossover study using the National Health Insurance Database of Korea. J Korean Med Sci. 2022;37:e248. https://doi.org/10.3346/jkms.2022.37.e248.
Gupte T, Knack A, Cramer JD. Mortality from aspiration pneumonia: incidence, trends, and risk factors. Dysphagia. 2022;37(6):1493–1500. https://doi.org/10.1007/s00455-022-10412-w24.
Sura L, Madhavan A, Carnaby G, Crary MA. Dysphagia in the elderly: management and nutritional considerations. Clin Interv Aging. 2012;7:287–298. https://doi.org/10.2147/CIA.S23404.
Rofes L, Arreola V, Almirall J, Cabré M, Campins L, García-Peris P, Speyer R, Clavé P. Diagnosis and management of oropharyngeal dysphagia and its nutritional and respiratory complications in the elderly. Gastroenterol Res Pract. 2011;2011:818979. https://doi.org/10.1155/2011/818979.
Author information
Authors and Affiliations
Contributions
This study was designed by HJ Kim. HJ Kim acted as a consultant and monitored the research process. Data collection was completed by HT Kim who also drafted the article. HJ Min completed statistical analysis, and interpretation. HJ Kim and HT Kim critically read the article and revised it for the final draft. The final version of the article was approved by all the authors.
Corresponding author
Ethics declarations
Ethical Approval
Ethical approval was obtained from the Institutional Review Board (IRB) of the Chungbuk National University Hospital (CBNUH 2022-06-012-002).
Conflict of Interest
The authors declare that they have no conflicts of interest concerning this research.
Informed Consent
Informed Consent was obtained from participants included in this study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kim, H.T., Min, HJ. & Kim, H.J. Reliability and Validity Analyses of the Practical Assessment of Dysphagia Test in Stroke. Dysphagia (2024). https://doi.org/10.1007/s00455-024-10708-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00455-024-10708-z