Introduction

Traumatic spinal cord injury (SCI) may result in profound impairments of motor, sensory and autonomic function, which can result in significant disability [1, 2]. To help predict the severity of SCI, the International Standards for the Neurologic Classification of Spinal Cord Injury (ISNCSCI) is widely used as a reliable and valid measure for documenting remaining motor and sensory function following SCI [3, 4]. While the ISNCSCI does provide an assessment of motor and sensory function after SCI, it does not assess SCI-related impairments of the autonomic nervous system (e.g. cardiovascular, temperature regulation, bowel, bladder, and sexual function). Abnormalities of autonomic function significantly affect the quality of life for individuals living with SCI, and detection is important as treatments are available to assist in mitigating these abnormalities [5, 6].

To enhance the detection of autonomic impairments after traumatic SCI, the autonomic standards were developed by a joint committee of the American Spinal Injury Association (ASIA) and International Spinal Cord Society (ISCoS) in 2009 [7]. The International Standards to Document Remaining Autonomic Function after Spinal Cord Injury first edition (ISAFSCIv1) was published in 2012 [8], and is used among clinicians and researchers worldwide [9,10,11,12]. A Spanish version of the ISAFSCIv1 was used in a study and reported to be a useful tool in clinical practice [13]. The inter-rater reliability of ISAFSCIv1, evaluated by Davidson et al. was reported to be moderate to strong [14]. Feedback received from clinicians, researchers and other relevant literature prompted a further revision to ISAFSCI and the second edition of ISAFSCI (ISAFSCIv2) was recently published in 2021 [15]. Currently, both versions of the ISAFSCI assessment form are available for download on the ASIA website [16].

A number of improvements were made in the ISAFSCIv2. In addition to changes in language and clarifications, definitions of abnormal conditions for the General Autonomic Function component and anticipated function based on ISNCSCI for the Lower Urinary Tract, Bowel and Sexual Function component were added to the assessment form. The scoring for the General Autonomic Function component changed from a nominal (‘present’ or ‘absent’) to an ordinal scale (normal = 2, altered = 1 and absent = 0 when applicable) to be consistent with the scoring method for the Lower Urinary Tract, Bowel and Sexual Function component [15] (see Supplementary Fig. 1 for the two versions of ISAFSCI assessment forms). To our knowledge, there have not been any previous studies that have examined the validity of either version 1 or 2 of the ISAFSCI.

Validity refers to whether a measure assesses what it intends to measure. Instead of being a property of a measure, validity is the extent to which the meaning of the results from a measure can be warranted for a particular purpose [17]. Construct validity assesses the theoretical relationship of the responses to items or scores for the measure to other measures assessing similar constructs referred to as convergent validity [18]. In addition, construct validity can be assessed by testing a priori hypotheses on how the measure will perform based on clinical variables such as injury level and severity.

The objective of this study was to assess the construct validity of the ISAFSCIv1. This was done by testing a priori hypotheses on the expected correlation between ISAFSCI items for the General Autonomic Function component and Lower Urinary Tract, Bowel and Sexual Function component and similar items and/or domains in other standard measures. In addition, clinical hypotheses on ISAFSCI responses to items based on clinical assessments such as neurological level and injury severity were tested. Validation studies using ISAFSCIv1 can inform future studies of ISAFSCIv2 given the similarities between the two versions (e.g., the Lower Urinary Tract, Bowel and Sexual Function component is the same in both versions of ISAFSCI).

Methods

Study design

We conducted an observational study between June 2011 and April 2014 at two Canadian SCI centers: Vancouver, British Columbia and Kingston, Ontario. The inclusion criteria for participants consisted of a diagnosis of traumatic SCI, at least one-year post-injury, age ≥18, and ability to read English and provide informed consent. Participants were required to have no changes in neurological function within the last three months and no change in their medications for the entire study period. Research Ethics Board (REB) approval from the hospital and university was obtained for both sites.

Study procedures

Study participants attended two research visits. During the first visit, participant eligibility was confirmed, informed consent was obtained, demographic information was collected, blood pressure and heart rate were measured, the ISNCSCI was performed, and the ISAFSCIv1 was completed by an assessor. At the second visit, a different assessor obtained measurements of blood pressure and heart rate, and administered the ISAFSCIv1. The ISNCSCI examination was completed for each participant from which neurological level, injury severity (AIS A to D), motor, and sensory scores were obtained. Although motor and sensory scores were obtained clinically, only ISNCSCI pinprick scores were used for analysis in this study. Specifically, an average of ISNCSCI pinprick scores at neurological levels T10, T11, T12, and L1 (ISNCSCI composite pinprick score) was used. Standardized measures of autonomic functions described below were administered. Full details of the study design and procedures have been described elsewhere [14].

ISAFSCIv1 assessment

The ISAFSCIv1 assessment consists of: (1) General Autonomic Function component and (2) Lower Urinary Tract, Bowel and Sexual Function component. The General Autonomic Function component has five sections including autonomic control of the heart (4 items), autonomic control of blood pressure (4 items), autonomic control of sweating (4 items), temperature regulations (3 items), and autonomic and somatic control of broncho-pulmonary system (4 items). Items in this component are evaluated with a nominal response of ‘present’ or ‘absent’ using a check mark. The Lower Urinary Tract, Bowel and Sexual Function component consists of three sections including lower urinary tract (2 items), bowel (3 items) and sexual function (5 items). Items are scored zero, one or two and evaluated using an ordinal scale where a score of two represents a normal function, one represents a reduced or altered function and zero means complete loss of control. Items that are unable to be assessed due to pre-existing or concomitant problems are coded as ‘NT’ (not testable) [8]. In order to standardize the study protocol, standard questions were developed by the project team to assist with performing the ISAFSCI assessment (see Appendix 1) [14].

Standard measures for comparison

Measures to assess the expected correlations were the Autonomic Symptom Profile (ASP) [19], the Quality of Life of Spinal Cord Injury Patients (Qualiveen) questionnaire [20], the Neurogenic Bowel Dysfunction Score (NBD) [21], the International Index of Erectile Function (IIEF) [22], and the Female Sexual Function Index (FSFI) [23]. The ASP has been validated to assess different components of the autonomic nervous system [19]. It includes 169 questions and measures 11 domains of autonomic function. The Orthostatic Intolerance domain of the ASP was used in this study which consists of nine items [19]. The Qualiveen assesses the impact of bladder function on the quality of life for individuals with SCI [20]. It is a 30-item questionnaire that assesses four aspects of patients’ lives: Bother with Limitations (9 items), Frequency of Limitations (8 items), Fears (8 items), and Feelings (5 items) [24]. In this study, the Overall Quality of Life score was used, which is the mean of the 30 items (Qualiveen Composite score) [24]. The NBD is a 10-item measure of bowel function and there is evidence to support its validity in individuals with SCI [21]. The NBD measure includes a Total score (NBD Total score) which was used in this study. IIEF and FSFI sexual function measures were recommended by an expert panel at the National Institute on Disability and Rehabilitation Research SCI Measures Meeting [22, 23, 25]. IIEF is a multidimensional scale for the assessment of erectile dysfunction. It consists of 15 items and five domains which are Erectile Function (6 items), Orgasmic Function (2 items), Sexual Desire (2 items), Intercourse Satisfaction (3 items), and Overall Satisfaction (2 items) [22]. IIEF Erectile Function domain score, Orgasmic Function domain score and ejaculation item were used in this study. FSFI is a self-report instrument for assessing female sexual functions. It comprises 19 items and six domains which include Desire (2 items), Arousal (4 items), Lubrication (4 items), Orgasm (3 items), Satisfaction (3 items), and Pain (3 items) [23]. FSFI Arousal and Orgasm domain scores were used in this study.

Mapping and a priori hypotheses generation

A priori hypotheses were generated on how the ISAFSCI items for the General Autonomic Function component and Lower Urinary Tract, Bowel and Sexual Function component were expected to correlate with items, domains and/or composite scores from measures considered to assess similar constructs (convergent hypotheses) as well as with clinical variables such as injury level and severity (clinical hypotheses).

General autonomic function

There was one convergent and five clinical a priori hypotheses for the General Autonomic Function component. The convergent hypothesis was that the orthostatic hypotension item under the autonomic control of blood pressure section would correlate with question 1 of Orthostatic Intolerance domain from the ASP measure. Five clinical hypotheses were generated for each of the five sections under the General Autonomic Function component based on neurological level and/or injury severity (T5 and AIS A/B vs C/D, T5 and AIS A/B vs C/D, L2, T1 and AIS A/B vs C/D and C3-C5 and AIS A/B vs C/D respectively).

Lower urinary tract, bowel and sexual function

There were seven convergent and five clinical a priori hypotheses generated for this component of ISAFSCI. For the convergent hypotheses, the ISAFSCI composite bladder score [i.e., on awareness of the need to empty the bladder and ability to prevent leakage (continence) items] was hypothesized to correlate negatively with the Qualiveen Composite score. Similarly, the ISAFSCI composite bowel score [i.e., on the sensation of need for bowel movement and ability to prevent stool leakage (continence) items] was hypothesized to correlate negatively with NBD Total score. For sexual function in males, it was hypothesized that the ISAFSCI psychogenic genital arousal item correlates positively with IIEF Erectile Function domain score, the ISAFSCI orgasm item correlates positively with IIEF Orgasmic Function domain score, and that the ISAFSCI ejaculation item correlates positively with IIEF ejaculation item. Similarly, for sexual function in females, we hypothesized that the ISAFSCI psychogenic genital arousal item would correlate positively with FSFI Arousal domain score and the ISAFSCI orgasm item would correlate positively with FSFI Orgasm domain score. For the clinical hypotheses, the ISAFSCI composite bladder score, composite bowel score, ISAFSCI psychogenic genital arousal item for males and females were hypothesized to correlate positively with ISNCSCI composite pinprick score. Lastly, the ISAFSCI sensation of menses item was hypothesized to correlate negatively with females being over the age of 50.

Statistical analysis

ISAFSCI responses for the General Autonomic Function component were coded as Present = 1, Absent = 2, Unknown = 3 and ISAFSCI responses of ’Unknown’ and ‘NT’ were treated as missing. The composite scores for some hypotheses were defined as the average score of specified items of the same sections for ISAFSCI and the same domains for the standard measures. For IIEF/FSFI, a domain score was calculated as the total score of specified items for the same domain as in Rosen et al. [22, 23].

A priori hypotheses were tested using either the Chi-square test (Fisher’s exact test if the expected cell counts were less than five) or a correlation test (Spearman’s rho). Specifically, since the General Autonomic Function component uses a nominal scale, the Chi-square test was used. Whereas because the Lower Urinary Tract, Bowel and Sexual Function component uses an ordinal scale, Spearman’s rho was used and both the strength and significance of the correlations were assessed. The strength of the correlations was assessed as: ≥0.70 = strong, 0.50–0.69 = good, 0.31–0.49 = fair, and ≤0.30 = weak [26]. Associations with p-values < 0.05 were considered statistically significant. All analyses were performed using SAS software, Version 9.4 of the SAS System for Windows. Copyright © 2013. SAS Institute Inc., Cary, NC, USA.

Results

Participant characteristics

Participants (n = 49) had an average age of 45 ± 12 years. Forty-two (85.7%) were male, 23 (46.9%) had a motor and sensory complete SCI, 37 (77.6%) had an at or above T6 level of injury, and 31 (63.3%) were living with tetraplegia. The median time since injury was six years (Table 1).

Table 1 Participant Characteristics.
Full size table

General autonomic function

For the convergent hypothesis test, the item on orthostatic hypotension under autonomic control of blood pressure section was significantly correlated with the ASP measure Orthostatic Intolerance domain question 1 (p = 0.01). In terms of the clinical hypotheses, the bradycardia item of autonomic control of the heart section was not associated with AIS and injury level T5 (p = 0.35). Autonomic control of blood pressure items was significantly associated with AIS and injury level T5 (p < 0.0001). Normal control of sweating items from the autonomic control of the sweating section was not associated with injury level L2 (p = 1.0). The hypothermia item under the temperature regulations section correlated significantly with AIS A/B and injury level above T1 (p = 0.04). The item on impaired voluntary breathing requiring partial ventilatory support under autonomic and somatic control of the broncho-pulmonary system section was not associated with AIS and neurological level C3-C5 (p = 0.65) (Table 2).

Table 2 General Autonomic Function Component Hypotheses Generation Details and Test Results.
Full size table

Lower urinary tract, bowel and sexual function

For the convergent hypotheses related to the bladder, the correlation between ISAFSCI composite bladder score and Qualiveen Composite score was fair and significant (rho = −0.33, p = 0.02). Similarly, the ISAFSCI composite bowel score had a fair and significant (rho = −0.37, p = 0.01) correlation with NBD Total score. For sexual function, the ISAFSCI psychogenic genital arousal item for males had a good correlation with IIEF Erectile Function domain score and the correlation was highly significant (rho = 0.59, p < 0.0001). The correlation between ISAFSCI male orgasm item and IIEF Orgasmic Function domain score was fair and significant (rho = 0.45, p = 0.0003). The correlation between the ISAFSCI and IIEF ejaculation items was good and highly significant (rho = 0.60, p < 0.0001). Tests for the two convergent hypotheses on ISAFSCI female sexual items did not have a large enough sample size to detect statistical significance (Table 3).

Table 3 Lower Urinary Tract, Bowel and Sexual Function Component Hypotheses Generation Details and Test Results.
Full size table

For the clinical hypotheses, the ISAFSCI composite bladder score correlated fairly well with the ISNCSCI composite pinprick score (rho = 0.45, p = 0.003). Similarly, the correlation between ISAFSCI composite bowel score and ISNCSCI composite pinprick score was good and significant (rho = 0.55, p = 0.0002). ISAFSCI psychogenic genital arousal item for males had a good and highly significant correlation with ISNCSCI composite pinprick score (rho = 0.68, p < 0.0001). Tests for the two clinical hypotheses on ISAFSCI female sexual items did not have a large enough sample size to detect statistical significance (Table 3).

Discussion

This is the first study to evaluate the validity of the ISAFSCI in individuals with traumatic SCI. The construct validity of ISAFSCIv1 for the General Autonomic Function component was considered to be weak based on the hypothesized correlation with other measures and our clinical hypotheses. For the six General Autonomic Function component hypotheses, two hypotheses (one clinical, one convergent) related to autonomic control of blood pressure and one clinical hypothesis for temperature regulation were statistically significant, indicating that on the whole, the General Autonomic Function component of the ISAFSCI does not have the validity that the clinical community would need in a bedside assessment of autonomic function. For the Lower Urinary Tract, Bowel and Sexual Function component, the construct validity was much stronger, and therefore more useful in the day-to-day clinical assessment of autonomic function. For this component, all of the hypotheses (five convergent, three clinical) were statistically significant except for the hypotheses on female sexual items (two convergent, two clinical), likely due to the sample size of female participants.

A priori clinical hypotheses indicated that the General Autonomic Function component of ISAFSCIv1 has weak construct validity. Bradycardia did not correlate with the hypothesized neurological level of T5 and AIS groups while autonomic control of blood pressure was significantly associated with the hypothesized neurological level of T5 and AIS groups. Inconsistencies between motor completeness and autonomic completeness for cardiovascular function in individuals with SCI have previously been reported in the literature [27]. In a systematic review, West et al. concluded that for chronic SCI, studies suggest that autonomic completeness of SCI is more related to cardiovascular function than neurological completeness of injury [28]. When the effect of neurological level, along with motor, sensory, and autonomic completeness of injury was assessed on cardiovascular control in Paralympic athletes with SCI, only neurological level and autonomic completeness were strong predictors [29]. Ravensbergen et al. reported that the odds of bradycardia and hypotension were significantly higher in those with cervical and high thoracic lesions compared to those with lesions below T5 while the completeness of injury as determined by AIS classification did not influence the odds [30]. Hypotheses on the associations of bradycardia and autonomic control of blood pressure with injury level and severity individually in this study were similar to the findings in these studies (statistically significant association with level but not significant association with AIS).

The Lower Urinary Tract, Bowel and Sexual Function components demonstrated construct validity, especially for the male sexual function items. The ISAFSCI composite bladder score was significantly negatively correlated with the Qualiveen Composite score. Welk et al. reported that the incontinence domain of Neurogenic Bladder Symptom Score, a scale developed for individuals with neurogenic bladder dysfunction including SCI, had a moderate positive correlation with the Qualiveen Composite score (Pearson rho = 0.46, p < 0.001) [31]. The different health conditions and measures validated may explain the variation observed.

The differences seen in the two major components of ISAFSCI assessment may be influenced by how items were designed and scored. In the ISAFSCIv1, items in the General Autonomic Function component use a nominal scale (with two nominal response options) while the Lower Urinary Tract, Bowel and Sexual Function component use an ordinal scale (with three numeric response options). With the latter option, it may be easier for the assessor to pick a more objective and accurate response.

In the recently published ISAFSCIv2, numerous improvements over the ISAFSCIv1 have been made. In addition to language changes and clarifications, definitions of abnormal conditions are now added in the new assessment form directly. With respect to scoring, items in the General Autonomic Function component are now scored using an ordinal scale (normal = 2, altered = 1 and absent = 0 when applicable) [15]. In our study, we generated standard questions to facilitate the ISAFSCIv1 assessment data collection. Similar standardized questions now have been added for the assessment recommendations for the Lower Urinary Tract, Bowel and Sexual Function component in ISAFSCIv2. The work conducted in this study provides a foundation for future studies that may assess the construct validity of the ISAFSCIv2.

This study has limitations. First, our study was a pilot study with only 7 females which led to a sample size that was too small to test any hypothesis for this group of participants. Future studies of ISAFSCIv2 should implement a recruitment strategy for females to increase sample size and assure sex and gender equity. Second, some clinical hypotheses may need to be re-considered or revised based on our results in the future. For example, hypothesis tests of bradycardia and autonomic control of blood pressure with hypothesized injury level and severity individually produced results that are consistent with the literature [30]. Future evaluations of the ISAFSCIv2 should consider neurology level and severity independently. Third, in this study, maintenance of pinprick at levels T10, T11, T12, and L1 was used as a surrogate measure of autonomic neurological function for some clinical hypotheses (e.g. bowel/bladder/sexual functioning) [32]. More recent literature has considered somatic and autonomically mediated reflexes such as the anal wink, bulbocavernosus, adductor, cremaster, abdominal, dartos and vasomotor reflexes to predict intact autonomic function at these levels (e.g. sexual function) [33]. Future research should consider including these reflexes when predicting responses on the ISAFSCIv2. Fourth, the availability of standard measures limited our ability to evaluate the General Autonomic Function component. While there were seven convergent hypotheses generated as priori for the Lower Urinary Tract, Bowel and Sexual Function component of ISAFSCI, there was only one similar convergent hypothesis in the General Autonomic Function component. Future studies evaluating the construct validity of ISAFSCIv2 could work on identifying more standard measures for the General Autonomic Function component. Lastly, there is very little research on autonomic function measures and the ISAFSCI, so it was difficult to compare our results.

In conclusion, the construct validity of ISAFSCI (2012 1st Edition) for the General Autonomic Function component was considered to be weak while it was much stronger for the Lower Urinary Tract, Bowel and Sexual Function component based on a priori hypotheses. For the clinical hypotheses, motor completeness did not correlate well with autonomic completeness. These results may inform future psychometric studies of the ISAFSCIv2 and further revisions to the ISAFSCI.