Background

The National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System® (PROMIS®) was designed to develop, validate, and standardize item banks that measure key patient-reported outcomes including, symptoms of chronic conditions, functioning, and health-related quality of life [1]. The goal was to develop item banks, short forms, and Computerized Adaptive Tests (CAT) to enable comparison across disease conditions and be openly available to researchers. As such, most PROMIS measures are not disease-specific but, instead, global measures of patient-reported outcomes. PROMIS created an initial domain framework [1] that included self-efficacy as a desired bank to be developed under the mental health/positive psychological health domain. Funding provided by the NIH for the second wave of PROMIS included our grant to develop the self-efficacy domain.

Self-efficacy is the belief that one can carry out a behavior necessary to reach a desired goal, even when a situation contains unpredictable and stressful elements [2, 3]. For many intervention studies [4, 5], self-efficacy is a primary or secondary patient-reported outcome, or it may be an important mediator variable. Self-efficacy is a major determinant of behavior and behavioral change, and acts as a key mediator of the acquisition of self-management skills in chronic disease [3, 6, 7]. Bandura theorized that an individual’s perception of capabilities may be more predictive of function than the actual level of impairment [8]. Both cross-sectional and longitudinal studies across a range of disease conditions have shown greater self-efficacy to be associated with better health outcomes and a greater sense of well-being [9–18]. Furthermore, self-efficacy has been shown to be a modifiable [6, 7] and an important mediator between self-management skills and other patient outcomes, including depression, pain, emergency department visits, and increased physical activity [12, 19–23].

Previously validated self-efficacy scales reflect a range of subdomains of self-management skills in chronic disease, including self-efficacy for daily activities, for effectively working with the clinician, and for managing the condition [24–26]. Most self-efficacy scales were developed to measure disease-specific self-efficacy [9, 27]. Only one scale, the Self-Efficacy for Managing Chronic Disease 6-Item Scale (SEMCD6 [28]), is widely used to measure self-efficacy for managing chronic conditions across multiple health conditions [7, 29]. While the SEMCD6 includes items related to important domains of chronic conditions, including fatigue, pain, emotional stress, symptoms, activity, and medication, it does not provide subscale scores nor is it calibrated for CAT applications.

The development of the PROMIS Self-efficacy for Managing Chronic Conditions domain and item banks followed standard PROMIS guidelines [1, 30, 31], including development of definition, identification of domains, patient focus groups, and qualitative item review based on cognitive interviews. The final definition was: “Self-efficacy for Managing Chronic Conditions is defined as an individual’s confidence in his/her ability to successfully perform specific tasks or behaviors related to one’s health in a variety of situations.” This definition was determined by an online Delphi panel of 23 international experts who have published on self-efficacy for chronic conditions [32]. The Delphi panel also prioritized the domains of interest, resulting in the development of five item banks (Self-Efficacy for Managing: Daily Activities, Medications and Treatments, Symptoms, Emotions, and Social Interactions). These domains were further reviewed and confirmed by 78 patient participants in 12 focus groups, where additional content areas were identified. Over 1000 potential questionnaire items drawn from the literature and focus groups were reviewed by the investigative team, which including binning of similar items and then winnowing of redundant items by teams of two investigators. Items mentioned by more than one participant in the focus groups were created and added if not found in existing literature. The final items selected were reviewed by the full research team and then subjected to cognitive interviews with 30 patients. PROMIS reviewers also examined the items before calibration testing in a representative sample that is described here.

The goal of this paper is to report initial validity testing of the PROMIS Self-efficacy for Managing Chronic Conditions final calibrated item banks.

The specific aims are

  1. 1.

    to validate five item banks of self-efficacy for managing chronic conditions across a range of chronic health conditions; and

  2. 2.

    to investigate the characteristics of the five measures of Self-Efficacy for Managing Chronic Conditions by demographic characteristics, disease severity, disability, and mental health.

Methods

Sample

A total of 1,087 subjects completed the PROMIS Self-Efficacy for Managing Chronic Conditions measures. All forms were self-completed. Two sets of participants were included, those with (1) chronic neurologic conditions and (2) general medical conditions.

Patients with chronic neurologic conditions recruited from the University of Maryland Neurology Ambulatory Center (n = 837) were 18 years or older and had one of the following five neurologic conditions: epilepsy (n = 171), multiple sclerosis (MS, n = 166), neuropathy (n = 163), Parkinson’s disease (PD, n = 170), or stroke (n = 167). The original sampling goal was 150 participants per condition. Data in this part of the sample were collected from April 2013 to April 2014.

To improve generalizability to a wider range of general medical conditions, 250 participants were recruited from a national Internet panel maintained by Op4G, a private research community with approximately 250,000 members who participate in Internet-based surveys using home computers. Op4G participants are volunteers, and although they are spread across the USA, the panel is not nationally representative (see Op4G.com for more detail). To be eligible for this part of the sample, participants had to be 18 year or older and have at least one of the follow 20 chronic conditions: angina, arthritis, asthma, cancer, coronary artery disease, epilepsy, diabetes, myocardial infarction, chronic obstructive pulmonary disease, congestive heart failure, HIV or AIDS, kidney disease, liver disease, migraines, MS, neuropathy, PD, spinal cord injury, or stroke. Pre-specified target demographic distributions were identified that were representative of the age, gender, race, ethnic, and educational distributions of the U.S. general population [33]. Due to the low number of Hispanic participants in Baltimore, a target of 20% was included in this part of the sample to increase the overall Hispanic representation. The Internet data were collected August 25 to September 5, 2013.

Measures

PROMIS Self-efficacy for Managing Chronic Conditions banks and scales included the following five domains:

  1. 1.

    Self-Efficacy for Managing Daily Activities The 36 items included assessment of the subject’s confidence in performing various activities of daily living (ADLs) without assistance. Items also assessed exercise, sexual activities, and managing activities in challenging situations (traveling, bad weather).

  2. 2.

    Self-Efficacy for Managing Medications and Treatments The 26 items (27 initially, 1 dropped) included assessment of the subject’s confidence in managing medication schedules of different complexities. Items also assessed managing medication and other treatments in challenging situations such as when traveling, when running out of medication, and when adverse effects are encountered.

  3. 3.

    Self-Efficacy for Managing Symptoms The 28 items included assessment of the subject’s level of confidence to manage/control their symptoms, to manage their symptoms in different settings (home, public place, an unfamiliar place), and to keep their symptoms from interfering with work, sleep, relationships, or recreational activities.

  4. 4.

    Self-Efficacy for Managing Emotions The 27 items (28 initially, 1 dropped) included assessment of the subject’s level of confidence to manage/control symptoms of anxiety, depression, helplessness, discouragement, frustration, disappointment, and anger. Items also explored the presence of strategies to manage stress and loss.

  5. 3.

    Self-Efficacy for Managing Social Interactions The 23 items (24 initially, 1 dropped) included assessment of the subject’s confidence in participating in social activities and getting help when necessary. Items also assessed managing communication with others about their medical condition, including communication with health professionals.

Items were scored on a 5-point Likert scale: 1 (I am not at all confident); 2 (I am a little confident); 3 (I am somewhat confident); 4 (I am quite confident); and 5 (I am very confident). Full-item banks, as well as 8-item and 4-item short forms, were created for the 5 domains. For all self-efficacy scales, a higher score meant more self-efficacy.

Testing and calibration of the banks followed the PROMIS standards for bank creation [1, 30, 31, 34]. Confirmatory factor analyses (CFA) were used to investigate the underlying structure (dimensionality) of the item bank using Mplus version 7.11 [35]. Because the data were treated as ordered categorical, all model parameters were estimated using the weighted least square mean-and-variance adjusted (WLSMV) estimator and a one-factor solution [36]. Dimensionality was investigated using model fit indices, including comparative fit index (CFI), Tucker–Lewis Index (TLI), and root mean square error of approximation (RMSEA) [30]. The residual correlation matrix from the single-factor CFA was used to evaluate local independence of the item bank. The criterion for the violation of the local independence was defined as a residual correlation greater than 0.2 with any of the remaining test items [30]. Item-total correlation and Cronbach’s coefficient alpha were used to evaluate internal consistency of the test items. The IRT analyses were conducted using IRTPRO version 2.1 [37]. A 2-parameter Graded Response Model was used to estimate item difficulty and item discrimination. Only 3 items were eliminated for violating local independence (1 in Managing Medications and Treatment, 1 in Managing Emotions, 1 in Managing Social Interactions).

The detailed psychometric results for one of the banks (Managing Daily Activities) have been previously published [34], but an overview of the findings from all the banks is presented here. Internal consistency of the five PROMIS Self-efficacy banks and short forms was high, with Cronbach’s alphas ranging from 0.96 to 0.97 for the full bank, 0.90–0.95 for the 8-item short forms, and 0.85–0.92 for the 4-item short forms. All measures demonstrated good item-total correlations (>0.55). The CFA model fit indices were moderate to high; observed CFIs were: 0.952 Daily Activities, 0.959 Emotions, 0.967 Medications and Treatments, 0.923 Symptoms, and 0.917 Social Interactions (where CFI > 0.95 is a good fit); TLI were: 0.949 Daily Activities, 0.955 Emotions, 0.953 Medications and Treatments, 0.917 Symptoms, and 0.909 Social Interactions (where TLI > 0.95 is a good fit); and RMSEA were: 0.090 Daily Activities, 0.127 Emotions, 0.089 Medication and Treatments, 0.127 Symptoms, and 0.118 Social Interactions (where RMSEA < 0.08 adequate fit). Factor loadings were all greater than 0.50. All short forms correlated highly with the full-item banks (r  > 0.90 for 8-item short forms, r  > 0.0.85 for 4-item short forms). All measures demonstrated good item-total correlations (r  > 0.55 8-item short forms, r > 0.85 for 4-item short forms). None of the items across the five banks demonstrated differential item functioning (McFadden’s pseudo R2 > 10%) for age (18–55 years vs. 56+), sex (male vs. female), race (white vs. other), or data source (chronic neurologic conditions vs. general medical conditions).

Legacy self-efficacy measure

The Self-Efficacy for Managing Chronic Disease 6-Item scale (SEMCD6) was developed as part of the Stanford Patient Education Center [28]. Items were rated from 0 = not at all confident to 10 = totally confident and scores are averaged across items (final score 0–10). The SEMCD6 contains 6 items related to managing (1) fatigue, (2) physical discomfort/pain, (3) emotional distress, (4) other symptoms/health problems, (5) tasks/activities needed to manage health conditions, and (6) things other than just taking medication.

PROMIS version 1.0 short forms

Data were collected on six previously validated PROMIS version 1.0 short forms (SF, with the number afterwards stating the number of items, and the letter indicating the version) [1, 30, 31]: Physical Function SF12a, Anxiety SF8a, Depression SF8a, Fatigue SF8a, and Global Health 10a (which includes scales on Global Mental Health, 4 items, and Global Physical Health, 4 items). These are all normed to the U.S population as T-scores with a mean of 50 and standard deviation of 10. Higher scores on depression, anxiety, and fatigue indicate worse health (e.g., greater depression), while higher scores on the other scales indicate better health (e.g., greater physical functioning).

Clinical scales

Three clinician-reported scales were completed for the participants from the neurologic ambulatory center (n = 837); all disease-specific ratings were done for the index neurologic condition. Physicians rated severity of disease as 0 = none, 1 = mild, and 2 = moderate or severe. Two disability scales were completed: the modified Rankin Scale [38], which rates overall disability from 0 = no disability to 6 = dead, and the Barthel Index,[39–41] which ranges from 0 to 100, with lower score indicating better functioning. In addition, duration of disease (number of years since diagnosis) was also completed by the physician.

Analyses

Description of derived banks

Descriptive statistics were used to describe the demographic characteristics of the sample. Scores for PROMIS scales are presented based on T-score units, with a mean of 50 and a standard deviation of ten. For the PROMIS Self-efficacy banks and scales, the T clin designation is provided, as our sample is a clinical sample (by definition, as the bank refers to managing a chronic disease condition). Differences in the means of the PROMIS Self-efficacy scales and the SEMCD6 legacy measure by the demographic groups were tested using Analysis of Variance (ANOVA) models.

Concurrent validity

Pearson product moment correlations of the five PROMIS Self-efficacy scales with each other and with the SEMCD6 legacy measure were computed. Correlations were also compared with the PROMIS short forms, as well as the clinical measures within the neurologic sample to examine convergent validity.

Results

Descriptive statistics

Table 1 presents sample demographics and the distributions of the PROMIS Self-efficacy scales by these demographics. The sample was 57% female, mean age = 53.8 (SD = 14.7, range 18–89), 76% white, 21% African American, 3% other race, with 6% Hispanic. Education level was above high school for 80% of the sample and income over $60,000 a year for 54%. The majority were currently married (59%) and not employed (59%). All subjects (by definition) had at least one chronic condition, with 45% having 4 or more. Most data came from the neurologic data source (77%).

Table 1 Demographics and Means/Standard Deviations for PROMIS Self-Efficacy Item Banks and the SEMCD6 Legacy Scale
Full size table

PROMIS Self-efficacy measures differed by most demographic characteristics. Younger age was significantly associated with better Self-efficacy for Managing Daily Activities, but worse Self-efficacy for Managing Emotions and Managing Social Interactions. Females scored higher on Managing Medications and Treatments, but did not differ for other measures. Those of non-white race and Hispanic ethnicity scored lower on all the PROMIS Self-efficacy domains. Married participants scored higher on all the PROMIS Self-efficacy banks, except for Managing Symptoms. Those who were employed and had higher incomes had higher scores on all PROMIS Self-efficacy domains. Those with more chronic conditions scored lower on all PROMIS Self-efficacy banks. Finally, the chronic neurologic sample scored higher than those with general medical conditions completing questionnaires on the Internet (although not significant for Managing Daily Activities).

Most observed differences by the demographics were less than 5 Tclin units (0.5 standard deviation), a less than medium effect size difference. The differences by Hispanic ethnicity were larger than 0.5 standard deviation for all the PROMIS self-efficacy banks (ranging from 7.9 to 10.4 Tclin units). Differences of around 5 Tclin units were seen for Self-efficacy for Managing Daily activities by income and by comorbidity (1 vs. 6+ conditions) and for Self-efficacy for Managing Symptoms by comorbidity. Self-efficacy for managing Emotions and Social Interactions also differed by more than 5 Tclin units by source of data, with higher scores for the general medical conditions sample.

For the SEMCD6 legacy measure, self-efficacy was higher among those with higher age, higher education, white race, non-Hispanic, being married, currently employed, greater income, and fewer comorbidities. The chronic neurologic sample also scored higher on the SEMCD6 than those with general medical conditions completing questionnaires on the Internet. Again, most comparisons were less than 0.5 standard deviations (about 1 point on the SEMCD6), with the exception of Hispanic ethnicity (1.1 points) and comorbidity (1.5 points difference between 1 and 6+ comorbidities).

Concurrent validity

Table 2 presents the intercorrelations among the PROMIS Self-efficacy banks and correlation with the legacy scale. All correlations within the PROMIS Self-efficacy banks were significant (r  < .001) with r  = 0.59 or higher. The highest correlation within the PROMIS Self-efficacy banks was between Managing Symptoms and Managing Social Interactions (r  = 0.76), and lowest between Managing Daily Activities and Managing Social Interactions (r  = 0.59). Correlations of the PROMIS Self-efficacy banks with the SEMCD6 legacy measure were all positive and ranged from r  = 0.57 (Managing Medications and Treatments) to r = 0.76 (Managing Symptoms).

Table 2 Pearson correlations among PROMIS Self-efficacy measures (banks), and with the SEMCD6 legacy measure
Full size table

Table 3 presents the correlations of the PROMIS Self-efficacy banks with the PROMIS short forms and with clinical measures, as well as the means for these comparison measures. As a whole, this sample scored close to the normed T-score of 50 for Fatigue, Depression, Anxiety, and Global Physical Health, but were about 5 T-score units lower than the national average on Physical Function and Global Mental Health. The PROMIS Self-efficacy banks correlated significantly (all r  < .001) with all the PROMIS short forms. The highest correlations for Self-efficacy for Managing Daily Activities was with Physical Function (r = 0.78) and Global Physical Health (r  = 0.75). The highest correlations for Self-efficacy for Managing Emotions was with Global Mental Health (r = 0.70) and Depression (r = −0.69). The highest correlation with Managing Medications and Treatments was with Global Mental Health (r = 0.52). The highest correlation with Managing Social Interactions and with Managing Symptoms (r  = 0.63) was also with Global Mental Health. The highest correlation with Fatigue was with Self-efficacy for Managing Daily Activities (r = −0.54) and Self-efficacy for Managing Emotions (r = −0.54). The SEMCD6 legacy measure had higher correlations with Fatigue (r =−0.65) than the PROMIS Self-efficacy banks (r = −0.39 to −0.54).

Table 3 Pearson correlations of PROMIS self-efficacy banks and the SEMCD6 legacy measure with other PROMIS short forms and clinical measures
Full size table

Correlations of the PROMIS Self-efficacy banks with the clinical measures (physician-rated) were lower than the correlations among the patient-reported measures. Self-efficacy for Managing Daily Activities correlated significantly with disease severity (r = −0.43), modified Rankin (r = −0.47), and Barthel Index (r = 0.46) and had the highest correlations with disease severity and disability measures as compared to the other PROMIS Self-efficacy banks. Disease duration was only significantly correlated with Self-efficacy for Managing Medications and Treatments (r = −0.11), Managing Emotions (r = 0.09), and Managing Symptoms (r = 0.09), and these correlations were low. The SEMCD6 legacy measure had significantly lower correlations with disease severity, modified Rankin, and Barthel Index than the PROMIS Self-efficacy for Managing Daily Activities measure (differences between correlations p < .001).

Conclusions

The newly developed PROMIS Self-efficacy for Managing Chronic Conditions domains include five item banks (Self-efficacy for: Managing Daily Activities, Managing Emotions, Managing Medications and Treatments, Managing Social Interactions, and Managing Symptoms) that have been calibrated across diverse chronic conditions and show good internal consistency and cross-sectional validity. Short-form versions of these measures have also been created with 4 items and 8 items (scoring and calibration information can be freely obtained from the PROMIS website: http://www.healthmeasures.net/explore-measurement-systems/promis).

The PROMIS Self-efficacy for Managing Chronic Conditions banks differed in expected ways on a number of demographic characteristics, with higher self-efficacy scores seen among participants who were white, non-Hispanic, married, employed, had higher income, and with fewer chronic conditions. The differences between Hispanic and non-Hispanic and by level of comorbidity were the largest. Previous studies have shown greater difficulty in disease management among Hispanics, perhaps due to language and cultural barriers, lack of access to preventive services, lack of insurance, and greater medical comorbidity [42]. The mean SEMCD6 rating for the Hispanic sample in this study is similar to those reported in two randomized controlled trials of disease management among Hispanics [42].

Younger age was associated with better Self-efficacy for Managing Daily Activities, but worse Self-efficacy for Managing Emotions and Managing Social Interactions. This result is consistent with other findings in emotional health and social functioning supportive of the socio-emotional selectivity theory [43]. This theory builds upon the observation that older persons maintain higher levels of emotional health and quality of social interactions in the face of declining health through selective focus on more positive emotions and even positive cognitive stimuli. The SEMCD6 legacy measure also showed higher scores (higher self-efficacy) among those over age 65. The fact that Self-efficacy for Managing Daily Activities was higher in the youngest age group (as opposed to lower as for Emotions and Social Interactions) may be supportive of the need for domain-specific self-efficacy measures as proposed by Bandura [8].

Surprisingly, the chronic neurologic sample reported greater self-efficacy for managing conditions on all domains compared to those with general medical conditions completing the self-efficacy measures on the Internet. Findings describing the differences within the neurologic conditions will be published in another paper [44]. We explored whether this finding could be attributed to the higher prevalence of Hispanics in the Internet sample, but the difference was seen among non-Hispanics also. Higher scores from the chronic neurologic sample than the general medical sample Internet sample was also seen in the PROMIS short forms for Depression, Anxiety, Fatigue, and Global Mental Health (but not Global Physical Health or Physical Function, results not presented) in addition to the self-efficacy measures. We are unsure whether these differences may be due to location of administration (more reluctance to disclose mental health issues in-person versus the Internet, even though both sources self-completed the items) or to selection effects based on use of a self-nominating online survey population. Previous PROMIS findings did not show differences across settings for self-completed questionnaires (e.g., paper and pencil in a clinic versus Internet at home) for the PROMIS physical function, fatigue, and depression banks [45]. Neither our study nor previous PROMIS work has tested administration of forms by interviewers versus self-completion. Differences may also be explained by better quality of care in the chronic neurologic sample recruited from a neurology practice than the Internet survey population with self-reported chronic conditions. This finding warrants further exploration about the use of Internet surveys as samples for patient-reported outcomes.

All of the PROMIS self-efficacy measures correlated highly with the legacy measure and with the PROMIS short forms, supporting convergent validity. The SEMCD6 legacy measure correlated the most with Managing Symptoms, which is expected since many of the SEMCD6 items are symptoms. The SEMCD6 legacy measure showed greater correlation with Fatigue than the PROMIS self-efficacy measures—not surprising since fatigue is one of the six items. The SEMCD6 did correlate well across the PROMIS short forms (r > 0.5), although not as highly as the PROMIS Self-efficacy measures.

The highest correlations among the PROMIS self-efficacy measures and the PROMIS short forms were with PROMIS Self-efficacy for Managing Daily Activities and PROMIS Global Physical Health and Physical Function, which again was consistent with our expectations. Managing Daily Activities also showed the greatest correlations with the physician-rated disease and disability measures among all the PROMIS Self-efficacy banks. The high correlation between the Self-efficacy for Managing Daily Activities bank and the Physical Function short form (r = 0.78) raises questions about the difference between these domains. The self-efficacy questions ask about confidence in doing daily activities, while the Physical Function short form asks about the ability to perform daily activities. It is not surprising that these two measures are highly correlated, but it is in the cases where these measures do not agree that the most interesting information is found about the potential value of interventions to improve self-management skills. Patients who overestimate their self-management skills and abilities may engage in risky behaviors (e.g., walking in dangerous situations that may lead to falls) and those who underestimate their abilities may self-limit and underperform (e.g., fear of falling resulting in inactivity that leads to muscle wasting and weakness). Self-efficacy is an import independent predictor of health outcomes, and research has shown that self-efficacy can predict health outcomes better than actual change in health behaviors [20].

The chief limitation of this study is the cross-sectional nature of the assessment of validity. Further work is needed to examine whether these self-efficacy measures are independent predictors of outcomes while controlling for functional status. In addition, assessment of responsiveness to change and minimally important differences will be important to see if the PROMIS self-efficacy banks are sensitive to changes over time after interventions or other changes known to impact functioning or health-related quality of life. Validation in other samples of chronic conditions is also necessary.

In conclusion, the PROMIS self-efficacy measures provide a set of five item banks and short forms that have been psychometrically developed and tested using item-response theory. They allow for computerized adaptive testing and provide norms based on a large sample of patients with chronic medical conditions. The item banks (including CATs) and short forms are freely available to clinicians and researchers who want to evaluate patient confidence in performing important tasks and behaviors for managing chronic medical conditions.