Introduction

The Edmonton Symptom Assessment System (ESAS) [1] is a self-reporting tool of symptom intensity, initially developed for advanced cancer patients. It is designed to enable repeated quantitative measurements of symptom intensity with minimal patient burden. The ESAS includes nine common symptoms of advanced cancer, with the option of adding a tenth patient-specific symptom. The original ESAS used visual analogue scales to rate symptom intensity. In a subsequent version, they have been replaced with 11-point numerical rating scales [2].

Although the ESAS was designed for self-reporting, a recent survey of palliative care nurses identified concerns about its feasibility and usefulness [3]. Forty-eight of 74 nursing staff working within the Edmonton Palliative Care Program (65%) participated in this survey. Only 14 (29%) agreed with the statement “The ESAS is easy for patients to understand”. The most frequent comments were that the concept of “well-being” is difficult for patients to understand (n = 11), and that patients often confuse the terms “tiredness” and “drowsiness” (n = 6). Garyali et al. further identified concerns about potential errors in patient self-assessments of symptom intensity using the ESAS [4]. This raises the concern that patients may be interpreting and completing the ESAS differently from what was intended. If so, then health care providers may misunderstand their patients’ symptom experiences, possibly resulting in suboptimal symptom management.

The ESAS has undergone widespread adoption in palliative care programs nationally and internationally, for clinical, research, and administrative purposes [5–8]. However, there are relatively few studies that have focused on the psychometric properties of this measure. In a recent comprehensive literature review, 13 of 87 studies screened had a primary focus on instrument development [9]. The types of reliability and validity evidence collected included reliability estimates [4, 10–16], content validity evidence [17], concurrent validity evidence [13–16, 18], predictive validity evidence[19] and sensitivity and specificity [4, 15, 20]. None of these studies focused on patients’ cognitive processes or perspectives as a source of validity evidence.

Purpose

The overall goal of this study was to gather validity evidence for the ESAS, by exploring the potential difficulties that advanced cancer patients may have in interpreting and completing the ESAS, using the think-aloud method [21]. Three specific objectives were identified:

  1. 1.

    To explore the cognitive processes that underlie completion of the ESAS by advanced cancer patients

  2. 2.

    To examine patients’ understanding of ESAS terminology and numerical ratings of symptom intensity

  3. 3.

    To elicit patients’ opinions of the ESAS as a self-reporting tool

Materials and methods

Participants

The study was conducted at the Cross Cancer Institute, a comprehensive tertiary cancer center located in Edmonton, Canada. Ethics approval was obtained from the appropriate research ethics board. English-speaking, cognitively intact advanced cancer patients of at least 18 years of age, who were new referrals to the Pain and Symptom Control Consultation Service, were eligible. Patients with complex symptom issues requiring assessment in the Multidisciplinary Pain and Symptom Control Clinic were excluded, due to the time constraints of this clinic.

The sample consisted of twenty evaluable patients. Determination of sample size for a qualitative design, using the think-aloud method, differs from quantitative studies. In this study, the estimation of sample size was based on the number of participants needed to achieve data saturation (i.e. when there are no new themes emerging from the data). Sample sizes for qualitative research tend to be small, ranging from as few as one, as with case study research, to ten or more [22]

Measures

ESAS [1]

The ESAS is a self-report measure for assessing the intensity of nine symptoms (i.e. pain, nausea, tiredness, drowsiness, depression, anxiety, appetite, well-being, shortness of breath), with the option of adding a tenth patient-specific symptom. Each symptom is rated along a numerical scale, ranging from 0 (not at all, no symptom or best) to 10 (worst possible; Appendix). Each scale is interpreted independently of the other scales, although it is also possible to calculate a total symptom distress score.

Palliative Performance Scale (PPS) [23]

The PPS is a measure of performance status in palliative care patients, based on ambulation, activity, evidence of disease, self-care, intake, and level of consciousness. Ratings are in 10% increments, ranging from 0 (death) to 100 (fully functional).

Folstein Mini-Mental Status Examination (MMSE) [24]

The MMSE is a screening tool for assessing five domains of cognitive functioning: orientation, memory, attention and calculation, recall and language. Scores range from 0 (total impairment) to 30 (fully intact), and were adjusted for age and education [25]

Edmonton Classification System for Cancer Pain (ECS-CP) [26]

The ECS-CP classifies cancer pain according to five features: pain mechanism, incident pain, psychological distress, addictive behavior and cognitive function.

Procedure

The think-aloud method or protocol analysis is a well-documented approach for gathering validity evidence by focusing on patients’ underlying cognitive processes [21, 27]. Using this method, respondents are asked to think aloud during the completion of a cognitive task, or retrospectively describe the thought processes that they used. These verbal reports are often transcribed and then analyzed using qualitative approaches [27]. The prospective form of this validation method was implemented in this study, in which patients were asked to think aloud while completing the ESAS.

Prior to data collection, a research nurse or assistant informed patients about the study and obtained written consent. All consenting patients independently completed the ESAS, in the presence of the research nurse or assistant, who prompted patients to verbalize their thoughts as they rated their symptoms. Thereafter, the research nurse or assistant administered a structured questionnaire to elicit patients’ opinions of the ESAS (Table 5). The sessions were audio-taped.

Specific guidelines for clinically administering the ESAS were originally developed by the Capital Health Regional Palliative Care Program [2]. Prior to initiating the study, the research nurse and assistant were trained in administering the ESAS, using these guidelines. A script was used to ensure consistency of instructions across patients. After each interview, the research nurse or assistant completed a field note summary. Field notes are commonly used in qualitative research to provide a detailed description of the researcher’s observations, “insights, interpretations, beginning analyses and working hypotheses” in the field [22]

The following demographics were obtained from the patients’ medical records: age, gender, marital status, primary cancer diagnosis, current cancer treatment, education level, time since initial diagnosis and time until death. Additional information gathered from medical records included PPS, MMSE, and ECS-CP scores.

Data analysis

The audio-taped interviews (n = 20) were transcribed. The transcripts were coded and analyzed using content analysis [22]

Trustworthiness

Trustworthiness refers to the credibility of qualitative research findings [22]. The following strategies were used to enhance the trustworthiness of the findings:

  1. 1.

    A detailed audit trail was maintained to document the study and data analysis process.

  2. 2.

    Each written transcript was compared with the audio-taped interview to ensure accuracy of transcription.

  3. 3.

    Three primary data sources were used: think-aloud interviews, a structured questionnaire and researcher field notes (data triangulation).

  4. 4.

    Data were coded and analyzed independently by at least two research team members (investigator triangulation).

Results

Participant description

Fifty-nine consecutive patients were screened between September 2006 and May 2007. Twenty-seven eligible patients were approached regarding their willingness to participate in the study. Twenty-two patients were enrolled, of whom 20 were included in the analysis (Fig. 1). Nineteen were outpatients. The main reason for referral was pain. Participant characteristics are described in Table 1. Eleven patients died during the study period.

Fig. 1
figure 1

Flow chart for patient recruitment

Full size image
Table 1 Patient characteristics (n = 20)
Full size table

Objective 1: what are the cognitive processes that underlie completion of the ESAS by advanced cancer patients?

When rating symptoms on the ESAS, advanced cancer patients relied on four main factors, as part of their decision-making process: current symptom profile, temporal changes, symptom experience history, and individual perceptions (Table 2). Current symptom profile included symptom characteristics, comparisons with other symptoms, and how medications and activities affect a symptom. Individual perceptions included beliefs about and acceptance of a symptom, comprehension of terminology and scales, and the influence of other people on patients’ responses. Temporal changes included changes over a period of time and comparisons with the usual baseline intensity of a symptom. Symptom experience history involved the range of symptom intensity a person may have previously experienced.

Table 2 Cognitive processes underlying completion of the ESAS
Full size table

Objective 2: what are the patients’ understandings of ESAS terminology and numerical ratings of symptom intensity?

During both the think-aloud interview and structured questionnaire, participants shared some of their difficulties in understanding the terminology of the symptoms being assessed. As they were thinking aloud, they would often describe each symptom in their own words (Table 3) and provide a qualitative description for a quantitative rating (Table 4), either spontaneously or when prompted. The same quantitative rating, for example, a rating of “7” for pain, would be “getting pain in a lot of areas” for one person, and “pain has really gone away” for another. Some participants would circle a range, rather than selecting a single number.

Table 3 Patient descriptions of symptoms
Full size table
Table 4 Numerical pain ratings and corresponding patient comments
Full size table

Of the nine symptoms, participants had most difficulty understanding and rating the following symptoms: (a) tiredness versus drowsiness, (b) depression and anxiety, (c) appetite and (d) well-being.

Tiredness versus drowsiness

For some participants, both tiredness and drowsiness were associated with sleep or impaired physical mobility (Table 3). Some participants suggested that these two symptoms were the same. One participant described both a physical and mental component to tiredness, which potentially could be rated differently. Some participants suggested that these two symptoms needed to be more clearly defined, so that patients could better understand the differences between them.

Depression and anxiety

Depression and anxiety were often identified as emotional symptoms. Some participants suggested that these two terms should be described in more detail or reworded (e.g. changing depression to “on top of the world versus the bottom” or “change in moods”). One participant suggested that these descriptors might make a person uncomfortable and unwilling to disclose being anxious or depressed. In addition, some participants believed that these symptoms could fluctuate depending on the context: a person may be depressed about a particular area, while still being happy about other areas, of life. One participant described depression in the context of pain, referring to it as “a pain in the head” (Table 3).

Appetite

In most cases, participants described and assessed their appetite in terms of quantity of intake or level of hunger (Table 3). One participant had difficulty rating this symptom, as he described himself as “overeating” due to corticosteroid use, wondering whether or not overeating was “bad.” He eventually rated his appetite as a “5.” In some cases, participants incorrectly reversed the rating scale, by circling a higher number to represent a better appetite.

Well-being

Many participants had difficulty understanding the term, well-being. When prompted, some participants were able to offer their own unique interpretation, such as general health, peace, being out of pain or outlook on life (see Table 3). Participants suggested that this term needs to be more clearly defined.

Objective 3: what are the patients’ opinions of the ESAS as a self-reporting tool?

A summary of responses to the structured interview questions appears in Table 5.

Table 5 Summary of responses to structured questionnaire
Full size table

Clarity of instructions

Eighty percent responded that the instructions were clear. Suggestions for improvement included one-on-one explanation to clarify terminology, and that the timeframe for rating symptoms should be specified as “now.”

Suggested symptoms to add to the ESAS

Suggestions for additional symptoms included irregular bowel movements (i.e. diarrhea, constipation, and/or gas), distance one can walk, concentration, sleep difficulties, unexplained pain or pain unrelated to cancer, financial problems; and emotional difficulties (e.g. irritation, anger, sadness, grief, or frustration).

Order of items

Most participants (75%) agreed with the item order of the ESAS. Some participants suggested that related symptoms should be grouped together (i.e. tiredness and drowsiness, nausea and appetite), and that well-being should be the last item since it comprises all other symptoms. Some participants agreed that the pain scale should be the first item, while others disagreed, preferring to initially focus on other debilitating symptoms apart from pain.

Ease of completion

Most participants (85%) reported that the ESAS was easy to complete. However, some suggested that the terminology needed to be simplified and more clearly defined.

Assistance of a health care professional

Sixty percent of participants reported that a health care professional should be present for clarification and advice, which would ultimately improve accuracy, efficiency, and ease of completion of the ESAS. Others, however, suggested that the presence of a health care professional would be costly and may influence patients’ responses (i.e. patients may be less likely to express their honest opinions).

Additional suggestions

One participant suggested that it would be helpful to be informed about the personal benefits of completing the ESAS. It would also be helpful to complete the ESAS over a period of time to obtain a more complete picture of the symptoms, which may fluctuate considerably over time. Some participants wanted pain to be divided into components, such as location. Some suggested that certain medications or pain may interfere with a person’s ability to concentrate on symptom ratings. Others thought that the direction of the scales for each symptom should be better defined, and the patient should record the worst symptom rating encountered. One participant suggested that there should be different versions of the ESAS for different cancer types and that more space and scales should be made available for additional concerns (of which examples could be given). However, another patient thought that the form should not be any longer.

Discussion

To our knowledge, this is the first study to gather validity evidence for the ESAS by focusing on the underlying cognitive processes and perspectives of the patient. It adds to the limited body of validation work that currently exists for this widely used tool, providing insights into how patients arrive at ratings for their symptoms, and suggesting opportunities for refining the instrument.

Patients considered multiple factors when deciding how to rate a symptom. The number chosen, therefore, represented a distillation of a complex symptom experience. This implies that, although the ESAS scores give a useful “snapshot” of a patient’s symptom profile, an in-depth clinical assessment is still necessary. In other words, the ESAS functions as a screening tool.

Despite the fact that a majority of patients stated that the ESAS is easy to complete, they struggled with interpreting some of the symptoms and numerical scales on the ESAS. Specifically, they had difficulty distinguishing between related concepts of drowsiness and tiredness, rating depression and anxiety, and understanding the term well-being. Also, the scale for “appetite” was often reversed, perhaps because the word “appetite” is not a symptom per se; the same phenomenon was observed by Garyali et al. in a study comparing ESAS scores that were completed independently by patients with scores that were revised after discussion with the physician [4]. Strategies to address these issues include using different words or adding definitions of symptoms to the tool. Another option, which was endorsed by the majority of patients, would be to have a health care professional assist with completion of the ESAS.

Some patients suggested that the timeframe for rating the symptoms (“now”) should be emphasized. The appropriateness of rating symptoms at a single point in time, rather than over a period of time, may be questioned, given that symptoms usually fluctuate in intensity. In fact, Garyali et al. changed the timeframe for assessing ESAS symptoms to 24 h [4]. However, the most reliable rating may be the one that is made in the present, since it is not dependent on accuracy of recall. Again, a comprehensive clinical assessment that considers changes in symptom severity over time is warranted.

Patients suggested that a number of symptoms could be added to the ESAS. However, making the ESAS more comprehensive must be balanced against making the instrument longer and more burdensome to complete. Therefore, it may be reasonable to include a single blank scale to accommodate an additional patient-specific symptom, but add common examples such as constipation.

Most patients did not express a need to change the order of the items. Nevertheless, altering the order may help with comprehension. For example, grouping “tiredness” and “drowsiness” together may highlight the fact that they are related but distinct symptoms. Also, placing “well-being” at the end of the tool, as Garyali et al. [4] have done, may better convey the intention of capturing how patients feel overall.

The main shortcoming of this study is that the patients had pain as the predominant symptom concern, and a low overall symptom burden. Patients with distress in multiple symptom domains were seen in the Multidisciplinary Pain and Symptom Control Clinic and were excluded from the study because of the time constraints of this clinic. Therefore, the findings may not be generalizable to patients who present with complex symptom issues. Also, this study included only patients with cancer, as opposed to other illnesses. Finally, the number of patients was small. Accordingly, replication of this study in other populations may provide further information to guide the use and development of this tool.

Based on the findings of this study, the following suggestions are made:

  • Completion of the ESAS should be accompanied by a comprehensive clinical assessment

  • An effort should be made to clarify the items “tiredness”, “drowsiness”, “depression”, “anxiety”, “appetite”, and “well-being”

  • Clear and consistent instructions should be provided, with emphasis on the timeframe

  • Consideration should be given to modifying the order of symptoms

  • It may be helpful to have the ESAS completed with a health care professional present, at least for the first occasion

It is hoped that the results of this study will inform the design of future validation studies of the ESAS and ultimately lead to improvements in the tool. The authors are currently undertaking a multi-center study, in which advanced cancer patients will compare the current version of the ESAS with one that has been revised in accordance with the findings from this initial study. The gathering of further validity evidence for the ESAS will strengthen the credibility of its use as one of the potential standard assessment tools for pain and symptom assessment in research and clinical practice.