Introduction

How to measure the activity of esophageal diseases

The activity of a particular esophageal disease can be assessed by the means of patient-reported outcome (PRO) and clinician-reported outcome (ClinRO) measures. The concept is depicted in Fig. 1.

Fig. 1
figure 1

Assessing activity of esophageal disease by patient-reported outcome (PRO) or clinician-reported outcome (ClinRO) measures. QoL quality of life.

Full size image

The PRO is an “umbrella term” that, among other things, encompasses various measures of symptom severity, functioning (disability), health status, health-specific quality of life, “general” quality of life, and general health perceptions. PRO instruments assess either a single underlying characteristic of a disease, in which case these instruments are termed unidimensional, or multiple constructs, in which case the instruments are termed multidimensional. Measures of symptom severity focus on a particular or several disease-related impairments. Measures of disability assess the degree of impairment that manifests itself when performing various daily activities, such as personal care, and locomotion. Health-specific quality of life measures tend to be multidimensional instruments that assess various aspects related to impairment and/or disability inherent to a particular disease. These instruments allow the users to stratify the subjects into various “health status” or “clinical states” based on responses to the items that query relevant information on patients’ perception of health. As such, these instruments enable the comparisons between groups of patients with a particular disease that are treated by different therapeutic regimens and allow for monitoring of changes in “health status” or “clinical state” of a given patient [1]. In contrast, the “general” quality of life instruments evaluate the patients’ ability to fulfill their needs and assess the impact of a restriction or impairment on emotional well-being. These instruments help researchers to compare across different conditions and populations [1].

In esophageal diseases, endoscopic, histologic, and laboratory findings, as well as other aspects of gastrointestinal physiology, such as esophageal motility (as assessed by manometry) or gastric acid reflux (as assessed by impedance-pH-metric study), are evaluated using ClinRO measures.

The development and validation of PRO instruments is an iterative process that has been described in a position paper by the US Food and Drug Administration (US FDA) [2]. In the first phase, items are generated using patient input. The generated items are incorporated into an instrument (questionnaire), which is evaluated in a first and then validated in a second patient group. During the entire process, patient feedback is sought on numerous occasions. The instrument should be content valid, which means that it measures the disease-associated dimensions relevant to the patients, and responsive to change in a health status. The US FDA and the European Medicines Agency (EMA) put a particular emphasis on the use of validated PRO instruments in the trials of medical products in order to support labelling claims [2]. Interestingly, the EMA more frequently grants PRO label claims when compared to the US FDA [3].

Why do we need validated PRO instruments for assessment of the activity of esophageal diseases? First, these instruments help the users to standardize the language of disease activity assessment. Second, such validated PRO instruments and respective scores serve as endpoints in clinical trials and observational studies. The use of these standardized measures facilitates the comparison of the efficacy/effectiveness of various therapies and aids in optimization of therapeutic algorithms used to identify the appropriate therapy for patients.

Should the activity of esophageal diseases mainly be measured based solely on PRO, or ClinRO measures, or a combination of both? When assessing activity of any disease, it is important to take into account PRO, so long the disease-inherent alterations lead to generation of symptoms and changes in psychosocial functioning that can be reliably assessed by PRO measures. Similarly, it is important to take into account ClinRO measures, if these assess biologic findings that may serve as biomarkers for estimation of disease severity. We have recently described this concept elsewhere [4]. In esophageal diseases, such as EoE or GERD or achalasia, both PRO and ClinRO measures should be assessed to get a complete picture on disease activity overall.

In this review, the currently available PRO instruments assessing symptom severity and quality of life in adults with EoE and achalasia are discussed. A systematic review of PRO instruments for assessment of gastroesophageal reflux disease (GERD) activity has recently been published [5••].

PRO instruments for use in eosinophilic esophagitis patients

Eosinophilic esophagitis (EoE) emerged as distinct disease in the early 1990s [6•, 7•]. In industrialized countries, an incidence rate of about 1 new patient per 10,000 inhabitants per year and prevalence rate of approximately 1 patient per 2000 inhabitants is observed [8–11]. In the consensus guidelines first published in 2007 and updated in 2011, the experts defined EoE as “a chronic, immune/antigen-mediated, esophageal disease, characterized clinically by symptoms related to esophageal dysfunction and histologically by eosinophil-predominant inflammation” [12••, 13••]. Given the fact that esophageal eosinophilia can be caused by other diseases, such as GERD, Crohn’s disease, or achalasia, these should be excluded prior to establishing the EoE diagnosis. Historically, different non-EoE-specific or nonvalidated instruments have been used in clinical trials and observational studies. As a consequence, relationship between EoE symptom severity and histologic activity was a matter of much controversy, as dissociation between severity symptoms and histologic findings have been described in some [14–16], but not other studies [17, 18]. The validated instruments for assessment of the severity of various EoE-specific symptoms and biologic findings as well as EoE-specific quality of life have recently become available.

When assessing symptom severity in EoE patients, one has to keep in mind a few particularities. First, EoE symptom presentation is age-dependent [19]. Feeding difficulties and failure to thrive are symptoms more typical of infants and toddlers, while vomiting and abdominal pain are symptoms more characteristic of school-aged children. Dysphagia and food impactions are principal symptoms of adolescent and adult EoE patients. Second, patients need to recall their symptoms over a defined time period. Although regulatory authorities emphasize the importance of using daily electronic diaries in the context of clinical trials sponsored by pharmaceutical companies, the use of these diaries may not be entirely feasible in the context of long-term observational studies. As such, other recall periods must be considered. Third, the severity of dysphagia and behavioral adaptations to living with dysphagia may vary depending on the consistency of foods consumed by the patients. Therefore, instrument developers should attempt to capture the full spectrum of various behavioral adaptations and dysphagia severity by separately examining these for each distinct food consistency.

An overview of the available PRO instruments for use in adult EoE patients is provided below.

EoE activity index (EEsAI) PRO instrument

In 2014, Schoepfer et al. described the development and validation of the EoE activity index (EEsAI) PRO instrument (copyrighted by the University of Bern, Switzerland) [20••]. The work was carried out by the international EEsAI study group in line with the recommendations of the US FDA [2]. Multivariable linear regression analysis and analysis of variance with patient global assessment as dependent variable and distinct symptoms as independent variables were used to develop a PRO score by quantifying the extent to which symptoms explained the variability in patient global assessment. The EEsAI PRO instrument was evaluated in a group of 153 adult EoE patient and validated in the group of another 120 patients. The EEsAI PRO score of 0 indicates lack of EoE-specific symptoms, while a score of 100 points indicates most severe symptoms. The EEsAI PRO instrument consists of the five following components: frequency of dysphagia episodes, duration of dysphagia episodes, pain when swallowing, severity of dysphagia occurring when eating foods of various consistencies, and behavioral adaptations, such as avoiding distinct food due to EoE or cutting food into small pieces. A median of 8 min is required for instrument completion. Patients deemed the instrument as easy to complete and capturing most of their EoE-related complaints (content validity) [20••]. The EEsAI PRO instrument is available with a symptom recall period of 7 days and 24 hours. The responsiveness of the instrument is currently evaluated in a number of clinical trials.

Dysphagia symptom questionnaire

In 2013, the development of the dysphagia symptom questionnaire (DSQ) carried out in collaboration with Meritage Pharma, Inc. (copyright holder) was published by Dellon et al. [21••]. The DSQ is an electronic daily diary (symptom recall period of 24 h) consisting of three items that report on solid-food-avoidance days, dysphagia days, and actions taken to resolve dysphagia episode. At the time of the publication, the food described as solid was not actually specified. The scoring algorithm is expert-based. A score of 0 indicates lack of dysphagia, while a score of 5 points indicates severe dysphagia. The field trial was completed by adult (n = 18) and adolescent (n = 17) EoE patients. The DSQ score has strong positive association with the number of dysphagia days (R = 0.96, P < 0.001). The instrument was shown to be content-valid and is currently evaluated in a number of clinical trials (personal communication).

Straumann dysphagia index instrument

Nonvalidated, the EoE-specific Straumann dysphagia index (SDI) instrument for adults with EoE was first used to assess symptom severity in a randomized, placebo-controlled trial evaluating the efficacy of swallowed topical corticosteroids [22]. The measure and the scoring algorithm (score ranges from 0 to 9 points) were developed by a physician (AS). The SDI assesses frequency and intensity of dysphagia events; neither dysphagia caused by foods of various consistencies nor behavioral adaptations to living with dysphagia are evaluated. Responsiveness, defined as a decrease in 3 or more points from the baseline score, was demonstrated in this trial [22].

Mayo dysphagia questionnaire

Grudell and coworkers published the paper on the development and validation of the Mayo dysphagia questionnaire (MDQ) [23]. The MDQ is designed to evaluate dysphagia severity in adult patients affected by different esophageal diseases. This non-EoE-specific PRO instrument consists of the 28 items that query the following: frequency and severity of dysphagia (17 items), concomitant allergies (3 items), the use of acid-suppressive medication (3 items), past esophageal surgery (2 items), presence of GERD (1 item), past esophageal dilations (1 item), and overall duration of trouble swallowing (1 item). The MDQ evaluates dysphagia severity and behavioral adaptations to living with dysphagia (food modification and time to eat a regular meal) in the context of consuming foods of different consistencies. The MDQ, 30-day version (a 30-day recall period), has shown to be reliable and was validated in a large group of patients experiencing dysphagia [24]. On average, patients take 10 min to complete the MDQ, 30-day version. The instrument is also available with a recall period of 2 weeks (MDQ, 2-week version) [14].

Instruments for assessment of quality of life in adult EoE patients

Adult EoE quality of life instrument

In 2011, Taft and coworkers published a paper describing the development of EoE-specific quality of life measure for adults (EoO-QOL-A; copyrighted by Northwestern University, Chicago, IL, USA) [25••]. Two hundred and one patients between ages of 18 and 70 years old were enrolled into the study. By the use of qualitative methods, authors examined the whole spectrum of the EoE-specific concerns and their impact on quality of life. The 37-item instrument is split into the five following subscales: eating/diet impact, social impact, emotional impact, disease anxiety, and choking anxiety. Patients that do not undergo dietary therapy complete only 32 items. Authors chose a recall period of 7 days. The EoO-QOL-A instrument was shown to be reliable (internal consistency, split-half, and test-retest). The QoL score closely correlated with symptom severity score as assessed by the adult PRO instrument (r = 0.610, p < 0.001) [26].

PRO instruments for gastroesophageal reflux disease

An excellent systematic review on this topic has been published by Vakil and coworkers in 2013 [5••]. Authors identified existing PRO instruments for assessment of severity gastroesophageal reflux disease (GERD) symptoms and evaluated whether these instruments have been developed in accordance with regulatory requirements of US FDA and EMA (e.g., have been shown to be reliable [content, construct and test-retest, internal consistency], and responsive to change) [27]. Of the identified instruments, 15 PRO instruments underwent psychometric evaluation. Of these, eight instruments were developed to assess severity of GERD symptoms, two were developed to diagnose GERD, four PRO instruments were developed for both evaluation and diagnosis of GERD, and one PRO instrument was designed to screen for the existence of GERD. Developers of only five of the PRO instruments followed most steps outlined in regulatory guidance documents. These five instruments were also used as endpoint measures in clinical trials. Nonetheless, the authors concluded that, as of yet, none of the PRO instrument developed to date met all of the regulatory requirements for an outcome instrument in GERD trials [5••].

PRO instruments for achalasia

Achalasia is a rare disorder characterized by a loss of peristalsis and a failure of the lower esophageal sphincter (LES) relaxation with swallowing. An annual incidence of approximately 1.6 patients/100,000 inhabitants and a prevalence of 10 patients per 100,000 inhabitants are observed [28]. Affected patients most often suffer from dysphagia for solids and liquids and regurgitation of undigested food or saliva [29, 30•]. A mean diagnostic delay (time interval from onset of first symptoms to diagnosis) of 4.7 years was documented in adult achalasia patients [31]. The PRO instruments used in achalasia are reviewed below.

Instruments for assessment of symptoms in adult achalasia patients

The most frequently used scores for clinical assessment in achalasia are the Eckardt score and the Vantrappen classification.

Eckardt score

The Eckardt score evaluates the frequency or severity of the following components: weight loss, dysphagia, retrosternal pain, and regurgitation [32, 33]. Each item ranks from 0 to 3 points with the final score ranking from 0 to 12 points. Clinical remission was defined as Eckardt score of 3 and less, persisting for at least 6 months [34]. The modification of the Eckardt score was also published. Specifically, chest pain was rescored on a scale from 0 to 4 (none, < monthly, monthly, weekly, daily), such as the entire instrument scores up to a maximum of 13 points [29]. Using this modified scale for chest pain, a definition of clinical remission of 4 points or less points was proposed. The scoring algorithm is physician-based. For the development of the score no systematic patient input, as proposed by regulatory authorities, was used. The score is completed by the caregiver during a structured clinical interview. The score is also used for clinical staging of achalasia with stages from 0 (Eckardt score 0–1) to 3 (Eckardt score >6). A Korean group found no significant correlation between the Eckardt symptom score and Eckardt dysphagia grade and radiologic indices of achalasia severity [35]. The Eckardt symptom score significantly correlated with the esophagogastric junction distensibility as measured by the functional lumen imaging probe (FLIP™) [36].

Vantrappen classification

Similar to the Eckardt score, the Vantrappen classification is a physician-based instrument [37]. Patients are interviewed about the frequency of esophageal symptoms, such as dysphagia, regurgitation, and chest pain. Depending on whether these symptoms occur never, occasionally, more than once a week, or daily, a symptoms score on a scale of 0–3 is applied. The total score ranges from 0 to 9 points. Clinical remission was arbitrarily defined as a score of ≤2 and failure of treatment (or relapse) as a score of ≥3 points. In a prospective trial randomizing 25 achalasia patients to undergo pneumatic dilation and 25 patients to undergo myotomy, patients were assessed clinically using the Eckardt score, the Vantrappen classification, as well as the Adam’s stages [38]. Authors noted a significant correlation between the Vantrappen classification and the Eckardt score.

Achalasia symptom questionnaire

The achalasia symptom questionnaire (ASQ) was published in 2010. The ASQ is a 11-item measure designed to assess symptoms typical of achalasia [39]. For the first nine items, the symptoms severity is assigned a score between 0 (better) and 4 (worse). The symptoms severity queried in items 10 and 11 is assigned a score between 0 and 1 and 0 and 3, respectively. The total ASQ score of 0 indicates the lack of achalasia symptoms, while a score of 67 points indicates severe achalasia symptoms. Item selection and scoring algorithm are based on physician input. Using the data from the group of 124 adult patients, the authors developed a mathematical model to predict achalasia in order to fast-track patient to manometric confirmation of the suspected diagnosis [39].

Other clinical scores for achalasia

Other physician-developed scores, such as the one described by Vaezi and colleagues, exist [40]. The authors performed interviews in patients. The total achalasia symptom score reaches a maximum of 15 points and is a sum of dysphagia, regurgitation, and chest pain subscores. The frequency of each symptom was graded on a scale from 0 to 5 (0 = none; 1 = once per month or less; 2 = once per week, up to 3–4 times a month; 3 = 2–4 times per week; 4 = once per day; 5 = several times per day). The authors found that timed barium esophagogram better predicted the long-term success after pneumatic dilation than symptom assessment [40].

In summary, all available scoring systems to assess symptoms in achalasia were developed without psychometric evaluation and most of them are completed by caregivers during patient interviews. This stands in contrast to the current US FDA guidelines for PRO development [2]. Nevertheless, it is important to point out that these scores were developed well before comprehensive guidelines on PRO development and validation were published by regulatory authorities [2].

Instruments used to assess quality of life in adult achalasia patients

As mentioned previously, quality of life can be assessed either by health-related quality of life measures that are specific for a given disease or by generic quality of life. The following instruments exist for assessment of quality of life in adult achalasia patients.

Health-related quality of life score by Urbach

The authors conducted semi-structured interviews with achalasia patients (n = 7) and sought expert input to create a list of items for development of disease-specific quality of life measure [41]. The instrument was evaluated in a group of 70 patients. A ten-item instrument covered the following domains: food tolerance, dysphagia-related behavioral modifications, pain, heartburn, distress, lifestyle limitation, and satisfaction. The total score is calculated by summing the score for each item (lowest score 10 points, highest score 31 points), where after these outcomes are transformed to a 0–100 point scale. A lower score represents a better health-related QoL. The instrument was reliable and showed construct validity; however, it was not validated in the initial form [41].

Frankhuisen et al. translated the measure developed by Urbach and coworkers into Dutch and validated it in a group of 171 achalasia patients [42]. The instrument proved to be easy in use. Authors concluded that this achalasia instrument was a reliable and valid measure for comparison of different groups of achalasia patients [42].

Disease-specific health-related quality of life questionnaire for achalasia

The disease-specific health-related quality of life questionnaire for esophageal achalasia (AE-18) is a self-administered measure that has been developed in a group of 104 patients and prospectively validated in a second group of adult achalasia patients. The qualitative methodology was published only in abstract form [43]. The instrument consists of 18 items encompassing the following four domains: physical functioning, psychological functioning, social functioning, and sleep. Patients were asked to provide the answers on a five-point Likert scale ranging from 1 (“always)” to 5 (“never”). Higher scores correspond to better quality of life. Scores for each domain are calculated as the median of the scores of corresponding items. The responsiveness of the AE-18 instrument was evaluated in 65 prospectively included adult achalasia patients, of whom 47 were dilated and 18 underwent laparoscopic Heller myotomy [44]. Following treatment, AE-18 global score and subscales scores increased; changes in quality of life were associated with improvement in symptoms.

SF-36® health survey

The SF-36 health survey is a generic outcome measure designed to examine a person’s perceived health status [45, 46]. The self-administered instrument takes 5–10 min to complete. It includes a multiitem scale that measures each of the following eight health concepts: (1) physical functioning, (2) role limitations because of physical health problems, (3) bodily pain, (4) social functioning, (5) general mental health (psychological distress and psychological well-being), (6) role limitations because of emotional problems, (7) vitality (energy/fatigue), and (8) general health perceptions. Answers to each item are scored. For each health concept, these scores are summed to produce raw scale scores, which are then transformed to a 0–100 scale. Generic quality of life in achalasia patients was evaluated in several studies. Frankhuisen et al. found that many achalasia patients remain severely symptomatic after treatment and also suffer from decreased health-related quality of life [47]. Liu et al. evaluated health-related quality of life in 35 adult achalasia patients undergoing transoral endoscopic myotomy (POEM) and found that, besides the safety and efficacy of the procedure in the short-term run, it also relieved symptom severity and improved health-related quality of life [48].

Conclusions and outlook

The first guidelines on the development and validation of PRO, reflecting the opinion of regulatory authorities, were published in 2007. Before 2007, multiple instruments have been developed to assess symptom severity and quality of life in adult patients suffering from EoE and achalasia. It is clear that these instruments did not incorporate all recommendations from regulatory authorities that are currently regarded as mandatory to establish a labeling claim for the efficacy of a particular therapeutic intervention that is measured by a given PRO instrument. For EoE, there are currently two PRO instruments (the EEsAI PRO instrument and the Dysphagia Symptom Questionnaire [DSQ]) for assessment of symptom severity available that were developed according to the guidelines of regulatory authorities. In addition, the EoO-QOL-A instrument was developed following these guidelines and allows measurement of EoE-specific quality of life. In achalasia, there is currently no PRO instrument available that was developed according to the guidelines from regulatory authorities. Further collaborative efforts between different stakeholders are needed in order to define to what extent therapeutic success should be defined by either PRO or biologic measures.