The World Health Organization recognizes “health” as a multidimensional concept, including in their latest classification (the International Classification of Functioning, Disability, and Healthy, ICF) the components of body structures and functions (Impairment level) and functioning in daily life from both individual (Activity Limitation) and societal (Participation Restriction) perspectives [1].

Speech pathologists working with people with dysphagia have long used physiological assessment, such as videofluoroscopic swallowing studies [2], for detailed measurement of function. The level of a swallowing impairment, such as delayed triggering of the swallow reflex, and the resultant activity limitation, such as changes in the type of oral intake, are also used as clinical indicators of outcome, whether they are recorded formally (e.g., by using a measure such as the Royal Brisbane Hospital Outcome Measure for Swallowing [3,4]), or informally (e.g., via descriptive clinical bedside evaluation).

However, as the authors of the SWAL-QOL [5] noted, “swallowing disorders evoke a host of distressing psychosocial responses such as anxiety, shame, embarrassment, fear, and reduced self-esteem” (p. 116), because of the socially symbolic nature of eating and drinking.

Outcome measures of dysphagia intervention need to address different facets. First, they must describe areas of importance to patients. Outcome measures should capture information about the sequelae of dysphagia such as client restrictions in social participation and/or feelings of distress; measuring only the impairment and activity limitation of the client may be limiting and fail to address the real concerns of the patient. It has been shown that an increase in societal participation may be more meaningful to a client than are changes at the impairment level [6–12]. Second, they must capture the areas in which speech pathologists focus therapy. Speech pathologists do not aim to improve merely the physiological functioning of the client; they are also concerned with the client’s limitations in everyday activities, restrictions in participating in society, and well-being or level of concern [13]. It is important, as the need for outcome measurement grows across healthcare systems worldwide [14], that we can capture this complexity of our services, and that this information is used when planning service delivery across healthcare systems. Third, in the modern clinical world of evidence-based practice [15] and clinical accountability, we need to fully describe “what it is that we do” with clients with dysphagia in order to fully examine the evidence for therapy efficacy.

Current Outcome Measures for Dysphagia Used in Australia

Currently, three tools are commonly used in Australia for measurement of therapy outcomes in clients with dysphagia. First, the Royal Brisbane Hospital Outcome Measure for Swallowing [3], or RBHOMS, has been used in acute care facilities. This measure, designed by a group of speech pathologists in Australia, requires the clinician to make a single-point rating from ten levels, which have been divided into four stages: A: Nil by mouth, and B: Commencing oral intake, C: Establishing oral intake, and D: Maintaining oral intake. Each level includes a description of specific clinical features which “allows the clinician to adequately differentiate between the rating levels” (RBHOMS, p. 16). Use of the RBHOMS enables outcomes for dysphagia to be described in terms of a client’s changes in oral intake. Thus, it entails a rating of Activity Limitation, in ICF terms. However, many of the clinical features used in this measure to differentiate levels also indicate function/dysfunction at an ICF Impairment level. For example, for someone to gain a rating of stage A, level 1, he/she must exhibit “pooling of saliva in the oral cavity” and “wet phonation.” Despite the use of a numbering system, the RBHOMs levels are not in fact ordinal (that is, a higher number does not necessarily imply an increase in functional capacity), particularly once the client reaches stage D (maintaining oral intake), where three different “optimal” outcomes are proposed, each of which is assigned a different number value but in fact all three are equal with respect to desirable outcomes.

The second measurement tool is the SWAL-QOL, which is used in clinics in Australia with adult clients who have dysphagia. This client-based (i.e., client-rated) measure purports to assess both “quality-of-life and quality-of-care” (p. 16) for patients with swallowing disorders [5]. The SWAL-QOL includes patient self-ratings of quality of life, an important concept that falls outside of the scope of the WHO ICF. Thus, the SWAL-QOL, if used as an outcome measurement tool, adds to, but does not replace, measurements of change across the domains of health as identified by the ICF.

A third measure, widely used within the Australian setting to demonstrate outcome [16], is the UK. Therapy Outcome Measure for Speech and Language Therapy (UK-TOM) [17]. In terms of swallowing outcomes, the TOM offers one scale devoted to Dysphagia, where four domains are rated: Impairment, Disability, Handicap, and Well-being/Distress. In Australia, the TOM scales for speech pathology have been available since 1997 and are used clinically across acute and subacute healthcare, education, and community bases.

These three current outcome measures each have strengths as measures of therapy outcomes for Australian clients with dysphagia. The RBHOMS was designed and validated in Australia, has good validity and reliability, and is reportedly responsive to clinical change [4]; the SWAL-QOL focuses on an area likely to be of great concern to patients—quality of life; and the UK-TOM offers a measure of therapy outcomes across four domains of health and function. However, each tool also demonstrates weaknesses. Specifically, although the RBHOMS has been reported by speech pathologists to be sensitive within acute hospital settings, there are accounts of ceiling/plateau effects once the patient reaches the rehabilitation setting [18]. The RBHOMS focuses on the Impairment and Activity levels of patient outcomes and is not fully ordinal, making description of degrees of change from assessment to discharge difficult. The SWAL-QOL has been validated only on a North American population of adults with dysphagia [19], and its applicability and usefulness to the Australian health setting remains to be assessed. The TOM scales were developed and tested by UK clinicians [10,20] and the direct transfer of this tool to Australian healthcare settings has, again, not been validated. Further, the UK-TOM tools were based on the World Health Organization’s International Classification of Impairment, Disability, and Handicap [21], which was replaced in 2001 by the ICF [22].

In this article we present preliminary data from a 6-month clinical trial of a new tool, the AusTOMs Swallowing scale, in Victoria, Australia. This includes an examination of the strengths and weaknesses of the tool when used to measure clinical outcomes for people with dysphagia. The AusTOMs Swallowing scale is one of six scales developed in Australia to measure speech pathology outcomes. The AusTOMs is based on work by Enderby and colleagues in the UK who, for over 10 years, have been developing and testing the Therapy Outcome Measures for allied health disciplines such as Speech and Language Therapy [10,13,17,20], and, later, for Physiotherapy, Occupational Therapy, and Rehabilitation Nursing professions [23,24]. Their work resulted in two published tools, with each having a set of separate scales for use in specific clinical conditions that are relevant to the professions who treat them. There are ten TOM scales for speech pathology: Child Speech Language Impairment, Dysarthria, Dysfluency, Dysphagia, Dysphasia/Aphasia, Dysphonia, Hearing Therapy/Aural Rehabilitation, Laryngectomy, Learning Disability/Mental Retardation, and Phonological Disorder [17]. The TOM scales all include four domains to be assessed: Impairment, Disability, Handicap, and Well-being/Distress.

The AusTOMs scales were developed in the Australian healthcare context. As with TOM, four domains were similarly incorporated, three of which are defined by the WHO ICF [1]: Impairment, Activity Limitation, and Participation Restriction. A fourth domain, Distress, Well-being, adapted from the UK TOM, was included to examine changes in the client’s presenting level of distress or concern.

Method

The development of the AusTOMs has previously been reported [25], and papers describing the psychometric properties of the tool are in preparation. The aim of this study was to analyze the data collected using the AusTOMs Swallowing scale over a six-month prospective time period by speech pathology clinicians across 14 healthcare sites in Victoria, Australia. The sites represented a sample of six public acute hospitals (including five metropolitan and one regional setting), six specialized subacute/nonacute services, one pediatric hospital, and one specialist cancer institute. These sites were considered by the AusTOMs research team to be representative of public health services provided in Victoria. In total, 56 speech pathologists participated in data collection. Each clinician was previously trained in the use of the AusTOMs scales via use of paper case vignettes.

When using the AusTOMs, clinicians were asked to make an initial rating at the onset of therapy (following assessment of the patient) and a final rating at the end of therapy (at discharge). Interim ratings could be made, if the clinician chose, as a means of monitoring the progress of clients, particularly of those people who were receiving long-term therapy. It was suggested that appropriate times for these interim ratings might be when clients were undergoing review (reassessment), or when new therapy goals were being set.

Clinicians were asked to choose the AusTOMs scale(s) that closely related to the goals of therapy for that patient. For example, a patient being treated following a stroke may require therapy goals set around speech, language, and swallowing and, therefore, these three scales would be used. Clinicians were encouraged to choose up to three AusTOMs scales, as appropriate to the client, based on their therapy goals, The number was limited to three because the team, in consultation with clinicians, decided that clinicians would rarely target more than three areas of deficit in therapy at one time. A rating was made on each scale chosen across the domains of Impairment and of Activity Limitation. The Participation Restriction and Distress/Wellbeing domains, however, were common (i.e., identical) across all scales and were rated by taking into account all of the person’s difficulties/abilities. Therefore, one rating (at each time point) was made for each patient, no matter how many AusTOMs scales were used. Each patient’s rating was made on a data collection sheet along with de-identified information, such as age, sex, etiology and disorder type, service and setting, and the number of speech pathology contacts at discharge. Data collection sheets were then returned (via post) to researchers at La Trobe University where they were manually checked for completeness.

Data collected using the AusTOMs Swallowing scale were analyzed to answer the following questions:

  • Does the Swallowing scale demonstrate change over time and, if so, in which direction (improvement versus deterioration) did change occur for these patients?

  • Does the Swallowing scale demonstrate expected differences in profiles of change over time for variables such as setting (acute versus subacute), diagnosis (progressive versus acquired neurological disorders), and input (number of speech pathology therapy contacts)?

Hypotheses were developed around these questions as follows:

  1. 1.

    There would be differences in the profile of change over time for patients with swallowing disorders seen in acute and subacute settings.

  2. 2.

    There would be differences in the profile of change over time for patients with acquired neurological disorders (such as stroke) and progressive neurological disorders (such as motor neurone disease).

  3. 3.

    There would be differences in the profile of change over time for people who were seen for a small amount of therapy (1–10 contacts) versus those who received a greater amount of therapy input (11+ contacts).

Wilcoxon signed-rank tests were used to describe the type of change (positive, negative, no change) for each sample and to examine these hypotheses.

Clinician Feedback

After the six-month trial, clinicians were asked to provide anonymous feedback about the clinical usefulness of the AusTOMs scales. A feedback form was provided to each department, allowing ratings of clinicians’ agreement or otherwise (using a Likert-type scale) with the following statements about the tool: The AusTOMs tool was quick to use (in terms of time); It was easy to use the AusTOMs tool (easy to understand, easy to make a rating); The AusTOMs tool would be a useful tool in clinical practice (did you like it; would you use it). A rating of one showed strong agreement and a five showed strong disagreement; clinicians were able to use half-points. Clinicians were asked to make ratings as a group/department. Open-ended comments, along with general feedback about the AusTOM scale, were encouraged. Data forms were sent to each participating speech pathology department by e-mail/and were returned by e-mail or post to the research team at La Trobe University for analysis. One feedback form was returned per department, with each returned form representing the combined opinions of up to 10 clinicians.

Results

Data Collected

Fifty-six speech pathologists were trained to use the AusTOMs speech pathology scales. In total, speech pathologists across participating sites in Victoria collected AusTOMs data on 257 client episodes of care during the six-month data collection period. Of these 257 episodes of care, the Swallowing scale was employed as one scale (of up to three) in 183 cases to rate any given client. The profiles of patients for whom the Swallowing scale was selected are shown in Table 1.

Table 1 Profiles of all patients rated using the AusTOMs swallowing scale during the 6-month data collection period
Full size table

In the majority of cases, an acquired neurologic disorder, such as a stroke or head injury, was the primary etiology for the patients who were being treated for swallowing disorders. The majority of these patients were seen in acute facilities as inpatients. Of the 183 ratings using the Swallowing scale, 92 ratings were of male patients, and 82 were of females, with clinicians failing to report the sex of the patient in 8 cases. The age range was 0–99 years (mean age-65.3 years; standard deviation-25.15 years; missing = 9 cases). Twelve clients were in the age range of 0–3 years, three were 16–18 years, seven were 20–28 years, with the remaining 152 clients (83.1%) being over 36 years of age.

Scales were selected according to the clinician’s priority for intervention, with the first scale selected indicating the primary target for therapy. Selection of the Swallowing scale, as compared to other speech pathology AusTOMs scales, is shown in Table 2. Data showed 156 clients chose swallowing as a priority target for therapy (i.e, the first scale chosen). Scale selection by clinicians was likely a reflection of the centers involved in the study (five of the centers were acute hospitals where there is both a high turnover of patients and a prioritizing of patients for therapy on the basis of swallowing dysfunction). In relatively few cases swallowing was prioritized behind other disorders (usually either language or speech).

Table 2 Selection of the swallowing scale
Full size table

At admission to therapy, ratings for the group for whom Swallowing was a priority (first scale selected) showed a wide range in severity across all four domains, from 0 (the most severe) to 5 (normal). Where swallowing was chosen as a second priority (23 cases), mean ratings for admission tended to indicate a slightly less severe group (shown by a higher rating on the AusTOMs), particularly for the Impairment and Activity domains. There were significant differences in mean admission Impairment scores for patients where swallowing was chosen as a priority versus when it was not (t = 2.895, p < 0.005), but no significant differences between mean admission Activity scores. This indicates that “prioritizing” therapy may occur on the basis of perceived severity, particularly when using the Impairment domain.

Does the Tool Demonstrate Change Over Time?

Data collected demonstrated a shift in the average rating on the AusTOMs Swallowing scale, from assessment to discharge, of approximately one point across both the Impairment and Activity Limitation domains and approximately half of a point (0.5) for the Participation Restriction and Well-being/Distress domains. Clinical experience has shown that real change (as noted and accepted by client, family member, and therapist) is recognized by a shift of one point in any domain (A. John, personal communication, June 2002).

Wilcoxon signed-ranks tests were used to determine the magnitude of change from assessment to discharge for all clients for whom the Swallowing scale was used.

Impairment and Activity Limitation domains saw the greatest percentage of clients making a positive change (58.1% and 59.8% respectively), while the Participation Restriction (48.2%) and Well-being (41.6%) domains had fewer clients showing improvement.

Hypothesis 1: There would be differences in the profile of change over time for patients with swallowing disorders seen in acute and subacute settings.

We expected differences in the profile of change over time for patients with swallowing disorders seen in acute and subacute settings since both the pattern of care and the client’s recovery will be different across each setting. Figures 1 and 2 show the profiles of patient change over time across acute and subacute settings for each domain of the AusTOMs Swallowing scale where this was the first scale selected. As can be seen in Fig. 1, many patients seen for swallowing disorders in acute healthcare settings showed positive change (from assessment to discharge) in the Impairment and Activity Limitation domains, with fewer patients showing positive changes from admission to discharge in the Participation Restriction and Well-being domains. By contrast, many of the patients from subacute settings (Fig. 2) showed positive changes in Activity Limitation and Well-being domains, with fewer patients in this group showing positive change in Impairment. The pattern emerging from these data illustrates differences in change over time across the four domains of the AusTOMs Swallowing scale for different settings. With greater amounts of data collected over a longer time, it would be possible to describe in more detail an emerging “plateau” for improvement in the Impairment domain, once the patient reaches the subacute setting.

Fig. 1
figure 1

Percentage change over time: swallowing in acute settings (N=116).

Full size image
Fig. 2
figure 2

Percentage change over time: swallowing in subacute settings (N=37).

Full size image

Hypothesis 2: There would be differences in the profile of change over time for patients with acquired neurologic disorders (such as stroke) and progressive neurologic disorders (such as motor neurone disease).

We expected that people with progressive disorders would show little positive change over time, particularly in the Impairment and Activity Limitation domains; rather, deterioration and/or static ratings would be seen. On the other hand, people with acquired neurological disorders (such as stroke) might be expected to show positive change over time across all domains. Therefore, the profiles of change for these two groups would be expected to differ. Figures 3 and 4 show change over time for these two groups in terms of the percentage of clients showing improvement, deterioration, or no change on each domain, where swallowing was the first scale selected.

Fig. 3
figure 3

Percentage change over time: swallowing in acquired neurologic disorders (N=69).

Full size image
Fig. 4
figure 4

Percentage change over time: swallowing in progressive neurologic disorders (N=10).

Full size image

As predicted, the profiles of change over time for these two groups are very different. Many people with acquired (nonprogressive) neurological conditions showed positive changes over time in all four domains. The greatest percentage of people showing positive change was in the Impairment domain (72.5%), with a majority also showing a positive change in the Activity Limitation (65.2%) and Participation Restriction (56.5%) domains. A smaller number of this group showed improvements in the Wellbeing domain; only 43.3% of patients improved, while 47.8% remained the same from time of assessment to discharge.

In contrast, the group of people with progressive neurologic conditions tended to have static ratings from assessment to discharge across all four domains. In particular, ratings of these patients in the Participation Restriction (77.8%) and Wellbeing (88.9%) domains tended to remain the same. Interestingly, the only people to show deterioration in any domain were those with acquired (nonprogressive) disorders. It may be that deterioration in people with progressive disorders is not demonstrable on a scale such as the AusTOMs over a short data collection period (e.g., 6 months).

Hypothesis 3: The profile of change over time would differ for people who were seen for a small amount of therapy (1–10 contacts) versus those who received a greater amount of therapy input (11+ contacts).

We expected that the profile of change over time would differ for people who were seen for a small amount of therapy (1–10 contacts) versus those who received a greater amount of therapy input (11+ contacts). As seen in Table 2, the range of contacts was wide (1–84), and grouping the number of contacts into two broad groups gives only a very basic measure of the impact of therapy (viz. the number of sessions) on change over time. Figures 5 and 6 show percentages of change over time for these two groups of patients where swallowing was the first scale selected.

Fig. 5
figure 5

Percentage change over time: swallowing and 1–10 therapy contacts (N=123).

Full size image
Fig. 6
figure 6

Percentage change over time: swallowing and 11+ therapy contacts.

Full size image

As can be seen from Fig. 5, change over time for the group with a relatively small number (1–10) of contacts tended to occur across the Impairment and Activity Limitation domains, rather than the Participation Restriction or Well-being domains. This result might be expected, given that this group mainly comprises clients seen in high-turnover acute hospital facilities. For clients seen on more than 10 (11+) occasions (including those seen up to 84 times), the trend was toward positive changes occurring across all four domains of the AusTOMs Swallowing scale. While these results are in no way conclusive, given the broad groupings of therapy contacts necessary for analysis of this sample, and that therapy contacts in these groups were not controlled for and so were not necessarily equal (in terms of actual time or resources spent on a patient), they nevertheless illustrate a potential application of this tool for clinical management. Figures 5 and 6 illustrate that there is little difference between the percentage of patients making positive change over time in the Impairment and Activity Limitation domains for the group with fewer contacts (1–10) when compared with the group who had more contacts (11+). These results show only the direction of change; the actual amount of change made would need to be taken into account when deciding the value of increased therapy input.

Clinician Feedback

Six feedback forms representing X clinicians from 8 sites were returned. In general, speech pathologists were positive about the usefulness of the AusTOMs scales in clinical practice. Using a 1–5 Likert-type scale (1 = strongly agree, 5 = strongly disagree), clinicians agreed that the tool was quick to use (mean rating = 2.3 mode = 2, range = 2–3.5) and easy to use (mean rating = 2.4, moda = 2, range = 2–3), although there was some ambiguity about whether the tool was useful in clinical practice (mean rating - 2.75, mode = 2, 3 [bimodal], range = 2–4). Comments indicated that the data collection period (6 months) was felt to be too short a time to fully assess the clinical usefulness of the scale. Speech pathologists who were working in acute care facilities felt that an outcome measure such as AusTOMs may be too broad to capture the small changes occurring during a short stay in hospital.

The usefulness of the Swallowing scale was not assessed separate from the other AusTOM scales; therefore, specific information about its clinical usefulness is not available. Comments indicated that clinicians were happy with the whole AusTOMs speech pathology set of six scales and did not want further changes made to them. Further detailed assessment of the clinical usefulness of the AusTOMs Swallowing scale remains to be undertaken.

Discussion

Dysphagia was the most common presenting disorder seen by speech pathologists in Victoria, Australia, who participated in this project. Patterns are evident within and across the data, demonstrating possible clinical applications of the tool. For example, we might expect different types of change (and hence different outcomes) from therapy in acute versus subacute settings. With larger volumes of data we could examine whether differing patterns of therapy and outcomes are established. Rigorous valid information about the expected outcomes across different healthcare settings, for different patient populations, with different amounts of clinical and resource input will give us better evidence for efficient clinical management. In this study we found preliminary data indicating that therapy outcome is different across different settings (i.e., acute, subacute) and from receiving differing amounts of therapy. Although the data set from clients with progressive neurologic disorders was very small, the pattern emerging from these data are of interest. Deterioration, which we would expect to see in this population, was not evident across any domain of the AusTOMs Swallowing scale. It may be that a larger sample size would have revealed deterioration in some patients. It should also be remembered that these data were collected over only six months; perhaps few clients with progressive neurologic disorders show a deterioration in swallowing function within such a relatively short period. Larger data sets collected over a longer period will inform us about likely therapy outcomes for these clients Specifically, if and when deterioration is evident, and across which domain(s)—Impairment, Activity Limitation Participation Restriction, or Distress/Wellbeing—this deterioration is manifest. We may begin to examine how speech pathologists have an impact on people with disorders of this type, for example, by seeing whether therapy input relates to either a change, or a stabilization of the Well-being domain, despite possible deterioration in other domains. The strength of this tool lies in its incorporation of four domains for evaluation of dysphagia outcomes. Outcome tools should be designed to measure both the clinical domains and those which are of importance to the patient. Butler [11] argued that clients are more concerned with the functional and societal limitations of their disorder, and, therefore, outcomes should reflect these domains.

The clinical usefulness of the AusTOMs Swallowing scale requires further investigation. As this scale was not assessed separate from the other six AusTOMs speech pathology scales, clinicians’ feedback may have been confounded by their views of other scales or of the tool as a whole. We believe that speech pathologists will use only outcome measures that are perceived as clinically valuable [18]. The AusTOMs, like other outcome measures, was not designed to be used as a client assessment tool and, therefore, does not necessarily add clinically relevant assessment information (e.g., for use in improving diagnosis). Rather, it provides value for the clinician by giving information regarding patient changes over time across the four domains that speech pathologists address clinically. Such data, acquired in large sets, may be useful when planning services and making decisions about efficient and efficacious care for clients with dysphagia.