Introduction

Palliative care focuses on an individual’s quality of life and aims to relieve any distressing symptoms and to offer emotional, spiritual and psychological support. In the context of health and illness, outcome is usually defined in terms of the achievement or failure to achieve a desired goal [1]. Therefore, outcome measures used in the palliative care setting should quantify areas of care that reflect the specific goals outlined above. A variety of clinical audit tools and outcome measures have been developed for use in palliative care in recent years. However, no single measure has been found to address all key domains in palliative care, and many in current use have never been properly validated [2].

It is important to measure and record the outcomes of our care provision. The Department of Palliative Medicine at the Royal Marsden Hospital has piloted several outcome tools, for example, the Edmonton Symptom Assessment Scale and the Rotterdam symptom checklist [3, 4]. None of the tools used had been found to be of significant benefit in everyday practice, primarily because they are time consuming to complete and therefore difficult for patients to complete when they are unwell. In addition, none cover all the domains which are felt to be important to palliative care [2].

In order to address this problem, the Palliative Care Outcome Scale (POS) was developed by the Palliative Care Core Audit Project Advisory Group. This outcome measure tool was based on data from a systematic review of outcome measures, which identified problems in current tools [2]. POS consists of ten questions that assess the physical, psychological, practical and existential aspects of quality of life. These include control of pain and other symptoms, patient anxiety, family anxiety, provision of information, level of support, life worth, self-worth, wasted time and personal affairs. In addition to these ten questions, patients are asked to list their “main problems”. This tool was validated in a number of palliative care settings [2].

This paper presents the results of a pilot study of POS as a potential audit tool within the Palliative Care Unit in a specialist cancer centre. We wanted to assess whether or not this tool identified problem areas and demonstrated change in patient quality of life over time. We wanted to see if staff and patient ratings correlated and to assess staff experiences of using the new tool.

Materials and methods

Study population

Following approval from the Trust’s Scientific and Ethics Committees, POS was launched for a 6-week period on the two palliative care wards of this specialist cancer centre. During the 3-month study period, all patients newly admitted to the palliative care unit were invited to participate in the study. Patients who were already inpatients and those under shared care with other units (i.e. not predominately palliative patients) were not included.

Outcome tool

The questionnaire used was that developed by Hearn and Higginson [2] and was not altered in any way. Permission to use the tool was obtained from the Palliative Care Outcome Scale Action Partnership. This tool records data on the patient from (1) the patient’s perspective and (2) the staff members’ assessment of how they think the patient has been feeling. It assesses ten important areas of palliative care: control of pain and other symptoms, patient anxiety, family anxiety, provision of information, level of support, life worth, self-worth, wasted time and personal affairs.

The response to each of the ten questions in POS is graded on a scale from 0 to 4. A score of 0 indicates that the particular issue is not a problem at all. A score of 4 indicates that the issue is overwhelming for the patient, i.e. the higher the score, the worse the patient feels. The first questionnaire is completed by the patient and the second by staff. Both questionnaires ask the same questions, but the staff questionnaire includes an additional question on performance status.

Patients were asked to complete a questionnaire on admission (to reflect on how they had been feeling over the previous 3 days) and then twice weekly until discharge or death. Patients were encouraged to complete their own questionnaires wherever possible or to ask for the help of a family member or a member of staff not directly involved in their care. If a patient was too ill to complete a form, the staff member looking after them completed the staff assessment form only, at the same time intervals (twice weekly).

In addition, we developed a staff opinion questionnaire. The questionnaire was used to identify staff opinion as to how easy, relevant and appropriate POS was to the unit’s practice and the patient population. This questionnaire was sent to all medical and nursing staff on the unit before starting and on completion of the study to ascertain staff experience of using POS.

Methods of statistical analysis

Each question of the POS questionnaire was analysed separately. Patients were grouped according to their initial score, i.e. 0, 1, 2, 3 or 4, and follow-up was divided into four intervals of ∼5 days each (days 1–4, 5–9, 10–14 and 15–19). The significance of the change in score from the initial assessment was calculated for each of the intervals using a paired test (Wilcoxon signed rank test). The pattern of change over time was displayed by showing the average scores with follow-up of patient groups defined by their initial score. Differences between patient and staff assessments were compared using the McNemar test to examine whether one group typically scored higher or lower than the other. Staff assessment questionnaires before and after the study were not directly comparable. Trends and generalisations are therefore presented.

Results

Of the 39 patients admitted to the unit in the study period, 30 agreed to participate in the study and completed a baseline assessment. Nine patients were excluded either because they refused (four), were too confused (three) or were too unwell (two) to participate. Thirty patients completed a questionnaire between days 1 and 4, 18 at days 5 and 9, 11 at days 10 and 14 and nine at days 15 and 19. Interpretation of the results should therefore be tempered by the fact that fewer patients were assessed at later time points so that lack of statistical significance at these time points may reflect the reduced ability to detect differences because of smaller patient numbers. Nine patients died within the study period, and 21 were discharged to either home or to a hospice.

Patient characteristics

The majority of patients were female, 23 (77%), and this reflects the high percentage of patients with breast or gynaecological cancer who were cared for at that time (57%). The median age was 65 years (range 25–91 years). The median length of admission was 7 days (range 1–34 days), hence the attrition in completion of questionnaires at later time points. Performance status (ECOG scores) at time of admission is shown in Table 1. The median performance status did not change significantly over the study period.

Table 1 Patient characteristics (n=30)
Full size table

Questions which identified significant problems for patients

Four “symptoms” or issues were scored as being important for our patients. For six out of the ten questions, the median score on admission was “0” or 1, i.e. the particular factor was not seen as a problem in the majority of patients.

Pain

On admission, the majority of patients, 22 (73%), felt that they had moderate or severe pain (score 2 or 3). Two patients complained of overwhelming pain. There was a significant improvement in pain for all patients at days 1–4, 5–9 and 10–14 compared to baseline (p<0.05). Pain control had improved, such that there was little further improvement in pain control from the third week onwards (p=0.21).

Other symptoms

Patients are asked whether “other symptoms” had been affecting how they felt. In 75%, other symptoms were affecting how they felt ranging from “slightly” to “overwhelmingly”. This significantly improved by days 5–9 and 10–14 compared to baseline (p<0.05). Seven patients (25%) had no other symptoms.

Anxiety

On admission, levels of anxiety were evenly spread across the scores from “not at all anxious” to “completely preoccupied with anxiety and worry”. At days 1–4, there was a significant improvement in the level of anxiety in all patients (p=0.006) that was maintained at days 10–14. Numbers were too small at later time points to make further deductions.

Patient’s perception of family anxiety

Twenty-four (80%) of patients were concerned that their family and/or friends worried about them either sometimes, most of the time or all the time. There was a significant improvement across all groups at days 1–4, 5–9 and 10–14 following admission (p<0.05). However, there was a suggestion that patients felt that their families were more anxious again by days 15–19, but this was a non-significant trend and patient numbers were small.

Changes in overall scores with time

The median patient- and staff-assessed scores on admission and over time are shown in Table 2. There was a significant improvement in overall patient-assessed scores by days 5–9 (p<0.05) but no further improvement after that time. Staff-assessed scores significantly improved by days 1–4 (p=0.005) but then stabilised.

Table 2 Change in overall score from baseline with time
Full size table

Changes in importance of individual factors over time

The highest median individual score (3) on admission was for family anxiety (Fig. 1). Other factors such as pain, other symptoms and patient anxiety were clearly also important (median score 2). There was no increase in median score for these factors at any time. There was an increase from 0 to 1 in the importance of “practical matters” in the few patients remaining at days 15–19. The changes in median scores over time are shown in Table 3.

Fig. 1
figure 1

Change in questionnaire score for each index as reported by the patient over time; data are displayed to show the trend for each group of patients according to baseline score of 0–4, where, for example, 0=“not at all”, 1=“slightly”, 2=“moderately”, 3=“severely” and 4=“overwhelmingly” (n=number of patients)

Full size image
Table 3 Median score over time
Full size table

Questions which were not scored highly by patients

Information given

Most patients felt well informed. Eighty percent of patients scored 0, i.e. felt fully informed on admission, and this was sustained throughout admission (week 2). Moreover, all patients were fully informed by the end of week 2.

Sharing feelings (support)

The majority of patients, 25 (83%), felt well supported on admission (i.e. they felt able to share their feelings with family or friends all or most of the time). The scores of the few patients who reported feeling unsupported on admission improved over time, but the numbers are too small to show significance.

Life worthwhile

On admission, 63% of patients (19) felt life was worthwhile all or most of the time, and this remained consistent throughout their admission. In those patients with higher scores on admission, i.e. those who felt that life was worthwhile only sometimes (six patients), occasionally (three patients) or never (two patients), there was a significant improvement over the first 4 days.

Self-worth

On admission, 65% of the patients felt good about themselves all or most of the time. No patients recorded a score of 4 (i.e. they did not feel good about themselves at all). There was no significant change over time.

Wasted time

On admission, the majority of patients (74%) felt that none of their time had been wasted. For the few patients who felt that there had been some time wasted, there was an improvement with time.

Practical matters

The majority of patients (74%) felt that any practical matters resulting from their illness had been addressed by the time of admission. The outstanding issues concerning two patients were addressed within the first week.

Comparison of patient- and staff-assessed outcomes

Overall, there was a good agreement between patient and staff assessments (Table 4). Importantly, at the first assessment, staff underestimated patients’ pain in 48% of cases (in 7% of cases by >1 point on the 0–4 scale). They overestimated problems relating to information giving and patients’ ability to share their feelings in 11% of cases (all of these were by >1 point on the 0–4 scale). There was the suggestion at both days 0 and 7 that staff underestimated the patients’ perception of the anxiety of family and friends.

Table 4 Staff-assessed outcomes, compared with patient assessmenta
Full size table

Staff experience of using POS

Prior to and after completion of the study, questionnaires were sent to 48 staff members (medical and nursing) to survey attitudes towards using an outcome scale. Thirty-four staff replied prior to (70% response rate) and 22 returned the questionnaire post-study (45% response rate).

Prior to the study, the majority of staff felt that although it was important to use an outcome scale, using an outcome measure would be unlikely to influence their clinical practice, would not alter the way they assessed patients and would not be helpful in clinical practice. These sentiments were not altered with the experience of using POS, although interestingly, problem areas which were identified were obviously addressed and improved during the study period. After the study, the general opinion was that although POS had been easier to use than anticipated by staff, it had been more difficult for the patients, more time consuming than originally anticipated and was not always relevant. However, none of the patients voiced difficulties in completing the scores.

Discussion

This study was designed to assess both the usefulness of POS as a tool to be incorporated into everyday practice and to gain insight into the practice of palliative care in this unit (i.e. what do we do well and what could be improved upon). POS was useful in identifying four problem areas for patients, and it showed that these areas were addressed by staff resulting in improvements in patient care. However, the other six areas addressed by this tool were less useful since both patients and staff recorded low scores for these areas.

We chose to look at each individual component of POS as well as a global score. There was no obvious way to group any of the questions or any suggestion that certain questions should be weighted more heavily than others; this reflects the design of the tool. The majority of patients (80%) were admitted with moderate to severe pain, but over a period of 10 days, there was a significant improvement. Similarly, “other symptoms” were classed as moderate to severe in 60% patients on admission but improved at each time point out to 2 weeks.

Patient anxiety was spread fairly evenly across the scores on admission and improved with time. The patient and staff perception of family anxiety was high in the majority of cases at admission (80%). In fact, this factor accounted for the highest median score (3) on admission. Palliative care incorporates the care of families and carers as well as care of the patient. This study has illustrated this need well. Tools need to be developed to facilitate and assess the needs of families. In all areas in which the median score on admission was ≥2 (namely, in pain control, control of other symptoms, patient anxiety and family anxiety), there was a statistically significant improvement over the first few weeks. Interestingly, the initial fall and then rise in the patient’s perception of family anxiety was noted in the original work examining POS [2].

In six out of the ten questions, the median score on admission was “0” or 1, i.e. the particular factor was not seen as a problem in the majority of patients. Most patients felt fully informed, were able to share their feelings with family and friends, felt good about themselves, felt that none of their time had been wasted and felt that any practical matters resulting from their illness had been addressed by the time of admission. In no area was there a significant deterioration with time. This is surprising in that in other studies, we have shown that many symptoms (e.g. mobility, drowsiness) get worse in the time period leading up to death [3]. The fact that this study continued up until the time of death reduced the bias inherent in using assessment tools that can only be used as long as the patient is well enough to complete them.

In patients with advanced cancer, doctors and patients do not always agree in their evaluation of symptoms [5]. Similarly, discrepancies occur between accounts by patients and their families [6] especially as to the degree of distress caused by symptoms [7]. In this study, there was a good agreement between staff and patient assessments in most areas. The areas in which there were inconsistencies were in the assessment of pain, the perception of information received and the ability to share feelings. This enabled us to identify areas for further staff training, and the effectiveness of this can be reassessed in a future audit, completing the audit cycle.

After using POS, the general impression from staff was that using an outcome measure was easier than anticipated but that POS was time consuming and did not always seem relevant. Staff felt that completion of the tool was more difficult for patients than they first anticipated; however, they did recognise that measuring outcomes was important.

We were aware that other centres had been concerned about two questions in the POS questionnaire, i.e. asking patients if “life was worth living” and the other is asking if patients had “felt good about themselves”. There was concern that these questions might cause distress to patients and relatives. In our group of patients and staff, there was no report of distress. All questions were completed in full, and no patient voiced concern. However, we are a specialist palliative care unit, and some of the other assessments were in broader medical environments. It is possible that further staff training needs may need to be addressed in such units. We recognise that patients who are terminally ill may feel that life is no longer worthwhile. We found that only 8% expressed this (felt life was worth living only occasionally or never), and this improved over the first 4 days after admission. This reflects the study by Chochinov et al. [8] who found that 7.5% of patients had loss of dignity associated with a loss of will to live.

Is POS a good tool? Many of the previous tools we have assessed have simply reflected the inevitable worsening of patient symptoms until death and have rarely changed clinical practice [3]. Whereas many tools have concentrated on symptoms, this tool does explore other areas thought to contribute significantly to the quality of life of palliative care patients. It did allow us to identify some problems that might not necessarily have been obvious otherwise, e.g. the few patients who reported being underinformed on admission, and these were addressed. It did not adversely affect the way in which patients were assessed, but it is unclear whether it had a significant effect on clinical practice. The amount of work involved in completing the paperwork may preclude it from being useful on a day-to-day basis, but it is a significant improvement on tools to date. As an audit tool, it could be useful to highlight particular areas for individuals, e.g. the degree of patient/family anxiety shown here.

As is so often the case in studies of this patient population, the high patient dropout rates prevented us from getting any meaningful results beyond 2 weeks. The patient numbers were then too small to show any significant changes.

POS was designed to be used in various settings in palliative care and to encompass factors other than symptom control that contribute to the global quality of life of a palliative care patient. A weakness in our experience was that six of the ten aspects tested were not of concern to the majority of our patients on admission. The tool was therefore not useful as a global outcome measure other than in its assessment of pain, other symptoms and anxiety. It may be that patients admitted to a palliative care unit within a major cancer centre are of a better performance status and perceive themselves to have different needs than patients within other palliative care settings, e.g. community care.

In conclusion, POS was useful in detecting some problems in individual patients and could be a useful audit tool. It did indicate that symptoms which were identified did improve and in this way could be seen as a useful outcome tool in this setting.