Key points

  • Describes the rationale for collecting patient-reported experience measures (PREMs) and patient-reported outcome measures (PROMs) from a commissioning perspective and the benefits of incorporating their use into clinical practice.

  • Presents the findings from a service evaluation carried out in an oral surgery department at a London dental hospital.

  • Discusses key factors to take into consideration regarding the long-term implementation of PREMs and PROMs and implications for future practice.

Introduction

In 2014, NHS England published Five Year Forward View, a document outlining some of the issues associated with the current climate of medical and dental service frameworks.1 While the document has now been superseded by the NHS Long Term Plan (2019), its message is still relevant today - that change is needed to restructure the current format of NHS services in order to deliver efficient patient care.2

The Guides for Commissioning Dental Specialties3 were produced in 2015 in the wake of the Five Year Forward View.1 The focus of the guides is on delivering better patient care through the effective commissioning and delivery of specialist NHS dental services.

The Guide for Commissioning Oral Surgery and Oral Medicine Specialties (2015) includes strategies for achieving better outcomes through modifying the commissioning of oral surgery services.4 The document recognises that significant change which involves venturing into uncharted territory is required and that inevitably faults with the proposed framework will arise and necessitate further changes. It is therefore important that we present and discuss our own findings and together work towards defining the exact form the ideal service will take, with an appreciation that this may have regional variations.

The document describes what the patient journey within this ideal oral surgery service might look like, outlining the extent and complexity of work which should be delivered by clinicians of varying competence in both primary and secondary care centres. One aspect of the framework involves the appropriate collection and analysis of patient-reported experience measures (PREMs) and patient-reported outcome measures (PROMs).

PREMs are indicators of a patient's perception of qualitative aspects of their healthcare experience. These include factors which patients either directly or indirectly perceive at the time of their treatment. PROMs are measures of patients' objective understanding of their health-status and associated quality of life. They provide information on clinical outcomes associated with a certain unit or treatment modality.

The collection and analysis of PREMs and PROMs is not a recent concept. In 2007, Professor Lord Darzi5 produced an interim report which claimed that 'the NHS has an unprecedented opportunity to focus on quality and that this opportunity should be seized'. Following on from this, in 2008, Darzi6 claimed that PREMs and PROMs are two of the greatest measures of quality. The routine collection of PROMs has already been introduced on a national scale for certain procedures (such as total hip and knee replacements).7 Furthermore, the many potential benefits derived from the utilisation of PREMs and PROMs have been described by a number of authors in a range of clinical contexts throughout recent years.8,9,10,11,12,13,14,15 The rationale for incorporating them into a framework from a commissioning perspective is to drive local quality improvement projects by gathering benchmarking data and enabling assessment and review of the quality of service provision.4

We investigated patient-reported aspects of care in the Department of Oral Surgery at the Royal London Dental Hospital (RLH) with respect to dental extractions carried out under local anaesthetic by members of staff. RLH is part of Barts Health NHS Trust, one of the largest NHS trusts in the UK, with five hospitals, over 16,000 staff, and 1.4 million outpatient appointments each year. The trust is responsible for providing hospital- and community-based care to a broad and diverse population spanning across East London and beyond.

We are a combined, comprehensive oral surgery and oral and maxillofacial surgery (OMFS) unit with access to an Accident and Emergency department and continuing 24-hour care. We also offer a 'Back in Trouble' service, whereby patients we treat can return to us should they experience any post-operative issues.

The patients we see for dental extractions are often referred due to anticipated surgical complexity requiring specialty input or complicating medical and/or social co-factors. Patients have an initial consultation appointment and are then booked on an appropriate treatment list as required.

Members of staff involved in the study included: Dental core trainees (DCTs), junior clinical fellows, staff-grade clinicians, associate specialists, specialty trainees and consultants. All involved staff members were aware of the ongoing service evaluation, which was registered as a quality improvement project with the Barts Health Clinical Effectiveness Unit before commencement. Advice was sought from the NHS Health Research Authority and the project was granted ethical exemption on the grounds that it does not qualify as research.

Materials and methods

This two-stage project initially involved dissemination of a PREMs questionnaire to patients attending the hospital for dental extractions under local anaesthetic over a three-month period. The second phase project was the PROMs study which involved contacting the same cohort from the first of the study 4-6 weeks post-operatively by telephone or email with a follow-up questionnaire.

Only patients over the age of 18 years were included in the study. Patients who did not speak English were only provided a form if there was an appropriate advocate present to interpret the questions and their answers.

Data collection was entirely voluntary, and patients were asked for permission to opt in before taking part. After completing the initial questionnaire, patients were asked if they would consent to being contacted 4-6 weeks later with the second questionnaire. Patients reserved the right to opt out of the study at any point.

Patients were given PREMs forms by their operating clinician to complete by reception away from the clinical environment. The completed questionnaires and contact details of each patient were stored securely in a box only accessible by the authors at all times.

The PREMs questionnaire invited patients to respond to the questions in Table 1 (taken directly from the commissioning guide).4

Table 1 PREMs questions
Full size table

For each question, patients were asked whether they agreed, disagreed or were unsure.

We adapted the PREMs questionnaire to include relevant aspects of the patients' medical history and whether or not the treatment involved a surgical approach. These details were recorded post-operatively by the operating clinician before the patient received the form. Table 2 and Table 3 demonstrate the relevant treatment details and medical factors, respectively, that we took into consideration.

Table 2 Treatment details form
Full size table
Table 3 Relevant medical history form
Full size table

We attempted to telephone patients up to three times each for the PROMs questionnaire before sending them an email survey if they could not be successfully contacted. Patients who did not provide telephone numbers were contacted by email only.

The PROMs questionnaire asked for responses from the questions in Table 4 (taken directly from the commissioning guide).4

Table 4 PROMs questions
Full size table

Patients were asked to respond with either 'yes', 'no' or 'unsure' to the first two questions, and 'days', 'weeks' or 'months' to the final question. We noted the reasons for seeking advice/further surgery and the specific timeframe patients reported it took for them to restore to normal function/appearance.

Results

One hundred and fifty-five PREMs questionnaires were collected over the three-month period. Forty-two out of 155 patients underwent treatment involving a surgical approach (flap raised and/or bone drilled and/or tooth sectioned). The remaining 113 patients had extractions which were considered 'simple' (carried out with luxators/elevators/forceps only).

The responses to each of the aforementioned PREMs questions (Table 1) are summarised in Figure 1. The age range of the patients who completed the PREMs questionnaire was 18-90 years, the mean age was 42 years, and the median age was 38 years.

Fig.1
figure 1

Patient-reported experience measures

Full size image

One hundred and thirty-eight out of 155 patients consented to being contacted post-operatively with the PROMs questionnaire. Of this group, five patients provided illegible contact details, two did not speak English and one withdrew consent. Of the remaining 130 patients, 87 (56.1% of the initial cohort) were successfully contacted for follow-up. Eighty-six were contacted by telephone and one responded to email (31 patients were emailed in total).

Of these 87 patients, 12 (13.8%) reported that they sought advice or assistance hours/days after the procedure. The complications are summarised in Figure 2.

Fig. 2
figure 2

Number of patients with post-operative complications

Full size image

Eighty-five out of 87 patients reported that they did not require additional surgery as a result of their treatment. However, one patient required treatment due to a sharp bone fragment (smoothing under local anaesthetic) and one patient required removal of an unintentionally retained root fragment (under local anaesthetic). No patients in this cohort were admitted into hospital.

The data for the time taken to achieve restoration of normal activities/appearance are summarised in Figure 3.

Fig. 3
figure 3

Time taken to achieve normal function/appearance

Full size image

Discussion

The use of PREMs and PROMs in oral surgery services is still in its infancy. Although their use has been trialled in primary care oral surgery services by Gerrard et al.,16 who challenged the suitability of the questionnaires as described in the commissioning guide,4 this is the first time to our knowledge that the measures have been used to evaluate the delivery of oral surgery secondary care services on this scale. The practicality of collecting data on a national scale is yet to be determined.

We included an extended follow-up period of 4-6 weeks for our PROMs survey to allow for the development and inclusion of post-operative complications that may potentially be excluded from a shortened timeframe.

Gerrard et al.16 followed patients up between 24 and 72 hours post-operatively, and while recognising that those contacted after only 24 hours may be yet to develop common late complications such as post-operative infection and alveolar osteitis, they do not acknowledge that contacting patients at 72 hours post-operatively may still exclude these complications which can arise even after this timeframe. A shortened follow-up period also means that there is no consideration of the time taken to achieve restoration of normal function and appearance unless the patient reaches this status by 24 to 72 hours post-operatively. It should be noted that the commissioning guide includes the options for 'weeks' and 'months' in response to this measure.4 While we did not contact patients earlier than at 4 weeks, all patients received a post-operative advice leaflet with 24-hour contact details and instructions on what to do if any complications or concerns arose.

Regarding the PREMs we collected, most disagreement took place in relation to pain and anxiety control. Ineffective pain management can broadly be split into two categories: (a) inadequate anaesthesia, and (b) inability for a patient to cope with the sensation of pressure. It is difficult to ascertain what the cause of discomfort was for the patients who claimed that their pain was not managed effectively. This highlights the potential need to improve communication and management of patients' expectations by explaining that the sensation of pressure cannot always be avoided.

Anxiety management is more challenging to analyse objectively as there are many potential sources and variables. There are some aspects to patient anxiety that are outside the clinician's control, such as waiting in reception, seeing a different clinician at each appointment, long periods between assessment and treatment, difficulty reaching the hospital and previous dental experiences.

Regarding the PROMs, seven (8%) patients reported post-operative infections which were considered on the basis of receipt of antibiotics. In all cases where antibiotics were prescribed, it is assumed that antimicrobial stewardship was adhered to.

It has been determined that the inferior alveolar nerve paraesthesia reported by one patient was transient in nature as the symptoms have resolved at the time of writing.

The baseline of a patient to achieve perceived normality was likely affected by factors such as pre-existing infection, post-operative pain and familiarity with dental procedures. This again ties in with the theme of managing expectations. Patients may be expecting different outcomes to what is considered 'normal' from a biological or physiological standpoint if they have never had a dental extraction before. It would seem that the majority of these issues could be overcome through improved communication with patients.

The majority of collected PREMs questionnaires were distributed by DCTs, who had not only the highest volume of local anaesthetic minor oral surgery (MOS) lists but were also the most compliant cohort at distributing forms.

While our unit generally accepts Level 2/3 oral surgery referrals as defined in the commissioning guide, a Level 1 procedure can be modified to a higher complexity status by complicating medical/social factors.4 Therefore, we receive and accept a number of referrals for extractions that are not sent to us on the grounds of anticipated surgical complexity. A significant number of the extractions carried out by the DCT cohort specifically would be classified as Level 1 complexity, such as extractions of erupted uncomplicated third molars, if it were not for complicating patient factors. The fact that from a surgical standpoint these extractions may be relatively less technically demanding may allow easier pain control, shorter appointment times and as a result improved anxiety management and confidence in the clinician. Surgical procedures of greater complexity are generally booked in to higher grade clinicians' local anaesthetic lists or intravenous sedation/general anaesthetic lists, which were excluded from this study.

It should also be noted that DCTs generally have the least experience out of all staff clinicians within the department and are therefore potentially more prone to encounter difficulties in managing pain and anxiety. Despite this, the fact that the majority of clinicians involved in the project were of a similar skill level adds a layer of homogeneity to the study and can be considered as a strength.

Another observation is that procedures were not standardised, and we only had an indeterminate understanding of the complexity of treatment through the simple criteria that were noted relating to surgical treatment details. The commissioning guide does not make reference to this.4 Despite adding these criteria ourselves in an attempt to gauge procedural complexity and apply context to our data, we lacked consideration for certain important factors such as angulation and depth of impaction, quantity and location of bone removed, size and design of flap - all variables which could impact outcomes yet are increasingly more complicated to standardise and measure.

Additionally, we did not note which tooth was being extracted, which is important in determining local inferior alveolar nerve injury rates.

Furthermore, staff members were trusted to hand out the forms to patients following treatment. A solution to mitigate bias in data collection is to have members of the reception team distribute forms.

We also found that data collection and collation of results was very time consuming and labour intensive, and alongside the aforementioned barriers associated with operator compliance, present as potential difficulties in the long-term use of PREMs and PROMs in the absence of adequate resources.

It should also be noted that two of our patients who did not speak English but completed PREMs forms were excluded from the PROMs questionnaire due to the language barrier. Consideration needs to be taken in including non-English speakers as in certain regions this may comprise a significant proportion of the patient base.

While we found value in using the oral surgery PREMs and PROMs as described in the commissioning guide4 in their current format, considering the aforementioned intrinsic limitations we have observed, we recognise a need for further development and expansion of comprehensive specialty and procedure-specific PREMs and PROMs in oral surgery.

We also recognise a need for incorporating specific quality of life (QoL) measures as an adjunct to the use of PREMs and PROMs. The benefit of using these measures in clinical practice is well established.17 Their use will enable patient-centred outcomes to be taken into long-term consideration along with clinical outcomes. Specific oral health related-quality of life (OHRQoL) measures have previously been used within the oral surgery setting, such as the Oral Health Impact Profile (OHIP),18,19 Oral Impact on Daily Performance (OIDP),20 and United Kingdom Oral Health Related Quality of Life (OHRQoL-UK)19 tools. While using these questionnaires can provide clinicians with valuable data, there is a need to develop standardised measures for widespread and routine use within oral surgery and indeed all of dentistry.

Conclusion

Based on our data, the vast majority of our patients reported that they were satisfied with their experience. Through presenting our results at local clinical governance meetings, our project paved the way for our department to begin regularly incorporating PREMs and PROMs collection and analysis into routine clinical practice. We have obtained valuable information pertaining to local patient satisfaction and complication rates which enables us to analyse the strengths and weaknesses of our service and provides us with scope to implement change and ultimately improve patient care.

Despite the aforementioned limitations, we believe that the format of our service evaluation has scope for implementation in other clinical environments, such as OMFS units in district general hospitals, and with other patient cohorts, such as those undergoing treatment with intravenous sedation or general anaesthetic. We reserve concerns over the long-term use of PREMs and PROMs in the current format due to the practicality and the logistics of collecting and processing data over an extended period and recognise a need to develop comprehensive measures within dentistry and oral surgery. We also recognise the benefit of developing an automated approach to data collection and analysis such as through text messaging services and recommend further research into both of these areas.