Clinical Audit

Abstract

In diagnostic imaging and radiotherapy practice clinical audit is particularly concerned with service provision, patient experience, patient safety, radiation dose and accuracy of treatment or diagnosis. Clinical audit is the assessment of activity against a standard, in order to check that the quality of services is met. It is a continuous cycle to ensure a high standard of service is delivered and is therefore not a one-off procedure. Attention needs to be paid to all aspects within the audit cycle in order to have a successful, safe and compliant service. Clinical audit must be implemented as a routine and necessary part of maintaining best practice and can provide valuable research evidence. This chapter provides an overview of the clinical audit process giving an understanding of what it involves and why it is important in diagnostic imaging and radiotherapy practice.

1 Introduction

Clinical audits are a good way for new researchers to participate in studies, using some research methods without the complexity of ethical approval, participant information sheets or gaining consent from participants; so if you are new to research this is a good way to start. Projects involving clinical audit are normally easier to perform compared to research, if you have a limited amount of time or resources, as they usually have a set methodology, and data analysis is normally via descriptive statistics making it easier to implement and conduct whilst adding value to the improvement of services. If you are an early researcher, you can gain valuable experience from participating in these investigations.

2 What Is Audit?

Audit is the assessment of an activity which is measured against a national or local standard (known as the gold standard) in order to check for compliance [1, 2]. This may be in the form of, for example, guidance notes, protocol, procedure, trial specification. The aim of an audit is to achieve and maintain a high quality of care and services through the process of setting standards, observing practice, evaluating results, communication of results and when necessary implementing changes. The audit is a cycle and should therefore be repeated at regular intervals. Documentation must be in place in clinical departments to assist the workforce with their audit activity and to ensure standardised practice. All such documents require ownership and recognition of their accuracy and worth. Audit is an umbrella term. It is an effective method of measurement and analysis of processes that are already in place and also provides a means of introducing improvements that can be assessed over a specific timeframe. All aspects of a quality management system can be tested for compliance through audit. Audit is an essential component of clinical governance. Under the clinical governance framework all clinicians are required to be involved in audit activity. According to the Health and Care Professions Council (HCPC), you should be participating in audit as part of your training and as a qualified radiographer [3].

3 What Is Clinical Audit?

Clinical audit focuses on the clinical practice of healthcare professionals to set high standards of practice. Clinical audit is defined as a quality improvement cycle that involves measurement of the effectiveness of healthcare against agreed and proven standards for high quality and taking action to bring practice in line with these standards so as to improve the quality of care and health outcomes [1]. The term ‘audit’ is widely used in the healthcare sector and there are numerous terms associated with audit activity. In health care most audit activity is categorised under the umbrella term of ‘clinical audit’.

In the healthcare sector audits aim to provide:

• a systematic review of the clinical practice of the whole multidisciplinary team involved in the patient’s care,

• a process of measuring current practice against specified standards aimed at improving patient care,

• a tool to enable healthcare professionals to disseminate good practice,

• a tool to demonstrate evidence-based practice (EBP),

• an organisational/management tool to assess activity,

• a quality assurance (QA) tool so that action can be taken to remedy discrepancies.

In England and Wales, there is a National Clinical Audit and Patient Outcome Programme (NCAPOP) managed by Healthcare Quality Improvement Partnership (HQIP) [4]. Local clinical audits should include a range of staff groups to provide a depth of understanding, so whether you are a student or a qualified radiographer you should be participating in audit in some way. Being a systematic process, a clinical audit allows us as radiographers to assess whether what we are doing is what we should be doing in relation to the service we provide to our patients and service users [4]. By performing audits, we are ensuring that patients are receiving an effective service or treatment. Audits are a useful way of evaluating our services and their impact on patient care, outcomes and service delivery.

Clinical audits are very important in both medical imaging and radiotherapy practice. Without audits it is difficult to understand the current standards of work within a department and what requires improvement. For example, waiting times in an Accident and Emergency (A&E) X-ray department. How long does it take from a patient’s arrival time in the department to the time their images are available on the PACS? Another important area which is commonly audited is the rate of rejected radiographs in imaging departments. This involves considering the number of times an examination is repeated before the final images are sent to the PACS, and the reason for the repeated radiographs. In radiotherapy, you could audit the time taken from a patient’s referral for radiotherapy treatment to their first actual radiotherapy session and assess the consistency in referral times, or, if there are delays, explore the reasons for the delays. Another example could involve the assessment of the number of patients who develop skin reactions within a set timeframe following radiotherapy treatment for breast cancer. Without carrying out these audits it is difficult to highlight the problems which are likely to occur when providing these services or making recommendations to improve such services. Getting involved in clinical audits as a Band 5 radiographer helps you to understand the demands of the services you work for and gets you started in feeling involved as a part of the department. This involvement should inspire you to make a difference. If you have a chance you should get involved with audits as a student. This will help you decide on the type of audits you would like to conduct when you qualify. The experience will also help you to learn the stages of the audit cycle as well as how to analyse audit data and report on the results. Audits make feasible undergraduate projects because you can undertake them within a set period of time, as a small-scale project.

4 Forms of Audit

There are many ways to conduct an audit and they can be categorised depending on the method in which the data is to be collected.

• 1. Compliance audit: this involves ensuring compliance with a set standard and could be in the form of professional guidelines, national protocols or local policies and procedures.

• 2. Audit trail or process audits: this involves assessing the results of an audit from input to output to ensure that the information it yields are being effectively and efficiently managed and explained. For example, a patient from a radiotherapy treatment localisation or diagnostic scan to provision of follow-up with a view to improve the efficiency or effectiveness of the process.

• 3. Improvement audit: this involves using either of the audit types above to review a process or procedure to identity improvements. This audit process is used primarily in an area where an issue has already been identified and a systematic approach is required to implement change.

• 4. Documentation audit: this involves a review of a specific document to ensure that the content is appropriate and relevant to current practice.

5 How to Plan and Conduct a Clinical Audit

Before conducting an audit ensure that you know which type of audit you would like to undertake and that you have a clear strategy in place. Consider whether you will do the audit retrospectively or prospectively? Retrospective audits involve auditing existing information sources and are less complicated as there is no direct patient involvement. The disadvantage of retrospective audits is that some of the data required could be absent or incomplete.

Prospective audits gather more complete and accurate data sets but can take longer since a predetermined amount of data is collected over a period of time. They provide a more structured audit giving greater depth. Often the audit cycle starts with a retrospective audit showing areas for improvement followed by a prospective audit after recommended changes have been made. Prospective data is more accurate. It allows for real time data which reflects current, rather than historic, practice. Case notes are easily assessable and the pro forma can be designed to ensure that all relevant data are collected. Again this is an activity in which time must have a bearing on sample size. If you are reliant on others to record the data for you, the data collection again may be inaccurate or incomplete. Colleague cooperation will be essential. Audit should not inhibit normal clinical activity and data collection by colleagues should not be laborious.

Before performing prospective audits, you must check that research is not being undertaken, which is especially important if there is any direct patient involvement or change to the service a patient would receive. This can affect the nature of the patient involvement including how much time is being asked of the patient to participate in both the research and audit activities. Chapter 6 gives detailed guidance on ethical considerations.

Each department should ideally have their own audit plan template which must be used before starting a new audit. The audit plan template allows you to identify the question to be answered from your audit, the methodology that you will use including the timeframe to carry out the audit.

In addition to mandatory audit topics, the choice of topic should be based on the standard criteria of areas with high volume, high cost and high risk. Audit is an effective tool for change in specific areas where compliance is weak and an improvement in practice or assessment of process is required. Audit activity needs to be appropriate for an individual’s level of influence, ensuring results will impact on changing activity. Traditionally persons independent of the task being audited undertake audit activity, for example, finance audits. This is to ensure that activity is transparent and without bias. This method would be effective if a review of services was taking place or several work areas are being compared for efficiency. However, within the healthcare sector it is common for an individual to assess an activity that relates specifically to his or her own level of work, or an activity that is having a fundamental effect on how they work. In this way the audit is more likely to influence and change current practice. Care must be taken not to conduct an audit where criticism or blame is directed at another staff group or department without their knowledge or involvement. Joint audit is far more effective where the goal is to improve the quality of care provided and not to pass on blame.

6 The Audit Timeframe

The timeframe in which the audit is to be conducted has a major impact on the audit itself. If the data collection period is too long, interest will be lost and data may no longer be accurate. Any enforced time constraints should be respected; hence, the scope of the data collection and type of analysis required need to be considered at original audit design. There is little point in generating 6 months’ worth of data requiring analysis of thousands of samples unless you have the means by which the statistical analysis can be conducted. Do not design separate data collection tools if the information is already recorded elsewhere. The sample size should be small enough to allow for rapid data collection but large enough to be representative. When designing an audit, potential seasonal fluctuations should be considered, for example, patient throughput may increase following the summer vacation period, hence a larger sample of data may be required to ensure data collection accurately reflects the current situation. A multidisciplinary approach will ensure that such patterns are recognised and the audit designed appropriately.

The aim of the audit must be defined from the outset. An awareness of what is to be achieved will keep continued focus on the audit and ensure that activity is worthwhile. An accurate definition of the audit question and the defined scope are essential to evaluate results within the context of the actual data collated. The audit questions need to be specific and unbiased. The data collection should be transparent to present facts that accurately reflect conclusions being made. The data collected should answer the audit question.

7 Who Should Be Involved in Clinical Audit?

A clinical audit should involve several staff groups within the healthcare environment. Patients, service users, carers, relatives, commissioners, managers and board trustees could also contribute. Having different groups of people involved could enrich the interpretation of the data since it will be viewed from a different perspective. Furthermore, for audit activity to be effective and meaningful it requires a coordinated multidisciplinary approach. There will be a named clinical governance lead for most departments performing a clinical activity. Departments with quality systems in place based on the International Organisation for Standardisation (ISO) requirements will also have a named quality manager to manage the approach towards audit activity. Audit is an effective method of measurement and analysis of processes that are already in place and also provides a means of introducing improvements that can be assessed over a specific timeframe. This will ensure that the requirements of the Trust, directorate, local users and patients are continuously monitored to ensure a quality service is being delivered.

However, undergraduate student radiographers are increasingly conducting audits that involve collecting retrospective data. These audits typically involve patient or radiographer data only and not the patient, service user or radiographer directly. Students wishing to conduct an audit using retrospective data must write to the department manager to seek permission to access the data required. The permission must be granted before data collection can begin. The department manager would normally recommend that the PACS manager be available to help you retrieve the data. If undertaking an audit as a student radiographer, some hospitals may require you to complete a clinical audit proposal form and submit it to the appropriate person prior to the audit being approved. The department manager would be able to advise if this is the case.

8 Management of Audit Activity

Many Trusts will have a department dedicated to clinical governance/audit activity.

These departments will play a central role in project design, project management, data collection, data analysis and report production and often hold a list of specific audit activity that has to be conducted. This is often related to the current political agenda, for example, infection control audits. Diagnostic imaging or radiotherapy departments will have a named clinical governance lead. Departments with a quality management system (QMS) will have an audit plan for the year that ensures the system documentation is audited on a regular basis. This will be managed by the quality manager. Other departments may favour a team approach with multidisciplinary team meetings to coordinate audit activity. Whichever method is used, the audit reports and recommendation will need to be managed by named individuals to ensure the audit process is effective and efficient at implementing a quality service. Under the clinical governance framework all clinicians are required to be involved in audit activity. For all other clinical staff, although it is not a compulsory activity, audit activity is often assessed during the performance appraisal process with managers.

9 The Audit Cycle

Clinical audit is not a one-off procedure but is a continuous cycle to ensure a high standard of service is delivered. Attention needs to be paid to all aspects of the cycle in order to have a successful/safe and compliant service. Successful audits follow a cycle with defined stages of completion. Figure 14.1 shows a typical audit cycle.

Fig. 14.1

The audit cycle

10 Identifying Standards

When you have decided on your audit you will need to refer that to a standard. The standard for an audit, as previously mentioned, may be derived from national guidance, local protocols, regulatory compliance, best practice, local expert opinion or previous audits. All audits require standards to measure the accuracy of the audit. Standards can be set nationally or locally depending on the improvement or clinical outcome being measured. You should ask members of staff in the department if the audit has been conducted before and if a local standard exists. High standards may only be achieved by a few, but they could encourage improvement. Where patient safety is a factor, for example, an audit to assess whether the World Health Organisation (WHO) checklist has being completed for all sedated, analgesia and anaesthesia patients in radiology, the target must be 100% [5]. The Royal College of Radiologists (RCR) in conjunction with HQIP has a list of completed projects with standards which can be used for local audit. You should have a look at these to find out if your audit is on the RCR list.

11 Indicator to Be Measured

The indicators are measurable variables that should be identified during the planning stage. If you are undertaking a local audit which has been performed before, you should use the same indicators. The indicator can be expressed as an absolute number, percentage, average or rate. Audits may need several indicators to assess a complex process. For example, you want to highlight to management that an extra X-ray room for the A&E X-ray department is required. To justify this you can do an audit to determine the work flow in the A&E X-ray department at different times of the day. You would have to consider the average number of patients coming in, including the minimum and maximum number at various time points. You may also need to look at the transportation of these patients such as, are they ambulant or on a wheelchair or trolley, and how long each one occupies the X-ray room for. Or in the case of radiotherapy, your department may have introduced a new type of scan prior to radiotherapy treatment, such as a new MRI sequence to demonstrate the position of the brachytherapy rods. In this case each scan will be assessed for the accurate position of the brachytherapy rods. You will have to record the number of patients having an MRI scan prior to radiotherapy treatment and the percentage of patients with accurate positioning of the brachytherapy rods.

12 Data Collection

The sample size is very important for the accurate results of any audit, along with the timescale used as this will have direct bearing on the sample size. If the data collected in the audit is to be representative of a wider population, then it is important to have a representative sample size. During any audit you want to ensure that you have high confidence of accuracy levels in your results to give an overall picture of the entire service. There are calculators available online that can be used to calculate the sample size required in order for the results of the audit to yield a high confidence level, i.e., a confidence level of 95%. A larger sample size is required to identify a small margin of error which is usually considered in the range of ±2.5%. For example, in a population of 500 people you may require a sample size of 380 to give a 95% confidence level with a ±2.5% margin of error, whereas a sample size of 80 would give a ±10% margin of error. The results will be more powerful if the margin of error is smaller.

13 Analysis

If the standard is attained, it is an assurance of the quality of the service provided and gives positive reassurance that no change is necessary. The audit can be repeated at a relevant interval though, such as 1 year or 6 months, to check for consistent compliance. When results do not meet the standards, all possible reasons for not meeting the standards should be examined such as target level, system, process, technical reasons and so on. The possibility of sampling bias must also be considered before recommending any changes. Only after this analysis should a change in practice be implemented, if necessary. It is always best to have a list of suggested changes put together by the team in order to improve outcomes of a re-audit. Following any change in practice, a follow-up or repeat audit is required at an appropriate time interval. An improvement in service can only be proved following the repeat audit and only if it shows an improvement in results.

14 Audit Report

The audit must be written up at the end of the process in the form of a report with aim/s, methodology, results, analysis and conclusion along with recommendations for change in practice, policy or protocol. Accurate recording of the audit procedure allows the same audit to be repeated by any individual at a later date. The report should be discussed with the relevant staff members and changes can be planned and then implemented in the next cycle. The structure and presentation of the report is similar to that of the dissertation but shorter in length and more concisely written. If you are conducting the audit as part of your undergraduate training course, follow the layout that is required by your educational institution but typically the audit report will cover the following areas.

• Contents page: This page lists the sections that are included within the audit report.

• Executive summary: This is the summary of your whole report. It is usually in the region of about 300 words. It allows the reader to identify the key information. It should include a summary of background information, key findings and recommendations. It should also include aims and objectives and keywords. Much like the abstract in a dissertation, the executive summary is written last.

• Introduction including background and rationale: This section should include essential background information which puts the audit in context; it should set the scene and describe the reasons for undertaking the audit. Recent audits or related publications should be critically examined in order to justify the need for the audit.

  Perceived benefits to practice should be included as well as the standards and guidelines you are comparing practice against.

  Aims and objectives should be clearly laid out in this section. Objectives should be a statement of what you are trying to achieve by undertaking the audit and should reflect a commitment to improve practice.

• Methodology: This section should include all the details of the data that is to be collected during the audit. It must be written in a clear and logical manner so that if the audit is to be repeated by another person, it would enable them to conduct it in the same way. The following points are not indicated as subheadings to be used but rather as a guide as to what should be addressed within this section:

  • Criteria: Here you should consider what is to be measured? You can refer to your aim and objectives to help you with this information.

  • Ethical considerations: Here you should consider confidentiality and sensitivity of data. Acknowledge that ethical approval is not required in clinical audits; however, ethical considerations such as confidentiality, anonymity and data protection need to be explicitly made.

  • Data collection: Here you must consider who is/ are to be involved in the supervision of the collection of the data. In the case of students conducting the audit, most hospitals will nominate a qualified member of staff to guide the student during the data collection process.

  • The design of the audit: Here you should consider whether the audit is prospective or retrospective and the type of audit design. You should also state justification for why you think so.

    • Data sources: Here you must specify where the data was obtained from, for example, PACS, RIS or DAP meters in retrospective audits or from radiographers, patients or service users if the audit is prospective. You can also provide details on who collected the data and whether the data were validated by the Trust staff involved in the audit. In addition, you should specify the time period that the data was collected and why.

    • Sample: Here you can consider questions such as, ‘what is the sample size?’, ‘how was this determined?’

    • Procedure: Here you are required to give a clear outline of the procedure you followed in collecting your data by providing a step by step explanation of the method that was used. The procedure is important for replicating the study in case of a re-audit. Information on data analysis methods used should be included, as well as details of computer packages if used.

• Results: This section should report on the compliance against each of the standards/audit measures that you are comparing your practice against. This should include the number of cases that were compliant and the percentage compliance. There should be further investigation/explanation of where any non-compliance has been identified.

  Present findings in a logical, sensible order. Be selective with the use of charts, remembering to use the most appropriate method to present the data, e.g. pie charts to show proportions and bar charts for easy comparison between different areas/time periods. It is important to be consistent in presenting results; do not mix bar charts, pie charts and line graphs for similar data. You may refer to HQIP (2018) [6] for help with analysing and presenting your results.

• Analysis: This is the part of the audit which most clearly demonstrates your ability to discuss, evaluate, analyse and interpret the results in relation to the original reasons for undertaking the audit and findings of previous audits.

  Take care to critically analyse your results and not simply describe them. Refer back to the results and your initial rationale for conducting the audit. Where your results differ from previous published evidence, you should explain and justify why this might be. Include any real or perceived weaknesses of your audit design.

• Conclusion and recommendations: The conclusions should summarise the key findings from your results. This should identify the areas of good practice where standards are being met and identify the areas for improvement where there is a gap in compliance. This is the only section where you can express an opinion based on the application of the results. This section should summarise the extent to which the aims were achieved. It should end with recommendations for implementation or a re-audit. In the case of a re-audit being recommended, a timeframe for this to take place should be specified.

• References: This should be a list of the resources, including publications, that were used within the report.

• Appendices: These should be presented in numerical order as cited within the body of the report.

15 Communication of Results and Any Changes

Once an audit has been completed it is important that the outcomes of the results are disseminated to the relevant groups along with any actions identified. This will affect the success or failure of the audit if it is to be repeated. This can be done in the form of a simple discussion in a staff meeting, email or a presentation depending on the nature of the audit. If the current audit has not met compliance, then further action is required, and staff should be made aware of what is expected from them. If the current audit has met compliance, staff also need to be made aware in order for them to maintain their standards of practice in running a successful department.

At this point you should announce whether the audit would be repeated or closed down.

16 A Step by Step Guide to Help You Design an Audit

Figure 14.2 below gives an outline of a ten-step guide to designing and conducting a clinical audit. Figure 14.3 gives an example of how to implement the steps within the guide.

Fig. 14.2

Ten-step guide to conducting a clinical audit

Fig. 14.3

An example of implementing the ten-step guide

17 Data Protection and Information Governance

When performing a clinical audit it is important that you gain relevant permissions prior to starting. An audit will not require ethical approval or consent from patients because it is a review of the care provided but permissions may be required to access patient information. Wherever possible, data collected for audit purposes should be anonymised. Where data cannot be anonymised, it must be stored appropriately and not removed from the hospital/clinic where it was accessed and collated. Any reports produced must be anonymous and any non-anonymous data must be destroyed once analysis of the data has been completed. Patients must not be individually identified. See also Chap. 6.

Prior to the collection of data, you may need to consult the data protection officer and the data may need to be collected by a responsible person. When setting up the clinical audit the need for a data protection impact assessment should be considered. This is a process which helps you to identity and minimise any potential risks of your audit, including considerations regarding how the data will be collected and stored.

18 Statutory and Mandatory Requirements for Clinical Audit

Healthcare professionals are expected to take part in regular local and national clinical audits [3]. When clinical audits are carried out in accordance to best practice it:

• Improves the quality of care and patient outcomes [2, 4, 7]

• Provides assurance of compliance with clinical standards [2, 4, 7]

• Identifies and minimises risk, waste and inefficiencies [2, 4, 7]

• Complies with the ionising radiation (Medical Exposures) Regulations 2017 (IR(ME)R17, 2017) regulations [8] which states that employers’ procedures must include provisions for clinical audit as appropriate.

The National Health Service (NHS) standard contract forms the agreement between commissioners and providers of NHS funded services who must do the following.

• Participate in national clinical audits within the NCAPOP relevant to their services.

• Make national clinical audit data available to support publication of consultant level activity and outcome statistics.

• Implement and/or respond to any outcome measures.

• Implement ongoing programmes in accordance with good practice.

• On request, provide the coordinating commissioner with the findings of any audits carried out especially in relation to locally agreed requirements such as Commissioning for Quality and Innovation (CQUIN) audits.

NHS England and NHS improvement, and National Institute for Health and Clinical Excellence (NICE) [7] provide the clinical governance frameworks which feature strongly within NHS Trust clinical audit programmes. In addition, the Care Quality Commission (CQC) requires healthcare providers to constantly monitor the quality of their services. Access to the guidance from these organisations is available freely within the public domain.

19 Succeeding in Clinical Audits

In both diagnostic imaging and radiotherapy, someone embarking on clinical audit needs to consider the following principles.

• Do ensure that you have the permission of clinical managers, including radiologists or oncologists if necessary, to undertake the audit. It is best to obtain a written permission letter.

• Do consider the timeframe needed to undertake the audit. Although a ‘snapshot’ of clinical activity or performance at a certain point of time may be useful in many situations, such as in ongoing quality assurance, it may be necessary to collect data over a period of several months. Is this feasible for you?

• Do remember that you may not always be present at the clinical site to collect data. In your absence, are there people who have the time and ability to undertake data collection for you? And if so, will the data be collected in the same way that you would do yourself?

• Do check that there are measuring instruments or procedures in place which are capable of gathering the data you need. Also, are you capable of using them, or will you need training?

• Do not assume that retrospective data will always be available, or accurate.

20 Conclusion

Clinical audits act as a starting point for novice researchers and play an important role in measuring compliance and quality. To ensure a successful clinical audit it is important that the right persons are involved and that efforts are focused towards the question that needs to be answered. You must pay careful attention to the timeframe of data collection and ensure that your dataset is large enough to give you a high confidence level in your audit results. The audit report should clearly communicate the conduct and findings of the audit. The results and recommendations must always be disseminated within the department team and re-audits should be carried out as much as it is required in order to prove that standards have been met for the maintenance of a quality service.