Background

Patients’ experiences of radiology services centre on the key issues of availability and waiting times [1]. The three key measures of patients’ waiting times within radiology departments are the time intervals between: (1) referral and examination, i.e. the pre-examination waiting time (PEWT); (2) examination and finalised radiology report, i.e. the report turnaround time (RTAT) [1] and (3) referral and finalised radiology report, i.e. the total radiology waiting time (TRWT), which is 1 and 2 combined. Unless otherwise stated patients’ waiting time is used to represent all three aspects of patients’ waiting experiences from now on in this paper.

There is a time boundary within which the radiology examination (report) is of clinical importance [2]. Increasing financial, political and clinical pressures to reduce the waiting times for radiology examinations have meant that many radiology departments are implementing a variety of service delivery initiatives (SDIs). The breadth of SDIs is wide, ranging from small scale inexpensive changes to practice, to large costly initiatives. There is a dearth of literature on how best to evaluate these SDIs within radiology departments, where pragmatic constraints often mean that randomised controlled trials are not feasible. Consequently, the methods and quality with which SDIs are evaluated within radiology settings is often mixed. In spite of, and perhaps even because of these constraints, a review of the type of SDIs, methods of evaluation used, and evidence of effectiveness, would be a useful addition to the literature. There has been no synthesis of evidence on the effectiveness of the frequently adopted SDIs at reducing patients’ waiting times within radiology departments. A few reviews on the causes of elongated hospital waiting times and the impact of various improvement strategies have been published [3–5]. However, many of these reviews were unsystematic [6] and have only focused on the waiting lists for surgical care.

Our aim was therefore to systematically review the literature to address how effective SDIs implemented within radiology departments are at reducing patients’ waiting times. Evidence of this form will allow for a more effective guidance to radiology service managers who are keen to improve their services and, those designing and conducting studies evaluating the effectiveness of SDIs within radiology departments.

The Global Radiology Workflow

The radiology workflow starts with the request for a radiology examination by a clinician and ends with the finalised radiology report (Fig. 1). The radiology workflow steps can be optimised with different type of SDIs, for example, the traditional hardcopy imaging request form is often replaced with a computerised physician order entry systems (CPOE). This should reduce the requests delivery times and intuitively lead to quicker turnaround times. A key workflow step is the transcription of radiology reports. The human transcriptionist might be replaced with a speech recognition reporting (SRR) system which converts dictated report to written text.

Fig. 1
figure 1

Global radiology workflow and possible workflow improvement strategies. TRWT total radiology waiting time; PEWT pre-examination waiting time; DT dictation time; TT transcription time; ST signature time; WDE waiting on the day of examination; RTAT report turnaround time; SDIs service delivery initiatives; ER electronic requesting; CPOE computerised physician order entry; RIS radiology information system; DR digital radiography, CR computed radiography; PACS picture archival and communication system; ESP extended scope practice; TR teleradiology; PNS pager-notification system; QMMs quality management methodologies; HIS hospital information system; SRR speech recognition reporting; WMS workflow management system

Full size image

Methods

A systematic review is a protocol-driven attempt to discover, evaluate and synthesize all the empirical evidence that meet pre-specified criteria, to address a given research question, using explicit methods to minimise bias, with an objective of producing a more reliable findings that can be used to inform decision making [7]. This review incorporates methods from the Cochrane Collaboration [7], Centre for Reviews and Dissemination [8] and the PRISMA statement [9]. The general structure of this review, organisation of search and the risk of bias assessment followed the Cochrane guideline. Data synthesis followed the Centre for Review and Dissemination guideline. We adopted the PRISMA guideline for reporting systematic reviews. The PRISMA guideline is widely endorsed [10]. These guidelines were combined because preliminary literature search revealed a diverse study designs and settings and we did not wish to impose a highly restrictive inclusion criteria.

Data Sources

We searched MEDLINE, EMBASE, CINAHL, INSPEC and The Cochrane Library for relevant articles published between 1995 and February, 2013. The search was organised in line with the PICO framework: Population/Problem; Intervention, Comparison (optional) and Outcome. The search strategy implemented on MEDLINE is shown on appendix I. The strategy combined Medical Subject Heading (MeSH) with free text terms.

Inclusion and Exclusion Criteria

We included articles that reported objective measures of the impact of SDIs on patients’ waiting times within routine clinical settings. This is expressed as the time waited from referral to either examination or finalised radiology report, from examination to finalised radiology report or the number/proportion of patients that waited above or below a specified time.

The type of SDIs included extended scope practice (ESP), quality management, SRR, electronic requesting etc. (Fig. 1). Only English language articles were included.

Studies which addressed diagnostic performances without reference to patients’ waiting times, clinical interventions, simulation, opinion, editorials and other non-empirical studies were excluded.

Study Selection Process

All identified articles were imported into EndNote X6™ and duplicates removed (Fig. 2). One of the reviewers (BO) screened the list of unique articles by title and abstract. The full text articles assessed as ‘potentially relevant’ were retrieved.

Fig. 2
figure 2

Articles filtration process

Full size image

The inclusion and exclusion criteria were independently applied to the potentially relevant articles by BO and YFC and reasons given for exclusions. Disagreements were resolved by discussions. Opinions were sought from a third reviewer (KH or AG) when required. The reference lists of the included articles were hand searched and identified articles were added to the review database. Data were extracted from the included studies by BO and YFC. We stratified and sub-stratified the studies by the type of SDIs and study design, respectively.

Risk of Bias Assessment

Risk of bias assessment addresses the key question of the extent to which the results of a study can be believed [11].

A bias is a systematic error, or deviation from the truth, in results or inferences … meaning that multiple replication of the same study will reach the wrong answer on average [11].

Tools have been developed to assess the risk of bias in studies, however these tools are developed for studies meeting certain minimum research design requirements. The Cochrane Collaboration Effectiveness of Practice and Organisation of Care (EPOC) review group risk of bias tool [12] was used to assess the risk of bias on studies that were: either randomised controlled trials (RCT), non-randomised controlled trials, controlled before-after (CBA) with 2 control and intervention sites or interrupted time series (ITS) [13]. The risk of bias was not assessed on studies that used either the uncontrolled before-and-after or post-intervention only designs because these designs are already known to be inherently susceptible to a high risk of bias [7, 13].

Data Synthesis

Data synthesis involves the collation, combination and summary of the results of individual studies included in a systematic review. Data synthesis can be done quantitatively using formal statistical procedures such as meta-analysis, or if formal pooling of results is unsuitable, through a narrative approach [8]. Pooling of results obtained from diverse non-randomised study types is not recommended [14]. Similarly, meta-analysis of poor quality studies could be seriously misleading as errors or biases in individual studies would be compounded and the very act of synthesis may give credence to poor quality studies [11]. Pooling of results in a meta-analysis is inappropriate for the current review due to a high level of heterogeneity. The narrative synthesis is widely used in this situation [15, 16]. Therefore, we present a narrative synthesis of our findings. This is a textual approach that provides an analysis of the relationships within and between studies and an overall assessment of the robustness of the evidence [8].

Results

Our search yielded 11,056 articles (Fig. 2). We screened 9,765 articles by titles and abstract after removing duplicates (n = 1,291). We excluded 9,621 as not relevant leaving 144 articles eligible for full text review. Full text for 3 articles could not be obtained from the British Library and were therefore excluded. Seventy-eight articles were excluded with reasons: did not report the outcome measures of interest [17–68]; did not report objective measure of patients’ waiting times, is an opinion piece, an editorial or theoretical paper [69–83]; did not report any intervention [84–89] and was withdrawn from publication [90]. Sixty-three articles (57 studies) met the inclusion criteria.

Most of the studies (61 %: 35/57) were performed in the USA, 14 % (8 of 57) each in the UK and EU, respectively and 6 (11 %) within rest of the world. The majority of the studies (60 %: 34/57) were published between years 2000 and 2009, 17 % (10/57) before the year 2000 and 23 % (13/57) from and including years 2010 to February 2013. Forty-five studies (79 % of 57) used the pre- and post-intervention design while 15 % (9/57) used the post-intervention only design (Table 1). The RTAT outcome measure was reported in 65 % (37/57) of the studies while PEWT was reported in 30 % (17/57) (Table 1). The characteristics and main findings of the included studies are summarised in Appendix II. The results of the studies by type of SDIs are presented below.

Table 1 Included studies stratified by type of intervention, research design and component of patients’ waiting times reported (percentage in parenthesis)
Full size table

Extended Scope Practice

An ESP radiographer is one who has significantly extended their role and accordingly has supplementary clinical proficiency in a specified area of practice [91], e.g. image reporting. Three (5 %) of the 57 included studies evaluated ESP and all were performed in the UK. Two of these studies used the uncontrolled pre- and post-intervention design and covered different patient groups: accident & emergency (A&E) [92], inpatients and outpatients [93]. The third study used a time series design on A&E patients [94]. Different components of the patients’ waiting times were reported: RTAT [92, 94] and PEWT [93]. All three studies reported improved patients waiting times. For example, regression analysis suggests that increased proportion of A&E examinations reported by ESP radiographers is associated with 36.8 % reduction in RTAT, p < 0.001 [94], and there was a 75 and 62 % drop in the mean PEWT for inpatient and outpatient video fluoroscopy, respectively following ESP implementation [93].

Quality Management Methodologies

Quality management is a general approach to delivering services that meet service users’ needs with a more effective use of resources [95]. There are different approaches to quality management. Twelve (21 %) of the 57 studies investigated quality management strategies including the Lean, Six Sigma and continuous quality improvement methodologies [96–101] and process/service re-design [102–107]. The type of study designs used include the controlled pre- and post-intervention [99], the post-intervention [104] and the remaining 10 studies adopted the uncontrolled pre- and post-intervention design. The PEWT were reported in seven studies [96, 98, 100–102, 104, 107], one study reported the RTAT [103] and two studies reported the TRWT [99, 106]. Two studies failed to define the timelines used in computing the outcome measures [97, 105].

Most of these studies reported improved outcomes [96–105, 107]. For example, the mean PEWTs were 56 (90 % CI 54, 57) and 36 (90 % CI 34, 38) min pre- and post-intervention with the Lean methodology, respectively [100]. However, one of these studies reported that the improvements were not sustained [99]. One study found increased waiting times following service re-design [106]: the TRWTs were 51 and 69 min for CT head; 69 and 82 min for body CT pre- and post-service re-design, respectively.

Outsourcing

Outsourcing is a situation where the radiology department (not the referring physician) contracts an examination or parts of it (e.g. reporting) to an outside agency [108]. One study evaluated the impact on PEWT of outsourcing radiology examinations [108]. This study used the post-intervention only design to compare the PEWT of outsourced examination with those performed in-house. The study found no statistically significant difference between the groups, in either the number of examinations that were not performed within the preferred time or the number of days that exceeded the preferred waiting time target. However, for examinations without a preferred timeframe, the waiting time was shorter for outsourced investigations than those not outsourced.

Pay-for-Performance (PFP)

Pay-for-performance (PFP) is a financial incentive intended to inspire providers to deliver higher-quality care [109]. A PFP programme comprising $5,000 annual bonus payment to radiologists who met specified RTAT targets was evaluated for its impact on RTAT [109]. This study found that the mean RTATs were 43 (SD 99), 32 (SD 78) and 16 (SD 54) h before, after and 2-year follow-up periods, respectively, p < 0.0001.

Productivity-Enhancing Technologies (PETs)

Productivity-enhancing technologies (PET) is a collection of technologies for optimising the radiology workflow steps. The effectiveness of PETs at reducing patients’ waiting times was investigated in 51 % (29/57) of the studies. The technologies examined included speech recognition reporting, picture archival and communication systems, teleradiology, radiology information systems, computerised physician order entry systems and other.

Speech Recognition Reporting (SRR)

The SRR system works by converting spoken words into digital signal which is then transformed into written text. SRR was evaluated in 11 (19 % of 57) studies [110–121]. Two of the studies used the post-intervention design [119, 121] and the remaining nine studies used the uncontrolled pre- and post-intervention design. All 11 studies evaluated different patient populations and measured the RTAT with different time lines. All 11 studies reported varying degrees of improvement in RTAT. One of the studies noted that 2 of 30 radiologists did not experience improvement in their individual workflow following SRR implementation [114].

Picture Archival and Communication System (PACS)

The picture archival and communication system (PACS) is a structure of hardware and software system for handling, storing, organising and distributing digital images within the health care environment. Five (9 %) of the 57 included studies evaluated PACS [122–127]. The uncontrolled post-intervention design was used in one study [123] and the remaining four studies used the pre- and post-intervention design. Different patient populations were included: these were based on imaging modality [124, 126, 127], or referral sources [122, 125]. The definition of outcome measures also varied. The results for PACS is mixed, for example one study [124] found that the mean RTAT increased from 4 to 7 days for MRI, p < 0.001, remained stable at 2 days for CT and dropped from 4 to 3 days for plain X-rays. However, the overall departmental RTAT improved from 6 to 5 days p < 0.001. Another study found a 9 % improvement in RTAT [126, 127]. Yet another study reported that the median PEWT was significantly longer for plain X-rays after PACS implementation: increasing from 20 to 25 min for A&E patients and three to 42 min for patients on intensive care [125].

Teleradiology

Teleradiology is the method of sending digital radiology images from one location to another for the purpose of consultation and interpretation. Two (3.5 % of 57) studies on teleradiology met the inclusion criteria. Both studies found that teleradiology improved RTAT [2, 128]. For example, the proportion of reports completed within 40 min increased from 34 (95 % CI 29, 38) to 43 % (95 % CI 39, 47) pre- and post-intervention, respectively, p < 0.01 [2]. The two studies used different research designs: controlled pre- and post-intervention design [2] and controlled post-intervention design [128]. Both studies measured RTAT in using different timelines.

Radiology Information System (RIS)

The radiology information system (RIS) is a software system for managing and keeping permanent records of patients’ journeys through the radiology department. Two (3.5 % of 53) studies investigated the RIS and both used the pre- and post-intervention design. Both studies investigated different components of patient waiting times: the TRWT for orthopaedic patients [129] and the RTAT for MRI and mammography [130]. The results were mixed. One study found that the mean RTAT for mammography improved from 4.06 (SD 2.34) to 2.17 (SD 1.43) h while the RTAT for MRI increased from 3.11 (SD 1.87) to 3.20 (SD 1.85) h pre- and post-intervention, respectively [130]. These results were statistically significant at 5 %. The earlier study found that the mean TRWT reduced from 26.8 (SD 6.8) to 3.6 (SD 2.5) h following the RIS implementation [129].

Computerised Physician Order Entry (CPOE) Systems

CPOE is a system for requesting radiology examinations electronically instead of the papers-based methods. Four (7 % of 57) studies assessed the impact of CPOE on patients waiting times. All four studies used the uncontrolled pre- and post-intervention design. One study measured the TRWT [131] while the rest measured the PEWT [132–134]. The study populations varied: patients that presented with chest pain in the A&E department and subsequently had chest a X-ray [131], patients on adult Intensive Therapy Unit (ITU) who had urgent CT or plain imaging [133], patient referred for either plain chest/abdominal X-rays or abdominal ultrasound from the transplant service [134], the fourth study included only very low birth weight (VLBW) babies on the Neonatal Intensive Care Unit (NICU) who had abdominal or chest X-rays [132]. Three of the studies reported positive findings: The adult ITU study [133] found reduced median PEWT from 96 to 29 min p < 0.001 with less variation around the median, while the study involving patients referred from the transplant unit found that PEWT reduced from 7 to 4 h (49 %) p < 0.05 [134]. It was not specified if these were mean or median values. The VLBW study reported reduced mean order-to-image-display time from 42 to 32 min [132]. The fourth and most recent study reported no improvement in patient waiting times: TRWT remained stable at 80 min, p = 0.49 despite increased volume of requests [131]. Two of the four studies [132, 134] were from the same institution.

Other Technologies

The remaining 5 of 57 (9 %) studies investigated a wide range of productivity-enhancing-technologies. These technologies included pager-notification systems [135–137], digital imaging [138], computer coded reporting [139], workflow management system [140]. Two studies used the controlled post-intervention design [138, 139] while the remaining studies used the uncontrolled pre- and post-intervention designs [135–137, 140]. All five studies measured the RTAT with different timelines and included different patient population as well. Most of the studies reported positive findings [135–137, 139, 140], however one study noted that the gains were not sustained beyond 1 week post implementation of a pager-notification system [136, 137]. Mixed results were reported on a digital radiography system [138].

Multiple Interventions

We identified 11 (19 % of 57) studies where more than one type of intervention was evaluated. Most of the studies combined multiple PETs [141–150]. The remainder combined quality management methodologies (QMMs) with PETs [151–154]. The studies used varied research designs including the post-intervention only design [147, 148] and the uncontrolled pre- and post-intervention design. In terms of outcome measure, most of the studies reported RTAT [141, 142, 146–148, 151–154], two studies measured TRWT [143–145] and one study reported the PEWT [149].

Most of the studies reported positive findings [141, 142, 145, 146, 150, 152–154]. For example, the average RTATs were 115 and 23 h pre- and post-intervention, respectively [141, 150]. However, one study reported that the improvements were not sustained [152]. Another study reported negative findings [149]. This study reported that implementation of PACS and automated scheduler increased the PEWT from 0.12 to 0.27 h. Three studies reported mixed results [143, 144, 147, 148, 151]. For example one study [151] evaluated a combination of interventions and found that better staffing level, technology (use of SRR) improved RTAT while proposed sanction on non-compliance with RTAT target and education of staff on the need to comply with RTAT requirements did not improve RTAT.

Risk of Bias Assessment

Only one study [94] fully met the minimum design standard for a Cochrane review. Two studies [2, 99] used the controlled pre- and post-intervention design without the recommended minimum of two control and two intervention sites thereby meeting the standard only partially [13]. The Cochrane EPOC risk of bias tool [12] was used to assess the risk of bias on these three studies (Table 2). We did not assess the risk of bias on the remaining studies for two reasons: as earlier stated, the studies used research designs that are already known to be inherently susceptible to a high risk of bias [7], and we did not find any assessment tool either.

Table 2 Risk of bias assessment on the three studies that met the minimum design requirement for a Cochrane EPOC systematic review
Full size table

Discussion

Patients’ waiting times are a major indicator of the quality of care within radiology departments [155, 156]. Several type of intervention are being implemented by radiology departments to improve waiting times. Some individual study estimates of the impact of the SDIs on waiting times have been published but there is yet no synthesis of the evidence of their effectiveness. Recent systematic reviews have examined the impact of a single SDI on a range of outcome measures. For example, CPOE system was found to impact on imaging requesting behaviours, adherence to guidelines, length of hospital stay, mortality, readmission rates and radiology turnaround times [157]; PACS within the intensive care setting was found to impact on image availability, image viewing patterns, clinical decision, etc. [158]. These reviews have not focused on patient waiting times. We have used a different approach in the current review by evaluating the impact of popular SDIs implemented within radiology on an outcome measure of topical interest (patients’ waiting times). The studies included this review are highly heterogeneous and most (95 %) of them used study designs that can potentially lead to biased estimated of effect size and the reporting quality was poor. In the next sections, we discuss each type of intervention in terms of the theory behind how it works, why it might work for which type of organisation, the results and relationships between the studies with a focus on the studies with lower a risk of bias. The subsequent sections examine the robustness/quality of the evidence and the causes of heterogeneity in the studies.

Extended Scope Practice

ESP radiographers reporting has been employed by NHS organisations experiencing increased demand and shortage of radiologists [159], with attendant increased RTAT. ESP allows radiographers to extend their roles into some tasks traditionally undertaken by radiologists (e.g. plain film reporting) as means of increasing (reporting) capacity [159, 160]. A previous review [15] of the evidence on the effectiveness of ESP concluded that most of the studies explored the acceptance of these roles by other professional colleagues; however, their impacts on services were not evaluated. We found three ESP studies, all performed within the NHS, UK. This is not surprising given that the NHS is one of the first healthcare systems to implement ESP [160, 161]. Only one of the three studies [94] used a robust research design [13] and we performed a risk of bias assessment on it. Table 2 shows that, generally, this study has a low risk of bias. All three studies reported positive findings, suggesting that where appropriate ESP might be an effective strategy to reduce RTAT for A&E plain film. However, amongst other considerations for implementing EPS, an assessment must be made that increasing RTAT is due to shortfall in reporting capacity.

Quality Management Methodologies

The objective of QMMs is to identify and remove wastage from a system. QMMs appear to have a huge potential to improve the global radiology workflow processes especially when combined with PETs [152–154]. Implementing PETs without QMMs is unlikely to yield the optimum results [112, 145, 152]. It is not surprising therefore that the NHS is paying a greater attention to the Lean and Six Sigma methodologies [162, 163]. Only one [99] of the 12 included studies partially met the Cochrane EPOC study design requirements. This study implemented a seven-step continuous quality improvement (CQI) strategy on the intervention site and a ‘traditional’ management technique on the control site. The seven steps included using expert team to map the process, identify and understand the problems, select, design, implement and monitor the process improvement. This led to 18 % reduction in the proportion of chest X-ray examinations breaching the 2 h target. One [106] of 12 reported negative results. These two studies differ; in terms of population, imaging modality, methodology and type of intervention. Whereas the latter was done in an A&E department the former was done in an outpatients setting. The second study involved a barrage of interventions but did not follow a problem identification procedure. This might explain the difference in results between this two studies. Any radiology department will benefit from QMMs because the radiology workflow processes is particularly suited to process improvement. However, sufficient time must be invested in identifying and understanding the problem as well as its root cause(s).

Productivity-Enhancing Technologies

PETs include a host of technologies employed to improve process flow within the radiology departments. PETs were evaluated in 51 % (29/57) of the studies of which only one study [2] evaluating teleradiology partially met the minimum design standard for a Cochrane EPOC review. Teleradiology is mainly used by NHS hospitals for outsourcing routine reporting to cover shortfall in reporting capacity and provide cover for remote communities [164]. Kennedy et al. [2] found that teleradiology improved RTAT. A second [128] teleradiology study also reported positive findings. The results of both studies suggest that teleradiology might improve RTAT, however, this must be balanced against other quality parameters such as costs and referring clinicians’ satisfaction [164].

The importance of SRR is limited to addressing the time delay between report dictation and its transcription. Theoretically, SRR should improve the speed of report production by instantly transforming dictated reports into text. Therefore, the SRR intervention might only be useful to an organisation struggling with its transcription workload, as opposed to a shortfall in reporting capacity. Some researchers have argued that SRR only shifted the burden of transcription to the radiologists with detrimental effects on their productivity, which might result in higher aggregate costs [55, 63]. Other concerns included high error rates [63] and the brevity of reports generated with SRR (24–39 % shorter in length) compared to conventional dictation [55, 57]. All the 11 SRR studies included in this review reported varying degrees of improvements on RTAT. Some reported cost savings as well [112, 117], others reported that SRR had not improved the RTAT of some radiologists within the practice [114]. It was therefore thought that human behaviour might play a significant role on the extent of improvements observed. Although all 11 studies used designs with a high inherent risk of bias. The results suggest that a ‘total’ (100 %) SRR implementation might be more effective than partial implementation [111] and even better when combined with QMMs [112]. However 100 % SRR adoption might be a difficult proposition for organisations with teaching commitments [113].

PACS and RIS are the bedrocks of any modern radiology department. Both technologies impact patient waiting time by improving process flow; reducing time wasted on tracking films, patients’ records and appointments. The impact of PACS on patient waiting times is mixed. One study reported mixed results depending on referral sources [124]. Other studies observed no impact on waiting times [126, 127], deteriorated waiting times [125] and positive results [122, 123]. The situation is similar with the RIS: two studies with mixed results. We have found the evidence of the impact of PACS and RIS on patients waiting times to be both inconsistent and insufficient. A previous review on PACS reached similar conclusion [16]. However, we feel that the overall importance of these two systems to a large radiology department might outweigh any considerations of their empirical impact on reducing waiting times. The dynamics might be different for smaller departments processing only a few thousand cases per year.

Other Interventions

We found a few other promising technologies including electronic requesting [145], CPOE [131–134, 142]. CPOE improve waiting time by ensuring that requests are received by radiology departments almost instantaneously. Again, this technology might be useful for large departments having problems of not receiving requests in a timely manner/loosing requests forms. The earlier the requests are received, the sooner the examinations can be arranged. Of the four CPOE studies, only one [131] reported no improvement in waiting times. This is probably because the study examined chest X-ray requests in the A&E settings. Care in the A&E is usually fast paced, usually with X-rays performed in adjacent rooms. Therefore, CPOE might not make a drastic impression on this setting.

We found one study each evaluating PFP [109] and outsourcing [108]. PFP might be useful when routine QMMs fail and an organisation decides that staff needed additional incentive to improve quality [109]. The PFP study reported statistically significant improvement in RTAT. This is a single study estimate. The implications of PFP are a current debate topic in many health economies [165, 166]. Some think that there are too many obstacles for it to work in radiology [167], others feel that it can be easily abused [168] but most importantly there are insufficient outcome studies [169]. The study that evaluated outsourcing [108] found no difference in waiting between outsourced examinations and those performed in-house, when a preferred time frame was specified. We feel that a predictable consequence for the development of teleradiology is the potential for the outsourcing of radiology reporting. In 2009, 37 % of UK radiology department were already outsourcing parts of radiology reporting as a means of increasing capacity [164]. This review has found insufficient evidence that either PFP or outsourcing of radiology examinations improved patients’ waiting times.

Robustness/Quality of the Studies

Only one of 57 studies [94] fully met the minimum design standard for a Cochrane EPOC review. Two studies [2, 99] partially met the standard by using the controlled pre- and post-intervention design. These two studies do not have the recommended minimum two control and two intervention sites [13]. The pre- and post-intervention and the post-intervention only design were adopted in 95 % (54/57) of the studies (Table 1). There is empirical evidence that these study designs produce biased estimates [11, 170–172] and are prone to overestimating the effect size of an intervention [172, 173].

Reporting quality was generally poor. For example, many of the studies that reported positive findings did not test and/or report the statistical significance of their findings [93, 97, 98, 101, 102, 105, 121, 122, 126–128, 142, 146]. Only a few studies reported the confidence interval on their results [2, 100, 104]. Many of the studies did not define the timelines used in computing the outcome measures [92, 94, 97, 105, 115, 116, 120]. Virtually all the included studies failed to give any information on the technical features of the implemented systems or the IT infrastructure and the levels of integration within these settings. The IT infrastructure and levels of integration have a significant impact on the effectiveness of radiology SDIs [114, 141, 150]. The results of the studies must be viewed with the above quality issues in mind.

Exploration of Heterogeneity

The results of the studies could not be pooled in a meta-analysis due to a high level of heterogeneity. The causes of heterogeneity included varied research design, the breadth/combination of SDIs and variation in the study population and setting. The study population included patients who had specified examinations e.g. chest X-rays [99], CT pulmonary angiogram [2]; patient referred from specified sources like A&E [94], or imaging modalities [152]. Most importantly, we also found inconsistent definition of the outcome measures. The importance of consistent outcome measure definition has been highlighted [174]. We illustrate the inconsistent definition of the RTAT outcome measures: the time interval between the examination and the finalised report. This was measured by different studies to start from the: time a patient arrived the X-ray reception desk [138], start of examination [122, 154], completion of image acquisition [135–137], completion of the examination on the RIS [126, 127] and time the image became available on the PACS [140] to the time of final radiology report. Many studies failed to define the timelines used in computing RTAT [92, 94, 97, 105, 115, 116, 120]. The time interval between the completion of image acquisition and completion of the examination on the RIS is frequently more than 1 h [175]. Given that many of the studies reported improvements in minutes [2, 112], it is easy to see how inconsistent outcome measure definitions might affect the results of any comparison. We have therefore proposed generic timelines for defining patients waiting times in clinical radiology (Table 3). In addition, the IT environment within which the evaluated systems were implemented and the levels of integration were different and/or not discussed.

Table 3 Proposed generic timelines for defining patients waiting times in clinical radiology
Full size table

Implications for Future Research

Evidence of effectiveness is clearly paramount in the implementation of appropriate SDIs in radiology as a means to improve the patients’ experiences. Studies to date have been mostly of low quality and future studies need to be of a higher quality. Higher quality studies might consist of interrupted time series evaluations [170. p 171–172] or, randomised designs. As there is obviously a need for pragmatic evaluation, one possible appealing randomised design might be the stepped wedge study [176]. The stepped wedge design is a cluster study, and so would involve multiple sites or modalities, which would sequentially be randomised to receive the SDIs. In addition, there is a need to harmonise the definitions of the timelines used in computing patients’ waiting times to reduce the level of heterogeneity in the studies. We propose that the timelines should be defined as shown in Table 3. We also recommend that future studies should include basic details of the IT infrastructure and levels of integration. We think that this will make both comparison and meta-analysis less restrictive.

Limitations

It is possible that we have missed articles indexed under different MeSH headings or key words. We excluded non-English language papers. This might lead to language bias.

Conclusions

This review has highlighted the type of SDIs implemented to improve patients’ waiting times in radiology departments. Most of the studies used the post-intervention only design and the pre- and post-intervention designs without a control group. These designs are prone to overestimating effect size. It is therefore not surprising that majority of the studies had positive results. There is a need for higher-quality studies to improve the evidence base.

We found the studies to be highly heterogeneous and the reporting quality was poor. We understand that SDIs within radiology departments will impact on more than one quality measure. Therefore, we suggest that interested parties should critically appraise the studies for their designs, results, and the description of the elements of the evaluated systems that they think are critical to achieving their objectives. We propose that the definitions of patients’ waiting times should be mapped to generic timelines as a starting point for moving towards a situation where it becomes less restrictive to compare and/or pool the results of future studies.