Background

Mobile radiological diagnosis utilizing wireless communication devices would be a desirable extension of imaging services, particularly as an additional feature of existing Picture Archival and Communication System (PACS), for after-office-hours consultation. A portable high-resolution display screen, running image viewing software, would be a key requirement to allow off-site radiologists to provide second line support to on-call trainee residents interpreting in-hospital emergency studies, allowing safe and timely communication of important radiological findings to the clinician that may affect the patient’s treatment while at the same time removing restrictions in physical location. In recent years, mobile computing devices such as laptop personal computers, personal digital assistants (PDA) and smartphones have been introduced into the broader consumer electronics market, and their wireless mobility has been explored for use in medical teleconsultation [1–11]. However, although laptop computers have large screens and computing power, they are hampered by excess weight and short battery life. On the other hand, PDAs and smartphones have seen limited user acceptance in radiological image review mainly due to small screen size, despite high pixel per inch resolution in some smartphones (such as the iPhone 4).

The recent introduction of tablet computers such as the iPad (Apple Inc., Cupertino, CA, USA) and Galaxy (Samsung Inc., Seoul, Republic of Korea), which incorporate technological improvements in display resolution and a touchscreen interface, suggests that this larger-screen yet lightweight class of portable devices may have potential to successfully fill the need for remote wireless image review of radiological studies. Unfortunately, most DICOM-viewing applications come with a disclaimer of “not for primary diagnosis”, and there is a paucity of published literature documenting the accuracy of primary radiological diagnosis made on these newer tablet computers. Our objective therefore was to audit the agreement of radiological diagnosis of multi-image CT and MRI studies reviewed on the iPad tablet computer compared to a full-featured diagnostic PACS workstation for some common after-hours conditions, as this is a scenario we think mobile teleradiological consultation could potentially be useful. We also assessed the user experience of the iPad running on OsiriX HD v 2.0.2 DICOM application as this would determine user acceptance of this new technology.

Methods and Materials

Institutional review board approval was obtained for this retrospective study using annonymised patient data stored on the Radiology Department PACS in an 800-bed tertiary-level general hospital.

Equipment and Software

We compared a reference set of radiological images reviewed on two display devices; a standard PACS workstation using 6 Mbp diagnostic display monitors (Barco N.V, Kortrijk, Belgium) installed in the radiology department RIS/PACS network of our hospital (Microsoft Amalga, V.5.1, Redmond, WA, USA), compared with the first generation iPad (Apple Inc.) mobile tablet computer. Table 1 shows the hardware technical specifications used in both display devices; these are also compared with older generation PDA and the 4th generation iPhone (Apple Inc.). OsiriX HD v 2.02 (Pixmeo SARL, Geneva, Switzerland) was the program chosen for DICOM image display as it was an open-source software that was easily available at reasonable cost at the time of the study. The OsiriX HD DICOM viewer application which features basic image manipulation tools such as windowing, zooming and panning, measurement of distance and ROI, was purchased online from the iTunes store, downloaded and installed onto three first-generation iPad tablet computers.

Table 1 Display device specifications
Full size table

DICOM Study Selection and Comparisons

We retrospectively selected a sample of multidetector CT and MRI studies reported after hours by one Reporting Radiologist (Senior Consultant board—equivalent with 15 years of experience in body and general imaging) between July 2009 and January 2011 for review. All CT studies were performed on a 64 slice scanner (Toshiba Aquilion, Toshiba, Japan) and MRI studies were performed on a 1.5 T scanner (Siemens Symphony, Erlanger, Germany) using standard imaging protocols. The cases selected represented a spectrum of common acute conditions encountered during a busy after-hours call at a general hospital and included a mix of noncontrast and contrast-enhanced studies, including CT and MR angiographic examinations and their corresponding postprocessed images of various anatomical regions. Emergent MRI examinations included brain and spine studies for stroke, spinal trauma or suspected cord compression with neurologic deficits. An arbitrarily selected small number of normal studies (24 cases) were also included in the case list in order to better simulate a typical after-hours call and avoid reader bias.

The DICOM images of the studies were annonymised and transferred from the PACS server to a portable hard disk and subsequently wirelessly transmitted to the iPads via a Macbook laptop (Apple Inc.) running OsiriX v 3.9.2. This method was employed even though OsiriX is capable of real time DICOM query and retrieve via the Internet. This was because we were unable to successfully connect our iPads running OsiriX to our PACS network due to security issues, IT policy restrictions, and technical limitations in our RIS/PACS system. These studies were then independently reviewed by three designated readers who were board-equivalent attending radiologists (each with at least 10 years experience in general radiology) on the iPad. The readers were chosen from among radiologists already familiar with using OsiriX HD on Apple products and also specifically familiar with the user interface (including intuitive finger gesture input features) and consumer applications on the iPad tablet computer. There was no time restriction imposed on the readers within which the scans had to be read and interpreted. The reporting environment, including location and lighting, were also not standardized, in order to best simulate a real-world application.

Readers then completed structured reporting sheets documenting pertinent abnormal findings. These were compared to the formal clinical reports of the initial reporting radiologist retrieved from PACS. A single independent adjudicator who was a board-equivalent attending radiologist with 10 years experience in general radiology reviewed both sets of reports and assigned discrepant findings as major (clinically important findings that would affect immediate management and outcome) and minor (incidental findings that would not impact immediate management or outcome) discrepancies, based on a modified categorization of errors in radiology [12]. As long as a single discrepant finding was noted, the reading of the entire study was considered discrepant.

The readers also completed a user survey on their experience with the iPad, using a five-point Likert scale to rate the iPad on the ease and speed of image retrieval, image display (format, size, resolution, and brightness), comfort and ease of manipulation, user interface, and portability. The questionnaire also provided for a free text response for additional comments on reader experience.

Results

Eighty-eight studies comprising 79 CT examinations and nine MRI examinations were selected and interpreted on the iPad by each of the three readers for a total of 264 readings. Table 2 summarizes the range of cases reviewed on iPad tablet computers according to modality and region imaged. There were 24 normal studies (22 CT and two MRI studies).

Table 2 Range of cases reviewed on iPad tablet computers
Full size table

Table 3 summarizes the list of major and minor discrepant findings between iPad tablet computers and PACS workstations. Compared to the reports of the one reporting radiologist who originally interpreted the studies on PACS, 240 out of the 264 readings (90 %) were in total agreement. The 24 discrepant findings comprised nine major and 15 minor errors. Of the nine (3.4 %) major discrepant findings, eight were CT studies and one was an MRI study; three were regarded as interpretative errors and six as detection errors. Two abnormalities were detected but were each interpreted incorrectly by one reader. These comprised a cerebral arteriovenous malformation misdiagnosed as a dural arteriovenous fistula as the nidus was very small, and a vesicoureteric junction calculus that was interpreted as a bladder calculus. One patient with pulmonary embolism was misdiagnosed as having features of right ventricular strain by one reader. The remaining six major discrepant findings were not detected by one reader each. These included two cases each of traumatic intracranial hemorrhage (both on a background of multiple hemorrhages), one small cerebral infarct, one missed cecal foreign body, and one case where midline shift was not described in acute intracranial hemorrhage. There was only one major discrepant finding on MRI, which was an acute right cerebellar infarct on the background of multiple acute brain infarcts that was not detected by one reader.

Table 3 Discrepancy between iPad tablet computers and PACS workstations
Full size table

There were 15 (5.6 %) minor discrepant findings, of which two were missed by all three iPad readers, including the scar of a prior hepatic wedge resection in a CT abdomen and pelvis and frontoparietal encephalomalacia in an MRI brain. However, review of PACS clinical reports showed that all these cases had prior comparison studies with older images showing the abnormal findings in the acute stage. None of the 24 normal studies were incorrectly diagnosed.

Table 4 summarizes the survey taken by the readers about their experience using the OsiriX application on the iPad. All three readers found the image display format, comfort of use, and ease of image manipulation acceptable, and two out of the three readers rated the system ‘Good’ to ‘Very Good’ for ease and speed of image retrieval, portability, user interface, image display size, resolution, and brightness. All three readers rated the application ‘Poor’ for stability. Readers reported in free text comments that ‘screen size was inadequate for comparisons between multiple MR sequences within a single study’, ‘return to the main study directory when attempting to shift between image series in a single study was repetitive’, ‘lacks 3D angiography tools’, and ‘frequent crashes’, which negatively impacted their user experience. One left-handed reader remarked that ‘the horizontal scrollbar is fixed in position on the right side; this means both hands would need to be on the right side of the screen, which makes holding the device very awkward’.

Table 4 Survey of readers’ experience using OsiriX software on iPad tablet computers
Full size table

Discussion

In a simulated mobile review of emergency after-hours CT and MRI, we found good agreement between diagnostic reports viewed on iPad tablet computers running OsiriX software and those viewed on PACSs. Our results are consistent with published literature comparing tablet computers and smartphones with dedicated diagnostic workstations. A recent study comparing the reporting discrepancies of 100 emergency CT brain studies between the iPad and a 3-megapixel diagnostic quality display monitors found that 75 % of reports were in complete agreement, with only three clinically significant misses [13]. In another study, no significant loss of information was detected between CT and MRI reviewed on smartphones compared to diagnostic displays [14]. Researchers investigating the accuracies of a smartphone client–server teleradiology system compared to a dedicated workstation for acute stroke diagnosis reported specificity of 100 %, sensitivity of 94–97 %, and accuracy rates of 98–99 % [15]. We did not restrict the selection of emergency cases by diagnosis or body region, e.g., stroke, spine trauma, or acute appendicitis as in these studies [8, 15–17] but included a range of common emergent pathologies encountered in our hospital, including large angiographic studies with post-processed images. Our results suggest that the iPad is a viewing platform that has high levels of reporting concordance compared to PACS, regardless of the body region scanned, imaging protocols, and diagnosis.

There were nine major discrepant findings: three interpretative and six detection errors, from one reader each. However, none of these errors were committed by all three readers, and in all cases, the correct diagnosis was achieved by two of the other readers. This would imply that the discrepant readings were likely due to reader factors such as inherent variability in interobserver interpretation rather than hardware or software limitations with the iPad and OsiriX [18]. In two cases of the undetected focal intracranial hemorrhages and one case of a missed acute cerebellar infarct on MRI, the abnormalities were part of a broader spectrum of similar findings and although in themselves clinically significant findings, would not have impacted immediate management. With respect to the 15 minor discrepant findings, two chronic findings of prior hepatic wedge resection in a CT abdomen and pelvis study and frontal lobe encephalomalacia in an MRI brain study were missed by all three readers (accounting for six errors in total). We speculate that these small abnormalities were probably difficult to appreciate in the absence of comparison studies and clinical history, and highlight the importance of being able to retrieve historical studies and relevant clinical information on the iPad, as an extension of a fully functional PACS and RIS network, rather than just as an isolated stand-alone image review platform. Our study did not allow us to retrieve in real time, comparison studies in a practical manner as our iPads could not be wirelessly linked to the PACS network. The remaining nine minor discrepant findings were correctly diagnosed by at least one reader, which again suggests that it was not necessarily hardware issues that impeded the perception of the abnormalities, but could be attributed to the variability in reporting by the participating radiologists [18].

The user survey results showed that the iPad and the OsiriX application scored favorably for ease and speed of image retrieval, portability, screen size, and user interface. The readers did not experience difficulty learning to use the OsiriX application due to the intuitive nature of the iPad controls, but generalizability of these results may be limited to early-adopters who are enthusiastic and familiar with tablet computers. All the readers found the image display format, and ease of image manipulation to be acceptable. The 10-in. display size of the iPad, although larger than the smartphone, is still smaller than that of a laptop or the standard diagnostic monitors, which are optimized for reading large volumes of imaging studies. This is an inherent limitation of the tablet computer platform but may be an acceptable compromise in favor of decreased weight and improved portability. Readers also noted that it was cumbersome to have to repeatedly switch to the study directory to select a different series in MRI or CT studies with multiple series. This limitation however, has since been addressed by the developer and the current version allows for two series in a study to be compared side by side in a single view, although this is still not the case for historical and current study comparisons [19]. The readers also noted the limited availability of 3D and angiographic post processing tools. However, as the current workflow in our department is to perform the standard image post-processing on a dedicated workstation, prior to uploading into PACS, this may not have much impact if the majority of studies do not require further image reconstruction or reprocessing in order for the diagnosis to be made remotely. One left-handed reader highlighted the need for user-customizable graphic user interface, and the option for a virtual or peripheral add-on device such as a scroll-wheel interface might be a possible solution. Finally, software stability issues with system crashes, although not logged or correlated for study size or number of images, would not be unfamiliar to radiologists working in the digital realm, particularly in the early iterations of any new technology. This should be addressed by the software developers in more recent software versions, and may represent an opportunity for vendors to incorporate both hardware and software improvements into an integrated, feature-rich and user-friendly diagnostic quality mobile tablet computer to function as a review extension of PACS.

Our study is limited by the relatively small number of participating readers and unselected cases that were reviewed, which warrants cautious interpretation of the results. We have sought to mitigate this limitation by having a larger number of data points per reader, with each reading 88 studies. Larger studies with more readers and representative cases would be desirable to allow more definitive conclusions. We made an assumption that the reports of the primary reporting radiologist were completely accurate; future studies that require the same radiologist to prospectively review studies both on PACS and the iPad using histological, surgical, or clinical findings and follow-up as the gold standard would avoid this limitation. We did not investigate the time taken to read a study on the iPad as compared to PACS, which might be expected to be longer and may place restrictions on the volume of studies that can be read consecutively on the iPad in a single session. However, this is not anticipated to be a major issue in the scenario of after-hours teleradiology consultation where typically fewer studies require consultation. As the cases were retrospectively uploaded and not an integral part of PACS, we did not evaluate the transfer or upload speeds of studies to the iPad, directly from the PACS servers. This is an important factor that should be investigated as it may significantly impact timely communication of results and reader experience, and this may vary depending on the network provider, accessibility, and signal strength. Finally, our study did not standardize reading environments and lighting conditions of the readers as we felt this was important in order to better simulate real-world conditions. That our results were comparable with other studies with standardized reading environments seems to suggest that the iPad is capable of performing well regardless of the reading conditions chosen by the radiologists [15].

Conclusion

Our study shows that the diagnosis of emergency conditions commonly encountered in after-hours calls on CT and MRI using tablet computers such as the iPad can be made with good agreement to those reviewed on dedicated PACS workstations. User experience was favorable, although there were shortcomings in software and application design that might represent an opportunity for commercial PACS vendors and tablet computer application developers to address. Tablet computers with their excellent portability and large screens may have potential as remote mobile radiological image review and teleconsultation devices.