Introduction

Since the discovery back in the XIX century, stereoscopy advanced to many different areas. The comprehensive analysis of papers and experiments studying the stereoscopic effect for the last 50 years shows that, in a majority of cases, usage of stereoscopic displays increases overall performance, if compared to conventional 2D displays [16]. For instance, it is known that the stereoscopic effect delivers better gaming experience [37, 42], enhances visual experience and naturalness of a visual image [20], increases the efficiency of a number of medical applications [44]. This could be explained by the way in which the stereoscopic depth influences our perception of visual information.

The stereoscopic depth could be seen as an approach to achieve more efficient use of an additional dimension in graphical user interfaces (GUI) layout. Stereoscopic depth planes could present a reliable solution for GUIs of medical applications, which could be heavily cluttered with functionality, controls and, in case of augmented reality UIs, real life objects. Enhanced visual experience and naturalness of a visual image [20] along with an increased level of immersion [26] (if combined with other technological components: e.g. screen resolution, field of view, interaction model) generated by the stereoscopic effect could in turn lead to more natural interaction with the system. A proper approach to this problem could enhance user experience of a system and efficiency of its UI.

The review [25] of 184 works researching impact of the 3D displays on human performance revealed that overwhelming 60 % of studies show a clear and definite performance benefit of S3D over 2D displays. This results are confirmed by a number of recent studies [4, 10, 11, 45] exploring the impact of the stereopsis on performance of user interfaces. Nevertheless, despite relative simplicity of the concept behind the stereoscopic effect, nowadays there are few effective and easy-to-use tools to implement stereoscopic depth into applications.

This paper presents a brief analysis of the state of the stereoscopic display technology and its application in medicine, reveals our endeavour in alleviation of the aforementioned problem by introducing a tool for development of stereoscopic user interfaces, and discloses results of the tool’s technical performance throughout the latest browsers and subjective performance among 30 participants.

State of the Art

Stereoscopic Display Technology

Stereopsis is a depth perception mechanism that is being thoroughly researched since its discovery more than a century ago [18]. Our binocular vision is what makes this effect possible. Differences and similarities between two images on left and right retinas of both eyes are processed by the brain to obtain the information on relative distances and a depth structure of an observed environment. Stereoscopic displays try to follow the same basic principle: provide the left and the right eye of an observer with two different images. Thus the illusion of depth is achieved. Such effect can be produced in a number of ways [17, 33], which all come to the simple idea of splitting a so called stereo-pair into two separate images and deliver both images to respective retinas, but the overwhelming majority of them are two-view 3D displays.

When it comes to the medical applications, 4 main types of stereoscopic two-view display systems (Fig. 1) are currently in use: (i) active, in which both left and right eye images are displayed on a single monitor, with the two images presented alternately on successive frames; (ii) passive, in which both left and right eye images are simultaneously delivered to the intended eye through independent channels, (iii) autostereoscopic, in which, as opposed to previous two, the stereo-pair distribution is done by display alone and does not require the observer to wear glasses or other headwear, and (iv) head mounted displays (HMDs), physically separating two images.

Fig. 1
figure 1

3D displays in medical applications: The DexVue screen (top left) using active shutter 3D technology, Planar StereoMirror SD2250 (top right) passive stereoscopic display system, Tianma NLT autostereoscopic display (bottom left), and Sony HMM-3000MT (bottom right) head mounted display

Full size image

Probably, the most used of these four is the time-sequential two-view displays. This technology uses interleaving of two images on the same display with a speed fast enough for the viewer not to notice the switching. Both images are then filtered by glasses equipped with an electronic shutter which delivers left or right images to the respective eye [13]. Currently this technology is used on most of 3DTVs and 3D-enabled PC monitors available on the market.

Stereoscopic Displays In Medical Applications

The first articles discussing possibilities of application of the stereoscopic effect in radiography, also known as stereoradiography, were published in the early XX century [2, 5]. The studies described the advantages that the stereoscopic vision can bring for better understanding of the spatial relationships within parts of a roentgenogram. In the 70s, with a discovery of slice-based volumetric imaging, such as magnetic resonance imaging, and algorithms for building 3D models out of obtained slices, the stereoradiography became close to be obsolete. Though with further increase of image resolution and quality, in some cases, the stereoscopic vision still has proven to have positive effect on a visual examination of those 3D compositions [22, 35, 48].

The same or even greater positive effect is observed in a number of medical areas. In angiography [8, 27] and neuroradiology [41], where proper perception of the spatial relations is crucial for judging the exact shape and the distance between the blood vessels. In ultrasound imaging, where stereoscopic displays help to decrease the noise in the images and better identify anatomical structures [14, 29]. Kockro et al. [1] found that the stereoscopic effect improves spatial orientation and confidence during the neurosurgical procedures, improves clarity of structural display and adds realism to the navigation process. The research [49] of impact of the stereoscopic displays on the performance of the robot assisted laparoscopy shown 40 % increase in teamwork speed. Applied on the visual ultrasound data evaluation [28], use of the 3D displays improved identification of fetal bony structures. Recent evaluation [24] of 3D display technology applied on visual-spatial tasks in medical context revealed statistically significant improvement in performance between distances of 5mm and 15mm. Further notable cases of application are observed in virtual endoscopy, ophthalmology, dermatology, and others.

Overall, stereoscopic displays are presented and successfully used in diagnosis, pre-operative planning, minimally invasive surgery, training and other medical domains. For more comprehensive overview of the effectiveness of stereoscopic displays in medicine see [44].

Platform

HTML5 was chosen as the main development platform due to the number of features that can help to build a stereo-pair and provide the end-user a way to interact with the output composition. Although the standard is still in development [15], HTML5 is already in use for several years, it is being supported by many different platforms, and has one of the widest developer communities. It worth mentioning the Proposal of Special Extension for Stereoscopic 3D Support [12] on W3C [46].

State of the Technology

Most of the tools for production of interactive stereoscopic HTML compositions are associated with WebGL technology [23]. WebGL elements can be seamlessly implemented into a user interface along with the rest of the HTML. There are several works showing successful implementation of this technology in different areas [21, 32, 47]. Nevertheless in order to achieve acceptable results WebGL requires expert knowledge of JavaScript and experience working with 3D modelling. This problem is alleviated to some extent by a number of WebGL frameworks, such as ThreeJS [43], nonetheless, the idea of using WebGL and its frameworks for the task in hand was declined due to (i) excessive functionality, (ii) complexity of the development process, (iii) lack of support from several targeted platforms.

Method

To increase the performance of the output and reduce the development time, it was decided to use jQuery [34], the JavaScript framework designed to do exactly that.

Conversion Algorithm

Based on the techniques behind the targeted stereoscopic displays and the requirements of HTML5 standard an algorithm [7] was designed and implemented. Its operation mode can be divided into 4 main stages: (S1) cloning of the existing content and creating a side-by-side composition; (S2) detecting stereoscopic content, such as images, videos etc.; (S3) mirroring the interactions from one side to the other; and (S4) distributing user predefined markup elements along the depth axis (z-axis), which determines the way the illusion of depth is achieved.

The Implementation

Throughout the development process and the initial testing, our implementation called 3DSjQ proved to be a stable easy-to-use tool. Due to the HTML in its core, the output composition can be viewed on 3D enabled projectors, TVs, PC monitors, tablets, smartphones, etc. The tool allows to design all type of interactions supported on a client side and work with many user input devices, including basic ones, such as mouse, keyboard, touch screens, and more sophisticated MS Kinect, Leap, and others equipped with a corresponding API.

Despite its stability, the number of functional features, 3DSjQ still has several restrictions for proper use. For instance, (i) it is strongly advised to maintain validity of the input code and its correspondence to the W3C requirements; (ii) all the interactions with the output stereo-pair should be specified after 3DSjQ was executed since all the javascript executed prior the tool was triggered could cause interaction fails; (iii) due to browser rendering issues some graphical stylings can look different from the input composition, thus visual properties of some elements need to be corrected correspondingly.

Operation

3DSjQ was designed so that operating it requires only the basic knowledge of HTML and related technologies. The procedure of enabling the framework on a page is similar to that of enabling any other jQuery plugin. The 3DSjQ.js file should be specified in a header of an HTML file to convert. Then, in order to start the conversion, one should launch the initiating function go3DS(elements, params), where elements is an array of elements to shift along with a respective depth level and params is an array of parameters of the stereoscopic workspace (e.g. stereo-pair orientation, depth budget, etc.).

Technical Performance Evaluation

Each browser has specific properties and rendering capabilities which can affect the output markup. Moreover, operating system that is hosting the browser can also influence the rendering behaviour. Thus, in order to evaluate the obtained tool in terms of performance and platform compatibility, it was tested on the most popular browsers (desktop and mobile) under the most popular operating systems for the passed year (those available within 2015–2016) [39]. The following browsers were included in the research according to their current market share: Google Chrome (45.34 %), Safari (12.98 %), Internet Explorer (10.86 %), Firefox (10.26 %), Android (6.13 %). Other browsers were also tested, but results were not included because of their low market share. Windows 7 (30.4 %), Android (24.87 %), iOS (10.82 %), Windows 8.1 (8.5 %), OS X (5.47 %) were included as one of the most popular operating systems.

The displays that have been tested include: Samsung 3DTV with screen size of 1920x1080px, Samsung UD590 with 2560x1440px, Apple iPhone 5S - 1136 x 640px, LG G3 - 2560x1440px. These devices were considered as a reference group because their screen sizes are currently the most widely used [40].

For the tests several responsive prototypes were created (Figs. 2 and 3) exploiting stereoscopic features provided by the 3DSjQ. The HTML markup of all the prototypes is done the way it can adapt its appearance to a current display screen size (also known as responsive). The prototype was developed on a desktop PC under Mac OS X 10.10.3 and tested on Google Chrome browser along the process. Thus this OS / Browser combination was used as a main reference.

Fig. 2
figure 2

The initial HTML prototype containing the 2D image (to the left) and the output stereo composition containing left and right parts of the stereo image located in the corresponding parts of the stereo-pair (to the right) as seen on Chrome Browser under Android OS. Please note the layout stability

Full size image
Fig. 3
figure 3

Output stereo-pair of the e-commerce application as seen on Google Chrome under Mac OS X (top) and PS4 browser (bottom). Please note the layout stability in both browsers

Full size image
Table 1 3DSjQ browsers compatibility
Full size table

Observations

As seen in Table 1 the framework performs flawlessly in terms of support of its features when tested under different operating systems. The final versions of the stereoscopic prototypes were compatible with almost all of the targeted platforms. Overall, 3DSjQ was able to run on all the test devices with most of the visual features displayed correctly. Some browsers were unable to correctly render the output composition due to incompatibility of the input HTML, though the framework performed as expected. The obtained results will be taken into consideration and will be useful for testing framework’s fallback strategies.

User Evaluation

To support the results of the technical performance evaluation a user study among 30 participants was conducted. The purpose of the test was to observe the influence of a specific operating system / browser (OS / Browser) setup on the usability of an HTML application converted into a stereo-pair using the 3DSjQ. If no significant interaction between the usability of the application and the environment is detected, 3DSjQ’s performance in all the tested environments can be considered stable.

Participants

Participants included 26 male and 4 female students and faculty staff of the Engineering School of the Autonomous University of Barcelona (M = 29.467, SD = 8.835, Rg = 19 - 52 years old). All the participants possessed no stereo blindness or any other sight defects preventing proper stereoscopic effect perception. Overwhelmingly the majority of the test subjects had occasional (1–3 times a year or more) 3D experience mostly limited to 3D movies, only two people never experienced 3D before. Windows users included 25 individuals, 4 were Linux users and only 1 was Mac OS X user. The most of the participants indicated Google Chrome as a browser of choice for entertainment purposes, leaving only 4 people using Mozilla Firefox and only 2 people using Opera and Microsoft Edge. As a compensation participants were offered a refreshment drink after the study.

Materials

Based on a current market share [30, 31] of desktop browsers (see Tables 2 and 3) two operating systems were chosen to host the browsers - Windows 7 (Win7) and Mac OS X 10.11 (OSX), and the most popular builds of 4 main browsers - Microsoft Internet Explorer 11.0 (IE11), Google Chrome 47.0 (Chrome), Mozilla Firefox 43 (FF), Apple Safari 9.0 (Safari). The remaining operating systems and browser versions were not considered due to their low market share.

Table 2 Operating systems market share for desktop browsing
Full size table
Table 3 Browser market share for desktop browsing
Full size table

The stimulus represented a stereoscopic HTML video clips browser (Fig. 4). The user interface contained video category tabs on the top of the page and a grid of video clips corresponding to the chosen category. Each video clip contained a thumbnail preview, a title, a category, and a duration in minutes and seconds. Click on one of the thumbnails triggered a corresponding video clip to play in a popup on top of the rest of the content. After playback is completed, the popup window closes automatically.

Fig. 4
figure 4

Stimulus used for the user evaluation in standby mode (to the left) and video playback mode, triggered when clicked on one of the thumbnail previews (to the right)

Full size image

The test was conducted in a lab environment using: (i) 24-inch Sony Playstation 3D Display [38] with Full HD resolution; (ii) Sony Playstation 3D glasses; (ii) HP Compaq Elite 8300 Microtower QV994AV [19] used as a host for the Win7 and (iii) Apple MacBook Pro A1398 MC975LL/A [9] as a host for OSX; (iv) a mouse as the way to interact with the application.

The usability was measured using the System Usability Scale (SUS) [3]. SUS is a 10 item questionnaire designed to quickly and accurately assess the perceived usability of a system. To complete the study the participants were asked to point out up to three things they liked or disliked about the user interface and their experience.

Procedure

Participants were randomly assigned to one of the 6 groups. Each group was dedicated to a specific OS/Browser combination. Before the study each of the participants was (i) given instructions about the purpose of the study and its workflow, (ii) exposed to a 3 minutes stereoscopic video clip to confirm correct perception of the stereoscopic depth and to get participants accustomed to the stereo effect perception, and (iii) introduced to the stimulus, while basic functionality was explained. Afterwards the observers were offered to find three video clips relevant to her interests and watch them. Later, after completing the task, the participants filled-in the SUS and answered the open-ended questions.

Results

A significance level of α = 0.05 was chosen for the study.

SUS Scores

Overall the stimulus performed good in terms of usability: the mean SUS score throughout all the participants was (N = 30, M = 83.75, SD = 8.8), which is well above the average of 68, that is considered a good usability score [36]. This threshold has been overcome only once, when the application’s usability was rated 55, by a participant in a Win7 / IE11 group. Kruskal-Wallis test for independent samples was applied for more detailed analysis of the obtained body of data, using the SPSS package. There was no significant interaction between the gender, age, previous experience with 3D, operating system and browser of choice, or any other. What is more important, no interaction was found between the OS / Browser combination and the SUS scores (N = 30, df = 5, p = .652) either (see Fig. 5).

Fig. 5
figure 5

SUS scores distribution within the sample groups displaying the overall mean score (M = 83.75) and the threshold for good usability (SUS a v = 68) [36]

Full size image

Further examination of the SUS questions one by one, revealed an effect of the OS / Browser setup on the question number 3 (“I thought the application was easy to use”), defining perceived ease of use (N = 30, df = 5, p = .035). The Windows / Chrome setup (M = 5, SD = .00) and the OS/Safari (M = 4.80, SD = .45) were perceived to be easier to use than the rest of the setups.

Based on the performed statistical analysis, it’s reasonable to conclude that the 3DSjQ performed stable in all the tested conditions.

Open-ended Questions

Generally, most of the complaints were related to a cursor behaviour and lack of functionality as 9 participants indicated to be concerned with that. From the sample 7 participants expected more profound 3D effect or disliked it overall. At the same time, unexpectedly, only 2 participants pointed that they disliked wearing the 3D glasses and one participant who would like to use the application with a greater display size. The platform specific complaints include (i) a text rendering quality, where all 4 complaints were detected in the Win7 setups and (ii) a browser specific behaviour, such as a scroll direction or browser’s UI layout, where 3 complaints were given in the OSX setups. The first could be explained by the Win7 rendering capabilities of the CSS transformations, which are used during the conversion process, and the second finding could be explained by the fact that only one out of 15 participants exposed to the OSX setups indicated herself as an OSX everyday user.

Among the features the observers liked 23 participants indicated (i) the ease of use, 13 participants indicated (ii) the 3D effect and (iii) the design quality, 1 participant mentioned (iv) the novelty of the application.

All negative feedback can be related to the rendering behaviour of the browsers in use, ergonomics of the 3D equipment, display size, etc. None of the registered complaints were addressed to the markup behaviour, elements displacement, interactions with the stimulus. At the same time the majority of the users indicated the ease of use as the main positive feature of the stimulus.

Conclusion

With this work the 3DSjQ [6] was proposed as a valid open source tool for production of passive and interactive stereoscopic compositions. The framework was in a state of development for several years, and, as shown in this work, performs really well under all the most popular browser versions and operating systems, with over 90 % of available platforms rendering the output composition as expected. At the same time, the results of the conducted user research confirms the finding. The S3D application built with 3DSjQ was tested among 30 participants and performed well above the SUS’s “good” performance threshold. Thus, the proposed method could be used for production of interactive stereoscopic graphical user interfaces, which could represent a software layer for web-based medical systems facilitating the stereoscopic effect.

Future Work and Discussion

Despite the outstanding performance, this study uncovered flaws in the Tool’s operation. Thus, there is still work to be done to improve stability and functionality of the framework. This can include: (i) further update of the functionality, (ii) closer work with the stereoscopic web standards, to support its features and contribute to the standard’s development; (iii) definition of the fallback strategies for browsers that can not (or not fully) support all the features provided by the HTML5; (iv) provide support for autostereoscopic displays. At the same time more sophisticated user studies with larger samples and tasks of greater complexity could reveal new problems and potentially contribute to the Tool’s performance.

We would like to encourage researches and software engineers working with the medical applications, to use the described method for development of S3D user interfaces and try to put the proposed tool in a good use. It could help establish guidelines and standards for efficient usage of the stereoscopic depth in different medical tasks (diagnosis, pre-operative planning, training) and domains (angiography, virtual endoscopy, ophthalmology, dermatology). Overall, the method could be used to test interaction paradigms with the stereoscopic workspaces, measure level of immersiveness and similar matters.