1 Introduction

Like other technologies in the hardware domain, display and interactive technologies are subject to continuous advancements and even technological realignments. When assessing the decades-long history of display technologies in general, the past decade in particular has seen a significant shift and a great diversification. Before the turn of the millennium, the majority of information-reception and processing by the user took place with the help of rather large and cumbersome displays, often in the form of desktop PCs Sedig et al. (2016), Liang and Sedig (2010a). Recently, this condition has changed due to progressing technologies in the field of mobile devices. Nowadays, mobile devices, such as smartphones, tablets, and laptops, are broadly used. These mobile devices are often equipped with small size, touch-based displays, which allow users to interact with virtual objects directly Chen et al. (2020). This impacts the overall interaction of the users with visual objects and how information is perceived and manipulated.

In addition to tactile mobile devices, virtual reality (VR) display technology is widely used in data visualization and exploration. Head-mounted displays (HMDs), such as the Oculus Rift/Quest and HTC Vive, allow users to experience a virtual environment to an even greater degree by placing themselves into an immersive virtual world controlled by and responding to the behavior of users Chen et al. (2021), Lu et al. (2018), Nanjappan et al. (2018). Similar to mobile devices, VR HMDs come with new input technologies, such as 3D controllers, which provide 6 Degrees of Freedom (DOF) input in the 3D environment Nanjappan et al. (2018), Bergström et al. (2021); Yu et al. (2021). While the touch screen for mobile devices is a widely accepted and highly efficient standard for this domain, the HMDs and VR are still in a phase of experimenting with various input technologies to determine which one fits best to the new form of presenting information to the user Bergström et al. (2021).

What all three display technologies—classical desktop systems, touch-based mobile devices, and HMDs—have in common is that they are all widely being used in numerous different domains for different purposes and tasks Monteiro et al. (2018), Xu et al. (2021), Luo et al. (2021). The newly resulting display and input device diversity have led to the need to develop applications for a number of different target platforms, visualization, and user interaction capabilities Chen et al. (2021, 2020, 2021). However, due to the fast development of new output (visualization) and input (interaction) technologies, the findings on how these different technologies affect users’ ability to perform given analytical tasks and acquire spatial knowledge are very limited. Engagement, as one of the most important factors to drive and motivate learning activities, has not been well studied with different display types Bryson and Hand (2007).

In this research, we attempt to address this gap by conducting an empirical user study to compare the effects of different display types on learning and engagement. Specifically, we focus on the acquisition of spatial knowledge, which refers to the ability to understand the shape, structure, and location of certain 3D spatial data. To investigate whether and to what degree the three mentioned display and input devices could impact users’ spatial knowledge acquisition, we designed and implemented an interactive visualization tool that allows users to explore 3D geometrical shapes. We conducted a pre-test and a post-test to study the usability of the visualization tool in supporting users to acquire spatial knowledge and eventually improve on analytical reasoning based on the underlying display technology (and input type) being used.

2 Related work

2.1 Interactive visualizations

Interactive visualization tools are designed to be supportive of users’ exploratory, interpretational, and sense-making activities Card (1999), Chen (2004), Jonassen (2006), Liang and Sedig (2010a), Spence (2001). These tools have two main characteristics: (1) they maintain and display information in the form of visual representations; and (2) they allow users to manipulate or interact with the visual representations through a human-computer interface Liang and Sedig (2010a). Such tools usually embed interactive techniques so that users can acquire information through manipulating and interacting with visually-enhanced symbols or objects Chuah and Roth (1996), Liang and Sedig (2010b). When designed for exploratory interaction, these tools often do not provide any explicit or direct instruction to users. Instead, users are encouraged to investigate, formulate, analyze and interpret their hypothesis by themselves using the functions designed in the tool, and draw their own conclusions, thus leading to a more proactive, interactive, and engaging experience.

The coupling of visually-enhanced objects with interactive possibilities can increase memory, analytical reasoning, and engagement. Because of this, researchers have been widely applying interactive visualizations in various domains, from business to medicine, from engineering to science Liang and Sedig (2010a), Petersson et al. (2009), Portmann and Lüthi (2000), Chen (2004), Chittaro (2006). Some guidelines point out that in order to design an effective interactive visualization tool for various platforms, it is crucial to investigate user feedback regarding the embedded interaction techniques that are provided for each respective platform. In this research, we aimed to find out how interactive visualization tools used with different displays and different input technologies could possibly influence learning activities and engagement.

2.2 Information visualization on different displays

People are used to relying on visualizations for understanding problems they have to solve and taking decisions in a shorter time. Historically, visualizations have been mainly presented on desktop computers because of their powerful graphic hardware, and interaction has happened through traditional input devices such as mouse and keyboard Roberts et al. (2014), Wenz (2009). In recent years though, the rapid development of processing power made mobile devices much more powerful than before, even slowly catching up with the capacity of desktop PCs Wenz (2009), Yoo and Cheon (2006). Referring to the recent report made by Computer Hope, the computing power of a smartphone or tablet nowadays can generally rival the best laptops or desktops of about five years ago Computer (2022). On mobile devices, users can interact with the visualizations by directly touching the display, and the devices can provide vibrotactile feedback when certain actions are performed. However, both desktop computers and mobile tablets utilize traditional 2D screens. Although sometimes data needs to be projected to lower dimensions for better understandability and pattern identification Xia et al. (2021, 2021), Yuan et al. (2021), Rauber et al. (2016), when it comes to 3D data, 2D displays could be less effective and intuitive Shneiderman (2003).

Recently, with the development of virtual reality, it is possible to place users into an immersive virtual environment containing 3D visualizations using a head-mounted display (HMD) with depth being rendered Rheingold (1991), Cruz-Neira et al. (1993), Lu et al. (2018). As such, users are allowed to reach into the immersive visualizations and observe 3D objects freely from different angles, instead of only ‘imagining’ the 3D structure through a 2D display paradigm Cai et al. (2013). Together with synthetic stimuli such as spatial sound, force, or tactile feedback on controllers or sometimes on the HMD, VR can provide users with a deeper level of interaction possibilities, which is claimed to improve motivation, understanding, and engagement Barab et al. (2005), Psotka (1995), Hoffman and Vu (1997), Bowman and McMahan (2007), Wang et al. (2022), Luo et al. (2022). Some studies have pointed out the advantages of high-fidelity displays, such as those integrated into a VR HMD, as compared to low-fidelity desktop systems Gruchalla (2004), Ruddle et al. (1999), Arms et al. (1999), Hwang and Hu (2013), while some studies also highlighted that traditional 2D displays could be faster and more precise in certain visualization exploration tasks Bach et al. (2017). In this study, we investigate the potential influence of different display types on knowledge acquisition and engagement levels while interacting with visualizations.

2.3 3D geometric learning and problem-solving

3D geometry has been considered a category worth studying since ancient times because of the inner structural beauties and regularities of 3D shapes Pugh (1976). The exploration of 3D geometry, including moving, positioning, orienting, and communicating 3D objects, is important to fully appreciate the real world and other fields such as computer graphics, architecture, and engineering Yeh (2007). Geometry education, raised by Baki et al. (2011), encourages students to understand and explain the physical world around them using geometry within the processes of pattern-finding and problem-solving. Among 3D geometry, the intriguing relationship between Platonic solids and Archimedean solids is particularly worth paying attention to. The three Platonic solids shown in Fig. 1, cube, octahedron, and tetrahedron, are 3D shapes that are composed of only one type of regular and congruent polygons HSM (1991). Because of this property, there can be only five Platonic solids Heath (1956). Archimedean solids are 3D shapes formed by two or more types of regular polygons. Platonic solids and Archimedean solids are closely interconnected and can be obtained from one another by truncating or augmenting vertices and edges. 3D Geometry, as one of the important branches of mathematics, embodies the concepts of symmetry, abstract symbolic algebra, and group theory HSM (1991).

2.4 3D geometric shapes and spatial knowledge

Navigation has been proved as an important way of interacting with, within, and between visualizations Liang and Sedig (2009). It is concerned with how learners acquire knowledge about an information space and form their knowledge of the space via interactions Spence (2001), Liang and Sedig (2009), Spence (1999). The transformational relationships between abstract 3D solids (i.e., Platonic and Archimedean solids) can be considered as an information space, which encodes three types of spatial knowledge: (1): Landmark Knowledge, possible stages that a Platonic/Archimedean solid can morph to; (2) Route Knowledge, how to morph the shape to obtain a desired stage; (3) Survey Knowledge, the entire landscape of the stages and transitions across multiple shapes Liang and Sedig (2010a). By navigating within the spatial informational space of the 3D solids, one will be able to not only think and imagine how the shapes could morph and transform themselves from one another but also have an entire landscape knowledge of all potentially related solids Liang and Sedig (2010a), Sedig et al. (2005). Previous studies have shown that viewing such abstract information as spatial could assist learners with their learning and comprehension Liang and Sedig (2010a), Sedig et al. (2016), Sedig and Liang (2006). Moreover, such diversity of representations for mathematical concepts, potential connectivity, and transformation properties from one representation to others are crucial to successfully learning mathematical concepts Gagatsis and Shiakalli (2004). However, it is hard for learners to imagine such abstract information, which has made visualization tools necessary for providing cognitive aids. Therefore, they could be a feasible testbed to investigate the effectiveness and viability of visualizations in supporting sensemaking, analytical reasoning, and knowledge acquisition. As such, our findings in this study can be generalized to sense-making tasks with visualizations of mathematics knowledge that can be represented as shapes in three-dimensional space. In this research, we are concerned with exploring the potential effects of display type on the learning of 3D geometry and sensemaking activities, particularly the relationships between Platonic and Archimedean solids.

2.5 Engagement in learning and with visualizations

Engagement plays a critical role in users’ learning activities, especially with visualizations. Research has demonstrated that learners who actively engaged with visualizations could substantially outperform those who were passive viewers Hundhausen et al. (2002). As argued by Naps et al. (2002), “visualization technology, no matter how well it is designed, is of little educational value unless it engages learners in an active learning activity.” As such, improving their engagement level is important to designing a successful visualization tool. Recent research has demonstrated the benefits of VR displays over traditional video or text-based platforms in engaging the learners Allcoat et al. (2018). However, there has been a lack of research comparing engagement levels according to different display types and platforms. In this study, we attempt to fill the gap by quantifying engagement level and comparing it across the three display types.

Fig. 1
figure 1

(Top to bottom rows) Octahedron, Cube (or Hexahedron), and Tetrahedron (LEFT) and all their derivable Archimedean solids (the octahedron, which is derivable from the tetrahedron, is not an Archimedean solid) (RIGHT)

Full size image

3 Research questions

The goal of our research is to study the effectiveness of different displays and input devices in spatial knowledge acquisition. We focus on three types of spatial knowledge of 3D geometric shapes (landmark knowledge, route knowledge, and survey knowledge) and three platforms (a desktop display, touch-based tablet, and VR HMD). Based on the literature discussed above, we set out to answer the following research questions:

  • RQ1. What are the differences in spatial knowledge acquisition among desktop, tablet, and VR HMD displays when using interactive visualizations of 3D geometric shapes?

  • RQ2. What are the differences in user engagement among desktop, tablet, and VR HMD displays when using interactive visualizations of 3D geometric shapes?

4 Hypothesis

In terms of visualization, both desktops and tablets rely on 2D screens, while VR HMDs offer embodied interactions with immersive and stereoscopic rendering of 3D visualizations. Based on the existing literature Ruddle et al. (1999), Arms et al. (1999), Hwang and Hu (2013), we believe that with VR HMDs, users should have a better learning experience (overall), as well as a structural understanding of individual 3D shapes (landmark). Therefore,

  • H1. VR users will obtain overall larger improvements in post-test questions than Desktop and Tablet users.

  • H2. VR users will perform significantly better on landmark questions than Desktop and Tablet users.

In terms of user interaction, a multi-touch-based tablet display allows users to make multiple inputs simultaneously to better understand the transitional properties between a pair of 3D visualizations. Traditional desktop relies on the indirect mouse input and provides only one action (e.g., rotation or translation) at one time. A VR HMD controller offers 3D interaction; however, to most common users, it may not be easy to explore a 3D object by using a controller. Therefore, we believe,

  • H3. Tablet users will perform significantly better on route questions than desktop and VR users.

Although a VR HMD provides immersive visualization of the data and could lead to a more engaging experience, the limited FOV (field of view) of modern HMDs might constrain the overall understanding of interrelationships. Therefore, we believe,

  • H4. Desktop and Tablet users will perform significantly better on survey questions than VR users.

  • H5. VR users will be more engaged as compared to Desktop and Tablet users.

5 Design of the interactive visualization system

In this section, we introduce the interactive system that is designed for studying 3D geometric shapes, such as cube, octahedron, tetrahedron, and all their derivable Archimedean Solids. With the interactive system, we conducted a user study to evaluate the effects of different types of displays and input devices on spatial knowledge acquisition. As demonstrated by prior research Yeh (2007), Cohen and Hegarty (2014), Garg et al. (2002), Chariker et al. (2011), viewing and manipulating virtual models is particularly beneficial for learning spatial configuration and complex 3D objects. Therefore, our interactive system is designed to be capable of: (1) visualizing the metamorphic transformations among three Platonic Solids: cube, octahedron, tetrahedron, and all their derivable Archimedean Solids; and (2) interacting with the solid visualization symbols through different interaction techniques.

5.1 Interface

The interface of our visualization system is shown in Fig. 2. For all three platforms, the interfaces are similar in layout and all can be generally divided into two sections: 1) the visualization area placed in the middle of the interface, which shows the selected 3D solid; and 2) the solid transition map (STM) placed below, which shows cognitive information on possible transitional stages. We put the three maps and 3D solid visualizations next to each other for two reasons: firstly, such an arrangement can facilitate the exploration process. Once the user decodes a map, he will also know how to understand and interact with the other maps; secondly, it is easy to gain knowledge by comparing 3D solid visualizations, such as structural similarities and differences, the existence of Twin solids and dynamic-linking of all visualizations (explained below). Users can focus on the relationships of solids across the three maps.

In the next sections, we introduce the visualization widgets and interaction techniques of the interactive system, and especially, how the system is designed to provide cognitive information on analytical reasoning of 3D geometries.

Fig. 2
figure 2

Interface of the interactive visualization tool for (LEFT) Desktop, (MIDDLE) Tablet display, and (RIGHT) VR HMD

Full size image

5.2 3D solid visualizations

As mentioned above, the 3D solid visualizations are placed in the middle of the whole interface, since they should serve as the most important part of the visualization tool. The 3D solid visualizations present direct structural information on the observed solids.

To further provide transformational information, the faces of solids are rendered with different colors, since color has been proved to play an important role in visual communication, which can significantly improve user cognition at a conceptual and subconscious level Scott (1992), Wilkinson (2012). Here, three colors (i.e., white, blue, and yellow) are used to render a solid, with each indicating the source from which it is obtained. For example, in Fig. 4, the rhombitruncated cuboctahedron, the existence of blue and yellow faces indicates the happening of truncation on both vertices and edges.

As shown in Figs. 2 and 3, an STM is showing under each shape to provide cognitive feedback on solid transition stages. Users are supported to select a desired stage (solid), which will be highlighted by a blue circle, on the transition map. The other stages are set to black and white [see Fig. 5 (Right)]. Moreover, to provide a more detailed observation, the 3D solid (stage) will also morph corresponding to the user selection, serving as an enlarged version of the selected solid. However, because the transition map and the enlarged shape are visually disconnected, it might be still difficult for users to make sense of the information presented separately in each of them. To resolve this issue, we used a visualization technique called context-enhanced magnification technique Sedig et al. (2003). The technique contains three visual elements to enhance user cognition. Firstly, the original Platonic solid is rendered in transparent wireframe, so that users can clearly see which parts of the solid have been truncated (see Fig. 5a). Secondly, a local STM is settled on one face of the transparent wireframe, by which the 3D solid visualizations and STMs are potentially connected (see Fig. 5b). The existence of local STM can provide a context for users to understand the changes to the 3D solids. This approach is inspired by semantic zooming, a visualization technique where the representation of an object is not only magnified but also added with additional details, which, often hidden from explicit, direct view, are unraveled and shown Spence (2007), Perlin and Fox (1993). Thirdly, one vertex of the enlarged solid is highlighted to provide semantic information on solid stages (see Fig. 5c). The position of the highlighted dot relative to the local STM is exactly the same as the position of the highlighted stage relative to the STM. With the aid of the three context-enhanced techniques, the STM and the 3D visualizations become visually and interactively bridged, where the change on one will instantly be linked to the other. Users can form a coherent knowledge of the relationships between the solids and their maps.

Fig. 3
figure 3

The solid transition map (STM) of the Cube, in which all stages that the Cube can morph to were represented in the map

Full size image
Fig. 4
figure 4

(LEFT) Cube and (RIGHT) rhombitruncated cuboctahedron, which can be obtained by truncating the vertices and edges of a cube

Full size image
Fig. 5
figure 5

(LEFT) Context-enhanced 3D solid visualizations and (RIGHT) Cube-Map with cuboctahedron selected on the Map

Full size image

5.3 Interaction techniques

Some interaction techniques are designed and implemented for the visualization system to provide users with alternative and complementary ways of facilitating their exploratory needs and sensemaking activities. Note that, although spatial interaction techniques, such as tangible interaction (e.g., Yu et al. (2022)), mid-air gestures and hybrid interaction paradigms Besancon et al. (2021), may have a significant impact on the knowledge acquisition and user engagement in learning, in this work, we focus on the effect of display platforms. Thus, we follow the standard interaction techniques provided by VR HMD devices.

5.3.1 Direct and indirect manipulations

For indirect manipulation, as mentioned before, users can morph a 3D solid visualization by selecting a desired stage on the map. The user-initiated morph will be instantly reflected on the local STM of the solids. Such interaction is referred to as “indirect manipulation”, where a transformation is performed on a set of graphical objects Roberts (2007), Thomas and Demczuk (2002), by using dedicated buttons, panels, or other mediums. However, direct manipulation with target objects is also important for spatial knowledge acquisition since it provides direct user interaction and engagement Ballas et al. (1992). Therefore, we developed interaction techniques to support users to interact with the solid directly and get immediate changes. Users can move the highlighted dots on the local STM and morph the solid directly by moving one of the vertices inside the semantically magnified STM.

5.3.2 Link view of multiple visualizations

In the exploration of Platonic and Archimedean solids, there is an important concept called “Twin solids”. A Twin solid is a shape that has the same structural properties as the selected solid but is different in how it is obtained [4]. Such property may lead to complex and abstract connections between multiple STMs, as identical Archimedean shapes could be obtained from different Platonic solids through different transformations. In our interactive visualization system, the STMs not only highlight the selected solid but also its twin solids. As shown in Fig. 6, when a solid stage is selected on a map, the corresponding twin solids will be also highlighted on the other two maps if exist. The 3D solid visualizations will morph together with the selected stage on the map. Such implementation can facilitate global knowledge acquisition across multiple maps. The solids, STMs, and the semantic STM on the solid visualizations all become dynamically linked to help visualize the existence and patterns of twin solids, which indicate structural similarities across the three STMs. It also ensures that the twin solids of a selected solid will always be displayed and moved synchronously during the exploration. By doing this, users can obtain a more comprehensive sense of whether and how the shapes are related to each other. For example, by observing Fig. 6, it is obvious that a truncated octahedron can also be obtained by truncating the vertices of a cube or truncating the vertices and edges of a tetrahedron.

Fig. 6
figure 6

Layout of application showing selected solids and twin solids

Full size image

5.3.3 Rotation and scaling of 3D geometries

Occlusion is a common problem for 3D visualization. It is inevitable that part of the 3D visualization might be occluded by themselves or other objects in 3D space, which might lead to a loss of structural information Roberts (2007), Elmqvist and Tsigas (2008). To gain a full picture of 3D solid, users are supported to rotate and scale solid by user interactions. Rotation allows users to freely observe solids from different perspectives. Scaling allows users to increase the size of the selected solid for a more detailed exploration, or decrease the size of the 3D shapes, which makes it easier for participants to fit multiple shapes in their FoV for side-by-side comparisons.

6 Experiment

The goal of the experiment is to study the effect of three different platforms on spatial knowledge acquisition. We used our interactive system as the testbed. In order to allow for comparisons among the groups, this experiment utilized a quasi-experimental between-subjects design. In the experiment, we use both quantitative and qualitative methods. We collected user performance data in completing tasks and subjective data from pre-test, post-test, questions, and interviews. By applying such a multi-method user study design, we can triangulate and cross-validate different observations. For example, the comparison of pre-test and post-test results would reveal the improvements that each platform leads to and why. The questionnaire and interviews would indicate data about user preferences on each platform or function we designed and the general usability. The study compared three groups, with each group interacting with one specific platform.

6.1 Apparatus

In the experiment, we use three platforms: a desktop computer, mobile tablet, and VR HMD (see Table 1). For the desktop computer, the experiment was run on the system with an i7 4 GHz processor and GTX 1080Ti dedicated GPU. The display used was an AOC U2879VF with a resolution of 3840*2160. Mouse input was the only allowed input. For the mobile tablet, the experiment was running on Google Pixel C tablets with a configuration of 3GB RAM and 64GB Memory with a display resolution of 2560*1080. Multi-touch input was enabled. For the VR HMD, we utilized the same desktop system. We used an Oculus Rift CV1 headset, a commercial VR device with a 1080*1200 display resolution for each eye. The Oculus Touch, the customized controller for the HMD, was used for user input and to allow more intuitive interaction and a rich set of interaction gestures.

Table 1 Display platforms and the input used in the experiment
Full size table

6.2 Participants

30 undergraduate students (15 males and 15 females) with an age range of 18–25 were recruited to participate in the study (Mean Age = 20.7, SD = 1.725). They are volunteers from a Sino-UK university located in a mid-size city in China. The study was firstly advertised by professors in their classes. Then, potential participants sent emails to experimenters directly for more information. Note that the participation was entirely voluntary. Participants were told that their participation and performance in the study would not affect their academic performance in any way. None of the students had used our interactive visualization system before. A between-group design was applied and the participants are randomly assigned to three groups: Desktop, Tablet, and VR group. Each group has 10 participants, with a balanced distribution of 5 males and 5 females.

6.3 Procedure

The whole experiment was divided into five phases (see Fig. 7): (1) understanding the content and procedures of the experiment: we gave a brief description of the procedure, content, and some mathematical background of the study (10 min); (2) pre-test: participants were asked to complete a questionnaire about 3D geometric shapes (20 min); (3) performing tasks with the system: the participants were asked to use the system and finish a set of question-based tasks (40 min); (4) questionnaire & interview (20 min) and (5) post-test: participants were asked to complete the same questionnaire used in the pre-test (20 min). In total, the whole experiment took about 2 h, which was similar to a typical weekly lab session. We used video and screen captures of participants’ interactions with our interactive system.

More specifically, in phase 4, the participants were asked to fill in an engagement questionnaire, VisEngage, which is a dedicated questionnaire for assessing users’ engagement level while interacting with visualizations Hung and Parsons (2017). The questionnaire consisted of 22 seven-point scale Likert questions about 11 different types of engagement measurements with visualizations. Afterward, we interviewed participants about their feelings and perceptions of the tool and its supporting functions and if they saw any advantages and drawbacks (see Appendix for more detail), which took 10–15 min. Finally, in the last phase, participants were asked to complete the post-test, which took about 20 min. The language used on the test questions and tasks was plain and easy to understand. If participants were unclear about anything, they were allowed to ask researchers for clarification at any point during the study.

Fig. 7
figure 7

An illustration of experiment procedure

Full size image

6.4 Task

The tasks were intended to provide subjects with predetermined goals to facilitate data gathering within the short duration of the study. A digital multi-choice test about Platonic and Archimedean solids was designed and given to participants before (pre-test) and after (post-test) interacting with the visualization tool. It contains 18 questions in total. Each question only has one correct answer. To discourage students from guessing the answers, we followed Liang & Sedig’s experimental design Liang and Sedig (2010a) and added an “I do not know” option for each question. The participants were told clearly that the user study results would not affect their academic performance in university in any way.

The questions were divided into three groups based on three types of spatial knowledge: Landmark, Route, and Survey. Each group has six questions: some of them were about the structural properties of the shapes (Landmark knowledge), some were related to the transformational properties between multiple solids ((Route knowledge), while others were about potential relationships between groups of solids ((Survey knowledge). The questions were placed in random order with varying degrees of difficulty Liang and Sedig (2010a). To avoid bias or discrepancies in the difficulty level when using two different sets of questions, the same questions were given in both the pre-test and post-test. However, correct answers were not given to participants. As such, it would not have affected their post-test performance.

For the Desktop and Tablet platform, the participants were asked to answer the questions on paper, while for VR, the participants were allowed to answer questions by annotating on a virtual canvas using the Oculus Touch controllers, since it was not practical to use paper and pen considering that they would be wearing the HMD goggles during the interactions. The answers were recorded automatically for later assessment of user performance, from which we calculated the overall accuracy of users’ answers in percentage for overall and individual knowledge categories.

7 Results

For data analysis, we conducted one-way ANOVA tests to test our hypothesis about conditions and used paired sample t-tests to compare the pre-test and post-test results. In general, the interactive visualization system led to a better performance in the post-test questionnaire. As shown in Fig. 8, the median of the post-test result is higher than the median of the pre-test. Although we noticed that some participants performed worse on the post-test, most of them achieved higher scores on the post-test with the highest improvement of 55.56%, after interacting with the system.

Fig. 8
figure 8

Boxplots for pre- and post-test results in % for all participants (left) and boxplots of the differences in % between pre- and post-test for all subjects (right)

Full size image

Table 2 shows the descriptive statistical results for all participants. As noticed, the standard deviation for both pre-test (13.819) and post-test (14.618) are large, which indicates that the level of performance varies before and after using the system. The pre-test scores differ from 0 to 61.11%, while the post-test scores vary from 16.67 to 83.33%. The overall differences between the two tests vary considerably with a standard deviation of 11.79%. The span of the test score and individual performance data shows some reasons: the participants who obtained a relatively lower pre-test score got a better chance of improvement after using the tool (for example from 0% to 55.56%), however, due to the limited space for improvement, such substantial improvements were harder to occur on the participants who already got relatively high scores on the pre-test.

Table 2 Overall descriptive data of user performance (accuracy) in the pre- and post-tests in %
Full size table

Table 3 presents the results of statistical analysis on all participants using each device. The mean increase of all participants is 25.925%. A one-sided paired-sample t-test was performed to check whether the score changed significantly between pre- and post-test. The t-test shows that in general, the participants performed significantly better (\(t_{{{\text{diff}}}} (29) = - 12.041,p{\text{ }}.001\)). Thus, we can confirm that the overall performance was significantly improved after interacting with the tool, which further proved the viability of understanding abstract mathematical content using visualization tools.

Table 3 Results of paired t-tests on the overall difference between pre-test and post-test in terms of user performance (accuracy) in %
Full size table
Table 4 Descriptive statistical data for Desktop, Tablet, and VR participants in terms of user performance (accuracy)in %
Full size table

7.1 Display category

As demonstrated in Fig. 9, participants generally experienced some degrees of improvement after the interaction. Among Desktop, Tablet, and VR, the VR group has the largest median on the overall improvement, but also with very high spanning; the Tablet group had the lowest median, and the Desktop group had the lowest deviation. Table 4 shows the descriptive data for both tests of each platform. The Desktop group had the lowest standard deviation on post-test, which indicates a more homogeneous performance. In contrast, the Tablet group had a high standard deviation on both pre-test and post-test scores. The same one-sided paired-sample t-test was performed for each group of participants. On all three platforms the participants significantly improved their comprehension on the transformation of 3D geometries (\(t_{{{\text{Desktop}}}} (9) = - 9.946,p.001\); \(t_{Tablet}(9) = -6.167, p<.001\); \(t_{VR}(9) = -6.280, p<.001\)). Similar to the result shown in Fig. 9d, the VR group had the greatest improvement from pre-test to post-test and the largest deviation (Mean = 29.44, SD = 14.826), which revealed the variance in the participants’ acquisition of knowledge during the interaction process. In contrast, although the Tablet group had a high standard deviation on pre-test and post-test scores, as seen in Table 4, they had a relatively low deviation on the test score improvement (Mean = 22.221, SD = 11.415). In terms of improvement, among the three platforms, the VR group had the most improvement, then the desktop group; the tablet group had the least improvement. The desktop group had the lowest standard deviation, indicating a relatively homogeneous level of learning (Mean = 26.110, SD = 8.310). To further investigate whether significant differences exist among the three platform groups, thus whether any of the group participants performed significantly better or worse than others, a one-way ANOVA test was conducted. No statistical difference was found among the three platforms (\(F_{227} = 1.012, p = 0.377\)), which rejects our hypothesis H1.

Fig. 9
figure 9

Boxplots of a overall test score and pre- and post-test score in percentage for participants using b Desktop PC; c Tablet display and d VR HMD

Full size image

7.1.1 Knowledge category

To investigate if there is any difference among one specific type of knowledge gained during the learning process, further analyses were conducted both generally and separately on test questions categorized by three types of spatial knowledge (i.e., landmark knowledge, route knowledge, and survey knowledge) and three types of platforms (i.e., desktop display, tablet display, and VR HMD). Highly significant improvement was detected on all three types of spatial knowledge after interacting with the system (\(t_{{{\text{Landmark}}}} {\text{ }}(29){\text{ }} = {\text{ }} - 8.752,{\text{ }}p{\text{ }} < .001\); \(t_{Route} (29) = -6.718, p <.001\); \(t_{Survey} (29) = -5.114, p <.001\)). Overall, participants’ comprehension of landmark knowledge improved the most with the lowest standard deviation (Mean = 29.999, SD = 18.774). In contrast, survey knowledge has the lowest improvement and the largest standard deviation among the three knowledge categories (Mean = 23.332, SD = 24.989).

7.1.2 Landmark knowledge

There were six questions related to landmark knowledge in both pre-test and post-test. They were mainly used to test participants’ knowledge about possible stages that a base shape could transform into. Generally, most subjects performed better on landmark questions after interacting with the tool. It is also indicated that the VR group might have facilitated a higher degree of Landmark knowledge than the other two groups. The t-test analysis on the difference between the two test results indicated that all three groups of users improved their landmark knowledge significantly (\(t{landmark-desktop} (9) = -6.708, p <.001\); \(t_{landmark-tablet (9)} = -4.129, p =.003\); \(t_{landmark-vr} (9) = -5.284, p <.001\)). Again, a one-way ANOVA test was conducted to compare whether there was a significant difference in participants’ performance on land-mark questions across these three platforms. Although VR users did obtain higher improvements, no significance was found (\(F_{227} = 2.423, p = 0.108\)), which failed to support our hypothesis H2.

7.1.3 Route knowledge

There were six questions involving route knowledge in both tests, focusing on participants’ comprehension of the transitional processes between Platonic solids and Archimedean solids. The boxplots indicate that the performance of the Desktop and Tablet groups was very similar. VR participants have the highest median and largest variance among the three. Some unexpected cases were observed, showing that the user performance for Route questions dropped after interacting with the visualization tool.

The statistics of the t-test shows that generally all three groups improved their performance on Route questions significantly after interacting with the tool (\(t_{route-desktop} (9) = -6.708, p =.001\); \(t_{route-tablet} (9) = -3.674, p =.005\); \(t_{route-vr} (9) = -4.358, p =.002\)). One-way ANOVA yielded no significant difference among the three groups (\(F_{2,27} = 2.314, p = 0.118\)), which did not reach what we expected in H3.

7.2 Survey knowledge

There were six questions related to survey knowledge on both tests. These questions were intended to test knowledge concerned with transitional relationships between multiple stages of the three solids. It can be observed that Desktop users yielded more homogeneous improvement compared to the other two groups of users. The Tablet group shows the highest median, but also the largest standard deviation, indicating the large variance in survey knowledge acquisition.

Figure 9 shows the boxplots of the overall user performance in terms of survey knowledge. From the t-test results showed, we can see that the Desktop and Tablet groups performed significantly better on the post-test after interacting with their assigned display. The Tablet and VR users had relatively higher variance than Desktop users. Although Desktop and Tablet obtained higher improvements, still no significance was found (\(F_{227} = 0.516, p = 0.603\)), which did not support our hypothesis H4.

7.3 Engagement ratings and scores

As mentioned above, we used an engagement questionnaire called VisEngage in this study to measure participants’ perceived level of engagement while interacting with each platform Hung and Parsons (2017). The questionnaire consists of 22 7-point scale Likert questions regarding 11 different types of user engagement measurements with visualizations. The questionnaire has the highest score of 154 when participants rate 7 (Strongly Agree) on all 22 questions and the lowest score of 22 when participants rate 1 (Strongly Disagree) on all 22 questions. Figure 10a shows the engagement score boxplots for the three types of displays. The VR group obtained a slightly higher median than the Desktop and Tablet groups. It is noteworthy that some outliers were found for the Desktop and Tablet groups. The reason might be the limited sample size.

Figure 10b–d shows the categorized ratings on each of the 11 categories. 6 out of 11 categories for VR were rated beyond 6 (Agree), which was quite surprising. The ratings for VR also had a lower standard error as compared to the other two platforms, which indicates that the participants were very engaged when they consistently perceived the information provided by the immersive VR environment (H5). The Desktop users rated higher on Discovery and Exploration, but lower on Creativity. The VR users rated higher on Control, Interest, and Discovery, while lower on Attention and Challenge. For the Tablets, it is interesting that it shows a more homogeneous rating as compared to the other two. All the ratings are between 5 and 6 except for challenge, which is slightly below 5.

Fig. 10
figure 10

Boxplots for a overall engagement score of Desktop, Tablets, and VR; the engagement ratings for b Desktop PC; c Tablet display and d VR HMD

Full size image

8 Discussion

In this section, we discuss the influence of the three platforms on the overall learning experience, engagement, and learning outcome.

8.1 Overall learning experience and engagement

It is noticeable that the participants who were new to VR generally showed different levels of amazedness when they first interacted with the visualizations. Some of the participants verbally expressed their excitement during the experiment. From the videos captured during the experiment, we observed that while interacting with the visualizations in VR, the participants seemed to stay focused and engaged very quickly with the immersive content. Sometimes they talked softly to themselves or slightly nodded their heads. The high interaction fidelity and immersive nature improved the engagement and learning outcome. It aligns well with the fact that VR obtained the highest engagement among the three platforms (H5), with especially higher ratings on Aesthetics and Interest. That was also the reason we expected that VR could bring significant benefits to observing structures of a 3D visualization as compared to the other two platforms (H2), which, however, was not the case found in the experiment. In the interview session, some VR users commented: “I feel tired after a while”, and “I feel uncomfortable after using (the tool) for a long time”. It reflects two common problems in modern VR HMD systems: visual fatigue and motion sickness Sharples et al. (2008), Cobb et al. (1999). The continuous wearing of the HMD may cause nausea and sore eyes, which may probably reduce the level of knowledge acquisition during the interactions. Furthermore, the VR participants asked more frequently for some rest during the task session. This could be the reason why VR generally has a higher standard variance in knowledge acquisition. In contrast, the Desktop and Tablet participants appeared to interact with the visualization tool in a more constant and stable manner, and most of them completed all the assigned tasks at one time.

In contrast, for the Desktop and Tablet groups, the participants appeared to be calmer and more silent during the interactions. A possible reason is that both desktops and tablets are familiar input devices to them. The statistics show that the Desktop and Tablet groups also improved significantly on their overall performance. The Desktop group users obtained an improvement slightly below that of the VR group. People’s familiarity with desktop PCs contributes to the respective knowledge acquisition and interactions process. However, because of the familiarity, it appears relatively harder for visualizations on a desktop display to “catch” users’ attention. For instance, through our observation, less excitement was exhibited from the Desktop participants while interacting with the visualizations. Furthermore, during the interview session, the desktop participants left more negative comments on general improvements, for example, “the background of the interface is too plain”, “the interaction is a little boring” etc., although the interfaces and functions of the three platforms were designed to be similar. The lower ratings on the “Creativity” category for Desktop also somewhat shed light on such a trend.

On the contrary, some positive comments were given by the Tablet participants, for example, “touch the dot (semantic dot) to morph is very straightforward and intuitive”. When it comes to tablet screens, multi-touch is arguably the most natural form of input, and participants’ positive comments on touch interaction showed such preference. However, some potential issues with tablet interactions were also highlighted by our participants. When the input device (finger) and the output device (screen) are coincident, the finger or hand can occlude parts of the display, causing a loss of information on the visualizations being presented Vogel and Casiez (2012). It was also suggested by researchers that occlusion may have a negative impact on performance Shneiderman (1991), Hancock (2004). As a matter of fact, we were able to confirm this issue because several participants commented during the interview: “... I cannot see the exact position of the dot (i.e., interactive dot on semantic STM) while I move it, my finger blocked my view”. This reflects a common problem with direct touch input, the “fat” finger covers the actual contacting point on the touch screen, which was not the case for Desktop displays with mouse input.

8.2 Learning outcome

The results of our experiment suggest that interactive visualizations presented on either desktop, tablet, or VR HMDs may assist in learning 3D geometry. Contradictory to H1 where there will be a significant difference in knowledge acquisition among the three platforms, users can generally obtain a similar level of improvements after interacting with the visualization tool on each platform. Our finding aligns with Sasik and Swindells Kasik et al. (2002), Swindells et al. (2004), which did not show a significant difference in different display types in assisting navigation and wayfinding in complex 3D models. However, this does not imply that the size, immersion, or interaction characteristics of the displays are unimportant. We were still able to draw some patterns from the interview and video captures of participants’ feelings regarding each platform in assisting their acquisition of knowledge from different categories.

For Landmark questions, the Desktop and VR groups received more positive feedback than the Tablet group. More than half of the participants commented that the size of the large display favored their learning experience: “I like the large display size, it just made the [solid] structure clear”, and “I like how the colorful solids look on the display”. The size of the large display makes the demonstration of 3D solids vivid and favorable, thus facilitating the acquisition of landmark knowledge. For Route questions, we expected a better performance for the Tablet participants as compared to the Desktop participants, since a multi-touch display could enable several actions simultaneously (H3). For instance, by using tablets and multi-touch gestures, users can morph the shape and rotate them simultaneously. However, by using a desktop display and mouse, only one action can be performed at one time. Our results showed no significant difference among the three platforms was detected. A possible reason could be that the Tablet participants rarely used the multi-touch feature in the experiment. One participant commented: “I tried to interact with two fingers, but lots of things are blocked by my hand”, “for transformations of 3D solids, I have to observe very carefully, but my hand makes it easy for me to lose some information”. Thus, we assume, that while performing activities that require a coherent observation of transformation properties, the “fat” finger occlusion prevents users from using the multi-touch feature. Instead, they tend to perform one action at a time. Thus, while consistent interaction and observation are both required on tablet multi-touch displays, finger/hand occlusion must be addressed properly.

For Survey knowledge, it is noteworthy that the participants in the VR group commented: “It’s hard for me to manipulate the map (STM) and observe the 3D solids simultaneously, the view [of the HMD] is very limited”, “Dynamic linking caused many unexpected moves (i.e., during the transformation of the solids), which I was not able to follow sometimes”. Here, by “unexpected”, the participants were referring to the object manipulation or transformations which were excluded in the first-person view. The challenge here, by our observation, was not that the propagation effects were not designed properly, but the fact that the field of view through the VR HMD was limited, which makes it participants difficult to see multiple visualizations simultaneously. The binocular field of view for the human visual system is beyond 180o horizontally, while current VR HMDs, such as the Oculus Rift we used in this research, have a field of view limited to 110\(^{\circ }\) Xiao and Benko (2016). That could explain why VR was rated lower in the “Attention” category. The participants felt hard to concentrate sometimes because multiple contents were presented simultaneously. This somewhat aligns with what we hypothesized in H4. However, for desktop PCs and tablets, every change is made visible on the 2D screen. They can constrain attention due to their limited screen size and interaction bandwidth, thus allowing more precise actions, analysis, and judgments Tory et al. (2005), Smallman et al. (2001), John et al. (2001). Therefore, around 70% of the Desktop and Tablet participants commented: “…everything is just clear on the screen”, “The layout is clear, it’s hard for me to miss something”. The participants were able to form a clearer overview of all visualizations being presented, and how they are potentially linked. It reflects that while visualizing multiple objects in a VR environment, such effects must be considered; otherwise, user performance could be compromised.

Based on our findings, we can distill the following lessons and recommendations for designing visualization tools for different display platforms:

  • When it comes to learning complex 3D geometry, there is no clear winner. All displays could be equally effective in supporting users’ learning of the content.

  • It is important to take into account the pros and cons of each display platform. While desktop and tablet displays may have a flatter learning curve, it may be more challenging to design interactive visualizations that appear novel and attention-grabbing for the users.

  • For VR displays, information overload and divided attention may be an issue given the relatively small FoV of current HMDs. Similarly, learning activities in VR need to take place in a more compact, focused way with short-term uses, given the possible discomfort users may experience when they use HMDs for a long period.

9 Conclusion, limitations, and future work

In this research, we have evaluated the effect of display platforms on engaging users in learning activities and knowledge acquisition while interacting with visualizations. An interactive visualization tool is designed and implemented on three different platforms: (1) Desktop display with mouse input; (2) Tablet display with multi-touch input; (3) VR HMD with the Oculus Touch controllers for input. An experiment was conducted with 30 university students (15 males and 15 females). Their acquisition of three different spatial knowledge was assessed after interacting with the tool. The whole interaction process was video-recorded. Participants were asked about their feelings on the interaction.

In general, our study contributes to a better understanding of the potential effects displays could have on analytical reasoning and learning experience. Our results show that all three platforms could improve analytical reasoning using interactive visualizations. Virtual reality head-mounted displays received positive feedback about their sense of immersion. However, visual fatigue and motion sickness may limit learning efficiency under a prolonged period of use. While presenting multiple visualizations, especially those with abstract relationships, the limited field of view on modern VR HMDs may make it harder for users to cover all the changes that happen. They might feel confused about where they should pay attention to, thus constraining the amount of knowledge gained during the interactions. Desktop and tablet displays are suitable for visualizing multiple visualizations because of their 2D display features. The desktop display is the most suitable way for information visualization because of its large display size and participants’ familiarity with the inputs. Desktop displays can also lead to a more homogeneous learning result. However, more efforts might be needed to make the visualizations creative and novel to users. For Tablet multi-touch displays, touch interactions are considered intuitive and easy to use. Nevertheless, finger/hand occlusion may yield some performance drop and prevent users from using the multi-touch feature. The mobile feature of tablets may also lower engagement and attention while learning alone. Performance-wise, although some previous studies comparing VR and traditional desktop displays concluded to be one over the other, we found no significant difference for learning 3D geometry, they could be equally engaging and efficient.

Some limitations of this research have been identified, including possibilities for future research. First, the sample size of our study is relatively small, which could be one reason why no significant differences were found between groups. Second, the participants were recruited from a university campus, which meant that they had similar backgrounds and educational levels. In the future, we plan to extend the study to involve participants with different levels of backgrounds and educational levels (e.g., high school students). Third, the participants were exposed to a lot of content within the two hours of the study, which could have made the effects of each platform less dominant. The abstract nature of the content made it challenging for learners to gain all the required information within 40 min of exploration. In the future, we plan to conduct longitudinal evaluations that involve exploration sessions across several consecutive days. This type of study will provide a deeper understanding of the limits and benefits of each platform. Fourth, the learning efficiency was not measured since in this study we focused on the effect of display platforms on spatial knowledge acquisition and user engagement. Therefore, we gave participants enough time to experience the tool and its features and learn the embedded concepts, instead of instructing them to complete the tasks as fast as possible. Future research could further explore multiple aspects of user performance. Fifth, factors such as possible learning effects, differences in interaction modalities, and the potential discomfort caused by wearing an HMD for a long period of time could influence the study results. Future research can consider studying these factors with more controlled setups. Last, the complex nature of the exploratory tasks made our findings potentially more applicable to abstract sense-making scenarios with 3D visualizations. Future research can consider further exploring the impact of display platforms on visualization tasks with a lower level of complexity and abstractness.