Keywords

18.1 Introduction

Anatomy education is fundamental in life science and health-related education. In anatomy education, it has been traditionally believed that cadaver dissection is the optimal teaching and learning method (Winkelmann et al. 2007). Cadaver dissection definitely provides tangible knowledge of the shape and size of the organs, bones, and muscles. However, dissection offers only a subtractive and deconstructive perspective (i.e., skin to bone) of the body structure. When students start with the complexity of complete anatomical specimens, it becomes visually confusing and students may have a hard time grasping the underlying basic aspects of anatomical form (Miller 2000). Consequently, many students have difficulties with mentally visualizing the three-dimensional (3D) body from the inside out (i.e., bone to the skin), as well as how individual body parts are positioned relative to the entire body.

Unfortunately, even with the availability of 3D interactive tools including virtual reality applications (Parikh et al. 2004; Temkin et al. 2006), the issue of visualizing movement still remains to be addressed. These interactive tools mainly focus on the identification of anatomical components and passive user navigation. Students must still mentally manipulate 3D objects using information learned from 2D representations (Pedersen 2012). For students’ planning for future careers such as orthopedic surgery, physical therapy, choreography, or animation, they need to know the muscle’s action, how it interacts with other muscles, and which normal movements it facilitates. This level of complexity is not easily conveyed via 2D static representations (Skinder-Meredith Smith and Mathias 2010). In addition, existing educational programs don’t provide a flexible learning environment that allows a student to make a mistake and then learn from it. Alternative learning materials that focus on constructivist approaches have been introduced in anatomy education: 3D printing (Li et al. 2012; Rose et al. 2015) and other physical simulation techniques (Myers et al. 2001; Waters et al. 2011). The constructivism theory of learning states that students should construct their own understanding by actively participating in their environment. However, these alternative methods also have limitations: it is difficult to create an anatomical model that makes movements, interactions are limited, and a single model can only present limited information.

Recent technical innovations, including interactive and immersive technologies, have brought new opportunities into anatomy education. Virtual reality and augmented reality technologies provide a personalized learning environment, high-quality 3D visualizations, and interactions with contents. However, the majority of anatomy education applications still focus on the identification of anatomical parts, provide simple navigations of the structure, and do not fully support 3D spatial visualizations and dynamic content manipulations (Jang et al. 2016). Even with the availability of 3D interaction tools, mentally visualizing movement remains problematic. Therefore, students still struggle to understand biomechanics to accurately determine movement caused by specific muscle contractions (Cake 2006).

We created Anatomy Builder VR and Muscle Action VR, embodied learning virtual reality applications, to promote embodied, multi-modal mental simulation of anatomical structures and functions. Anatomy Builder VR is a comparative anatomy lab that students can enter to examine how a virtual reality system can support embodied learning in anatomy education. The backbone of the project is to pursue an alternative constructionist pedagogical model for learning canine and human anatomy. Direct manipulations in the program allow learners to interact with either individual bones or groups of bones, to determine their viewing orientation, and to control the pace of the content manipulation. Muscle Action VR pursues interactive and embodied learning for functional anatomy, inspired by art practices including clay sculpting and dancing. In Muscle Action VR, a user learns about human muscles and their functions through moving one’s own body in an immersive VR environment, as well as interacting with dynamic anatomy content.

18.2 Background and Prior Research

18.2.1 Challenges in Traditional Anatomy Education

For over 400 years, cadaveric dissection has been the mainstay for teaching gross anatomy (Azer and Eizenberg 2007). However, in recent years there has been a trend in both the medical and veterinary schools toward reducing contact hours dedicated to traditional dissection-based anatomy courses (Drake et al. 2009; Heylings 2002). This reduction is the result of many factors, both pedagogical and practical. Traditional anatomy teaching methods have emphasized learning by deconstructing anatomical structures through full-body dissection. However, there is an argument that practitioners will be treating living patients, therefore the need is to learn anatomy in that context (McLachlan 2004). Additionally, there has been a shift toward an integrated and/or system-based approach to medical education. The argument is that there are other pedagogical practices that more readily promote student engagement and active learning (Rizzolo et al. 2006; Turney 2007).

Other reasons for the reduction in contact hours include the more practical aspects of teaching anatomy utilizing cadavers, such as the costs associated with maintaining a dissection laboratory as well as the ethical considerations for legally obtaining and storing cadavers (Aziz and Ashraf 2002). Another important factor is student and instructor safety. Typical embalming solutions contain formaldehyde at levels ranging from 0.5% up to 3.7%. Formaldehyde exposure can lead to many known health hazards such as respiratory and dermal irritation (Ajao et al. 2011; Tanaka et al. 2003). Recently, formaldehyde has been designated as a carcinogen (NTP 2011). All of these factors have resulted in some medical schools teaching anatomy with either limited or even no dissection of cadavers (McLachlan 2004; Sugand et al. 2010). To date, there is an ongoing debate as to the most effective way to teach anatomy, and no individual tool has the demonstrated ability to meet all curriculum requirements (Kerby et al. 2011).

18.2.2 Prior Research

When viewing the process of anatomy learning from the students’ perspective, an additional set of challenges arise. While there is more than one way to approach dissection (regional versus system based), the process of dissection is one of deconstruction. Layers of tissues are sequentially peeled away from most superficial to deepest. Additionally, muscles are cut and connective tissue and even organs are removed. This is necessary in many cases to find and view underlying, hard to access structures. However, this means that students are unable to replace the structures in their natural spatial arrangement, and any errors made during the dissection cannot be repaired.

In order to design and build student-centered embodied learning applications, we began by asking students what they would like to have. Texas A&M University offers a cadaver based undergraduate anatomy course to students in the Biomedical Sciences major. Students enrolled in this anatomy course (N = 23) were invited to participate in an anonymous online survey that asked them open-ended questions pertaining to their views on learning anatomy. We didn’t collect the participants’ identification information. Students were asked to list their primary study methods used to learn anatomy. Two main methods emerged from their responses: cadaver review and print notes. All of the students stated that they spent time in the anatomy laboratory reviewing using the cadaver. The second most relied upon method was to read and memorize their notes.

Students were also asked to respond to questions relating to what types of study aids they would have liked to use and what they would create as an additional study aid. One primary theme emerged from these two questions: Interactivity. Students wanted an aid that they could interact with that had diagrams, pictures, and physical elements. Here are representative student quotes from the survey:

“An interactive, virtual lab that correlated with our lab directly”.

I would love a picture that you can flip up the layers of the muscles, like in the labs. That way, it is easier to visualize while we aren’t with the cadavers.

Some kind of physical model that can allow students to visualize the origins/insertions and actions of the muscles.

While this was not stated directly, the implication is that they wanted a study tool that could be accessed outside of the lab and could be directly manipulated. We have performed a thematic analysis of the student feedback and laid the foundation of our creation of Anatomy Builder VR and Muscle Action VR, on the pedagogical theory of constructivism.

18.2.3 Constructivist Approaches in Anatomy Education

According to the constructivist learning theories, learning is the personal construction resulting from an experiential process. It is a dynamic process, a personal experience that leads to the construction of knowledge that has personal meaning (Mota et al. 2010). The use of physically interactive tools to aid in learning is supported by constructivist learning principles. In addition, characteristics of learning anatomy such as its visual, dynamic, 3D, and tactile nature present a unique environment for the implementation of such tools (Mione et al. 2016; Jonassen and Rohrer-Murphy 1999; Winterbottom 2017). Typically, gross anatomy courses utilize cadaver dissection to facilitate learning specific structures as well as spatial relationships of one structure to another. The deconstructive nature of dissection, however, directs students to first examine the big picture and then discover underlying details. While this approach may be useful for many, some students are more successful when they are able to build-up knowledge from the smallest details to the larger picture. Further, if students are able to successfully work through the deconstructive process of dissection, then mirror the process in a constructive way, they are more likely to have a comprehensive understanding of the location of structures and the spatial relationships between them (Malone et al. 2018).

While diagrams, drawings, and cadavers are sufficient tools for learning and recognizing structures, all of these aids have one common disadvantage—they cannot demonstrate movement (Canty et al. 2015). In addition, these tools do not allow students an opportunity to come to their own conclusions and construct their own understanding of the material. Incorporating aspects of visualization sciences that allow students to be engaged in the construction of their own knowledge could provide valuable new tools for teaching and learning anatomy (Canty et al. 2015; Malone et al. 2016a). Our interdisciplinary research, called Creative Anatomy, initially pursued this approach in the undergraduate gross anatomy classes via utilizing tangible and embodied methods. Here are our prior works that guided us toward constructivist learning using virtual reality.

18.2.3.1 Building Musculoskeletal System in Clay

We utilized the Anatomy in Clay Learning System® in our classes to evaluate how sculptural methods could benefit students to learn three-dimensional anatomical structures (Fig. 18.1). Jon Zahourek is a traditionally trained fine artist who created this entirely hands-on approach to learning anatomy in the late 1970s. The Anatomy in Clay Learning System® is now used in more than 6,000 classrooms nationwide. The system allows students to build essentially any gross anatomical structure out of modeling clay and place it on a model skeleton. In regard to learning muscles, the system is especially efficient for illustrating origins, insertions, shapes, and relationships of muscles to one another. Students are able to isolate each muscle and build them up from the base level of the skeleton to the surface of the skin (Anatomy in Clay Learning System 2019).

Fig. 18.1
figure 1

Building muscle system using “Anatomy in Clay” at the biomedical anatomy class: Students were asked to build muscles of the arm in clay for both the dog and the human after learning about the muscles in a lecture video, via class activities, and a canine cadaveric dissection. Students were guided through this activity with a step-by-step packet as well as aided by multiple instructors trained in human and canine anatomy

Full size image

To evaluate how supplementing learning by dissection with a constructive analogy affects students’ knowledge acquisition and application, 165 undergraduate anatomy students were asked to build pelvic limb muscles in clay following dissection of the same muscles. Prior to the clay building activity, students had completed the following class assignments: (1) watched lecture videos presenting information regarding pelvic limb muscles, (2) participated in class activities in which they were asked to apply information from the videos, and (3) completed a dissection of the pelvic limb on canine cadavers. During one lab period, students participated in a guided activity involving building muscles on a skeletal model of the dog or human (Anatomy in Clay® CANIKEN® & MANIKEN®) in order from the deepest muscles to the most superficial (Malone et al. 2018). Students’ feedback from this activity was extremely positive. Some of their written feedback includes quotes listed below.

It was really helpful to be able to see the muscles being drawn from nothing, to see where they were in relation to one another and learning about their actions.

I think this was a really good idea because it makes learning anatomy more fun and provides a change of pace for us. I definitely remember the information from that studio session better than most dissections.

I liked this activity most as far as helping learn because it helped put into perspective each muscle in the limb from insertion to origin to everything in between.

18.2.3.2 A Kinetic Model for Learning Gross Anatomy

To address how a constructivist approach could aid students with an understanding of biomechanical concepts, the movement was simulated with a kinetic model of a canine thoracic limb (Fig. 18.2). Students in an undergraduate anatomy course were asked to interact with the model, guided by an activity designed to help them construct their own understanding of biomechanical concepts. Anatomical structures such as bones, ligaments, tendons, and muscles were simulated in order to create movement. The simulated bone was made from plastic casting resin and built to withstand pressure from all angles while maintaining a small, delicate appearance. Simulated ligaments and tendons were created with elastic bands so that they were able to withstand pressure while simultaneously giving and stretching with movement. Simulated muscles were created from a string so that they allowed attachment of tendons, stretch and contract, and smooth, continuous motion (Malone and Pine 2014).

Fig. 18.2
figure 2

Kinetic model: This kinetic model represents the thoracic limb of a canine and the strings can be pulled to create the actions that muscles would create in life. Students used this model to study basic concepts in biomechanics such as flexion and extension of joints

Full size image

18.2.3.3 An Interactive Simulation Model to Improve the Students’ Ability to Visualize Movement

As an expansion of the previous model, our team created a physically based kinetic simulation model (Fig. 18.3) of the canine pelvic limb that provided student interaction via a computer interface (Malone et al. 2016b). Bones of the pelvic limb were molded and cast using the same technique employed for the thoracic limb model. Components such as structural support, muscle function simulation, simulated muscle structure, and simulated skeletal structure all had to be considered during model construction. Four servo motors were mounted onto the model. Two of these motors were mounted directly to the femur of the model, while the remaining two were mounted on the top of the Plexiglas stand. Screw eyes were put on the model and the stand at major origin and insertion points. White nylon string was attached to the furthest point from the center of rotation on the servo motor arm. The other end of the string was fixed onto a screw eye representing a point of insertion or origin common to a muscle group. A computer was connected to an Arduino microcontroller via a USB port and a serial monitor, or text-input window was displayed on the computer screen. The user would type the name of a muscle into the serial monitor and hit the return button on the computer. Motors that represent that muscle’s action would then be activated, turning the motor arm 90–180 degrees, thus creating the action of the muscle that was typed into the program (Malone et al. 2017).

Fig. 18.3
figure 3

Interactive Simulation Model being used in anatomy lab: This model utilized servo motors mounted to bones cast in plastic to create the actions of muscles in the canine pelvic limb. Students typed the name of a muscle into a computer and the simulation model would create the action of the specified muscle

Full size image

Students were divided into two groups—control and experimental. Both groups completed a short quiz after the unit on pelvic limb musculoskeletal anatomy. The students in the experimental group then were provided with an opportunity to interact with the model during the lab. The average quiz score for the experimental group improved significantly from a mean of 49.83–65.26%. The average quiz score for the control group did not improve significantly (49.50–58.60%). We received positive feedback from students. 75.55% of the students found the model easy to use, and 68.4% of students agreed or strongly agreed that the model helped with at least one concept related to movement.

Even though these physical and kinetic models provided benefits in the classrooms, we have encountered numerous challenges that cannot easily be resolved. The physical systems usually have fixed structures so that it is difficult to move around and replace parts as you wish. Physical materials have limitations to visualize movements and deformation of muscles. In addition, it is very difficult to provide personalized feedback to students using physical systems. Therefore, we started looking into incorporating embodied actions and interactive computer graphics using immersive technology.

18.2.4 Immersive Applications in Anatomy Education

Immersive technologies such as virtual reality and augmented reality are becoming popular tools in instructional technology, and educators are beginning to utilize them in their classrooms. Many meaningful efforts toward immersive learning technology have been made into anatomy and medical education. First, the creation of highly detailed and accurate three-dimensional models using computed tomography (CT) scans and magnetic resonance imaging (MRI) has significantly improved (Nicholson et al. 2006; Noller et al. 2005). Educators, taking notice of the increased accessibility and accuracy of these models, have begun to rely upon virtual 3D models to illustrate structures and relay concepts that are difficult or impossible to show on a cadaver.

Methods to increase the level of student interaction have been explored across multiple different platforms for more than two decades. In 1994, researchers in Greece published an article about a computer-based veterinary anatomy tutoring system. While the program was developed before the advent of accurate and easily accessible 3D models, the researchers concluded that the unique ability of an interactive computer program to individualize the learning experience, focus and combine certain aspects of veterinary anatomy and provide immediate feedback was undoubtedly beneficial (Theodoropoulos et al. 1994). Additionally, researchers at Linkoping University in Sweden developed a web-based educational virtual reality (VR) tool to improve anatomy learning. The program was well-received by medical students studying gross anatomy and generally preferred over study with a textbook and cadaver alone (Petersson et al. 2009).

Once the ability to incorporate virtual 3D models into interactive programs and virtual environments was fully developed, the development of interactive computer-based programs began to increase. A collaboration between the Oregon Health and Science University and McGill University used MRI data from a scan of a human cadaver to create a 3D model of the inner ear. Students who had access to a web-based 3D tutorial, scored an average of 83% on a post-instructional quiz while students without access to the 3D model only scored an average of 65% (Nicholson et al. 2006). Researchers at Oregon State University are exploring the use of a virtual 3D model of the canine skull and hyoid apparatus to allow interactive virtual articulation (Viehdorfer et al. 2014). At Texas A&M University Catherine Ruoff developed a computer program that demonstrated the anatomy of the equine paranasal sinuses. She concluded that anatomical structures that are difficult to visualize can be sufficiently illustrated and understood by allowing the user to interact with 3D models by rotating them in space and choosing which aspects of the model to focus on (Ruoff 2011).

A team of anatomists and computer scientists in Munich, Germany, have created what they refer to as an “augmented reality magic mirror” (Blum et al. 2012) which they have named “Mirracle.” Users stand in front of a large display which is equipped with a depth-sensing and pose-tracking camera. Different views of a virtual 3D model are displayed on the screen overlaying the image of the user based on the user’s position and gestures, essentially providing a mirror that allows the user to interactively explore their own anatomy (Blum et al. 2012). With the increasing use of virtual methods for visualizing and experiencing anatomy, many educators felt that the inherently physical nature of anatomy might soon be overlooked (Preece et al. 2013), however, it was not long before interplay of haptics and virtual tools were introduced. The Ohio University Virtual Haptic Back provides both visual and haptic feedback combining the use of a virtual 3D model with haptic feedback technology. Data collected during a study of this model showed that the accuracy of identification increased and required palpatory examination time decreased (Howell et al. 2005).

At the University of Magdeburg, Germany, a survey was conducted to evaluate methods involving visualization and interaction in anatomy education and how these methods are integrated into virtual anatomy systems. The researchers cite many learning theories that support the use and design of virtual anatomy systems. These theories include constructivism and embodied cognition, problem-based learning, and blended learning. Based on their analyses, these researchers concluded that virtual anatomy systems play an essential role in allowing students to explore shapes and spatial relationships (Preim and Saalfeld 2018). More recently veterinary students teamed up with visual arts students at Virginia Tech to create an immersive dog anatomy environment. The system allowed students to explore anatomical structures beyond the confinement of surgical views. Students were even able to zoom into certain organs to view layers of tissue (Virginia Tech 2019).

With the advancement of new technologies, virtual reality systems enable users to interact directly with anatomical structures in a 3D environment. This raises a new question: does manipulating the anatomical components in a virtual space support the users’ embodied learning and ability to visualize the structure mentally? Our goal is to develop virtual reality learning environments that support a constructivist learning approach and a flexible learning environment. These environments allow a student to make/manipulate a musculoskeletal system, as well as learn from any mistakes made throughout that process. The recent development of body movement tracking ability in virtual reality has allowed us to implement this idea. We investigated how virtual reality technology with hand controllers and body tracking benefit students’ learning while studying human/canine anatomy.

We present two case studies (Anatomy Builder VR and Muscle Action VR) in this chapter. Anatomy Builder VR allows the user to experience different components of human/canine anatomy by physical manipulations: recognizing bones, selecting bones, and putting bones together in the 3D orientation that they would be in a live animal. Muscle Action VR provides embodied interactions to learn about human muscles and their functions.

18.3 Case Study One: Anatomy Builder VR

18.3.1 Overview of Anatomy Builder VR

Anatomy Builder VR examines how a virtual reality system can support embodied learning in anatomy education through spatial navigation and dynamic content manipulations. In the VR environment, a user can walk around and examine anatomical models from different perspectives. Direct manipulations in the program allow learners to interact with either individual bones or groups of bones in order to determine their viewing orientation and control the pace of the content manipulation. Anatomy Builder VR consists of four main labs: Pre-Med, Pre-Vet, Sandbox, and Game room. A user can access each lab from the main lobby (Fig. 18.4) of the application.

Fig. 18.4
figure 4

Main Lobby of Anatomy Builder VR

Full size image

Pre-Med and Pre-Vet Labs provide guided lessons and quizzes about directional terms (Fig. 18.5), upper/lower limbs of a human skeleton, thoracic/pelvic limbs of a canine skeleton (Fig. 18.6). Students also learn skeletal systems through 3D skeletal puzzle questions.

Fig. 18.5
figure 5

Guided lesson about directional terms in the Pre-Med Lab

Full size image
Fig. 18.6
figure 6

3D puzzle quiz of a thoracic limb in the Pre-Vet Lab

Full size image

Sandbox (Fig. 18.7) includes major activities in Anatomy Builder VR. This provides an unstructured learning environment where a student can freely assemble human and canine skeletons in the “anti-gravity” field. Placing a bone in the anti-gravity field suspends it in place. Picking up another bone and placing it near a connection on the already field-bound bone will make a yellow line appear. When the user lets go of the controller trigger, the two bones snap together. The user repeats this action until the skeleton is assembled to satisfaction. Reference materials to complete the articulation of the limb are displayed on a side. Individual and grouped bones can be scaled to provide extreme details.

Fig. 18.7
figure 7

Skeleton assembly in the anti-gravity field of the Sandbox

Full size image

Game room is a playful space where a student can learn the names and shapes of bones (Fig. 18.8). A user can select certain body regions of human and canine to test their knowledge about the bones that belong to the regions. Individual bones are encapsulated in basketballs. Once the game starts, a user can shoot a ball with a bone that is displayed on the board.

Fig. 18.8
figure 8

A user is shooting a ball with a human radius in the Game Room

Full size image

18.3.2 Development of Anatomy Builder VR

The Anatomy Builder VR program utilizes the HTC VIVE virtual reality platform. VIVE is a consumer-grade virtual reality hardware, primarily developed for use with video games. The platform comes with a high definition head-mounted display (HMD), two motion controllers, and two infrared tracking stations. The tracking stations, placed on opposite ends of a room, allow for room-scale virtual interactions. The project was been developed in Unity3D, a real-time development platform. All scripting is done in C#. Unity’s development environment allows for easy integration of the VIVE with pre-built scripts and API. This allowed us to rapidly develop a functioning prototype and begin design on the user-specific interactions. Interaction with the virtual environment is primarily done with the VIVE controllers. The controllers have several buttons that are available to be programmed for various interactions.

18.3.3 Interaction Tasks in Anatomy Builder VR

There are multiple interaction tasks that would specifically support embodied learning of skeletal contents. All these tasks have been realized with VIVE Controllers.

Recognition of bones. For optimal identification of the bones, it is crucial that the user can view the bones from all angles. Therefore, natural head movement is required to be able to inspect individual objects.

Selection of bones. The prerequisite for 3D interaction is the selection of one of the virtual bones. Placing a bone in the anti-gravity field suspends it in place.

Transformation of bones. The transformation task includes rotating and translating the 3D bones. Since this is a task that the student is required to spend most of the time on, the success of learning the spatial relationships highly depends on the selection and interaction techniques.

Assembly of bones. Selecting and transforming a set of 3D bones is the process of assembling bones in the correct positions. When a user picks up a second bone and places one of its ends near a connection on the already field-bound bone, a yellow line appears. When the user lets go of the controller trigger, the two bone ends immediately attach together (snap) creating a joint. The user repeats this action until the skeleton is assembled to satisfaction. Assembly is entirely up to the user, allowing for incorrect bone combinations. This allows the user to make mistakes, learn, and try again.

18.3.4 User Study

In the pilot study, we investigated how a virtual reality system with direct manipulation may affect learning anatomy. The main focus of the study was to identify and assemble bones in the same orientation as they would be in a live dog, using real thoracic limb bones in a bone box and digital bones in the Anatomy Builder VR. For the purpose of the study, we recruited 24 undergraduate students. 66.7% of participants were females and the remaining 33.3% were males. The age range of these individuals spanned from 18 to 23, and each age was represented by roughly the same amount of people. However, a mere 8.3% of participants were 23 years old. Their majors were from departments across the university and they had never taken a college-level anatomy class before. The participants took a pre-study survey, experienced Anatomy Builder VR, and a post-study survey. We used a built-in processing system that recorded the duration of each participant’s use and quiz scores. During the VR session, the participants were given a brief introduction to how a VR system worked and then fitted with the VIVE headset and controllers. Upon entering the Anatomy Builder VR program, the student was given time to become comfortable with the controls before beginning the tasks. All participants tried the Pre-Vet lab. Each student’s study ended with a short interview about their learning experience.

On average, each participant spent 13.4 min in the VR system. The participants’ experiences with the VR system were very positive. In the surveys, most of the participants (90%) rated as Strongly agree for the statement, “I enjoyed using virtual reality to complete the activity.” and 8.7% as Agree. Using the method with a constructivist focus, 63.6% of the participants responded as Agree on the statement, “I was able to manipulate bones and put them together with ease in VR”, 27.3% responded as Strongly agree and 9.1% responded as Neutral. In the written responses, some participants expressed difficulties in certain interactions: rotating and scaling bones. However, most participants (88%) expressed positive aspects of learning the canine skeleton system using Anatomy Builder VR:

“This is so great. I think now anatomy students can learn things in a totally interactive world. Maybe they don’t need to go to the lab” (ID 09)

“…being able to leave the bones in a specific orientation in VR was a good compromise for me mentally because I didn’t have to continually revisit each bone or use energy holding them in the right place two at a time.” (ID 10)

“It actually made it easier because I was able to better manipulate the bones because they were held up “in space”. Also, it made more sense when the bones “connected” to each other.” (ID 11).

18.4 Case Study TWO: Muscle Action VR

Muscle Action VR pursues interactive and embodied learning for functional anatomy, inspired by art practices including clay sculpting and dancing. In Muscle Action VR, a user learns about human muscles and their functions through moving one’s own body in an immersive VR environment, as well as interacting with dynamic anatomy content. This allows learners to interact with either individual muscles or groups of muscles, to identify parts of the muscular system and control the pace of the content manipulation. Muscle Action VR utilizes the HTC VIVE virtual reality platform including VIVE trackers (Fig. 18.9).

Fig. 18.9
figure 9

VIVE trackers setup

Full size image

18.4.1 Overview of Muscle Action VR

Muscle Action VR consists of four activity labs: Muscle Tracking Lab, Sandbox Lab, Guided Lesson Lab, and Game Lab. The Muscle Tracking Lab requires six body tracking points using VIVE headset, two VIVE controllers, and three VIVE trackers attached at the waist and ankles. A user can experience other labs without trackers.

18.4.1.1 Muscle Tracking Lab

In this lab, a user with VIVE equipment including a VR headset, two controllers, three motion trackers, becomes a moving male écorché figure in VR (Fig. 18.10). The user is able to directly move their own body and see what muscles are contracting via a virtual mirror in the environment. Our system infers what muscles are being activated, and then highlights and displays these muscles so the user can get instant visual feedback. Users can learn about different muscles by using them directly and therefore gain an embodied understanding of muscle movements. The mirror system works by using a virtual camera that projects the figure onto a texture in front of the user. This provides more functionality than a regular mirror, giving the user the ability to switch to different angles. By switching angles, the user is able to see muscles from a different angle, which is crucial when viewing back or side muscles that are typically blocked from view.

Fig. 18.10
figure 10

A user can see themselves as a male muscle figure that moves based on their tracked movement

Full size image

The experience in this lab starts with a tutorial session. Going through the tutorial, the user learns how to rotate mirror views. The user can change the mirror views by selecting a different camera in the lab. Therefore, the user can view muscle details from other sides without turning their body. In addition, the user can change the muscle views by clicking the toggle button. The user can choose a mode to see either all muscles with highlighted muscles or specific muscles that are activated caused by a motion. After the tutorial, the user enacts motions that are demonstrated on the mirror and examines the visualization of the muscle activation via the mirror screen (Fig. 18.11). The last part of the session allows the user to freely move their body parts and learn how their movement affects muscle activation.

Fig. 18.11
figure 11

The tutorial sessions show an animated visual reference to guide the user through the motion control system

Full size image

18.4.1.2 Sandbox Lab

The Sandbox room allows the user to interactively generate muscles on different portions of the body to activate the muscles and to create specific actions. Each bone in the skeleton is a simulated physical object using a RigidBody component in Unity with joint connections to other bones. Users can see the mechanics behind how bones are connected and how muscles drive their motion and movement. The positioning of muscles relative to bones is a critical feature of what kind of motion will occur when the muscle is contracted or activated.

The Sandbox muscular system in the Sandbox Lab is comprised of three key components. The first component is the preparation step, where a developer marks all critical collision regions on the connecting bones objects. This is done to emphasize what sections of the skeleton the muscles should distort around. The deformation spheres used to mark these regions guide generated muscles around details such as joint connections or protruding bone features. The second key component is a convex hull algorithm. It takes a 3D line drawn by a user as input to generate a planar muscle that bends around the skeleton’s deformation points. The start and endpoints of the user’s line mark the insertion and origin points, respectively, of the generated muscle. Meanwhile, the line in between determines which side of the skeleton the algorithm will generate a muscle (Fig. 18.12). The third component is to activate each muscle to create movement. Each muscle can be individually contracted or relaxed, to produce flexion and extension of the skeletal joints. After a muscle is drawn, a slider is automatically created to allow the user to control the flexion and extension of the associated muscle. In addition, the entire skeleton’s movement can be generated by activating multiple muscles simultaneously (Fig. 18.13).

Fig. 18.12
figure 12

Muscle drawing and generation

Full size image
Fig. 18.13
figure 13

Muscle activation interface

Full size image

18.4.1.3 Guided Lesson Lab

Muscle Action VR also provides an interactive, but guided lesson lab about directional terms, and the basic biomechanics of muscles and their movements (Fig. 18.14).

Fig. 18.14
figure 14

Guided lesson lab

Full size image

18.4.1.4 Game Lab

The Game Lab allows the user to test their knowledge of human muscles and movements, covered in our application, through the form of a dodgeball game (Fig. 18.15). Users are challenged to move certain muscles by using a provided skeleton to deflect dodge balls. The user simply points at the provided muscles on the skeleton to contract and relax, so that the skeleton can move appropriately to play in the game. The user will continue to receive points for each dodge ball deflected, until three dodge balls have passed the player’s perimeter, thus ending the game.

Fig. 18.15
figure 15

Game lab

Full size image

18.4.2 Development of Muscle Action VR

Similar to the Anatomy Builder VR project, Muscle Action VR was also developed in Unity3D through C# scripting and used the HTC VIVE as its VR platform. This development environment offered the same advantages from the previous project, such as being compatible with HTC VIVE through the OpenVR SDK and the SteamVR plugin, allowing easy integration and rapid prototyping. In addition to these advantages, the HTC VIVE was also chosen to utilize the VIVE tracker hardware, which is essential for the Body Tracking Lab to be possible.

The Muscle Tracking Lab utilizes six tracking points to drive the rig of the virtual body or “Muscle Man”: a VIVE headset, two VIVE Controllers, and three VIVE trackers. Based on the positional and rotational values of these trackers, an IK solver from the FinalIK plugin was used to estimate and control different parts of the virtual body to mimic the user’s movements in the real world. To create different muscles contracting and extending from certain actions, blend shapes were used to drive this effect, along with using the Shape Driver plugin to correctly create this effect based on the positional and rotational values of the virtual body’s joints.

18.4.3 User Experiences in Muscle Action VR

We received very positive responses from preliminary studies done with several university students and anatomy experts. Most participants didn’t have prior VR experience but they were able to navigate the application through and successfully learn key concepts and detailed visualizations. Students pointed out that this application would be greatly beneficial for learning anatomy and they would share their experience with peer students. In the open-ended interviews, participants described that the application was intuitive and engaging by providing innovative learning methods. Here is some feedback from participants:

I wish we had this kind of learning aid when I was taking anatomy. This is fantastic.

Even after I finished the VR experience, I still have an image in my mind so I can navigate the structure through.

This looks so realistic. I feel like I am in the lab and dancing with the muscle man. I would like to show this to my roommate.

18.5 Conclusion

Our studies have focused on ways to utilize constructivist principles to design VR applications for learning anatomy. We created a learning platform that allows students the opportunity to not only visualize the individual components of the skeletal system but also interact with them in meaningful ways. Participants in our study enjoyed putting the skeleton together because the process was similar to that of completing a puzzle or building blocks. The virtual bones remained in place within the anti-gravity field, and joints were visually represented in a dynamic way. Another exciting aspect of the Anatomy Builder VR program is the inclusion of the “Sandbox Lab” for novices so that participants have an opportunity to place bones into an imaginative skeleton. Learners were encouraged to assemble the bones themselves, and they were free to make mistakes. This also provided a safe environment for active exploration.

For Muscle Action VR, we incorporated aspects from traditional art practices such as clay modeling and sculpting and dance to create an interactive and embodied learning environment. VR and traditional learning methods lack a way for learners to actually visualize and create movement. The musculoskeletal system is important because it is dynamic, yet cadaveric dissection and diagrams are static. Our VR application encourages learners to use their own body to visualize what is happening beneath the skin. We are extending the understanding of virtual reality design for anatomy education. In the future, Anatomy Builder and Muscle Action VR will include an even richer environment for self-evaluation and collaboration.