Abstract
We describe the framework for capturing the internal view of complex anatomical spaces via multiple media and haptic platforms, exemplified by realistic and conceptual representations of the pterygopalatine fossa (PPF). A realistic three-dimensional (3D) mesh of the PPF was developed by segmenting the osseous anatomy on computed tomography (CT) using Materialize InPrint. Subsequently in Autodesk 3D Studio Max, the realistic mesh was enhanced with graphically designed neurovascular anatomy and additionally a conceptual representation of the PPF with its connections and contents was created. An interactive web-compatible Adobe Flash tutorial using ActionScript was developed, allowing users to advance through a series of educational slides that contained interactive rotatable interior camera views and scrollable CT cross-sectional content, incorporating both the realistic and conceptual models. Both models were also 3D printed using polyamide material. In the realistic model, the neurovasculature was colored with water-based acrylic paint. A 3-piece modular design with embedded magnets allows for internal visualization and seamless assembly. A serious gaming environment of the conceptual PPF was also developed using Truevision3D application programming interface, where users can freely move around rooms and hallways that represent various spaces. Lastly, the realistic model was incorporated into a headset-based virtual reality environment, Surgical Theater, allowing visualization and fly-through inside and outside the model. Multiple 3D techniques for visualization of complex 3D anatomical spaces from within were described, with the necessary software and skills detailed. A rough estimate of the time and cost needed to develop these tools as well as multiple supplementary source and end result files are also made available. Educators could utilize multiple advanced delivery methods to incorporate custom digital 3D models of complex anatomical spaces understood from inside.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Advances in three-dimensional (3D) visualization and printing have provided many clinical innovations, including surgical models, medical devices, and tissue engineering [1]. Additionally, 3D imaging remains foundational for medical education and is challenging the traditional paradigm of anatomic instruction, in which trainees study two-dimensional (2D) anatomic illustrations that are correlated with cadaveric materials [2]. Although 3D imaging technology can render virtual or physical models of any anatomic volume, it is most practically used to re-create anatomy that is spatially complex [3]. In this work, we describe the use of 3D digital technologies to uniquely re-create the spatially complex pterygopalatine fossa (PPF) as viewed from within the fossa. This approach may be similarly applied to re-create other spatially complex anatomy.
Nicholson et al. [4] and Abid et al. [5] reported increased exam scores with 3D computer-based modules detailing ear anatomy [4] and organogenesis [5], respectively. Although some studies have reported no additional benefit of 3D stereoscopic computer models [6] or instructional videos [7] on anatomy exam score, the results of Nicholson et al. [4] and Abid et al. [5] suggest that 3D imaging is of benefit when re-creating anatomy that requires greater visuospatial understanding, such as ear anatomy [4] or organogenesis [5], or when it is more efficiently processed by learners with high spatial ability [3]. The latter point highlights the significance of personal learning style, as learners with high spatial ability may prefer dynamic 3D models, while those with low spatial ability may prefer static 2D images [3]. In addition, these models can provide a conceptual analogy to learning 3D anatomy by depicting the orientation of anatomic structures as they arise from a conceptual geometric base. For instance, the middle ear cavity has been previously illustrated as a cube-shaped space consisting of a roof, floor, four walls, two windows, and a door with each plane containing specific anatomic structures [8].
With increased affordability, greater computer processing ability, and higher spatial resolution of monitors and smartphones [9], alternative learning platforms are gaining widespread acceptance and adoption. E-learning, in the form of web-based learning resources, multimedia, and social networking, is the most accessible platform and, consequently, is more seamlessly integrated in learning [9] with extensive reach. Mathiowetz et al. [10] showed that e-learning via an online anatomy software resulted in statistically similar laboratory practical exam scores and written exam scores versus gross anatomy learning. Virtual or physical 3D models may be used as supplements to or potential substitutes for cadaveric materials, as Lim et al. [2] found that external cardiac anatomy was better learned by trainees instructed with 3D printed models versus cadaveric materials. In addition, 3D models have also been shown to significantly increase understanding of spinal fractures [11], hepatic segmental anatomy [12], and renal tumors [13] in comparison to computed tomography (CT) counterparts. Furthermore, the customizability intrinsic to 3D visualization technology can be used to re-create patient-specific 3D anatomic models that depict critical spatial relationships in the setting of pathological conditions [14].
Virtual reality (VR) and serious gaming immerse the trainee in a computer-simulated environment with real-time interaction to varying degrees. “Serious games” are interventions that implement elements of gaming to achieve a serious purpose, such as health or educational goals [15]. Both are simulations that allow the trainee to experience learning in a more engaging fashion. Although not implemented in our work, augmented reality, the addition of digital information to enhance the trainee’s sense(s), has also been applied for anatomic understanding by superimposing digital datasets onto corresponding real-world anatomy [16, 17].
In this article, we describe the development of static and dynamic learning platforms for the spatially complex PPF as an example, which is a major neurovascular communication site located in the deep face and skull base [18], as viewed from within the fossa. Given that these communications can inherently act as a channel for spread of pathology, a keen understanding of the anatomy of the PPF is of utmost importance for multiple specialists. Recently, Bannon et al. [19] used axial slices from a head CT scan to create a “negative” space model of the PPF, in which the fossa and its associated foramina and canals were uniquely depicted as a solid volume. Here, we have developed an alternative, novel perspective of depicting the PPF using three distinct media platforms: an interactive Adobe Flash tutorial, a conceptual and a hybrid realistic 3D printed model, as well as a fly-through VR/serious gaming environment.
Materials and Methods
An Institutional Review Board (IRB) approval was not necessary as only anonymized freely available source data was used, and other models were graphically designed.
3D Digital Design of Realistic and Conceptual PPF
An Apple Macintosh device (MacBook Pro Retina Display Intel Quad Core 2.4 GHz, 16 GB RAM) was utilized with VMWare Fusion installed to allow for use of Microsoft Windows compatible software. A sample high-resolution maxillofacial CT dataset was obtained from the anonymized repository available on the Osirix website (http://www.osirix-viewer.com, Pixmeo SARL, Geneva). Materialize InPrint (Leuven, Belgium) was used to import the sample maxillofacial raw CT dataset (0.625 mm in bone tissue kernel) to perform 3D reconstruction of an initial model of the osseous PPF after initial automatic and subsequent slice-by-slice manual segmentation (Fig. 1a). Segmentation was performed by a neuroradiologist with 3 years of experience at the time (author RJ). In this case, the use of a CT bone algorithm is the preferred form of source data as opposed to a soft tissue kernel, since the intricate foramina, fissures, and canals resolved more accurately without running the risk of separate bony walls becoming attached during 3D reconstruction. However, since contrast resolution is lower in the bone kernel compared to the soft tissue kernel, there is added noise when thresholding bone during automatic segmentation (Hounsfield Units above 226), which has to be manually eliminated. After defining the initial bounding box in Materialize InPrint, the “Keep Largest Segment” option of the thresholding tool can be selected or the “Isolate” tool may be utilized to separate the main 3D object from the surrounding noise. Subsequently, the slices have to be inspected for any errors that need manual correction in the coronal, sagittal, and axial planes. Lastly, the “Smooth” tool is applied to smoothen the rough noisy surface of the bone.
Autodesk 3D Studio Max 2018 (San Rafael, CA, USA) was then used to enhance the initial 3D model’s design by adding the pterygopalatine ganglion with the emanating nerves (Fig. 1b), the trigeminal ganglion with its branches as well as the third segment of maxillary artery with its branches, conformed to the osseous anatomy (Fig. 1c). This step, performed by a neuroradiologist (author RJ), requires skills in 3D digital sculpting using graphic design tools, such as ones that allow for pushing and pulling of vertices, extrusion of surfaces, bending of objects, and smoothing of meshes. Specific details regarding how to perform each of these tasks are beyond the scope of this manuscript and can either be learned through online tutorials or by obtaining the services of a 3D graphic designer [20]. A supplementary video file is also available online to view a flythrough of the 3D model and into the PPF where the neural anatomy has been added to the native 3D reconstructed osseous anatomy. This model serves as the realistic version of the PPF for use in the multiple delivery methods described in the manuscript.
Additionally, the PPF is conceptualized as a room, its communications as hallways and doors. This approach breaks down the 3D anatomic relationships between the PPF and the adjacent communications into the six directions of medial, lateral, anterior, posterior, superior, and inferior. Initially, a simple computer-enhanced hand-drawn illustration was created with proper perspective view to provide the “big picture” of a negative space conceptual model (Fig. 2). Subsequently, graphic design software, Autodesk 3D Studio Max, was used to create a negative space set of 3D meshes, with the PPF depicted as a cone with tubular connections representing fissures, foramina, and canals connecting to cubes that represent the surrounding anatomical spaces. Additionally, hollow volumetric 3D meshes were made with interconnecting communications and their contents. This step can be accomplished using simpler graphic design software since the volumetric meshes created are predominantly simple geometric shapes, such as cubes, spheres, and cylinders. This significantly simplifies the process of graphic design, allowing users with essentially no prior experience to start creating customs 3D models without the need to learn time-consuming steps for producing complex and advanced 3D shapes. One important consideration to keep in mind regarding this step pertaining particularly to visualizing models from within is that the interior of each model has to be lined with renderable surfaces, meaning that the walls have to actually have a true thickness with surface triangles defining both the external and the internal aspects of each wall. This can be challenging depending on the graphic design tools that are used. For example, a solution for creating a 3D printable cube may be to create each wall separately as an object and to digitally assemble them to create the final cube. It is also possible to create two identical objects such as spheres, one made smaller and placed inside the other and subsequently inverting the surface triangles of the smaller object toward the inside and finally merging the surfaces of the two objects to create a final sphere with a thickness given to the wall, making the object 3D printable. In Autodesk 3D Studio Max, this can be performed using the “Attach” tool of the “Edit Mesh” modifier, allowing for creation of a final sphere with a wall that has defined inside and outside surfaces.
Adobe Flash Tutorial
Adobe Flash provides the capability to create heavily interactive and powerful web-compatible tutorials. In Flash Builder or Flash Professional, developers can create a user interface with customization of buttons and components using ActionSctipt code (performed by author RJ, a radiology resident at the tiem). The details regarding the ActioScript 3.0 syntax codes and commands are beyond the scope of this article. The source FLA file and the final web-compatible SWF files are made available for download as supplementary material. As it pertains to the main goal of this article, i.e., for the purposes of viewing a 3D anatomical space from within, users can rotate the “camera view” inside the conceptual PPF by scrolling the mouse-wheel or dragging the mouse left/right in order to see the connections and structures. These interactive animation Flash-based modules are created by sequentially placing rendered bitmap image files of the conceptual PPF from Autodesk 3D Studio Max with the camera view inside the PPF rotated every few degrees and obtaining an image. For example, for creating a 360° rotation performed through 30 clicks or steps, there needs to be a capture made by rotating the camera every 12° to obtain a total of 30 images. Text elements, graphics, buttons, audio, and other elements can be incorporated into this environment. Scrollable correlative axial, sagittal, and coronal thin-slice CT images of the PPF are also provided in addition to instructional slides.
3D Physical Models
The size of the previously described realistic PPF model was scaled to 2.5 times of normal, appropriate split planes were applied, spaces for magnets were incorporated, and the resultant 3D mesh was exported as an STL (STereoLithography) file for printing (performed by RJ, a neuroradiologist). Determining the ideal split planes can be challenging, depending on important considerations such as specific anatomic area of interest that is to be revealed and physical separability of the pieces. The latter can be especially difficult yet is of utmost importance. All iterations of physical assembly and disassembly must be considered during the digital design stage and potential problems predicted. With respect to designing spaces for magnets, it is important to create them slightly wider than the size of the magnet by approximately 0.5 mm, incorporating large enough magnets that can withstand the force created by the weight of each piece, as well as avoiding placement on any important anatomic landmarks or structures. We used a commercially available online service (iMaterialise.com, Lueven, Belgium) for 3D printing using selective laser sintering (SLS) with polyamide (nylon) material, which is a sturdy and versatile material with 0.3 mm surface detail and a 1-mm recommended minimum wall thickness requirement. Water-based acrylic color was used for painting the neurovasculature. As part of a large skull base conceptual 3D printed model, the PPF was 3D printed as a de-roofed hollow, near-pyramid shape structure with the surrounding connections and their contents also included. This was accomplished after some graphic design modifications to the previously mentioned conceptual PPF model. Again, polyamide material was used for 3D printing purposes due to its versatility and durability.
Serious Gaming
The conceptual PPF model was utilized as part of a “first-person shooter” serious gaming program, where the trainee can freely fly-through rooms and hallways that represent anatomical spaces of the skull base. Microsoft Visual Basic 6.0 programming language was used to access libraries from TrueVision 3D 6.3, which is a 3D gaming engine that is layered on top of the Microsoft DirectX 8.0 application programming interface (API) used for creating game-play. Extensive amount of coding was required for creating this gaming environment (performed by BSJ, a medical student at the time), the details of which are beyond the scope of this article. The graphics design of the 3D objects was done using Autodesk 3DStudio Max (performed by RJ, a medical student at the time), and conversion to DirectX files was performed by the Panda DX Exporter 4.9 plug-in. Teaching material in the form of optional pop-up menus, including interactive quizzes, cross-sectional CT sequences, pathology images, and slide presentations were created for major structures as teaching modules using Adobe Flash CS3 Professional.
Virtual Reality
A Surgical Theater Surgical Planner (SRP) workstation running 64-bit Microsoft Windows 10 Pro (Intel® Core™ i7-5930K CPU 3.50 GHz, 3 GB RAM) was utilized along with Surgical Theater proprietary technology version SRP 7.4.0 to render 3D models in the VR environment. This technology not only allows for the viewing of volumetric 3D reconstructions of CT and MR DICOM datasets in 360° Virtual Reality, but also grants the ability to import and view 3D objects as STL or OBJ files into the scene. The scene can be viewed on a flat touch screen and simultaneously with a variety of consumer ready VR headsets (Oculus Rift, HTC Vive, HP Windows Mixed Reality Headset, etc.). Previously generated 3D objects of the realistic PPF were imported into the Surgical Theater scene using the built-in polygon object importer. The PPF was imported as multiple separate objects in order to individually control the color, opacity, and visibility of each structure in the scene (performed by AHR, a company technologist under the supervision of RJ, a neuroradiologist). An Oculus Rift with Oculus Touch controllers and a Logitech F310 gamepad were used to view and control the scene in VR. Specific details regarding how to perform each of these tasks are beyond the scope of this manuscript and can be performed by a Surgical Theater technician. A supplementary video file is available to view a fly-through of the VR space into the PPF.
Results
In order to illustrate the 3D complexity of the PPF, we have developed both conceptual representations and realistic anatomic models. As delivery methods, three distinct platforms are implemented: web-compatible Flash-based interactive tutorials using the conceptual models, physical 3D printed models of both the conceptual and realistic versions of the PPF, and serious gaming through the conceptual PPF and VR exploration of the realistic PPF.
Adobe Flash
The PPF is conceptualized in the Adobe Flash tutorial (available under the supplementary online material), which consists of a non-interactive component with 2D illustrations of the PPF, as well as the relevant anatomy, and an interactive component. The interactive portion of the tutorial includes a rotatable negative space conceptual view of the PPF with connections to surrounding anatomical spaces (Fig. 3a), scrollable CT axial, coronal, and sagittal cross-sections (Fig. 3b), and a rotatable view from inside to understand the walls of the PPF with their associated foramina, canals, and fissures (Fig. 3c, d). Interaction is by pressing the left/right arrow keys, left-clicking the mouse while dragging to left/right, or clicking the “Previous/Next” buttons for rotation of the camera-view in the anatomic space or by scrolling through the CT cross-sections. A “Show/Hide” toggle option is also available for the label of the structures.
3D Printing
Physical anatomical representations of the PPF include both realistic and conceptual models (Figs. 4, 5, and 6). The realistic 3D printed model consists of a 3-piece osseous base along with the neurovascular anatomy (Fig. 4a). The 3D printed model polyamide material had specified dimensions of 10.3 × 12.9 × 22.1 cm (459.2 cm3 of material). Polyamide is a white-colored material that is durable, rigid, and has a granular surface. As the 3D printing technique used here, SLS on polyamide, cannot provide multicolor prints, the nerve and blood supply depictions were painted manually in yellow and red water-based acrylic color respectively using miniature paintbrushes, with robust results and no long-term chipping of the paint. The model also contains 3 × 1-mm round disc magnets in order to perform assembly/disassembly of the pieces to allow for visualization of the contents of the PPF (Fig. 4).
The conceptual PPF as a de-roofed hollow space (Fig. 6) was also 3D printed as part of a large model of the skull base and temporal bone with individual structures portrayed either conceptually as geometric objects or somewhat mimicking their realistic appearance. The material used was again polyamide due to its versatility, durability, and slight flexibility, and the approximate dimensions of the PPF portion of the model with the surrounding spaces was 10 × 14 × 14 cm. Because the PPF was printed as part of a large model of the skull base and the PPF is relatively a very small space, the nerves had a thickness of 1 mm, which led to breakage of one of the nerves in the PPF. No coloring was possible in this case due to the small size of the structures and difficulty reaching them in this one-piece model. Overall, this model mirrors the one used in the Adobe Flash tutorial.
Serious Gaming
An extension of this conceptualized space is a serious gaming representation of the skull base which also includes the PPF (Fig. 7). This offers the most interactive experience to the user by allowing exploration of the fossa in a manner similar to that of a first-person shooter video game. The game environment initially begins as a complete skull with anatomic entry points represented as transparent blue squares (Fig. 7a). After this stage, the game transitions into a space containing a 3-story building where the PPF is conceptualized as a room with the associated foramina, fissures, and canals as hallways and surrounding anatomical spaces as other rooms (Fig. 7d). In order to navigate and interact with the environment, the player performs a combination of simultaneous keyboard and mouse inputs. Keyboard inputs provide directional movement and are executed when the player presses the respective arrow keys. Mouse inputs allows for directing camera view and for targeting elements of the environment. Left-clicking simulates shooting of approaching pathogens and malignant cells, i.e., “enemies” (Fig. 7b) and right-clicking identifies a selected structure (Fig. 7d, e). Scrolling the mouse wheel allows for up and down strife movements. Pressing the “Alt” key opens any Flash-based pop-up tutorial that may be linked to a given structure, which can be animation, text, instructional videos, quizzes, or scrollable CT or MR images. Additional Flash modules can be added any time by simply placing them in a designated file folder with the filename matching the name of any given structure, for example “Vidian_Nerve.swf”.
Additional game features include “hit points” which represent the amount of damage sustained during engagement with pathogens, which when reaching zero would indicate user’s trial to have ended (“Game Over”). Pathogens (Fig. 7b) which have infinite amount of ammunition can be destroyed by using appropriately chosen “antibiotics” or “antibodies” that are obtained by completing quizzes. The pathogens also move away from the user when approached. A dynamic compass represents the direction of the view anatomically at all times as an overlay transparent circle in the right lower corner (Fig. 7d–f). Proper collision with solid objects has been incorporated to prevent passing through the walls and surfaces. A 2D map of the three stories of the skull base as a building is available with the PPF being on the middle floor (Fig. 7c). There is no specific end goal in the game except for exploration, navigation of the anatomic spaces, identifying structures, and learning during this process. Sound effects have also been added to the background theme, including during shooting and with movement of the user’s spacecraft. This high degree of interactivity with the addition of the gaming features may promote learning and improve retention rate, while entertaining the learner.
Virtual Reality
The virtual model (Fig. 8) simulates the real, rather than conceptualized, anatomic space of the fossa through a VR environment, representing exactly the same realistic 3D printed model described earlier. The simulation provides the trainee a more immersive experience within the fossa, as the environment can be observed from any point within the fossa at any angle while wearing the VR headset. The trainee interacts with the environment by use of the headset and two hand-held controllers. The headset tracks the trainee’s head position and simulates the camera angle for viewing the environment. The left controller’s thumb-controlled joystick allows for left/right and up/down movements, while the right controller’s thumb-controlled joystick allows for forward/backward movement. Both controllers have an index finger-controlled trigger which activates a 3D pointer/vector in the scene, useful for identifying structures. The scene that the user sees is also projected on a monitor, allowing for teaching and group discussions to take place. A unique feature of the VR environment is that up to twenty objects can be labeled with words, while words remain static relative to their associated 3D object, labels rotate relative to the camera view to remain perpendicular to the trainee’s line of sight.
Time and Cost Analysis
An important question that may naturally arise is the amount of time and expenses associated with developing the described tools in a practical sense. While it is very difficult to provide an accurate assessment due to the development of these tools in our case over a number of years through interconnected projects, one leading to another, an attempt has been made to provide a rough analysis (Table 1). Note that the 3D Studio Max files (.MAX) and the Adobe Flash Builder file (.FLA) have been made available as supplementary material to this article.
As mentioned earlier, the digital 3D model of the PPF in its realistic and conceptual forms have been implemented in multiple mediums. Therefore, those compromise a very important initial step in creating these tools. The conceptual 3D model is developed based on initial hand-drawn illustrations at no cost by one of the authors and through the use of graphic design software, Autodesk 3D Studio Max. While this is a commercial software, it is available for free under an education license for students and educators. The skills necessary to create the geometric shapes and their modification was acquired by the same author with some prior interest and background in 3D graphic design, therefore, at no cost. This powerful and advanced software provides numerous tools for creating highly customized 3D models depending on the skills of the user. Targeted learning of the software capabilities through online tutorials and videos to eventually implementing them, required approximately 8 weeks. However, students in graphic design may be recruited or freelance graphic designers can be hired depending on needed skills and complexity of models for prices ranging from $25–50/h to $75–150/h. The realistic version of the model was initially 3D reconstructed from CT using Materialise InPrint, available through institutional commercial license at a hospital already involved in 3D printing, therefore at no additional cost related to these projects requiring approximately 2 weeks to prepare. Modifications and enhancements of the realistic PPF model were done again using the free educational Autodesk 3D Studio Max software, requiring an additional 4 weeks. There is freeware software such as Slicer3D and Meshmixer which can alternatively be utilized, although perhaps with less manual and semi-automatic segmentation tools. The cost of the conceptual 3D printed model was $150, and the cost of the realistic 3D printed model was $200, each requiring approximately 2 weeks for outsourced online 3D printing and shipping. The cost of small magnets, miniature brushes, and acrylic paint used in enhancing the realistic PPF model was approximately $25.
Adobe Flash Builder Standard 4.7 costs $249 and is available on the Adobe website. Targeted learning of user interface and design of Flash modules utilizing the Actionscript 3.0 programming language, required approximately 2 months for one of the authors who had computer programming background and was done with the help of freely available online resources. Creating Flash tutorials may also be done by recruiting computer science students or freelance Flash programmers at prices ranging $15–50/h. Once again, the Flash Builder (.FLA) file provided in the supplementary material can be used as a template to create similar interactive scrollable Flash tutorials and speed up the process.
The VR environment was developed in a system already utilized by the institution, specifically for neurosurgical guidance and interdisciplinary consultations, therefore, at no additional cost. Import of the conceptual PPF model into the VR system environment required only a few hours. There are numerous free VR software development kits, such as Forge and Vimeo360, the use of which requires programming skills or, again, freelance programmers can be hired. Lastly, the serious gaming experience of the PPF was part of a larger project encompassing the entire skull base anatomy, programmed and graphically designed by two of the authors over an 8-month period requiring intermediate skill levels and, therefore, would be immensely difficult and time consuming to create by interested non-professional individuals. Microsoft Visual Basic as part of the Visual Studio Development Kit was acquired free of charge at the time for an educational institution, as was Autodesk 3D Studio Max. Once again, freelance programmers and graphic designers can be hired to use gaming design software such as Unity or Unreal Engine to help create a serious gaming environment.
Discussion
Mastery of anatomy is of utmost importance to radiologists, surgeons, and educators. Understanding the PPF, as an example of complex and intricate anatomy, can present a challenge to trainees [18]. The PPF is an important space as it communicates with the middle cranial fossa, orbit, nasal cavity, oral cavity, pharynx, foramen lacerum, and the infratemporal fossa via eight foramina, fissures, and canals. These include the sphenopalatine foramen, foramen rotundum, pterygoid canal, greater and lesser palatine canals, inferior orbital fissure, Vivian canal, and the pterygomaxillary fissure. Each of these contains important neurovascular components. The fossa itself can play an important role in the spread of infection and tumor. Therefore, understanding this intricate anatomy can prove essential in understanding the routes of spread of disease, in cross-sectional image interpretation, and in planning surgery and radiation treatment. This knowledge can also serve as a reference point when attempting to understand and learn the surrounding anatomic spaces and neuroanatomy.
With the continued development of 3D technologies, their utilization for clinical and educational purposes is at the forefront of medical innovation and personalized medicine [21,22,23,24,25]. In this article, we have described the development of three distinct media platforms, an interactive Adobe Flash tutorial, two 3D printed models using high-resolution CT and custom graphic design, as well as a virtual reality/serious gaming environment of the PPF, that uniquely render an internal approach to visualizing the PPF and its associated communications and contained neurovascular structures. The significance of this work is that it delivers a novel perspective for understanding spatially complex anatomy combined with a framework for designing both static and dynamic 2D and 3D educational platforms to render this unique view. At this time, no objective data has been acquired regarding implementation of these techniques, such as trainee opinions or performance before and after implementation of these platforms.
Adobe Flash
Although the Adobe Flash tutorial may appear to be the simplest platform, it contains the most comprehensive information about the PPF. As the tutorial was developed as a self-directed instructional module, it sequentially illustrates more nuanced detail of the PPF along with CT correlates. Though less dynamic compared with the other methods in function, the tutorial does provide an interactive experience for visualizing the interior of the PPF and scrolling through CT cross sections containing the PPF. The primary benefit of this platform is that it is interactive on any computer due to the minimal hardware and software requirements to run the tutorial. Additionally, the Adobe Flash tutorial can be widely distributable if made Web-accessible, is cheap to use, and is immediately usable [26].
A critical disadvantage of Adobe Flash is that it is not supported on iOS, the mobile operating system of Apple hardware, and, consequently, Flash-based programs will not run on iPhones or iPads. Potential solutions to this issue include making the program compatible with HTML5 (hypertext markup language) or running it on a third-party Flash-enabled mobile applications, such as Photon Browser [27] or Puffin Browser [28]. Additionally, both of these potential alternatives have their drawbacks: creation of a HTML5-compatible program will require rewriting the Flash-based code (ActionScript) into the HTML5-based code (JavaScript), while the performance of Flash-enabled mobile applications is a variable.
3D Printing
The realistic 3D printed model demonstrated here provides a number of added benefits compared with classical teaching methods. The PPF model demonstrates not only the osseous anatomy of this otherwise poorly visualized space, including from the inside, but also shows the neuroanatomy related to the pterygopalatine ganglion and the associated vascular anatomy passing through the canals and foramina. Additionally, a physical model provides hand-held tactile feedback, which may reinforce learning [29]. Furthermore, the model developed here, scaled to 2.5 times of normal, was printed at a reasonable total cost of $200 via SLS, which yields greater structural strength versus other printing techniques [25].
A general disadvantage of 3D printed models is their limited accessibility, as creating a physical model requires access to a 3D printer. A major event that rendered 3D printers more affordable was a key 1989 intellectual property patent expiring in year 2009 a decade after its conception [30]. Although costs of 3D printing continue to down-trend [21,22,23,24], use of commercial printers may still be a financial burden to the learner. However, the STL file of the complete model, if made available online, can be viewed and virtually/digitally interacted with using free software, such as 3DSlicer [31]. As mentioned earlier, we utilized a commercial online per diem service in order to circumvent the need for owning a 3D printer with the specific desired technology, i.e., SLS. Although the cost of any given model is significantly more expensive than the material cost necessary—at times up to 10 times based on our experience—it is significantly cheaper than the cost of owning such a printer, especially if only few models are being printed annually.
A design limitation of the 3D printed model is that polyamide is available in only one color. Of the current multicolor 3D printers, one is powder-based ColorJet technology, and the other is with a rubber-like PolyJet technique. Neither can create a model that can contain thin long structures, such as that of neurovascular anatomy incorporated in this model and therefore have certain design limitations. Furthermore, ColorJet models are generally quite fragile, and PolyJet models incur higher cost [32, 33]. There is continued development of multi-material multicolor printers, such as the PolyJet Stratasys J750 printer that can print up to 7 materials, one of which is the support material [33]. Another design limitation in the model, or any 3D anatomic model for that matter, is difficulty in labeling the intricate structures. Perhaps providing a labeled virtual equivalent of the 3D model, which can be rotated and zoomed on, either as an Adobe Flash module or 3D portable document format (PDF) file, through Web-based browsers or mobile-application technology, would be a valuable adjunct tool. Certain parts, such as the osseous landmarks can be labeled on their surface by first graphically designing the letters and numbers onto the desired parts, which would subsequently be 3D printed and visible as engravings or raised markings.
Virtual Reality and Serious Gaming
Considering the abovementioned limitations, the simulated VR/serious gaming environment can provide an immersive learning experience with the ability to label/identify surrounding structures. Both types of media allow for repeated walkthroughs of the simulated anatomic environment, promoting memorization and understanding of anatomic structure and orientation. Serious gaming can uniquely provide a relatively richer user experience through gamification, which refers to the process of enhancing the gaming experience to support the user’s value creation [35]. Conventional techniques to enhance the gaming experience are through the addition of gaming elements, such as challenges, rewards, or experiences of exploration. The PPF serious game developed here could be stylistically further developed as the popular first-person shooter, potentially incorporating a handheld controller in the game as well as giving users the ability to manipulate and change the game’s 3D world and target objects. Furthermore, if the PPF serious game was multi-player accessible, then the reward points system could stimulate competition. In addition, Adobe Flash tutorials, informational PowerPoints, or scrollable CT correlates can be continuously added into the PPF serious game as educational supplements, without the need for recompiling in the coding environment.
A limitation to VR and serious gaming, though rapidly improving, is their higher technological cost due to the specialized hardware required to run these simulated environments. Both technologies are conceptually similar, comprised of an input device (a hand-held controller) that transmits signal about the user’s actions to the processing engine, the processing engine (a computer) that recalculates input data about the virtual environment on a millisecond scale to produce real time simulations at adequate frames per second, and the output device (a head-mounted device or monitor) that renders processed data from the engine as sensory stimuli to the user [36]. Of the note, the cost of an adequate processing engine (computer) ranges from $799 to $2099 [37]. In addition, the software to run these virtual environments can similarly be expensive but free alternatives exist, including Unreal Engine [38] and Unity 3D [39] without compromising functionality. The design aspect of these simulated environments cannot be avoided, as development of these environments is highly technical and requires knowledge in computer programming and graphic design. However, self-design of virtual environments is becoming more user-friendly with “drag and drop” interfaces available on software, such as ENTiTi creator [40]. This approach still requires prerequisite knowledge of 3D design, as created STL files are exported to the VR design software for placement within the virtual environment. For individuals without knowledge of 3D design, freely available STLs of anatomic models may be found online via the National Institute of Health (NIH) portal [40] or TurboSquid [41].
Relative to the partially interactive Adobe Flash tutorial, the simulated VR/serious gaming environment likely provides the most engaging learning experience to the user. However, this needs to be verified with a crossover study, in which learners are sequentially subjected to the Adobe Flash tutorial, the VR simulation, and the serious gaming simulation with spatial anatomy knowledge tested after each exposure. User satisfaction scales should also be tested after each media exposure. In addition, the physiological effect of simulated environments on the user needs to be more thoroughly vetted; although Moro et al. [42] found statistically increased reports of headaches, dizziness, and blurred vision in learners using VR in comparison to those using AR or a tablet-application, the study was of a limited sample size.
In conclusion, a multimodality approach to teaching complex anatomical concepts from within may provide an ideal approach in disseminating information to the new generation of medical trainees, many of whom have grown alongside technologically advanced tools, computers, and mobile devices their entire lives. In today’s environment of such readily available and customizable means of three-dimensionally representing structures, educators can take advantage of this opportunity to supplement the traditional forms of teaching through textbooks, 2D illustrations, and cadaver dissection. Many of these tools can be created at relatively low costs and are easily distributable. While the development, programing, and graphic design skills that are required cannot necessarily be easily learned, a team approach can be taken and the services of experts in other fields, such as computer scientists and graphic artists can be acquired. It is also true that a significant amount of time may at times be necessary in creating these tools. However, as noted in this manuscript, once a 3D object is graphically developed, it can be implemented in multiple mediums, such as Flash tutorial, 3D printing, or a VR environment. Similarly, once the environment is created, the 3D model can be replaced by other meshes, for example the middle ear or the cardiac chambers. An expansion of the work described in this article is to determine the benefit of each of the developed educational platforms for learning anatomy, i.e., identification of neurovascular structures and foramina within the PPF, and spatial ability anatomy knowledge, i.e., describing neurovascular structures and foramina relative to each other. Objective data may be acquired through pre- and post-test evaluations along with subjective data through the Likert scale questionnaires and surveys as a future direction.
Abbreviations
- 2D:
-
Two-dimensional
- 3D:
-
Three-dimensional
- CT:
-
Computed tomography
- HTML:
-
Hypertext markup language
- PPF:
-
Pterygopalatine fossa
- SLS:
-
Selective laser sintering
- STL:
-
STereoLithography
- VR:
-
Virtual reality
References
VanKoevering KK, Hollister SJ, Green GE: Advances in 3-dimensional printing in otolaryngology a review. JAMA Otolaryngol - Head Neck Surg 143(2):178–183, 2017. https://doi.org/10.1001/jamaoto.2016.3002
Lim KHA, Loo ZY, Goldie SJ, Adams JW, McMenamin PG: Use of 3D printed models in medical education: a randomized control trial comparing 3D prints versus cadaveric materials for learning external cardiac anatomy. Anat Sci Educ 9(3):213–221, 2016. https://doi.org/10.1002/ase.1573
Hoyek N, Collet C, Di Rienzo F, De Almeida M, Guillot A: Effectiveness of three-dimensional digital animation in teaching human anatomy in an authentic classroom context. Anat Sci Educ 7(6):430–437, 2014. https://doi.org/10.1002/ase.1446
Nicholson DT, Chalk C, Funnell WRJ, Daniel SJ: Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Med Educ 40(11):1081–1087, 2006. https://doi.org/10.1111/j.1365-2929.2006.02611.x
Abid B, Douard R, Hentati N, Ghorbel A, Delmas V, Chevallier J-M: L’enseignement de l’anatomie entre la craie et la pédagogie numérique. Résultats d’une étude comparative. Morphologie 91(293):107–108, 2007. https://doi.org/10.1016/j.morpho.2007.09.064
Hopkins R, Regehr G, Wilson TD: Exploring the changing learning environment of the gross anatomy lab. Acad Med 86(7):883–888, 2011. https://doi.org/10.1097/ACM.0b013e31821de30f
Saxena V, Natarajan P, O’Sullivan PS, Jain S: Effect of the use of instructional anatomy videos on student performance. Anat Sci Educ 1(4):159–165, 2008. https://doi.org/10.1002/ase.38
Agur AMR, Dalley II AF. Grant’s atlas of anatomy. 12th ed. Philadelphia: Lippincott Williams & Wilkins, 2009, pp 709–710.
Trelease RB: From chalkboard, slides, and paper to e-learning: How computing technologies have transformed anatomical sciences education. Anat Sci Educ 9(6):583–602, 2016. https://doi.org/10.1002/ase.1620
Mathiowetz V, Yu CH, Quake-Rapp C: Comparison of a gross anatomy laboratory to online anatomy software for teaching anatomy. Anat Sci Educ 9(1):52–59, 2016. https://doi.org/10.1002/ase.1528
Li Z, Li Z, Xu R et al.: Three-dimensional printing models improve understanding of spinal fracture-a randomized controlled study in China. Sci Rep 5(January):1–10, 2015. https://doi.org/10.1038/srep11570
Kong X, Nie L, Zhang H et al.: Do three-dimensional visualization and three-dimensional printing improve hepatic segment anatomy teaching? A randomized controlled study. J Surg Educ 73(2):264–269, 2016. https://doi.org/10.1016/j.jsurg.2015.10.002
Knoedler M, Feibus AH, Lange A, Maddox MM, Ledet E, Thomas R, Silberstein JL: Individualized physical 3-dimensional kidney tumor models constructed from 3-dimensional printers result in improved trainee anatomic understanding. Urology 85(6):1257–1261, 2015. https://doi.org/10.1016/j.urology.2015.02.053
Andolfi C, Plana A, Kania P, Banerjee PP, Small S: Usefulness of three-dimensional modeling in surgical planning, resident training, and patient education. J Laparoendosc Adv Surg Tech 27(5):512–515, 2017. https://doi.org/10.1089/lap.2016.0421
Fleming TM, de Beurs D, Khazaal Y et al.: Maximizing the impact of E-therapy and serious gaming: time for a paradigm shift. Front Psychiatry 7(APR):1–7, 2016. https://doi.org/10.3389/fpsyt.2016.00065
Khor WS, Baker B, Amin K, Chan A, Patel K, Wong J: Augmented and virtual reality in surgery—the digital surgical environment: applications, limitations and legal pitfalls. Ann Transl Med 4(23):454, 2016. https://doi.org/10.21037/atm.2016.12.23
Li L, Yu F, Shi D, Shi J, Tian Z, Yang J, Wang X, Jiang Q: Application of virtual reality technology in clinical medicine. Am J Transl Res 9(9):3867–3880, 2017
Tashi S, Purohit BS, Becker M, Mundada P: The pterygopalatine fossa: imaging anatomy, communications, and pathology revisited. Insights Imaging 7(4):589–599, 2016. https://doi.org/10.1007/s13244-016-0498-1
Bannon R, Parihar S, Skarparis Y, Varsou O, Cezayirli E: 3D printing the pterygopalatine fossa: a negative space model of a complex structure. Surg Radiol Anat 40(2):185–191, 2018. https://doi.org/10.1007/s00276-017-1916-x
Javan R, Zeman MN: A prototype educational model for hepatobiliary interventions: unveiling the role of graphic designers in medical 3D printing. J Digit Imaging 31(1):133–143, 2018. https://doi.org/10.1007/s10278-017-0012-4
Mitsouras D, Liacouras P, Imanzadeh A, Giannopoulos AA, Cai T, Kumamaru KK, George E, Wake N, Caterson EJ, Pomahac B, Ho VB, Grant GT, Rybicki FJ: Medical 3D printing for the radiologist. RadioGraphics 35(7):1965–1988, 2015. https://doi.org/10.1148/rg.2015140320
Kamio T, Hayashi K, Onda T, Takaki T, Shibahara T et al.: Utilizing a low-cost desktop 3D printer to develop a “one-stop 3D printing lab” for oral and maxillofacial surgery and dentistry fields. 3D Print Med 4.1:6, 2018
Lupulescu C, Sun Z: A systematic review of the clinical value and applications of three-dimensional printing in renal surgery. J Clin Med 8(7), 2019
Shilo D, Emodi O, Blanc O, Noy D, Rachmiel A: Printing the future—updates in 3D printing for surgical applications. Rambam Maimonides Med J 9:e0020, 2018
Chepelev L, Wake N, Ryan J et al.: Radiological Society of North America (RSNA) 3D printing Special Interest Group (SIG): guidelines for medical 3D printing and appropriateness for clinical scenarios. 3D Print Med 4:11, 2018
Trelease RB, Nieder GL: Transforming clinical imaging and 3D data for virtual reality learning objects: HTML5 and mobile devices implementation. Anat Sci Educ 6(4):263–270, 2013. https://doi.org/10.1002/ase.1330
AppsVerse Photon Browser - Flash Player for iPad, iPhone, iOS and Android. http://www.appsverse.com/Index. Accessed 6 May 2018
Puffin Browser. https://www.puffinbrowser.com/mobile/. Accessed 6 May 2018
Preece D, Williams SB, Lam R, Weller R: “Let’s get physical”: advantages of a physical model over 3D computer models and textbooks in learning imaging anatomy. Anat Sci Educ 6(4):216–224, 2013. https://doi.org/10.1002/ase.1345
Crump SS. Apparatus and method for creating three-dimensional objects. US Patent 5,121,329; October 30, 1989
3D Slicer. https://www.sslicer.org/. Accessed 6 May 2018
Stratasys J750 3D Printer | Stratasys. http://www.stratasys.com/3d-printers/j735-j750. Accessed 6 May 2018
Javan R, Bansal M, Tangestanipoor A: A prototype hybrid gypsum-based 3-dimensional printed training model for computed tomography-guided spinal pain management. J Comput Assist Tomogr 40(4):626–631, 2016. https://doi.org/10.1097/RCT.0000000000000415
Huotari K, Hamari J: Defining gamification. Proceeding 16th Int Acad MindTrek Conf - MindTrek ‘12. 2012:17. https://doi.org/10.1145/2393132.2393137
Bamodu O, Ye XM: Virtual reality and virtual reality system components. Adv Mater Res 765–767:1169–1172, 2013. https://doi.org/10.4028/www.scientific.net/AMR.765-767.1169
Oculus Ready PCs | Oculus. https://www.oculus.com/oculus-ready-pcs/. Accessed 6 May 2018
Unreal Engine. https://www.unrealengine.com/en-US. Accessed 6 May 2018
Unity. https://unity3d.com/. Accessed 6 May 2018
WakingApp AR Studio. https://www.wakingapp.com/#!news/cy44l. Accessed 31 Jan 2019
NIH 3D Print Exchange | A collection of biomedical 3D printable files and 3D printing resources supported by the National Institutes of Health (NIH). https://3dprint.nih.gov/. Accessed 31 January 2019
3D Models for Professionals: TurboSquid. https://www.turbosquid.com/Search/3D-Models/free. Accessed 31 Jan 2019
Moro C, Štromberga Z, Raikos A, Stirling A: The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat Sci Educ. 10(6):549–559, 2017. https://doi.org/10.1002/ase.1696
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
AHR is an employee of Surgical Theater. No conflict of interest for any of the other authors.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Javan, R., Rao, A., Jeun, B.S. et al. From CT to 3D Printed Models, Serious Gaming, and Virtual Reality: Framework for Educational 3D Visualization of Complex Anatomical Spaces From Within—the Pterygopalatine Fossa. J Digit Imaging 33, 776–791 (2020). https://doi.org/10.1007/s10278-019-00315-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-019-00315-y