Introduction

For over a century, medical imaging techniques have been used to analyze the human face. 2D imaging diagnostics, such as photography and radiographic imaging, were initially integrated in the fields of orthodontic therapy and maxillofacial surgery [1], and are now routine imaging technologies in dental medicine. In addition, video motion is becoming established as a tool for analyzing movement and capturing patients’ emotions. Recent technological developments have led to a broader implementation of a variety of digital applications in dental medicine. Consequently, clinical practices and laboratory techniques are shifting from manual-based to virtual-related workflows [2, 3]. This trend in digitization has led to the implementation of computer-generated data to additional dental disciplines, such as prosthodontics and implant dentistry [4].

Besides influencing clinical diagnostics, treatment processes, and the technological design and production chains, digitization has also influenced patient management with their individual requests for information and knowledge flow using the World Wide Web and social media. In addition, modern dental education requires novel teaching concepts to supplement classical seminar-style curricula with digitally mediated theory transfer that is accessible 24/7 [5, 6].

Overall, the digital revolution is enabling individualized patient treatment concepts in dental medicine. The objective of this literature review is to summarize the current status and latest progress in the field of virtual patient science, and to identify further research that is needed before these techniques can be translated into routine clinical practice.

Data Acquisition and Technical Application

To create a realistic virtual dental patient, a simulation has to replicate the superficial surface of the skin combined with the underlying bony structures of the skull and teeth and the involved oral soft tissue layers [7, 8, 9••]. Different technologies and devices are necessary to generate virtual patient simulations for diverse diagnostic indications. The major developments in digitizing a patient are 3D cone-beam computed tomography (CBCT), optical scanning systems for intraoral (IOS) and extraoral (EOS) imaging, and computer-assisted-design/computer-assisted-manufacturing (CAD/CAM) of dental prostheses [10]. The 3D digital files obtained from CBCT, IOS, and EOS differ in their formal data structure:

  • CBCT uses DICOM (Digital Imaging and Communications in Medicine), a general standard format for handling, storing, printing, and transmitting information in medicine (ISO 12052:2006);

  • IOS uses STL (Surface Tessellation Language), which describes triangulated surface geometries of 3D objects without color information;

  • EOS captures facial extraoral skin and data are usually stored as OBJ files (developed by Wavefront Technologies), a widely accepted geometry definition format for representing 3D texture and color information.

CBCT can provide accurate 3D volumetric views of craniofacial hard tissues; however, imaging of facial soft tissue contours is limited due to lower contrast resolution with high scatter radiation [11]. Consequently, high-resolution surface details of the patient’s face, such as color and texture, cannot be captured using CBCT [12, 13]. Instead, laser and optical devices must be used to digitize a realistic image of the patient’s face [14, 15]. Furthermore, metallic restorations in dental patients can cause scatter artifacts which compromise CBCT image quality [16]. In this situation, additional IOS can be used to produce scatter-free intraoral imaging of the dental situation plus surrounding soft tissue [17, 18].

Data Management and Processing

The different technologies for hard and soft tissue imaging, such as CBCT, facial scanner, and digital impression devices for IOS, typically each generate a single data file output. Combining the individual data outputs gathered from each technique to create a composite 3D virtual patient is a logical continuum of the digitization trend in dental medicine [19••]. Superimposing the diverse tissue structures to generate a triad of the facial skeleton, the extraoral soft tissue, and the dentition including the surrounding intraoral soft tissue architecture is a complex process.

The selection which individual techniques and data formats to combine affects the final virtual patient model generated, and therefore, also the clinical value of the model. In general, the dental field of interest determines the data file composition. The individual data sources can be merged in multiple combinations of two or more files: i.e., DICOM + STL. 3D data can then be matched using point-related, surface-related, and voxel-related fusion methods. Point-related matching is based on identifying and aligning corresponding landmarks, while the other two matching methods use congruent surfaces or voxels of manually selected regions [20,21,22,23]. The more data files in total that are involved, and their specific file format, the higher the potential risk of an incorrect composition. At present, these image fusion and modeling techniques are mainly limited to the research setting. A manually based fusion is currently difficult to integrate in routine practice due to the multifaceted workflow involved, technical complexity, and questionable reproducibility. More rapid and more user-friendly techniques are needed before fusion models can be routinely implemented in daily clinical practice [19••].

Despite an increase in studies investigating virtual technologies over recent times [24], high-quality randomized controlled clinical trials investigating the accuracy, precision, and reproducibility of these models are scarce. Gkantidis et al. (2015) investigated the applicability, accuracy, precision, and reproducibility of different superimposition techniques for 3D radiographic data [25]. Five superimposition techniques were tested using eight pairs of pre- and post-treatment computed tomography (CT) radiographs. Three examiners independently matched the paired datasets and the entire process was repeated 4 weeks later. The superimposition techniques analyzed were (1) three-point triangulation, (2) anterior cranial base, (3) anterior cranial base and foramen magnum, (4) bilateral zygomatic arches, and (5) single zygomatic arch registration. Superimposition of 3D datasets using surface models created from voxel-based fusion facilitated accurate, precise, and reproducible outcomes. Superimposition using anterior cranial base registration or both zygomatic arches achieved the best accuracy and precision. These results, however, were limited to CBCT-generated DICOM, which was the only data source used [25].

Fluegge et al. (2017) investigated the influence of the operator and image artifacts on the accuracy of superimposition of DICOM + STL under clinical conditions [26]. The mean deviation between CBCT models and EOS scan was 0.54 mm. The segmentation procedure (default versus manual), the operator, and the number of restorations (image artifacts on CBCT) had a significant influence on the registration accuracy. The operator’s manual segmentation of 3D models was shown to be significantly better than the automatic segmentation procedure, especially, in patients with multiple restorations. Therefore, the authors call for specialized import algorithms for CBCT data with an automated presetting, which might help to minimize the influence of the identified factors and improve reliability of the workflow [26].

In a proof-of-principle single case report, three data sources—a full-arch IOS merged with CBCT and stereophotogrammetric facial imaging (EOS)—were used to build a 3D virtual patient. All media were superimposed to a unique data pool by means of a surface-based registration method. This technique successfully demonstrated the feasibility of building a craniofacial virtual reality model by triple image fusion of DICOM, OBJ, and STL files [27••].

Clinical Translation and Relevance

Digital medicine, and in particular the creation of a 3D virtual patient, is of great interest in dentistry and associated medical disciplines for use in diagnostics, advance treatment simulation, and during post-treatment follow-up [28]. A superimposition model displaying all three structures comprising the bony skull, extra- and intraoral soft tissues, and dentition would be particularly valuable in interdisciplinary complex cases of orthognathic surgery because the therapy will influence the skeletal situation, the occlusion, the facial appearance, and function [29,30,31]. A triple approach limited to the anatomical regions of the mandible and maxilla, including the maxillary sinuses, could provide sufficient information for virtual treatment planning in complex patients, including those requiring dental implants. Simulating only a limited field of interest would reduce the amount of radiation the patient is exposed to [32, 33]. A 3D model would enable anatomical structures to be analyzed and prosthetic outcomes simulated in advance. For example, pre-treatment evaluation could evaluate whether adequate lip support could be achieved with or without removable prostheses in demanding esthetic-functional rehabilitation protocols of edentulous or extended partly dentate situations. Algorithms for the virtual 3D simulation of post-treatment soft tissue changes have to be developed for this application [34, 35].

For most conventional prosthodontic treatment concepts, a two-file procedure of the extraoral lip-framing plus intraoral dental situation will be sufficient to simulate the intended prosthetic treatment outcomes. The advantages of this technique include fast image acquisition and easy alignment of a 3D model. Moreover, the procedure could be repeated at any time and documented for follow-up records. The patient would also benefit as the technique is completely non-invasive and does not use ionizing radiation.

A representative example of superimposition and simulation techniques is the prosthetically oriented digital smile design. Medical history, clinical and radiographic standard examinations, study model analysis, and 2D photographs build the groundwork for diagnosis and the treatment plan in reconstructive dentistry. However, they do not offer all the information necessary to analyze the smile and create harmony with the lips and face without excessive intraoral adjustments in esthetically challenging cases. Further developments facilitate the elaboration of a 2D smile frame that will guide a 3D digital smile design for the rehabilitation of complex prosthodontic treatment. The use of dynamic documentation of the smile allows an esthetic rehabilitative plan considering a facial perspective, improvement of communication with the patient due to in advance visualization, interdisciplinary integration between the specialists, and the predictable quality of the treatment outcome. Today, various commercial systems and software solutions are available on the market [36].

Many different technological approaches have been studied in virtual dentistry; however, most publications are technical case reports that have focused on the feasibility of different devices, applications, and software solutions [27••, 37,38,39,40]. Furthermore, trial reports are limited to data on 3D virtual simulations under static conditions. No study has yet described dynamic models that would allow visualization of functional actions of the jaws, lips, and muscles to build a complete 4D replication of a human head, integrating skeleton, extra- and intraoral soft tissues, and dentition. While it is feasible to extract a single frame of 3D data from a captured 4D video sequence and export this for superimposition with CBCT data, no currently available commercial system can fuse a 4D sequence of facial movements onto DICOM, OBJ, STL, or any other 3D medical file format. This will be crucial to develop a real-time animated virtual dental patient capable of realizing 4D movements.

Further investigations are required to validate the accuracy of the different technologies before virtual patient simulations can be implemented into routine dentistry. There is also a need to develop standardized and reproducible evaluation scales with which to compare the different superimposition techniques and clinically validate the virtual patient simulations. For routine use, it will be important that systems are capable of easily incorporating new additional patient images to existing data pools, both to avoid the need to repeat imaging when updating a patient simulation and also to allow for follow-up documentation. To facilitate this, an easily transferable and uniform file format has to be developed and established for medical 3D digital media. The workflows for superimposition techniques also have to be simplified before the technology can gain widespread adoption. In this context, a single-step approach with one device would be a major improvement for the generation of all 3D media at the same time; a single device would also likely increase the accuracy of the simulations.

Implication on Dental Curricula

In addition to the impact on clinical practice, the trend in digitization is also changing undergraduate and postgraduate education [41]. Dental teaching has evolved over recent years but to date this has been mainly limited to the introduction of technological tools to train and improve fine motor skills and hand-eye coordination in pre-clinical settings. Virtual reality simulations can help the transition from theoretical learning to a real patient situation and are becoming an essential part of modern education [42]. The use of virtual reality simulation is most effective when the goal of education is to transform and apply knowledge to real-world problems [43]. Virtual-based dental training comprises high-end dental simulation and skill improvement. Up to date, haptic training systems for dental education committed to helping students’ progress faster and evaluating in an objective way for an efficient increase of the students’ training. Future dental education should combine these existing haptic skill trainings with virtual reality technology, such as patient replication for reality simulation with 4D hologram technology.

Summary and Conclusions

The superimposition of 3D media types of the skull, the facial soft tissue, and the dentition to create a virtual dental patient appears to be a promising novel tool that will be useful in various clinical scenarios and indications: (1) pre-operative clinical assessment; (2) simulation of treatment planning; (3) post-operative follow-up documentation; (4) to facilitate more effective interdisciplinary communications and patient communications, and for discovering patients’ expectation in general; (4) as a non-invasive imaging technique for high-precision anatomical documentation; and (5) as a training tool in dental education. The technology will have wide-ranging applicability in the fields of prosthodontics and oral rehabilitation, implant dentistry, orthodontics, dentofacial orthopedics, maxillofacial, and plastic surgery, as well as interdisciplinary treatment protocols [19••].

In the end, how far along are we in virtual dentistry? The simulation of a 3D virtual dental patient requires the digital fusion of different tissue structures. Superimposition of different 3D media data sets is a feasible technique to create a virtual dental patient under static conditions; however, the dataset superimposition techniques needed to do this are still experimental and the optimum techniques have not yet been established. The selection of a particular technique affects the outcome of the virtually generated patient model, and therefore, also the field of interest and its clinical translation. None of the currently available craniofacial imaging techniques can capture the complete triad of the skull, the facial skin, and the intraoral dental situation with optimal quality in a single-step procedure.

The current status of the field is promising and many research groups and medical companies are collaborating on virtual patient simulation. The technology is improving continuously in terms of user-friendliness and safety. Nevertheless, high-quality randomized controlled clinical trials are still needed. For implementation into routine clinical practice, simplified and reliable protocols that can generate virtual patient simulations in a single-step approach are needed. Future research should also focus on 4D technology with simulated dynamic movements.