1 Introduction

Orthognathic surgery is a bony surgical procedure specially designed to correct jaw deformities. Although facial soft-tissue is not directly operated on, it naturally follows the movements of underlying bony segments. Due to the complex nature of facial anatomy, orthognathic surgery requires extensive surgical planning. To date, surgeons can accurately plan bony surgeries by virtually cutting deformed jaws into pieces and repositioning them individually to a desired position. However, they are still unable to predict soft tissue changes following the bony procedure because of complex facial anatomy, physical interaction of the facial structures, and inaccurate prediction results.

Current methods of predicting three-dimensional (3D) facial changes according to the bony movements are convoluted [1,2,3,4]. Among them, finite element method (FEM) is reported to be the most common and accurate method [4]. However, prediction accuracy in the clinically critical regions, i.e., the lips, is beyond the acceptable range [3]. Lip geometry, especially the lower lip, is one of the most prominent features determining facial aesthetics. However, in patients with jaw deformity, the lower lips are often severely deformed (strained, Fig. 1). After orthognathic surgery, the shape and position of the upper and lower lips are changed individually, and the strained lower lip is automatically restored to its relaxed status. In the past, we have made significant progress on developing FEM prediction methods using a realistic mucosa sliding effect in order to improve the accuracy in clinically critical regions [1]. While adding the mucosa sliding effect had significantly improved the prediction accuracy of the facial changes including the lips [1], we treated the upper and lower lips as a whole. The prediction accuracy in the lip, especially the lower lip, thus still requires further improvement. This is especially true for patients with severe jaw deformities and asymmetry [1].

Fig. 1.
figure 1

Strained lip in (a) mandibular hypoplasia, (b) mandibular hyperplasia.

Full size image

In this study, we hypothesize that the simulation of the lip sliding is a key factor for the improvement of soft-tissue-change prediction accuracy in the lip regions. Therefore, we propose the following approach: (1) lip-detailed patient-specific FE mesh generation, and (2) incremental FEM simulation method that incorporates the realistic lip sliding effect along with the mucosa sliding effect. Our method was quantitatively evaluated using clinical datasets. The contribution of our new method is the advancement in the FE mesh modeling and the simulation method to improve facial change prediction accuracy, which can innovate the surgical planning procedure for orthognathic surgery.

2 Methods

Our new facial change prediction approach consists of two stages. In the first stage, a lip-detailed patient-specific mesh is generated to further improve our previously developed patient-specific mesh generation method. In the second stage, facial soft-tissue change is predicted using incremental FEM simulation method with the realistic sliding effect of the lip and the mucosa. Both are described below in detail.

2.1 Lip-Detailed Patient-Specific FE Mesh Generation

The initial patient-specific and anatomically-detailed hexahedral FE mesh is generated using our previously developed method, which has been proven efficient and accurate [5]. The lip region of the initial FE mesh is further detailed, and the upper and lower lips are separated by the following steps: (1) digitization of lip surface points, (2) selection of lip nodes to be modified, and (3) generation of detailed lip nodes.

Digitization of Lip Surface Points.

Digitization of lip surface points was necessary because the image quality in the lip regions is generally poor, e.g., the lip inner surface along the labial surface of the teeth is difficult to detect on CT images due to orthodontic braces and artifacts of amalgam fillings. To overcome this challenge, CT parasagittal views are generated along and perpendicular to the dental arch between the labial commissure (the two lip-end points). Surgeons then manually digitize a group of points along the cross section of the upper and lower lip surfaces, respectively, to form smooth lip contours (Fig. 2).

Fig. 2.
figure 2

Digitization of lip surface points on the parasagittal CT slices. Contours for upper and lower lip surfaces are generated based on the digitized points (red). (Color figure online)

Full size image

Selection of Lip Nodes to Be Modified.

Once the points on the upper and lower lip surfaces are digitized, the mesh nodes corresponding to the inner and outer surfaces of each lip are selected in 2-manifold. Each mesh node is then assigned with a 2D grid coordinate \( \left( {l\_u_{i} ,l\_v_{j} } \right) \) (Fig. 3a). To select the mesh nodes to be modified in the lip region (Fig. 4a), a digitized lip boundary is defined as the contour enclosing the digitized lip points (black dots in Fig. 3b and c). Among the selected lip nodes, the ones inside of the digitized lip boundary are the nodes to be modified based on the digitized lip points (Fig. 3b). The mesh nodes outside of the lip boundary will remain unchanged and designated as the reference points for the lip surface modification next.

Fig. 3.
figure 3

Lip surface mesh generation based on the digitized landmarks (illustrated by upper lip). (a) Frontal view of lip surface extracted from FE mesh. Lip outer surface (blue) and lip inner surface (red). (b) Lip surface and the digitized landmark represented in the grid coordinate system. Digitized lip boundary is shown in black dots. (c) Detailed lip nodes represented in 3D Cartesian coordinate system. (d) Final lip-detailed mesh surface (Color figure online)

Full size image
Fig. 4.
figure 4

An example of lip-detailed mesh. (a) Initial mesh without lip-detailed geometry. (b) Lip-detailed mesh with lip opening.

Full size image

Generation of Detailed Lip Nodes.

The nodes inside of the digitized lip boundary is modified based on the digitized points. However, the digitized points cannot directly be used for the mesh because the numbers and locations of the points are not correlated to the exact numbers and locations of the existing mesh grids. Generating mesh nodes directly from the digitized lip points can cause invalid or severely distorted mesh elements. Therefore, lip-detailed mesh needs to be generated while preserving original structure of the initial FE mesh. This is achieved by following procedure. First, 2D grid coordinates of the digitized lip points \( \left( {lp\_u,lp\_v} \right) \) are interpolated using 3D Cartesian and 2D grid coordinates of the neighboring mesh nodes. As explained above, the grid coordinates of the digitized lip points \( \left( {lp\_u,lp\_v} \right) \) are not aligned with the existing mesh grid pattern (Fig. 3b). Second, given the coordinates (3D Cartesian and the 2D grid) of the digitized lip points and the reference points, the detailed lip nodes are acquired by approximating 3D Cartesian coordinates of the lip nodes on the existing mesh grid pattern using thin plate splines (TPS). The lip nodes inside of the digitized lip boundary are then modified by replacing them with the acquired detailed lip nodes (Fig. 3c and d and, green and orange in Fig. 4). Finally, to complete the detailed lip mesh generation, the mesh nodes inside of the modified lip region (Fig. 4b) are also modified according to the lip surface modification using TPS. The modification is completed for the upper and lower lips, respectively. The area of the detailed-lip improvement is limited to the lip region without harming the geometrical accuracy of the initial FE mesh.

2.2 Incremental Simulation of Facial Changes with the Realistic Sliding Effect

The facial soft-tissue change is predicted incrementally using realistic sliding effect of the lips and the mucosa. First, material property and boundary condition are defined for the lip-detailed patient-specific FE mesh. Second, facial soft-tissue change simulation is performed using FEM according to the surgical plan.

Material Property and Boundary Condition.

Linear non-homogeneous material property is assigned to the lip-detailed patient-specific hexahedral mesh, a total of approximately 40,000 elements and 50,000 nodes. The mesh elements inside of the lip regions are selected as lip muscle elements to represent the Orbicularis Oris muscle. They are assigned with transversely isotropic material property to maintain the shape of the lip during the simulation. (Young’s modulus for longitudinal (vertical) direction: 9,000 Pa, Young’s modulus for transverse direction: 3,000 Pa, Poisson’s ratio for both direction: 0.47) to simulate the stiffness of the lip muscle. Nearly incompressible isotropic linear properties (Young’s modulus: 3,000 Pa, Poisson’s ratio: 0.47) are applied to the rest of the FE mesh elements. Three different types of boundary nodes are defined on the mesh innermost surface; fixed, moving, and sliding nodes (Fig. 5). Fixed nodes (in red) are defined in the region which is not altered during the surgery. Moving nodes (in green) are nodes that are assumed to be attached to the bone and move together with the bone. They are located on the superior part of the maxillary segment and the inferior part of the mandibular segment to simulate the bony movements during the surgery. Sliding nodes (in pink) are defined around the regions which slide along the bone surface, i.e., the lips and the intraoral mucosa. Unlike the other boundary nodes that are defined on the mesh innermost surface, the lip sliding nodes are defined on the upper and lower lip surfaces. Assignment of boundary nodes is patient-specific and manual. The rest of the nodes are free nodes (in green-blue), and their movement is unconstrained. We assumed the postoperative face is fully relaxed. Therefore, the movement of the free nodes is determined by FEM.

Fig. 5.
figure 5

The boundary condition (represented on bony surface and FE mesh for illustration purpose. Red: fixed nodes; Green: moving nodes; Pink: sliding nodes; and Green-blue: free nodes. (Color figure online)

Full size image

Incremental Facial Change Simulation.

The incremental facial change simulation is performed with the mucosa and lip sliding effects using FEM [1]. The incremental simulation is implemented to facilitate a natural transition of the facial change and to enhance the effectiveness of the lip and mucosa sliding movements. It is achieved by evenly dividing the entire bony movement (the surgical plan) into 3 increments, in which each bony segment is moved from its preoperative position to the postoperative one in 3 increments. The acquired incremental bony movement is applied as a boundary condition for the moving nodes. The results of the previous incremental simulation serve as the input for the next incremental simulation, till all 3 are completed. For each incremental simulation, the stiffness matrix and the lip sliding nodes are updated accordingly.

Each incremental simulation is implemented in 2 steps. In the first step, facial change according to the incremental bony movements is simulated with nodal force constraint on the mucosa sliding nodes. The mucosa sliding effect is applied by forcing the mucosa sliding nodes to move along the bony surface by considering only the tangential components of the nodal force of the mucosa sliding nodes [1, 2]. In addition, the lip sliding effect is applied along with the mucosa sliding effect, by allowing the upper and lower lip sliding nodes to move freely according to the bony movements without penetrating each other. This is implemented by a master-slave approach [6] which constrains degrees-of-freedom (DOF) of the movement of the lip sliding nodes. The collisions between the upper and lower lip sliding nodes are detected by point-wise collision detection algorithm (threshold: 0.2 mm). This is to ensure the upper and lower lips does not collide to each other during the vertical movement, while the movements in other directions are not constrained. As a result, the upper and lower lips slide against each other and move independently without penetration. No friction is assumed. However, the result of the first step may include a geometrical mismatch between the bone surface and the mucosa sliding nodes, which is unrealistic [1].

In the second step, this geometrical mismatch is resolved by applying the nodal displacement boundary condition, which is acquired by finding the closest point from each mucosa sliding node to the postoperative bony surface, for the mucosa sliding nodes [1]. No bony movements are applied in the second step. The lip sliding effect is also applied as described above. As a result, the mucosa sliding nodes are ensured to match with the bony surface. In this study, FEM simulation is implemented with our in-house code using Matlab.

3 Experiment and Results

We evaluated our lip-detailed facial change prediction method on five randomly selected patients who are suffered from jaw deformity, underwent an orthognathic surgery, and had complete sets of pre- and postoperative CT data (3 females and 2 males with 21 ± 2.5 years old) from our digital archive [IRB0413-0045]. The facial change prediction was performed using three different simulation methods. Method #1 was the traditional FEM simulation approach without sliding effect [1]. All FE mesh nodes contacting the bony segments were assumed to be fixed to the bony segments. Method #2 was our previously developed three-stage FEM simulation with the mucosa sliding effect using nodal spatial constraint [1]. The sliding effect was applied only to the intraoral mucosa, in which the upper and lower lips were treated as a whole. In both methods, high-quality patient FE meshes were generated using our previous method described in [5]. Method #3 is our incremental simulation with the new lip-detailed FE mesh, and the realistic lip and the mucosa sliding effects developed in this study.

Both pre- and postoperative CT scans were segmented, and 3D models were reconstructed. Postoperative CT models were registered to corresponding preoperative ones based on surgically unaltered regions, i.e., cranium, and served as a “blueprint”. Virtual osteotomies were performed on the preoperative model, and the movement vectors of each bony segment, from pre- to postoperative positions, were computed based on the blueprint. Each acquired movement vector was applied to the corresponding moving nodes as a boundary condition to predict the facial soft-tissue changes.

Since the accuracy improvement was mainly in the lip region, only the prediction error of the upper and lower lips was evaluated in this experiment. The upper and lower lips were defined by the anatomical landmarks: subnasale, cheilion, and labiomental fold. The surface deviation error was measured using an average of absolute Euclidean distances between the predicted and the postoperative outcome. Friedman test with Wilcoxon Signed-Rank test was used to detect whether there was a statistically significant difference in accuracy among the three methods.

The results showed that the prediction accuracy in the lip region was improved with our method. Specifically, the results of Wilcoxon Signed-Rank test showed that using our method, the prediction accuracy of the lower lip was statistically significantly improved (P < 0.05). (Table 1 and Fig. 6). Our method achieved a 46.0% of accuracy improvement over Method #2 (Table 1 and Fig. 6). In addition, the lower lip prediction accuracy achieved sub-millimeter accuracy (0.85 mm), while the other two methods showed error significantly greater than 1 mm (Method #2: 1.53 mm; Method #2: 1.57 mm) according to Wilcoxon Signed-Rank test (p < 0.05). Finally, although it was not statistically significantly different, the prediction accuracy for the upper lip also showed a trend of 14% of improvement over Method #2 (Table 1 and Fig. 6).

Table 1. Prediction accuracy evaluation of three different simulation methods using surface deviation error. * Significantly greater than 1 mm (P < 0.05), ** Significant improvement compared with Method #1 and #2.
Full size table
Fig. 6.
figure 6

Prediction results using three different methods: Traditional method without sliding effect (Method #1); Mucosa sliding effect only (Method #2); Our approach (Method #3) with lip and mucosa sliding effect. Color-coded surface deviation errors between the postoperative face and the prediction results using our approach are in the rightmost column. (Color figure online)

Full size image

4 Discussion and Conclusion

We have significantly improved our previous FEM simulation method with the addition of realistic lip sliding effect to predict the facial soft-tissue change following orthognathic surgery. The evaluation results also confirmed our hypothesis that the application of lip sliding effect can improve the prediction accuracy in the lip region.

The realistic lip sliding effect plays a major role in the facial change prediction in addition to the mucosa sliding effect. The inaccurate prediction are mainly occurred in the lip region, especially in patients with severe jaw deformity, which the low lips are severely deformed [3]. While the mucosa sliding effect improves the prediction accuracy [1, 2], the upper and lower lip movements are still treated as a whole and not simulated independently. In our proposed method, the FE mesh of the upper and lower lips are detailed and separated, allowing to simulate the upper and the lower lip changes individually. Finally, the sliding effect was effectively enhanced by applying the incremental bony movements, facilitating a smooth and natural soft-tissue change.

In the future, our method will be investigated with an extended number of patients for more intensive validation. The prediction accuracy also needs to be evaluated with qualitative evaluation method that reflects clinician’s visual evaluation because current quantitative evaluation, i.e., surface deviation error, does not fully reflect the clinician’s qualitative assessment [1]. In addition, the manual generation of the lip-detailed mesh can prevent clinicians from adopting it to daily clinical use, and this needs to be addressed in the future.