1 Introduction

Facial appearance impacts human’s social life. Orthognathic surgery is a surgical procedure of treating patients with dentofacial deformity to improve jaw functions and facial aesthetics. It is a bone procedure in which the jaws are cut into pieces and then repositioned to a desired position (called osteotomy), resulting in a significant facial appearance change. To date, only the osteotomy can be accurately planned prior to surgery [1]. The facial soft-tissue-change following the osteotomies cannot be accurately predicted even though it is a direct result of an osteotomy. The major challenge of accurately predicting facial soft-tissue-change is due to the complex nature of the facial soft tissue anatomy. Traditionally, the soft-tissue-change simulation is based on bone-to-soft tissue movement ratios, which has been clinically proven inaccurate [2]. There are a few published reports on three-dimensional (3D) facial soft tissue prediction. The most common methods are finite element method (FEM) [3, 4], mass-spring model [5] and mass tensor model [6–8]. FEM is reported to be the most accurate and biomechanically relevant method [5, 7]. Nonetheless, the prediction results are still less than ideal. This is especially true in the nose, lips and chin regions, which are extremely important for orthognathic surgery. Therefore, there is an urgent clinical need to have a reliable method of accurately predicting soft tissue changes following osteotomies.

Traditional FEM for facial soft tissue simulation assumes that the FEM mesh nodes move together with the contacting bone surfaces without considering sliding movement [3]. The nodes contacting the corresponding bone surfaces are first acquired and then translated the same amount as the bone movement. However, this assumption can lead to significant errors when a large amount of bone movement and occlusion changes are involved. In human anatomy, the cheek and lip mucosa are not directly attached to the bone and teeth; they slide over each other. However, the traditional FEM does not consider this sliding movement, which we believe is the main reason for inaccurate prediction in the lips and chin.

Implementing the realistic sliding movement into FEM is technically challenging. It requires high computational power and long time because the sliding mechanism in human mouth is a dynamic interaction between two surfaces. The second challenge is that even if the sliding movement with force constraint is implemented, the simulation results may still be inaccurate, because there is no strict nodal displacement boundary condition applied to the sliding area. The soft tissues at sliding surfaces follow the buccal surface profile of the bones and teeth. Thus, it is necessary to consider the strict displacement boundary condition for sliding movement. The third challenge is that the mapping between the bone surface and FEM mesh nodes needs to be reestablished after the bony segments are moved to a desired planned position. This is because the bone and soft tissue relationship is not constant before and after the bone movement, e.g., a setback or advancement surgery may either decrease or increase the total soft tissue contacting area to the bones and teeth. This mismatch may lead to the distortion of the resulting mesh. The fourth challenge is that the occlusal changes, e.g., from preoperative (preop) cross-bite to postoperative (postop) Class I normal bite, may distort the mesh in the lip region where the upper and lower teeth meet. Therefore, a more advanced sliding effect method is required to increase the prediction accuracy in these critical regions.

In this study, we have successfully solved these technical problems. We developed a FEM simulation method of realistic sliding effects. The facial soft tissue changes following the bony movements were simulated with an extended sliding boundary condition to overcome the mesh distortion problem in traditional FEM simulations. The nodal force constraint was applied to simulate the sliding effect of the mucosa. Next, strict nodal displacement boundary conditions were implemented in the sliding areas to accurately reflect the postop bone surface geometry. The corresponding nodal displacement for each node was recalculated after reassigning the mapping between the mesh and bone surface in order to achieve a realistic sliding movement. Finally, our simulation method was evaluated quantitatively using 30 sets of preop and postop computed tomography (CT) datasets from the patients with dentofacial deformity.

2 Our FEM Simulation Algorithm with Realistic Sliding Effects

Our facial soft-tissue-change simulation incorporated with realistic sliding effects, which was applied by sequentially satisfying a nodal force and displacement boundary conditions. In this algorithm, a patient-specific FEM model was generated using a previously developed FEM template model [9]. Subsequently, post-operative bone movement according to the surgical planning was applied to the FEM model together with a sliding effect with nodal force constraint to simulate the facial change. Next, sliding effect with strict nodal displacement condition was implemented to efficiently mimic the realistic sliding of the soft tissue. More sophisticated boundary condition and mapping method were developed to implement a realistic sliding movement for prediction accuracy improvement in critical areas.

2.1 Tissue Property for FEM Model

In our study, homogeneous material FEM model was used for computational efficiency. This was based on the results of previous studies investigating optimal tissue properties for facial soft tissue simulation. They found that the effect of tissue property on facial tissue deformation was minimal even if the tissue property changed tremendously [10, 11]. The average prediction error varied less than 0.02 mm when Poisson’s ratio varied within 0 and 0.5 [11]. Moreover, the selection of value for Young’s modulus is irrelevant to the FEM model deformation for homogeneous material model under displacement boundary condition. Therefore, we utilized 3000 (Pa) for Young’s modulus and 0.47 for Poisson’s ratio for tissue properties.

2.2 Boundary Condition Assignment for Sliding Effect

Nodes of the FEM mesh were classified into the boundary nodes and free nodes (Fig. 1). The nodal displacement of the free nodes (GreenBlue in Fig. 1b and c) was determined by the displacements of the boundary nodes using FEM. The boundary nodes were further classified into fixed, moving and sliding nodes. The fixed nodes did not move during the surgery and FEM simulation (red in Fig. 1), thus, having zero nodal displacement. The lower posterior regions of the soft tissue mesh were assigned as free nodes for the sliding effect simulation with nodal force constraint. This boundary condition, together with the sliding effect of the partial ramus, ensured the soft tissue flexibility and smoothness in the posterior and inferior mandibular regions when an excessive mandibular advancement or setback occurred.

Fig. 1.
figure 1

Boundary condition. (a) Mesh inner surface (illustrated on bones) for the sliding effect with nodal force constraint. (b) Mesh volume and fixed boundary condition for the sliding effect with nodal force constraint. (c) Mesh inner surface (illustrated on bones) for the realistic sliding effect with nodal displacement constraint. (d) Mesh volume and fixed boundary condition for the realistic sliding effect with nodal displacement constraint. Fixed nodes: red; Moving nodes: Blue; Sliding nodes: pink; Free nodes: GreenBlue; Scar tissue: green. (Color figure online)

Full size image

The nodes on the mesh inner surface contacting movable bony segments were designated as the moving nodes. The moving nodes were assumed to move along with the bony segments during the simulation (blue in Fig. 1a). The corresponding moving nodes on the mesh were determined by finding the closest nodes from the vertices of the .STL bony segments using a closest point search algorithm. The movement vector of each bone segment according to the surgical planning was then applied to the moving nodes as a nodal displacement boundary condition. Additionally, the nodes corresponding to the area where two bony segments (proximal and distal) collided each other after bone repositioning (actually removed in real surgery) were excluded from the moving nodes and reassigned as the free nodes. Together with the sliding effect of the partial ramus, this reassignment further solved the mesh distortion problem at the mandibular inferior border when an excessive mandibular setback movement involved. Furthermore, the soft tissue geometry change from a scar formation was considered in the simulation. The regions degloved intraoperatively were selected as a moving boundary (green in Fig. 1a) and the corresponding nodes were shifted in anterior direction by 2 mm by the nodal displacement boundary condition.

The sliding nodes (pink in Fig. 1a) were selected on the mesh inner surface of the mouth (mucosa), including the cheek and lips, to simulate the sliding movement of the soft tissue. The sliding boundary condition in mucosa area was adopted from [7, 8, 12]. In our study, the definition of the sliding nodes was further extended to partial inferior ramus, preventing mesh deformation problem in posterior of the mandible.

2.3 Implementation of Sliding Effect with Nodal Force Constraint Using Iterative FEM Solving Algorithm

First, the sliding movement of mucosa was implemented by applying nodal force constraint on the sliding nodes. An iterative FEM solving algorithm was developed to solve the FEM with nodal force boundary condition. The general form of global FEM equation is:

$$ K\delta = f $$
(1)

where K is a global stiffness matrix, \( \delta \) is a global nodal displacement vector, and \( f \) is a global nodal force vector. The above equation can be rewritten as:

$$ \left( {\begin{array}{*{20}c} {K_{11} K_{12} } \\ {K_{12}^{T} K_{22} } \\ \end{array} } \right)\left( {\begin{array}{*{20}c} {\delta_{1} } \\ {\delta_{2} } \\ \end{array} } \right) = \left( {\begin{array}{*{20}c} {f_{1} } \\ {f_{2} } \\ \end{array} } \right) $$
(2)

where \( \delta_{1 } \) is the displacement of the moving and fixed nodes, \( \delta_{2 } \) is the displacement of the free and sliding nodes to be determined by FEM, \( f_{1} \) is the nodal force of the moving and fixed nodes, and \( f_{2} \) is the nodal force of both free and sliding nodes. Stiffness matrix (K) was reorganized accordingly. The nodal force of the free nodes was assumed to be zero, and only tangential nodal force along the contacting bone surface was considered to determine the movement of the sliding node [7, 8, 12].

The final value of \( \delta_{2} \) was calculated by iteratively updating \( \delta_{2} \) using Eq. (3) until the converging condition was satisfied [described after Eq. (6)].

$$ \updelta_{2}^{(i + 1)} =\updelta_{2}^{(i)} +\updelta_{{2_{ update} }}^{(i)} ,(i = 1,2, \ldots ,{\text{n}}) $$
(3)

The details of \( \updelta_{{2_{update} }} \) calculation was as follows. Equation (4) was derived from Eq. (2) to acquire current value of \( f_{2} \). \( f_{2} \) was composed of nodal force of the sliding nodes (\( f_{2\;sliding} \)) and the free nodes (\( f_{2\;free} \)). At the first iteration (i = 1), the initial \( \delta_{2 } \) was randomly assigned and substituted for \( \updelta_{2 } \) to solve Eq. (4) and \( f_{2} \) was calculated by substituting current \( \delta_{2 } \) into Eq. (4).

$$ f_{2} = K_{12}^{T}\updelta_{1 } + K_{22}\updelta_{2 } $$
(4)

Then, \( f_{2} \) was further processed by transforming only the nodal force corresponded to the sliding nodes (\( f_{2\;sliding} \)) among \( f_{2} \) using Eq. (5).

$$ f_{2\;sliding}^{t} = f_{2\;sliding} - \left( {f_{2\;sliding} \cdot N} \right)N $$
(5)

where \( f_{2\;sliding}^{t} \) is tangential component of nodal force of the sliding nodes, and N is a normal vector of the bone surface corresponding to the sliding nodes. Now, \( f_{2 }^{t} \) composed of the nodal force of the free nodes (\( f_{2\;free } \)) and a tangential component of the nodal force of the sliding nodes (\( f_{2\;sliding}^{t} \)).

Finally, the nodal displacement to update the current \( \updelta_{2 } \), \( \left( {\updelta_{{2_{update} }} } \right) \), was calculated from \( f_{{2_{update} }} \). \( f_{{2_{update} }} \) was the required nodal force to make \( f_{2\;free } \) zero and calculated by the difference between \( f_{2}^{t} \) and \( f_{2} \left( {f_{2} - f_{2 }^{t} } \right) \). \( \updelta_{{2_{update} }} \) was acquired using Eq. (6):

$$ \updelta_{{2_{update} }} = - K_{22}^{ - 1} (f_{{2_{update} }} + K_{12}^{T}\updelta_{1} ) $$
(6)

Then, \( \updelta_{2 } \) was updated using Eq. (3). The iteration continued until the maximal absolute value of \( f_{{2_{update} }} \) converged below 0.01 N (i = n). The final values of \( \delta \) (\( \delta_{1 } \) and \( \delta_{2 } \)) represented the displacement of all mesh nodes after the bone repositioning and the sliding effect application with nodal force constraint. The resulted \( \delta \) was designated as \( \delta_{intermediate} \).

2.4 Implementation of Advanced Sliding Effect with Nodal Displacement Constraint

A strict nodal displacement boundary condition was further applied to the result of the iterative FEM solution. Nodal force constraint had limited control on the nodal displacement. Thus, the result of the iterative FEM solution might have geometrical discrepancy between the mesh inner surface and the bone surface (Fig. 2). In real clinical situation, the mucosa should exactly match to the geometry of the teeth and buccal surface of the bone. Therefore, it was necessary to apply strict nodal displacement condition to enhance the iterative (intermediate) result of the nodal force constraint. It was also necessary to redefine the boundary condition mapping between the bone surface and mesh nodes in the sliding area. This was because the relationship between the bone surface vertices and the mesh nodes was changed after the bony segment was repositioned. Moreover, clinically the postop lower teeth were always located inside of the upper teeth (as a normal bite) despite of the preop occlusal relationship. Therefore, more advanced sliding effect with redefinition of boundary condition and strict nodal boundary condition was required to improve the accuracy of the result of sliding effect with nodal force constraint (intermediate simulation).

Fig. 2.
figure 2

Assign nodal displacement. The dot line depicts an artifact gap between mucosa and teeth after the sliding effect with nodal force constraint is applied.

Full size image

The sliding effect with nodal displacement constraint was implemented as follows. First, classification of boundary nodes was redefined. The free nodes at the inferoposterior surface of the soft tissue mesh were assigned as fixed nodes unlike the previous assignment. Then, the nodes on the mesh inner surface corresponding to the maxilla and mandible were assigned as the moving nodes (blue in Fig. 1c). The rest of the nodes are assigned as the free nodes (GreenBlue in Fig. 1c and d). Unlike the previous boundary assignment, there were no sliding nodes.

The nodal displacement for the moving nodes was assigned from superior to inferior direction by finding the closest bone surface vertex from the each of the moving node (Fig. 2), instead of finding correspondence from the bone to the mesh. Once the mapping between the vertex and nodes was computed, the vector between each node and its corresponding closest vertex on the bone surface was assigned as the nodal displacement of the moving nodes. This sequential assignment of nodal displacement prevented the two different nodes from having the same nodal displacement. The reestablishment of boundary condition mapping between the nodes and vertex solved the aforementioned postop mismatch problem between the bone surface and corresponding inner mesh surface due to the bone repositioning and occlusal relationship change.

In Eq. (1), the global stiffness matrix (K), the nodal displacement (δ) and the nodal force (f) were reorganized according to the new boundary conditions. The result was acquired by directly solving redefined Eq. (2) without iterative method. Here, the nodal force of the free nodes, \( f_{2} \), was assumed to be zero. The nodal displacement of the free nodes, \( \updelta_{2 } \), was calculated by reorganizing Eq. (2) with this assumption:

$$ \updelta_{2 } = - K_{22}^{ - 1} K_{12}^{T}\updelta_{1 } $$
(7)

Then, the resulted δ (\( \updelta_{1 } \) and \( \updelta_{2 } \)) was designated as \( \updelta_{\text{final}} \). The final simulation result with realistic sliding effects was acquired by integrating the resulted nodal displacements of the intermediate (\( \delta_{intermediate} \)) and the final (\( \updelta_{\text{final}} \)) FEM simulations.

3 Prediction Accuracy Evaluation

Thirty patients with dentofacial deformities were randomly selected from our datasets [IRB0413-0045]. Both preop and postop CT scan datasets were used for facial tissue prediction and its accuracy evaluation. The soft tissue prediction was completed using 2 methods: (1) traditional FEM without considering the sliding effect; and (2) our FEM method with the realistic sliding effects.

The patient-specific FEM mesh was generated from a template mesh, instead of manually segmenting the tissue volume from CT dataset. The template mesh was previously created from a Visible Female dataset [9]. Both inner and outer surfaces of the template mesh were registered to the patient’s skull and facial soft tissue surface respectively using anatomical landmark-based thin-plate splines (TPS) technique. Finally, the total mesh volume of the template was transformed to the patient data by interpolating the surface registration result using TPS again [9].

The actual movement vector of each bony segment was computed from the preop and postop CT datasets. First, the postop patient’s bone and soft tissue 3D CT models were registered to the corresponding preop data respectively by matching the surgically unchanged part (the cranium). Then, the osteotomies were performed on the preop models according to the postop CT data. The actual movement vector of each bony segment was calculated by moving each osteotomized segment from its preop original position to the postop position [13]. These vectors were used by both FEM methods as a moving boundary condition.

Finally, the simulated results were evaluated quantitatively. In order to acquire the error of each methods, displacement errors (absolute mean Euclidean distances) were calculated between the nodes on the simulated facial mesh and their corresponding points on the postop model. The evaluation was completed for the whole face and 8 sub-regions (Fig. 3). Repeated measures analysis of variance and its post-hoc tests were used to detect the statistically significant difference.

Fig. 3.
figure 3

Sub-regions (automatically divided using anatomical landmarks)

Full size image

The results of the quantitative evaluation showed that our FEM method with realistic sliding effects significantly improved the accuracy of the whole face and the critical areas (i.e., lips, nose and chin) comparing to the traditional FEM method, although the chin area only showed a trend of improvement. The malar region also showed a significant improvement due to the modeling of scar tissue. The improvement rates over the traditional FEM simulation were presented in Table 1 sliding.

Table 1. Error of the traditional method (mean ± SD) and improvement of the realistic sliding effects simulation over the traditional FEM method (mm).
Full size table

In addition to the quantitative accuracy evaluation, the predicted results were also observed visually. The traditional FEM simulation without the sliding effect resulted in a mesh distortion problem in the mandibular inferior border region in the patient underwent severe amount of mandibular movement (Fig. 4(a)). The collision between the two bony segments (proximal and distal) led to the distortion of the mesh in corresponding area, which was clinically unrealistic. On the other hand, the result of our realistic sliding effects clearly solved the mesh distortion problem by showing smooth inferior boarder (Fig. 4(b)).

Fig. 4.
figure 4

An example of mesh distortion after bone repositioning (inside of circle). (a) Mesh collision in inferior border of the mandible in the traditional FEM simulation without sliding effect. (b) Smooth inferior border line of the mandible of our method with the realistic sliding effects.

Full size image

Figure 5 illustrates the predicted results of a typical patient. Using the traditional FEM method, the upper and lower lip moved together with the underlying bone segments as a whole without considering the sliding movement (1.4 mm of displacement error for the upper lip; 1.6 mm for the lower). This resulted in large displacement errors (Fig. 5(a)), apparently clinically unrealistic. The sliding effect with nodal force constraint moderately improved the accuracy in the upper lip, while the lower lip still showed a larger error. The upper and lower lips were in a wrong relation. In addition, the mesh inner surface and anterior surface of the bony segment were also mismatched that should be perfectly matched clinically (Fig. 5(b)), apparently also unrealistic. Finally, the result of our realistic sliding effects achieved the best prediction results, accurately predicting clinically important facial features with a correct lip relation (the upper lip: 0.9 mm of the error; the lower: 1.3 mm) (Fig. 5(c)), apparently clinically realistic.

Fig. 5.
figure 5

An example of quantitative and qualitative evaluation results. The predicted mesh (pink) is superimposed to the postop bone (green) and soft tissue (grey). (a) Traditional FEM (1.6 mm of error for the whole face, clinically not acceptable). (b) Sliding effect with nodal force constraint. (c) FEM with realistic sliding effects (1.4 mm of error, clinically acceptable). (Color figure online)

Full size image

4 Discussion and Future Work

We developed a novel FEM simulation method with realistic sliding effects to accurately predict facial soft tissue changes following the osteotomies. Our approach has been quantitatively evaluated using 30 patient datasets. The clinical contribution of this method is significant. Our approach allows doctors to understand how the bony movements affect the facial soft tissues changes prior to the surgery. Incorporating such prediction tool into the surgical planning will also allow doctors to revise the plan as needed in order to achieve a best-possible treatment outcome. In addition, it also allows patients to foresee their postop facial appearance before the operation (patient education). The technical contributions include: (1) Efficient realistic sliding effects were implemented into the FEM simulation model to predict realistic facial soft tissue changes following the osteotomies. (2) The extended definition of the boundary condition and the ability of changing node types during the simulation clearly solve the mesh distortion problem, not only in the sliding regions, but also in the bone collision areas where the proximal and distal segments meet. (3) The patient-specific soft tissue FEM model can be efficiently generated by deforming our FEM template, thus reducing the time from 20 h to 10 min in MATLAB. It makes the FEM simulation feasible for clinical use.

There are still some limitations in our current approach. Preoperatively strained lower lip is not considered in the simulation. The strained lower lip can be automatically corrected to a reposed status in the surgery by only advancing (for Class II) or setting back (for Class III) the bony segments without any vertical movement. The same is not true in the simulation. We are currently working on solving this phenomenon. In addition, we are also improving the accuracy evaluation method. The current quantitative results do not necessary reflect the visualized results. Human eyes always have tolerances in judging facial appearance changes, usually within 2–4 mm. As shown in Fig. 5(c), although the prediction of the lower lip is improved tremendously with realistic sliding effects, the quantitative analysis only shows a 0.2 mm of improvement, which is a clinically nonsignificant improvement. Moreover, the prediction error of the lower lip using the traditional FEM is only 1.6 mm in quantitative analysis - an error also does not have any clinical significance. Nonetheless, Fig. 5(a) and (c) indicate the otherwise. Ultimately, our two-stage FEM simulation is the first step towards achieving a realistic facial soft-tissue-change prediction following osteotomies. In the near future, it will be fully tested in a larger clinical study.