10.1 Introduction

Facial expression is important in both verbal and non-verbal communication. The ability to express emotion through facial gestures impacts social and mental health (Swearingen et al. 1999). It has been previously described that more than 10 muscles of facial expression play a biomechanical role in the deformation of tissues surrounding the lips, which is essential for smiling and frowning (Chabanas et al. 2003). The production of facial gestures is the result of the individual function and complex synergistic activities of the muscles of facial expression.

Visualization and modelling techniques provide insight into how the muscles individually and collectively contribute to the shaping and stiffening of facial soft tissues during functional activities. Computer models used in early facial animation efforts contained parameterized representations of the muscles, their dimensions and geometric lines of action defined according to images found in anatomy textbooks (Platt and Badler 1981; Waters 1987; Terzopoulos and Waters 1990). An increasing number of studies use medical imaging to reconstruct the geometry of the face, muscles, and bones (Gladilin et al. 2004; Sifakis et al. 2005; Kim and Gomi 2007; Barbarino et al. 2009; Beldie et al. 2010; Nazari et al. 2010; Wu et al. 2014; Flynn et al. 2017). However, the thinness and complexity of the facial muscles can make delineating muscle boundaries in serial images a challenge (Som et al. 2012; Hutto and Vattoth 2015). Often, images are segmented from only one side of the face and either used to create a half-facial model reflected across the sagittal plane to create a full-facial model with right-left symmetry. Moreover, without more involved imaging techniques (e.g., diffusion tensor imaging, micro-computed tomography with iodine staining), modellers are left to heuristically define the inner structure of each muscle. This typically takes the form of some relatively homogeneous distribution of muscle fibres, which may not accurately reflect the architecture of a particular muscle.

An important determinant of muscle function is its architecture, the arrangement of contractile and connective tissue elements within the muscle volume (Zajac 1989; Gans and Gaunt 1991). The contractile elements consist of muscle fibre bundles, whereas the connective tissue elements that passively transmit forces include aponeuroses and tendons. Few studies have implemented techniques to reconstruct the architecture of select facial muscles (Liu et al. 2016; Wu and Yin 2016; Falcinelli et al. 2018; Sun et al. 2018). Thus, the 3D internal architecture of the muscles of facial expression remains largely unknown, impeding the development of high fidelity 3D visualization models and simulation of facial muscle contraction. Importantly, modelling muscle actuated facial deformation is challenging and requires an accurate representation of the internal anatomic elements and their interactions (Mazza and Barbarino 2011).

10.2 Methods

Recent technological advances have enabled the reconstruction of entire muscles in 3D space, as In situ, using dissection, digitization and 3D modelling at the fibre bundle/aponeurosis level (Ravichandiran et al. 2009; Li et al. 2015). More recently, collaborators incorporated the volumetric musculoaponeurotic 3D digitized data of the human masseter into a prototype finite element model that would better represent larger attachment areas and internal stresses of the muscle during contraction, i.e., chewing (Sánchez et al. 2017). This was the first reported application of digitized volumetric 3D data in the development of simulation models. Results demonstrated that the incorporation of 3D digitized data into their prototype finite element model “increase simulated maximum bite forces to more realistic levels” (Sánchez et al. 2017). Although presently available, this 3D digitization methodology has not been used to quantify morphology and architectural parameters of the muscles of facial expression, nor have the data been used to construct more detailed finite element models at the fibre bundle level.

The dissection, digitization and 3D modelling protocol has been developed in our laboratory over the last 20 years and has been used to model the musculoaponeurotic architecture of upper limb, lower limb and masticatory muscles (Agur et al. 2003; Li et al. 2015; Castanov et al. 2019). Approval was received from the University of Toronto Health Sciences Research Ethics Board (Protocol Reference #27210 and #28530). The protocol was adapted to study the muscles of facial expression. To enable digitization of the musculoaponeurotic elements of the muscles of facial expression, the head was first stabilized in a casing of polyurethane foam (Great Stuff™, Dow Chemical Co, Midland, Michigan, USA), leaving the face exposed. Next, three screws were inserted into the frontal, right and left temporal bones of the skull to provide a reference frame for reconstructing digitized data into 3D models. The skin of the face and neck was meticulously removed to expose the superficial muscles of facial expression. Fibre bundles of each muscle were exposed and digitized throughout the muscle volume, from superficial to deep, using a MicroScribe G2X Digitizer (0.05 mm accuracy; Immersion Corporation, San Jose, CA).

A custom software program, named “Fibonacci,” was created to digitize the fibre bundles of each muscle (Fig. 10.1). This program models the captured fibre bundle data as a hierarchy of muscles, muscle heads, layers, fibre bundles, and points. The first three levels of the hierarchy were created at the user’s discretion, whereas the latter two were defined using the MicroScribe digitizer. The MicroScribe pedal attachments send commands to the software. Specifically, pressing one pedal captures the current position of the digitizer and adds a point to the current fibre bundle. Pressing the other pedal completes the current fibre bundle and begins a new one. The digitizer was calibrated using the three screws embedded in the bones of the skull.

Fig. 10.1
figure 1

Fibonacci user interface

Full size image

To capture a 3D representation of a fibre bundle, the user first placed the tip of the digitizer on the fibre bundle and pressed the pedal to capture the current location as a point. This process was repeated while incrementally advancing along the entire length of the fibre bundle. Once digitized, the fibre bundle was carefully removed from the cadaver, exposing the deeper fibre bundles for digitization.

Fibonacci provides a simple user interface comprised of three panels. The main panel shows the captured 3D fibre bundle data. One smaller panel controls the connection to the MicroScribe scanner, and another panel displays a tree-view representation of the hierarchy of muscle data. In this latter panel, the user can modify attributes of each fibre bundle, such as the display colour, or add comments. Users can also use this panel to hide parts of the hierarchy or reorganize various components. Once the muscle data have been captured, it can be exported into the Autodesk® Maya® format for further processing and visualization.

The following muscles of facial expression were digitized bilaterally: risorius, platysma, zygomaticus major/minor, nasalis, levator labii superioris alaeque nasi, levator labii superioris, orbicularis oculi, levator anguli oris, depressor anguli oris, depressor labii inferioris, mentalis, orbicularis oris, frontalis, procerus, buccinator and corrugator. Additionally, digitized were the muscles of mastication including masseter, temporalis, and medial/lateral pterygoids. Digitized data of individual fibre bundles of each muscle, collected using Fibonacci, were imported into Autodesk® Maya® (Autodesk Inc., San Rafael, California) and reconstructed into a 3D model. The three-step modelling process is summarized below (Fig. 10.2):

Fig. 10.2
figure 2

Muscle fibre bundle digitization. a MicroscribeTM digitizer. b Cadaveric dissection of masseter muscle with digitized fibre bundles (data points are blue, and points are connected by polylines making up the fibre bundle). c Digitized fibre bundle of masseter as seen on Fibonacci software used to capture muscle fibre bundle trajectory. Reprinted with permission from Musculoskeletal and Peripheral Nerve Anatomy Laboratory, University of Toronto

Full size image
  1. 1.

    Digitized fibre bundles were represented as points connected by line segments (Fig. 10.2b, c). Using the built-in function available in Maya®, the trajectory of each fibre bundle was approximated using a cubic uniform B-Spline curve for smoothness and stored as a non-uniform rational basis spline (NURBS) curve (Ravichandiran et al. 2009). Each NURBS curve (representation of a fibre bundle) was reconstructed into a cylindrical tube for visualization purposes.

  2. 2.

    High resolution computed tomography (CT) scans of the skull and cervical spine were used to volumetrically reconstruct the skeletal elements to act as the base for the 3D models.

  3. 3.

    All of the muscle models were combined and registered to the CT-based skeletal model using the digitized reference frames.

Using this process, a complete, 3D model of the muscles of facial expression and mastication was generated.

10.3 Results

In total, 22 muscles were digitized and modelled. The number of fibre bundles digitized ranged from 22 to 1265, depending on the size of the muscle (Table 10.1). The smallest number of fibre bundles digitized was in risorius and the largest in masseter.

Table 10.1 Number of digitized fibre bundles in each muscle modelled
Full size table

The high-fidelity 3D model enables visualization of the relationships of the musculotendinous elements of the muscles and their attachment sites to the underlying skeleton, precisely as found in the specimen. The muscles of facial expression are superficial to the more deeply located muscles of mastication (Figs. 10.3 and 10.4). The muscles of facial expression, as seen in the 3D model, can be divided into five main groups: (1) muscles of the orbit; (2) muscles of the nose; (3) muscles of the scalp and ear (auricle); (4) muscles of the mouth; and (5) platysma. Generally, the main function of each muscle can be determined by the direction and spatial arrangement of the fibre bundles. For example, the main muscle of the orbit, the orbicularis oculi, has three main functions depending on the location of the part of the muscle that contracts. The palpebral, orbital and lacrimal parts of the orbicularis oculi function to gently close the eye, tightly close the eye and compress the lacrimal sac, respectively (Fig. 10.5). Muscles of the nose will compress (nasalis) or dilate (levator labii superioris alaeque nasi) the nostrils. The anterior muscles of the scalp, the frontalis, elevate the eyebrows and wrinkles the skin of the forehead. Muscles that attach to the upper lip (levator labii superioris) will raise it, while muscles that attach to the lower lip (depressor labii inferioris) will depress the lip. Muscles that attach to the angle of the mouth superiorly (levator anguli oris) will contribute to smiling, whereas muscles (depressor anguli oris) that attach inferiorly will contribute to frowning. The orbicularis oris is a sphincter-like muscle that closes the mouth, whereas the transversely orientated fibres of the buccinator, inserting into the angle of the mouth, are antagonists that retract the angle of the mouth. The platysma is the most extensive of the muscles of facial expression, spanning between the neck and the mandible, and functions to tense the skin of the neck (e.g., when shaving).

Fig. 10.3
figure 3

Anterior view of 3D model of the individual muscles of facial expression and mastication at the fibre bundle level. Reprinted with permission from Musculoskeletal and Peripheral Nerve Anatomy Laboratory, University of Toronto

Full size image
Fig. 10.4
figure 4

Anterolateral views of 3D model of the individual muscles of facial expression and mastication. a Right side. b Left side. Reprinted with permission from Musculoskeletal and Peripheral Nerve Anatomy Laboratory, University of Toronto

Full size image
Fig. 10.5
figure 5

Anterior view of 3D model of the 3 parts of orbicularis oculi muscle. Reprinted with permission from Musculoskeletal and Peripheral Nerve Anatomy Laboratory, University of Toronto

Full size image

More specifically, the comprehensive 3D model provides novel insights into the asymmetry and complex interrelationships of the individual muscles of facial expression. When comparing the right and left pairs of the muscles of facial expression, notable asymmetry existed in their size and shape. For example, the fibre bundles of the platysma were visibly longer on the left than the right (Figs. 10.3 and 10.4). Also, the fibre bundles of depressor anguli oris extended from the superior margin of the orbicularis oris and coursed inferiorly as far as the inferior border of the mandible on the right side. In comparison, the fibre bundles on the left side were shorter, extending from the angle of the mouth to the inferior border of the mandible (Fig. 10.4).

Furthermore, most of the muscles of facial expression exhibited complex interrelationships of surrounding muscle bellies and interdigitation of fibre bundles. For example, the fibre bundles of levator labii superioris lay deep to the oribularis oculi and zygomaticus minor as they course inferiorly to their attachment to the connective tissue superficial to the superior part of the orbicularis oris on the left side. On the right side, the fibre bundles were markedly shorter with a longer connective tissue attachment (Fig. 10.4). Interdigitation of fibre bundles of the muscles of facial expression was common. Some examples of fibre bundle interdigitation are listed below:

  • Peri-oral musculature at the angle of the mouth: the buccinator, levator anguli oris, depressor anguli oris, risorius and zygomaticus major all interdigitated with the orbicular oris (Fig. 10.6).

    Fig. 10.6
    figure 6

    Serial images of lateral views of peri-oral musculature, from deep to superficial (a–f). All images include temporalis (TM) and masseter (M) muscles. Reprinted with permission from Musculoskeletal and Peripheral Nerve Anatomy Laboratory, University of Toronto

    Full size image

  • Buccinator was found to have extensive interdigitations with the inferior part of orbicularis oris (Fig. 10.6b). Whereas the superior part of orbicularis oris lay superficial to buccinator.

  • Inferior part of levator anguli oris interdigitated with depressor anguli oris to form a continuous band of muscle at the angle of the mouth (Fig. 10.6d).

10.4 Discussion

Currently, no complete datasets exist that contain digital representations of the 3D internal musculoaponeurotic architecture of, and intermuscular relationships among, the muscles of facial expression. Studies to date have been more general and limited to numerical data about the average length and width of selected muscles (Balogh et al. 1988; Happak et al. 1997). In contrast, the current 3D model generated from digitized data could function as a digital atlas that provides 3D coordinate data of the musculotendinous anatomy of the muscles of facial expression and mastication. By digitizing fibre bundles from the same specimen, any architectural differences that existed In situ were real differences and not due to interspecimen variation. Furthermore, this model enabled quantification and visualization of the intricacies of the 3D spatial relationships among muscles, tendons, and aponeuroses, including the arrangement of fibre bundles within each muscle. These data possess great value to improve the anatomical fidelity of biomechanical models, and subsequently simulations, of facial gestures. Additionally, such data could prove useful for advancing imaging and image processing techniques that are used to derive models.

The current data revealed important geometrical information regarding asymmetry that exists between homologous muscles of the right and left sides of the face. Using the 3D model, morphological comparison of the left and right muscles of facial expression demonstrated marked asymmetry. The asymmetric features included differences in fibre bundle length, attachment sites, and interdigitation patterns between the right and left muscles. In the modelling literature, facial models constructed from medical images sometimes assume muscular symmetry (Beldie et al. 2010; Wu et al. 2014). This reduces the time required to construct models, as segmented images from one side of the face are either used to create a half-facial model or are duplicated and reflected to represent the other side in a full-facial model. Accentuation of muscle asymmetries could occur due to use patterns, pathology, injury or deformation of the underlying skeleton. Thus, creating 3D models that assume symmetry of the muscles of facial expression would likely produce inaccuracies in the model’s predictive capabilities. The architectural asymmetry demonstrated in the current 3D model provides a unique opportunity to further investigate the functional impact of facial muscle asymmetry, and to potentially move toward more realistic simulation.

As expected, interdigitating fibres existed in many of the muscles modelled. Due to the 3D data collection, the current model contains the precise location, morphology, interrelationships, and interdigitations of the muscles of facial expression, as In situ. While previous studies have reported such crossing of fibres, the specific geometry of the interdigitations remained undefined (Shim et al. 2008; Al-Hoqail and Abdel Meguid 2009; Yu et al. 2013; Kim et al. 2015). As determined in studies of the brain and tongue, standard diffusion tensor imaging fails to reconstruct crossing fibres (Wiegell et al. 2000; Ye et al. 2015; Voskuilen et al. 2019; Yang et al. 2019). Thus, the current model indicates the need to implement multi-tensor diffusion models when imaging the muscles of the face. As noted with the tongue, the fibre orientations and interdigitation greatly influence real and simulated tissue motion, particularly in the absence of internal joints (Gomez et al. 2018). Like the tongue, much of facial expression involves stiffening and movement of soft tissues. Including realistic architecture can also significantly alter muscle force predictions and resulting geometries of contraction (Sánchez et al. 2014; Sánchez et al. 2017). Consequently, accounting for interdigitation and heterogeneous fibre orientations are likely essential for improving the realism of facial appearance in animations and simulations.

10.5 Conclusions

By generating 3D representations of the musculoaponeurotic geometry of the face, we can move beyond the often idealized 2D depictions presented in anatomy texts; representations that are commonly used for learning and/or interpreting clinical and experimental findings. The current framework enables 3D structure and relationships to be explored and quantified, rather than simply interpreted from collections of images. High fidelity computerized representations of the human anatomy, as In situ, present the opportunity to develop advanced and interactive visualization techniques (e.g., augmented/virtual reality). Researchers and clinicians require resources to which image-based models can be compared, particularly when validating new techniques and/or understanding and handling the appearance of potential image artefacts. Advanced understanding of how musculoaponeurotic anatomy attaches to, and interacts with, neighbouring hard and soft tissues is fundamental for modellers looking to improve the realism of animations and simulations. The current 3D modelling process provides a common framework upon which these and other advances can occur, in a unified way.