Abstract
This paper provides a comprehensive survey on the techniques for human facial modeling and animation. The survey is carried out from two different perspectives: facial modeling, which concerns how to produce 3D face models, and facial animation, which regards how to synthesize dynamic facial expressions. To generate an individual face model, we can either perform individualization of a generic model or combine face models from an existing face collection. With respect to facial animation, we have further categorized the techniques into simulation-based, performance-driven and shape blend-based approaches. The strength and weakness of these techniques within each category are discussed, alongside with the applications of these techniques to various exploitations. In addition, a brief historical review of the technique evolution is provided. Limitations and future trend are discussed. Conclusions are drawn at the end of the paper.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Abrantes, G.A., Pereira, F.: MPEG-4 facial animation technology: survey, implementation, and results. IEEE Trans. Circuits Syst. Video Technol. 9(2), 290–305 (1999)
Allen, B., Curless, B., Popović, Z.: The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003)
Badler, N., Platt, S.: Animating facial expressions. ACM SIGGRAPH Comput. Graph. 15(3), 245–252 (1981)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM SIGGRAPH, New York (1999) (URL: http://www.kyb.tuebingen.mpg.de/bu/people/volker/)
Blanz, V., Scherbaum, K., Vetter, T., Seidel, H.-P.: Exchanging faces in images. Comput. Graph. Forum 23(3), 669–676 (2004)
Chai, J.-X., Xiao, J., Hodgins, J.: Vision-based control of 3D facial animation. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 193–206. EUROGRAPHICS, San Diego (2003) (URL: http://faculty.cs.tamu.edu/jchai/projects/face-animation/)
Debevec, P.E.: Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 189–198. ACM SIGGRAPH, New York (1998)
DeCarlo, D., Metaxas, D., Stone, M.: An anthropometric face model using variational techniques. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 67–74. ACM SIGGRAPH, New York (1998)
Du, Y., Lin, X.: Emotional facial expression model building. Pattern Recognit. Lett. 24, 2923–2934 (2003)
Farkas, L.: Anthropometry of the Head and Face, 2nd edn. Raven Press, New York (1994)
Gelder, A.V.: Approximate simulation of elastic membranes by triangulated spring meshes. J. Graph. Tools 3(2), 21–42 (1998)
Guenter, B., Grimm, C., Wood, D., Malvar, H., Pighin, F.: Making faces. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 55–66. ACM SIGGRAPH, Boston, MA (1998)
Hiwada, K., Maki, A., Nakashima, A.: Mimicking video: real-time morphable 3D model fitting. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 132–139. ACM SIGGRAPH, Osaka (2003)
Joshi, P., Tien, W.C., Desbrun, M., Pighin, F.: Learning controls for blend shape based realistic facial animation. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 187–192. EUROGRAPHICS, San Diego (2003)
Kähler, K., Haber, J., Yamauchi, H., Seidel, H.-P.: Head shop: generating animated head models with anatomical structure. In: Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 55–63. EUROGRAPHICS, San Antonio, TX (2002) (URL http://www.mpi-inf.mpg.de/∼kaehler/slides/sca02-headshop_files/ v3_document.htm)
Kähler, K., Haber, J., Seidel, H.-P.: Reanimating the dead: reconstruction of expressive faces from skull data. ACM Trans. Graph. 22(3), 554–561 (2003)
Kalra, P., Garchery, S., Kshirsagar, S.: Facial deformation models. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Handbook of Virtual Humans, chap. 6. John Wiley & Sons, West Sussex, England (2004)
Koch, A.: Structured design implementation – a strategy for implementing regular data paths on FPGAs. In: Proceedings of the 1996 ACM 4th International Symposium on Field Programmable Gate Arrays, pp. 151–157. ACM SIGDA, Monterey, CA (1996)
Kshirsagar, S., Egges, A., Garchery, S.: Expressive speech animation and facial communication. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Handbook of Virtual Humans, chap. 10. John Wiley & Sons, West Sussex, England (2004)
Lee, Y., Terzopoulos, D., Waters, K.: Realistic modeling for facial animation. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp. 55–62. ACM SIGGRAPH, New York (1995)
Lee, W., Goto, T., Kshirsagar, S., Molet, T.: Face cloning and face motion capture. In: Magnenat-Thalmann, N., Thalmann, D. (eds.) Handbook of Virtual Humans, chap. 2. John Wiley & Sons, West Sussex, England (2004)
Litwinowicz, P., Williams, L.: Animating images with drawings. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, pp. 409–412. ACM SIGGRAPH, New York (1994)
Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 271–276. ACM SIGGRAPH, New York (2001)
Magnenat-Thalmann, N., Thalmann, D.: Handbook of Virtual Humans. John Wiley & Sons, West Sussex, England (2004)
Marschner, S.R., Greenberg, D.P.: Inverse lighting for photography. In: IST/SID 5th Colort Imaging Conference, pp. 262–265. IS&T, Scottsdale (1997) (URL: http://www.graphics.cornell.edu/pubs/1997/MG97.html)
Noh, J., Neumann, U.: A Survey of Facial Modeling and Animation Techniques. USC Technical Report 99-705, Integrated Media Systems Center, University of Southern California (1998)
Parke, F.: Computer generated animation of faces. In: Proceedings of the ACM Annual Conference, pp. 451–457. ACM, Boston, MA (1972)
Parke, F.I.: A Parametric Model for Human Faces. PhD Thesis, University of Utah, Salt Lake City, UTEC-CSc-75-047, USA (1974)
Parke, F.I., Waters, K.: Computer Facial Animation. AK Peters, Wellesley, MA (1996)
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23(3), 664–672 (2004)
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, pp. 75–84. ACM SIGGRAPH, New York (1998)
Pighin, F., Szeliski, R., Salesin, D.: Resynthesizing facial animation through 3D model-based tracking. In: Proceedings of the 7th IEEE International Conference on Computer Vision, vol. 1, pp. 143–150. IEEE Computer Society, Los Alamitos, CA, USA (1999)
Pratscher, M., Coleman, P., Laszlo, J., Singh, K.: Outside-in anatomy based character rigging. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 329–338. Eurographics, Los Angeles (2005)
Seo, H., Magnenat-Thalmann, N.: An automatic modeling of human bodies from sizing parameters. In: Proceedings of the 2003 Symposium on Interactive 3D Graphics, pp. 19–26. ACM SIGGRAPH, Monterey, CA (2003)
Seo, H., Cordier, F., Magnenat-Thalmann, N.: Synthesizing animatable body models with parameterized shape modifications. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 120–125. Eurographics, San Diego (2003)
Shashua, A., Riklin-Raviv, T.: The quotient image: class-based re-rendering and recognition with varying illuminations. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 129–139 (2001)
Sifakis, E., Neverov, I., Fedkiw, R.: Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24(3), 417–425 (2005)
Terzopoulos, D., Waters, K.: Physically-based facial modeling, analysis, and animation. Vis. Comput. Animation 1, 73–80 (1990)
Tu, P.-H., Lin, I.-C., Yeh, J.-S., Liang, R.-H., Ouhyoung, M.: Surface detail capturing for realistic facial animation. J. Comput. Sci. Technol. 19(5), 618–625 (2004)
Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. ACM Trans. Graph. 24(3), 426–433 (2005)
Ward, K., Bertails, F., Kim, T.Y., Marschner, S.R., Cani, M.P., Lin, M.C.: A survey on hair modeling: styling, simulation, and rendering. IEEE Trans. Vis. Comput. Graph. 13(2), 213–234 (2007) (URL: http://www.cs.unc.edu/∼wardk/research.html)
Waters, K.: A muscle model for animating three-dimensional facial expression. Comput. Graph. 22(4), 17–24 (1987)
Wilhelms, J., Gelder, A.V.: Anatomically based modeling. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 173–180. ACM Press/Addison-Wesley Publishing Co., New York (1997)
Williams, L.: Performance driven facial animation. Comput. Graph. 24(4), 235–242 (1990)
Yin, L., Weiss, K.: Generating 3D views of facial expressions from frontal face video based on topographic analysis. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, pp. 360–363. ACM SIGMULTIMEDIA, New York (2004)
Zhang, Q., Liu, Z., Guo, B., Shum, H.: Geometry-driven photorealistic facial expression synthesis. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 177–186. Eurographics, San Diego (2003)
Zhang, Q., Liu, Z., Guo, B., Terzopoulos, D., Shum, H.: Geometry-driven photorealistic facial expression synthesis. IEEE Trans. Vis. Comput. Graph. 12(1), 48–60 (2006)
Zhang, Y., Sim, T., Tan, C.L.: Rapid modeling of 3D faces for animation using an efficient adaptation algorithm. In: Proceedings of the 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, pp. 173–181. ACM SIGGRAPH, Singapore (2004)
Zhang, L., Snavely, N., Curless, B., Seitz, S.M.: Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. 23(3), 548–558 (2004) (URL: http://grail.cs.washington.edu/projects/stfaces)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ersotelos, N., Dong, F. Building highly realistic facial modeling and animation: a survey. Visual Comput 24, 13–30 (2008). https://doi.org/10.1007/s00371-007-0175-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-007-0175-y