Psychological research in face perception has benefited greatly from advances in computer graphics. For example, morphing software allows us to test hypotheses about the ways faces are recognized by creating high-quality images that blend pictures in sophisticated ways. That technique can be employed by psychologists with a wide range of interests—for example, the perception of identity, expression, or social attributes (e.g., Beale & Keil, 1995; Calder, Young, Perrett, Etcoff, & Rowland, 1996; Young et al., 1997). Other image manipulation techniques allow us to alter images in ways that we predict will affect judgments of gender, age, race, or any number of other psychologically relevant dimensions (e.g., Busey, 1998; Oosterhof & Todorov, 2008; Stewart et al., 2012; Walker & Tanaka, 2003). In short, the facility to manipulate images in well-specified ways opens up the opportunity to design perceptual experiments that were impossible in the era before widely available graphical computers.

A second benefit of computational graphics is the ability it brings to analyze large sets of face images. Rather than generating novel pictures, some research questions can best be addressed by a statistical analysis of large sets of unmodified images—for example, when asking which physical properties of faces predict consistent social attributions (Nestor, Plaut, & Behrmann, 2013; Scheuchenpflug, 1999; Tredoux, 2002). Of course, these two approaches are related—if the analysis of image sets throws up a statistical regularity (let us say, a systematic difference between kindly and threatening faces), then it should be possible to use this finding to manipulate a novel set of images—perhaps rendering them more kindly or more threatening.

Although these techniques have been very prevalent in the past 20 years of face-processing research, they are not widely available to the whole research community. Laboratories specializing in such research have typically developed in-house bespoke software, and its use normally requires programming ability. Furthermore, because labs do not typically generate software for use by others, these programs are not generally user-friendly or well-documented enough to make sharing easy, even between collaborating partners. Although some face-related software is commonly available (e.g., Fantamorph—www.fantamorph.com; or Psychomorph—Tiddeman, Burt, & Perrett, 2001), the programs tend to be highly specific in their function, or not straightforward for novice users.

In this article, we describe a software suite to support research in face perception. InterFace is a program that can be used by any researcher. Its use does not require knowledge of programming, and all functions are interactive, contained in an easily understood graphical user interface (GUI). The program is written in MATLAB but can be run across Mac and PC platforms using the freely available MATLAB Runtime environment. It is freely distributed, along with a detailed manual including many examples. InterFace has both graphical manipulation and statistical analysis facilities. Its main functions are:

  1. 1.

    Shape and texture remapping Any face can be warped to a different shape. For example, the face of one person can be manipulated to the shape of a second person, or to a shape that is particularly masculine, friendly or smiling, and so forth. Faces can also be morphed together to produce blends.

  2. 2.

    Facial averaging Many faces can be averaged together. This technique can be used to observe regularities in different groups—for example, an average of faces that have been rated friendly might be compared to an average of faces that have been rated fierce. Alternatively, many different images of the same person might be combined to provide a single, average version of that person.

  3. 3.

    Principal components analysis This is a technique for extracting the key dimensions of face images. The program delivers these dimensions for further analysis, and also provides an interactive “reconstruction” tool, allowing users to manipulate the facial components independently—a facility that is useful for projects aiming to understand how different sources of facial information are coded in images.

The InterFace manual provides detailed instructions and examples. Here we will describe the approach taken in the design of the software, and illustrate some of its uses at a conceptual level. We will describe its main properties, though many more detailed features are available to users of the software.

Main concepts: Shape and texture

At the heart of InterFace is a distinction between shape and texture. For many graphical techniques, as well as for many psychological hypotheses, this is an important way of segmenting an image (e.g., Beymer, 1995; Craw & Cameron, 1991; Vetter & Troje, 1995). A face image shape refers to the positions of a set of fiducial points corresponding to key feature locations, such as the corners of the eyes, mouth, and so forth. InterFace uses 82 key points, which are shown in Fig. 1. The program contains a graphical tool for helping users to position these points, and the specifications for their placement are defined in the user manual. InterFace requires that these points be identified for all faces.

Fig. 1
figure 1

Segmentation of a face image into its shape and texture. This image of Barack Obama is by Pete Souza [Licensed under CC BY 3.0]

Full size image

We next consider the texture of a face. “Texture” is a shorthand label for all the information in a face that is not carried by the position of the key (fiducial) points. This includes information about the reflectance properties, the lighting and surface information, and information due to the camera characteristics. To consider this information separately from shape, all faces are warped to a standard shape. InterFace provides a standard shape that can be used, but users also have the option to define their own template for this purpose. The key issue is that within any set of faces under analysis, the “texture” of each face will be defined as the image resulting from morphing the original to a standard shape. We refer to the resulting images as shape-free faces (Craw, 1995; Craw & Cameron, 1991). This is because shape does not discriminate between the faces in the set—following the standardization, they all have the same shape. Figure 1 illustrates the separation of a particular image into its constituent shape and texture (for further examples, see Hancock, Burton, & Bruce, 1996; Itz, Schweinberger, Schulz, & Kaufmann, 2014; Schulz, Kaufmann, Walther, & Schweinberger, 2012).

InterFace provides a landmarking tool that allows an easy way to compute the shape and texture of an image. Figure 2 shows an example. The fiducial points are located by hand by using a mouse to align them until the user is satisfied. At this point, the user selects buttons on the GUI to save the shape and texture of the face. These are stored in standard directories as a text file for shape (a list of x–y positions of the fiducial points) and a graphical file for the texture (the face reshaped to the standard shape). This initial separation of a face into shape and texture forms the basis of all further operations, and we find that operators with a little experience can perform the landmarking of a face in under 5 min. The algorithm used in InterFace is bicubic interpolation (see Wolberg, 1998), though this is not under the control of users, who have access only to the input and output images.

Fig. 2
figure 2

Example image landmarked using the landmarking tool. The photograph is manually landmarked (left) with 82 fiducial points (right). The original image is by R.S.S.K. [CC BY-SA 2.0]

Full size image

Use of the software

Reshaping a face

As we described in the previous section, InterFace provides a simple tool for reshaping a face to a standard shape. However, users also have the option to use any shape they choose. For example, a researcher may wish to warp one person’s face to the shape of another person’s, a technique that has been used to study the different signals involved in the perception of identity (Andrews, Baseler, Jenkins, Burton, & Young, 2016; Burton, Kramer, Ritchie, & Jenkins, 2016). Alternatively, one might be interested in whether social judgements can be affected by shape change (e.g., Oosterhof & Todorov, 2008). In that case, a face that viewers have rated highly trustworthy could be reshaped to the shape of someone rated as untrustworthy. Another study might involve reshaping highly masculine faces to the shapes of highly feminine images. InterFace allows for all of these possibilities. Using the same tool illustrated in Fig. 2, users can reshape a face into any shape they choose, provided that the shape is stored in a text file corresponding to the simple x-y-coordinate structure required by the program. Figure 3 shows some examples of faces warped to different shapes.

Fig. 3
figure 3

Examples of images that have been reshaped: An image of Matt Damon warped to the average Tom Cruise shape (left); an image of Gwyneth Paltrow warped to the average female shape (middle); and an image of Brad Pitt warped to the shape of a different Brad Pitt image (right). The original images are by Nicolas Genin [CC BY-SA 2.0], Georges Biard [CC BY-SA 3.0], and Eva Rinaldi [CC BY-SA 2.0], respectively

Full size image

Face averages

A number of different types of averages can be created within InterFace, each with potential uses in psychological research.

Shape averages

Since the shape of faces is coded as an ordered set of x-y-coordinates, it is straightforward to compute the average of any set of these. The average fiducial points for particular sets can be useful in a number of research settings. For example, how does the average shape of a set of men differ from the average of a set of women? Such questions have previously been answered through laborious measurement (Bruce et al., 1993; Burton, Bruce, & Dench, 1993). However, this software delivers the ability to compute and display shape averages very easily.

The construction of face averages is also very useful in other settings. For example, the standard shape template in InterFace was derived as the average of a large and diverse set of faces. But some research questions might require normalization by more restrictive criteria—that is, norms based on a single sex, race, or age of a face. Similarly, all of the uses of face reshaping, described in the previous section, could be used with average face shapes. One might want to ask how a European face looks when it is morphed to the average shape of a set of Chinese faces, or the average of a set of men or women, young or old people. In short, this facility allows one to derive “norms” on which to base further image analysis.

Texture averages

In the same way that averages can be taken of shape information, it is also possible to combine textures. Any set of shape-free images shares the same feature layout (by definition), so averaging these together can be achieved simply by computing the average intensity at each point in the image, and InterFace provides this facility. This technique could be used for a number of research purposes. For example, Fig. 4 shows the average textures of a set of images of two celebrities, so-called within-person texture averages (Burton, Jenkins, Hancock, & White, 2005). Other uses might be to compare the textures between groups of different people, in exactly the same way that we described for shape comparisons in the section above.

Fig. 4
figure 4

Two celebrities whose average textures have been warped to the same common shape template. The images depict Gwyneth Paltrow and Tom Cruise

Full size image

Full averages

The computation of separate shape and texture averages is sometimes useful. However, a more general technique, likely to have wider use in research, is to compute averages of sets of faces that combine both shape and texture. This is very easily achieved in InterFace. To compute a full average of a set, its average texture is simply reshaped to its average shape. In this way, it is possible to derive full averages of very different images. Figure 5 shows two uses of this technique. In Fig. 5 (top row), a full average has been created for a set of men and a set of women: that is, for all of the items used in our celebrity database. In Fig. 5 (bottom row), we show within-person averages, which we have used in our own research as a means of eliminating superficial differences between different images of the same person—differences that make computer recognition very difficult across images (Jenkins & Burton, 2008, 2011; Robertson, Kramer, & Burton, 2015).

Fig. 5
figure 5

Example averages. Top row: The average man (left) and woman (right). Bottom row: Averages of 35 images of Hugh Jackman (left) and Anne Hathaway (right)

Full size image

Morphing

InterFace includes a facility to morph between two face images. The smooth graphical transition between faces has become a standard tool in psychological research (e.g., Beale & Keil, 1995; Calder et al., 1996; Young et al., 1997), and InterFace allows users to combine images in any proportion (50/50, 90/10, etc.). As with the techniques described above, this facility relies on the landmarking of both images. The shape and texture information for each facial image is stored in a standard location and combined in the morphing process. Figure 6 shows an example of two facial images and a 50/50 morph between them.

Fig. 6
figure 6

Example morph: Tom Cruise (left), Hugh Jackman (right), and a 50/50 morph of the two images (center). The left image is by Ian Morris [CC BY 2.0], and the right image is by Grant Brummett [CC BY-SA 3.0]

Full size image

Principal components analysis: PCA

PCA has become a very important part of face perception research (Kirby & Sirovich, 1990; Phillips, Moon, Rizvi, & Rauss, 2000; Turk & Pentland, 1991). The technique provides a statistical description of a set of face images, by extracting the dimensions of variability (eigenvectors, or “eigenfaces”) in the order of the variances they explain. Thus, early components capture gross variation in the image set, and later components capture more fine-grained variation. This technique, along with other, related techniques such as factor analysis, is useful in data reduction when a relatively small number of dimensions captures a large proportion of the set variance.

PCA is especially popular in psychological face research because it provides an operationalization of face space: a space with metric dimensions into which faces can be placed (Valentine, 1991). The typical use of PCA takes a large number of faces to derive a relatively small number of dimensions, in which any face image can be described, as either a set of coordinates in that space or (equivalently) a weighted sum of the eigenvectors. For an introduction to this technique, see Valentin, Abdi, and O’Toole (1994), or for a full mathematical account, see Gong, McKenna, and Psarrou (2000).

InterFace provides the facility to carry out PCA straightforwardly. As with the techniques above, it requires a set of faces that have been landmarked. The program performs separate PCAs on the shape and texture of the set, and users are prompted to specify how many components they wish to extract (up to a maximum of the set size minus one). Following PCA, eigenvectors and the corresponding eigenvalues are written to files, which can then be used in subsequent analysis of the original or novel images (see below). The texture eigenvectors are also represented in an image file, giving a visualization of each component. Reconstruction values of each of the contributing face images are also written to file (i.e., their values on the derived dimensions), along with measures of reconstruction error (e.g., the cosine between the original and the reconstruction).

Having derived a novel set of dimensions with which to describe faces, it is possible to use these faces in a number of ways. Most simply, one can ask whether the distribution of the faces in PC space has any correspondence with human face perception. The data files derived from the PCA—which give a location in space for each contributing image—are simple text files that can be analyzed in any way the researcher wishes. This approach has been used to examine notions of face similarity and distinctiveness—asking whether the faces that are close together in PC space are those that human perceivers find most similar (e.g., Nestor, Plaut, & Behrmann, 2013; Scheuchenpflug, 1999; Tredoux, 2002), or whether the faces perceived as “distinctive” by human viewers are those that lie in sparsely populated regions of space (e.g., Burton, Bruce, & Hancock, 1999; Hancock et al., 1996; O’Toole, Deffenbacher, Valentin, & Abdi, 1994).

A further possibility is to use the PC space to “reconstruct” novel face images. A property of the technique is that the novel space can be used to represent any image (as long as it is the same size as the originals). This property is at the heart of PCA as a tool for face identification. In short, the images for recognition are “reconstructed” in the low-dimensional PC space and then compared to known faces. If a novel image lies sufficiently close to a known face in this space, then the novel face is taken as being recognized (e.g., Moon & Phillips, 2001; Turk & Pentland, 1991). InterFace provides the facility to code any novel image in the PC space derived from a previous PCA, and thus supports this use.

To allow exploration of “face space,” InterFace also provides a graphical visualization tool for the interactive manipulation of items within the space. Figure 7 shows an example. In this case, we have a picture of Tom Cruise and its reconstruction in 30 texture components and 30 shape components. The component values (coefficients in the reconstruction) are shown to the sides, and slider bars invite the user to change the values. This allows the user to explore the derived space—the reconstruction image changes automatically as the coefficient values are altered. Thus, by altering one dimension at a time and leaving other component values unchanged, the user observes the effect that a particular dimension has—that is, the dimension it codes.

Fig. 7
figure 7

Example of an image of Tom Cruise reconstructed in the PCA Viewing Tool. The original image is by Ian Morris [CC BY 2.0]

Full size image

We now provide an example of using InterFace for carrying out a PCA in realistic research.

Example: Within-person PCA

The traditional use of PCA in face recognition research is to use images of different people to extract the major ways in which faces vary (e.g., Moon & Phillips, 2001; O’Toole, Abdi, Deffenbacher, & Valentin, 1993; Zhao, Chellappa, Phillips, & Rosenfeld, 2003). The intuition behind this approach is that a statistical description of real face images is more likely to reveal the true underlying dimensions of “face space” than is an intuitive language-based factorization relying on easily labelled metrics such as “distance between the eyes” or “width of the mouth”.

In our own research, we have used PCA in a different way, to explore the representations of familiar faces (Burton, Jenkins, & Schweinberger, 2011; Burton et al., 2016; Jenkins & Burton, 2011). Simply, the idea is that a separate PCA is carried out for each identity (on multiple images of that person), with the goal of deriving a multidimensional space specific for that face, independent of variability between people (Aishwarya & Marcus, 2010; Chiachia, Falcão, Pinto, Rocha, & Cox, 2014; Shan, Gao, & Zhao, 2003). We now describe an example use of InterFace for this purpose.

  • Step 1: Creating a set of images In this example, we used 30 ambient/unconstrained images of the same identity, collected from personal photographs.

  • Step 2: Landmarking We next manually landmarked each of the images using the InterFace tool (see Fig. 2). This process creates two new files for each original face: a shape file and (optionally) a texture file (see above). These are stored in separate shape and texture directories.

  • Step 3: PCA The InterFace program was used to conduct a PCA on the 30 images. In this example, we requested all possible 29 dimensions of shape and similarly all 29 of texture. This process created a number of new files, including the eigenvectors of the PCA, the associated eigenvalues, and the reconstruction values (coefficients) needed to code each original face in the new 29 + 29 dimensional space.

  • Step 4: Reconstruction After running a PCA on a set of faces, we can reconstruct those images using a simple weighted sum of the resulting components/eigenvectors. An example is shown in Fig. 8. This tool is useful to gain an intuitive understanding of the reconstruction, but we can also interrogate the quality of the representation more formally.

    Fig. 8
    figure 8

    Example image reconstructed using the PCA Viewing Tool. The projection values for texture (left column) and shape (right column) illustrate where this particular image falls on each component. The original image is by R.S.S.K. [CC BY-SA 2.0]

    Full size image

Figure 9 shows the first three shape and texture components from this analysis.

Fig. 9
figure 9

Variance captured by the first three components of shape and texture. The contributions of these components are illustrated by adding a low and a high value (±1.5 SDs) to the person’s average

Full size image

The first texture component (which explains the largest amount of variance) represents a general change in the brightness of the photographs. The first shape component captures head rotation along the longitudinal axis (“roll”), as well as a slight change in camera distance. These components are, of course, specific to the images of this identity, so other sets will likely depict different transformations for their components. What seems to be common to all within-person PCAs that we have tried is that the first three shape dimensions tend to describe rigid head rotations in three-dimensional space in some order/combination (Burton et al., 2016; Jenkins & Burton, 2011). This remains a topic of ongoing research.

Image presentation

The figures in this article show faces cropped to a standard shape, which we have used for much of our own work. For the purpose of statistical analysis, it is not important that this standard shape is somewhat angular—consistency of use being the most important feature. However, we are aware that this shape is not very aesthetically pleasing, and if researchers plan to use the outputs of these image manipulations as experimental stimuli, it may be better to use a more naturalistic outline. For this purpose, we have included a smoothing operator in InterFace, which will render norm-shaped faces less angular. Figure 10 shows some examples of this transformation, which simply adds extra vertices between outer points on the standard shape used by InterFace.

Fig. 10
figure 10

Averages of Keira Knightley and Tom Hanks, “cropped for display” to appear less angular

Full size image

Conclusions

We have provided an outline of the main features of InterFace, a software package intended to support researchers in face recognition. We have shown that the package offers a number of standard image manipulation tools (shape-warping and morphing), as well as statistical analysis (PCA and a visualization tool). We have outlined some potential uses of this tool in psychological research, and we hope it will support further work in this field.

This tool can be downloaded from https://www.york.ac.uk/psychology/interface. It contains a runtime app, which can be used on either PC or Mac, and is accompanied by a software guide describing all of its features.