1 Introduction

The human face abounds with numerous information of one’s subjective experience of emotion. Spontaneous facial expressions are temporally revealing emotional states of their host, much like a mirror. In recent decades, researchers have been endeavoring to seek the essential features of human facial expressions and to identify emotions by behavioral measures (e.g., self-report, rating systems, video analysis) or by electrophysiological approaches (e.g., electromyography (EMG), electroencephalogram (EEG), galvanic skin response (SCR)) (Mauss and Robinson 2009). The approach used in the current study is facial EMG because it is non-invasive but sensitive to capture fleeting and subtle facial muscle changes in an ongoing emotional process, and wherein visual observation is unavailable or ambiguous (Neta et al. 2009; Tassinary et al. 2007).

EMG responses over facial muscle regions like corrugator supercilii which draws the brow downward and medialward to form a frown and zygomaticus major which elevates the corner of the mouth superiorly and posteriorly to produce a smile (Ekman et al. 1980) can effectively discriminate the valence (pleasure) and intensity of emotional states (Cacioppo et al. 1986). In other words, facial EMG is a prominent “tool” to infer mood situations (Cacioppo and Petty 1981; van Boxtel 2010) and has been employed commonly to examine emotional responses elicited by affective visual stimuli (e.g., Lang et al. 1993; Bradley et al. 1996; Larsen et al. 2003; Neta et al. 2009), especially based on the three dimensional model of emotion (pleasure-arousal-dominance, PAD) which provides highly satisfactory and general descriptions of emotional states (Mehrabian 1995). As the principal dimension, the bipolar valence is effective to describe and identify emotions during affect processing by means of classification techniques (e.g., Cacioppo and Berntson 1994; Larsen et al. 2003). Previous studies demonstrated that pictures (e.g., see also Cacioppo et al. 1986; Lang et al. 1993; Larsen et al. 2003) or facial expressions (e.g., Dimberg 1990; Sato et al. 2008) with aversive content induced more corrugator supercilii activity than those with pleasant content. On the contrary, pictures with pleasant scenes or smiling faces generated more responses over zygomaticus major.

It is well-known that the international affective pictures system (IAPS) has been widely used to elicit emotion due to its standardization and universal availability. Although the aim of the IAPS in experimental settings is the induction of total emotion space, the distribution of PAD-ratings not all areas of the emotion space are equally possible (Davis, et al. 1995; Bradley and Lang 2008). Therefore, Walter et al. (2011) partially filled the gaps (low valence/low arousal, high valence/low arousal and neutral valence/high arousal) of Lang’s affect space of the IAPS in two dimensions (valence and arousal) with 30 additional standardized affective visual stimuli (called Ulm pictures) which were selected from the image database of the German Press Agency. Given this background, in the current study, we chose visual stimuli from these merged pictures to induce various emotional states.

In addition, the repeatability of surface EMG was often reported in human kinetics (e.g., Rainoldi et al. 1999; Falla et al. 2002; Granata et al. 2005); Additionally, Suvinen et al. (2009) reported the repeatability of portable EMG activities on the sites of masseter and anterior temporalis muscles in serial assessment performed on asymptomatic subjects. However, little is known about repeated facial EMG over corrugator supercilii and zygomaticus major in response to affective pictures, even though these two sites are prevalently employed in leading EMG activities to recognize emotional states.

To summarize, the primary goal of the present study was to investigate the potentially distinct effects of emotion on facial EMG responses over corrugator supercilii and zygomaticus major in response to various affective visual stimuli in two dimensions of valence and arousal. We hypothesized that all participants would show intensive corrugator supercilii activities when they are exposed to negative pictures, compared to neutral pictures which occupy medium corrugator activities and positive pictures which elicit little corrugator activities; Additionally, we hypothesized that all participants would reveal increased zygomaticus major actions while they looked at positive pictures, compared to neutral pictures and negative pictures. In addition, we assumed that facial EMG responses are differentiable in high and low arousal emotions. Our second objective was to examine the consistency of facial EMG activity and certain emotional states that participants were experiencing. For this purpose, we compared the facial EMG reactions with rating of valence assessed by participants and standardized Lang’s and Ulm’s samples. Finally, we aimed to test the repeatability of facial EMG activities over corrugator supercilii and zyomaticus major, respectively. Therefore, we performed the second session similar to the first session but with self-assessment manikin (SAM) in each trial after participants accomplished another experiment.

2 Materials and methods

2.1 Participants

A total of 23 healthy adults took part in this research for pay. Three participants were excluded, due to the fact that inaccurate facial EMG recorded on the interest site in the first participant and two absences of corrugator supercilii recording in session 2, respectively. Therefore, 20 participants ranging in age from 18 to 64 years (M = 41.5, SD = 14) are reported in this paper. All participants signed informed consent prior to the experiment, and they were told the goals of the experiment upon completion of all tests. The experiment was designed and implemented according to the ethical guidelines of the University of Ulm and has been approved by the Ethical Committee (number: 245/08-UBB/se).

2.2 Stimuli

Stimuli consisted of 8 positive, 8 negative and 8 neutral pictures, were chosen from the IAPS (Lang et al. 2008) and Ulm pictures (Walter et al. 2011). According to Lang et al. (1990) methods, we divided them into three categories in terms of the mean rating scores in the dimension of valence. In detail, pictures with rating scores ranging from 1.00 to 4.00 were considered as negative pictures (n = 8, M = 3.17, SD = 0.65). Pictures with rating scores between 6.00 and 9.00 were considered as positive pictures (n = 8, M = 7.00, SD = 0.55) and ones with rating scores in between 4.00 to 6.00 as neutral pictures (n = 8, M = 4.92, SD = 0.50). In the dimension of arousal, pictures above or below 5 in terms of their rating scores were considered high or low arousal (n = 12, M = 5.68, SD = 0.64, high arousal; n = 12, M = 3.67, SD = 0.69, low arousal, respectively). The code numbers of selected pictures from IAPS were 1810, 2130, 2410, 2490, 2745.2, 2870, 2980, 3060, 3550.2, 5130, 5260, 5300, 5920, 6010, 7361, 7400, 7502, and 7510. The code numbers and meanings of selected stimuli from Ulm pictures are as follows: 09, sleeping child with teddy bear, 14, gravestone in winter scenery, 15, young woman with depressed expression, 21, house demolition, 22, worker hanging from a helicopter, and 27, man sits at a rocky cliff.

2.3 Apparatus

A NeXus-32 physiological measurement system (http://www.mindmedia.nl/english/nexus32.php) was running in a DELL desktop computer. It provides 32 physiological channels wherein at least two channels can be used for the acquisition of EMG data. The software of Biobserve (http://www.biobserve.com) was used for recording the trigger data and signals from fixed channels (on- and off-sets of the pictures). Additionally, the software of IPhys programmed by one of the authors was employed for presenting stimuli.

2.4 Procedure

After the participants were introduced into the experiment and signed informed consent, they were seated stably in a comfortable reclining chair in a sound attenuated room of the Ulm emotion lab. They were told to relax and pay attention to the pictures during the experiment. In addition, they were instructed on how to assess their emotional condition using the SAM as Lang et al. (2008) recommended.

The pictures were presented on a 17-inch monitor 50 cm in front of participant’s eyes in both of session 1 and session 2. 24 pictures were presented randomly and each picture lasted 6 s after onset. There was a 5–13 s randomly changing interval with a fixation point in the center of the screen with gray background in session 1 (see Fig. 1, left). After a 2 s interval, each picture was followed by a SAM rating in session 2. Subsequently, a 5–13 s randomly changing interval like session 1 was offered before next picture (see Fig. 1, right). There was a subsequent experiment (it will be reported elsewhere) between session 1 and session 2. Physiological information (e.g., EMG, EEG, SCR) was recorded during all of the experiments.

Fig. 1
figure 1

Procedure of pictures presentation in session 1 and session 2

Full size image

2.5 EMG recording

Facial EMG activities were acquired by using bipolar miniature silver/silver chloride (Ag/AgCl) skin electrodes with 4 mm diameter attach space filled with gel. Bipolar electrodes were respectively placed on corrugator supercilii and zygomaticus major muscle regions in the left side of participant’s face, according to the guidelines for EMG placement recommended by Fridlund and Cacioppo (1986). EMG signals were recorded at a sampling rate of 512 Hz.

2.6 Data reduction and statistical analysis

Raw EMG signals were filtered by using a 50 Hz notch filter to decrease power interference and a 10 Hz high-pass filter to exclude noise which may induced by participants’ movements. EMG signals were then displayed on a computer monitor with high resolution to reject non-stimuli related artifacts (e.g., caused by cough, voluntary pouts) manually through inspection visually. Consequently, on the site of corrugator supercilii, 1 trial was removed from 480 trials in session 1, 1 trial was deleted from 480 trials in session 2. On the site of zygomaticus major, 1 trial was excluded from 480 trials in session 1, 6 trials were rejected from 480 trials in session 2, respectively. The remaining artifacts were reduced by following steps. EMG signals afterwards rectified and smoothed by means of the root mean square (RMS) technique with a 125 ms sliding average window. Furthermore, we calculated the mean amplitude of facial EMG for each second in the period of picture viewing (6 s), and 1 s preceding picture onset immediately. Finally, we considered the 1 s picture pre-onset as baseline in each trial, took the ratios between picture viewing period and baseline level into statistic analysis.

Facial EMG activity was submitted to a three-way analysis of variance (ANOVAs) with repeated measure using valence (positive, neutral and negative) and arousal (high and low) as between-subjects factors, and time (6 intervals of 1 s) as within-subjects factor. Two-sided Dunnett post-hoc tests were used for the multiple comparisons on valence (reference group: positive).

The mean EMG activities of each picture across participants in each site were fed to the analysis of correlation between EMG responses and participants’ rating of valence.

The Z-score of facial EMG responses of each picture across participants in each session and each site were forwarded to examine the repeatability of EMG using the principal measurement of repeatability—intraclass correlation coefficient (ICC) which indicates the proportion of the variance among groups or classes and the global variance (variance among groups or classes and variance within groups or classes) (Wolak et al. 2011). In practice, the ICC value above 0.8 is considered as ‘excellent repeatability’, an ICC value below 0.6 implies ‘poor repeatability’, and one in the range of 0.6–0.8 is accepted as ‘good repeatability’ (Rainoldi et al. 1999).

3 Results

3.1 Session 1

3.1.1 Facial EMG

Figure 2 (upper image) depicts corrugator EMG activities are much higher during participants viewing negative pictures, compared to neutral pictures which were greater than positive pictures. The ANOVAs with repeated measures reveled a significant main effect on valence (F (2, 476) = 5.65, p < 0.01), post-hoc tests indicated that negative pictures produced more intensive corrugator EMG reactions than positive pictures (p < 0.01). Conversely, zygomaticus EMG responses (see Fig. 2, lower image) are stronger in the period of positive pictures presenting, compared to neutral and negative situations. The ANOVAs with repeated measures also demonstrated a significant main effect on valence (F (2, 476) = 3.13, p < .05), post-hoc tests showed significant effect between positive and negative pictures (p < 0.05). There were no other significant main or interaction effects.

Fig. 2
figure 2

Mean EMG amplitude in ratio of baseline level over corrugator supercilii (upper image) and zygomaticus major (lower image) in response to the visual stimuli (negative, neutral and positive) in session 1

Full size image

3.1.2 Correlation of EMG and ratings of pictures

Figure 3 presents the scatter plot of EMG amplitude in ratio of baseline level over corrugator supercilii and zygomaticus major between the periods of picture viewing and baseline periods versus the standardized rating of valence from IAPS and Ulm pictures. The lower the rating of valence, the higher corrugator EMG responses were (see Fig. 3, upper image), whereas the higher rating of valence, the higher zygomaticus EMG activities were (see Fig. 3, lower image). The Pearson correlation’s coefficient revealed a significant negative correlation between corrugator EMG responses and standardized ratings of valence (r = −0.51, p < 0.05), and a significant positive correlation between zygomaticus EMG reactions and standardized ratings of valence (r = 0.52, p < 0.01), respectively.

Fig. 3
figure 3

Scatter plot of mean amplitude in ratio of baseline level over corrugator supercilii (upper image) and zygomaticus major (lower image) versus the rating of valence from standardized IAPS and Ulm pictures

Full size image

3.2 Session 2Footnote 1

3.2.1 Facial EMG

As we expected, the facial EMG activities were sensitive to various visual stimuli also in session 2. On the site of corrugator supercilii, EMG responses were much higher when participants viewed negative pictures, compared to positive pictures which occupied lower EMG responses than neutral pictures condition (see Fig. 4, upper image). The ANOVAs with repeated measures illustrated a main effect (F (2, 476) = 5.91, p < 0.01) in the dimension of valence. Post-hoc tests demonstrated significant effects between negative and neutral pictures, negative and positive pictures (p < 0.01). Opposite to corrugator supercilii, zygomaticus EMG reactions were greater during subjects exploring to positive pictures (see Fig. 4, lower image), compared to neutral and negative pictures conditions. The ANOVAs with repeated measures also indicated a main effect (F (2, 471) = 6.02, p < 0.01) in the dimension of valence. Post-hoc tests demonstrated significant effects between positive and neutral pictures, positive and negative pictures (p < 0.01), respectively. There are no other significant main or interaction effects.

Fig. 4
figure 4

Mean EMG amplitude in ratio of baseline level over corrugator supercilii (upper image) and zygomaticus major (lower image) in response to affective visual stimuli (negative, neutral and positive) in session 2

Full size image

3.2.2 Correlation of EMG and ratings of pictures

The correlations between facial EMG activities and the ratings of valence by participants are shown in Fig. 5. It is clear that the higher ratings correlated with lower corrugator EMG responses, whereas the lower ratings correlated with intensive corrugator EMG activities. In contrast, the lower ratings correlated with lower zygomaticus EMG, the higher ratings correlated with higher zygomaticus EMG. The Pearson’s correlation coefficient implied a significant negative effect between corrugator EMG and subjects’ rating of valence (r = −0.63, p < 0.01) (see Fig. 5, upper image), and a significant positive effect between zygomaticus EMG and participants’ rating of valence (r = 0.51, p < 0.05) (see also Fig. 5, lower image).

Fig. 5
figure 5

Scatter plot of mean EMG amplitude in ratio of baseline level over corrugator supercilii (upper image) and zygomaticus major (lower image) and the rating of valence assessed by participants

Full size image

3.3 Repeatability of facial EMG activity

The results of ICC indicated a high level of repeatability of corrugator EMG activities in session 1 and session 2 (ICC = 0.82, F (1, 23) = 5.55, p < 0.001), and a poor level of repeatability of zygomaticus EMG responses in session 1 and session 2 (ICC = 0.53, F (1, 23) = 2.15, p < 0.05).

4 Discussion

The primary hypotheses of this study were partially confirmed. In the dimension of valence, as we expected, negative pictures elicited increased stronger facial EMG activities over corrugator supercilii than neutral pictures which occupied medium EMG responses, whereas positive pictures generated very low EMG reactions. In contrast, on the site of zygomaticus major, facial EMG activities induced by positive pictures were significantly greater than neutral and negative pictures. In line with previous studies (e.g., Cacioppo et al. 1986; Lang et al. 1993; Larsen et al. 2003; Larsen and Norris 2009), corrugator and zygomaticus EMG are effective to identify negative and positive emotions, respectively. Therefore, corrugator and zygomaticus EMG can be considered as indicators of negative and positive emotional states accordingly.

Moreover, corrugator EMG was much sensitive to subtle emotions in the dimension of valence (negative, neutral and positive), whereas zygomaticus EMG was ambiguous with neutral and negative emotions. An alternatively explanation is that zygomaticus major is more likely to be involved in voluntary actions and more probably to be influenced by cross talk as its location lies close to various muscles like zygomaticus minor, masseter and buccinators (Tassinary et al. 2007) which links to negative emotions such as frustration and skepticism (Unz and Schwab 2005), but corrugator supercilii is less possibly to be engaged in voluntary behaviors like unfelt smile for masking emotional responses (Ekman and Friesen 1975).

Considering the consistency of facial EMG and subjective emotional experience, we compared the mean EMG activities of each picture cross participants with the rating of valence. There were significant correlations between facial EMG responses and the ratings of valence from standardized IAPS and Ulm pictures at the site of corrugator supercilii and zygomaticus major, respectively, in session 1. This illustrated that facial EMG reactions represent well emotional states elicited by pictures accordingly. Similar to prior research (e.g., Lang et al. 1993), in addition, the correlations between the rating of valence assessed by participants and facial EMG activities were also pronounced in session 2, which provides evidence for the consistency of facial EMG and emotional states that participants were experiencing. In total, these data suggested that IAPS and Ulm pictures what we chose are reliable to generate positive, neutral and negative emotional states, and participants’ emotional states were well represented by facial EMG activities.

The repeatability of facial EMG responses over corrugator supercilii and zygomaticus major in response to visual stimuli were proved by our data. Corrugator EMG responses has been shown to be well repeatable, and zygomaticus EMG reactions only provided a poor level of repeatability in identifying various emotions in the dimension of valence. There was no statistical evidence to prove that facial EMG responses are differentiable in the dimension of arousal since high arousal emotions cannot be easily elicited by pictures in laboratory situation. In other words, strongly arousal emotions could be induced by specific stimuli or situations like playing a game. Other autonomic measures, such as skin conductance level (SCL) and heart rate (HR), may better indicate emotions on the arousal level (Teixeira-Silva et al. 2004). Future studies using facial EMG over corrugator supercilii and zygomaticus major should be combined with other physiological channels to recognize more complex emotions. Individual characteristics like personality and gender also must be considered.

In conclusion, our data implied that: bipolar facial EMG over corrugator supercilii and zygomaticus major can effectively discriminate negative and positive emotions from others in the dimension of valence. Furthermore, facial EMG activities are significantly correlated with subjective emotional experience. Last but not least, corrugator EMG responses are excellently repeatable, and zygomaticus EMG reactions are only poorly repeatable.

Concerning these results, the procedure of measuring EMG data evoked upon viewing a standardized set of affective visual stimuli, may serve as a means to identify individual affective states in various situations especially in human machine interaction. It may build a basis for future applications such as real-time classification methods for recognizing emotional states particularly facilitating recognition for people in daily life and subjects with certain disabilities (Frantzidis et al. 2010) in healthcare.