1 Audiovisual Cuts and Viewers’ Connectivity

The development of how brain manages the perception of audiovisual content started in the 1950s (Cohen-Séat et al., 1954); however, it has recently got an impulse in neuroscience research (Cha et al., 2015; Kang et al., 2015; Nakano et al., 2009). Many investigations have been focused on finding what are the effects of the fragmentation and of the event boundaries of the media works in viewers’ perception (Andreu-Sánchez et al., 2017a; Francuz & Zabielska-Mendyk, 2013; Germeys & d’Ydewalle, 2007; Zacks et al., 2010). Some other investigations have approached synchronization between subjects watching films (Hasson et al., 2004; Lahnakoski et al., 2014; Nakano et al., 2009). There are also investigations studying the dynamics of brain networks related to aesthetic appreciation (Cela-Conde et al., 2013).

Here, we propose to study viewers’ connectivity related to watching media works containing cuts, paying attention to the main styles of edition in which those cuts tend to appear in our audiovisual works. The brain connectivity variable is addressed with functional and effective connectivity.

Functional connectivity refers to temporal correlation between two electro/neurophysiological measurements from different parts of the brain (Friston et al., 1993). It alludes to the dependence between signals and is defined in terms of correlations or covariance (Friston, 1994). A way to approach functional connectivity is with a phase synchronization index. In this investigation, we employed the phase locking value (PLV) for this purpose. PLV uses responses to a stimulus that is presented repeatedly and looks for latencies with a phase locking (the phase difference between the signals varies little across trials) (Lachaux et al., 1999).

Effective connectivity is understood as a time-dependent circuit that replicates the timing relationship between the recorded sources (Aertsen, 1991). Here, we employed the Granger causality for analyzing the effective connectivity of viewers when they were watching cuts in media content. This index is based on the idea that for two simultaneously measured signals, if one signal can be predicted better by incorporating the past information from the other signal than using only information from the former, then the latter signal can be called causal to the first (Granger, 1969; Niso et al., 2013; Wiener, 1956).

2 Methods

2.1 Participants

Thirty-six participants with normal or corrected-to-normal visual acuity participated in this study. Subjects were aged 28–56 (43.97 ± 8.07) years. The study had the approval of the Ethics Commission for Research with Animals and Humans (CEEAH) of the University Autònoma de Barcelona, Spain. All experiments were performed in accordance with relevant guidelines and regulations. Written informed consent was obtained from all participants.

2.2 Stimuli

We analyzed here brain activity associated with cuts. Movies are made up of shots, continuous short film sequences, spliced together with cuts. The term cut refers to the interruption of the shot with another shot with a change in space, time, or action, due to narrative or aesthetic edition requirements. We presented two video stimuli with the same narrative but different style of edition. Both stimuli had a duration of 198 s. Stimulus 1 had 33 shots, with an average shot length (ASL) of 5.9 s, and had been edited according to classical rules of edition, based on Hollywood films (smooth transitions, visual continuity, clear presentation of visual information, and among others) (Bordwell et al., 1996). Stimulus 2 had 79 shots, with an ASL of 2.4 s, and was edited breaking all the classical rules of edition; it was based on the so-called post-classic style with a more restless camera, greater number of shots, and framing jumps (Bordwell, 2002)—well represented in musical video clips that first appeared on the MTV channel, some decades ago.

2.3 Experimental Setup

The stimuli were presented on a 42-inch HD LED display (Panasonic TH42PZ70EA, Panasonic Corporation) and synchronized with EEG-recorded data using paradigm stimulus presentation (perception research system incorporated). Continuous EEG was recorded using the wireless Enobio® system (neuroelectrics). Twenty electrodes [(O1, O2, P7, P3, Pz, P4, P8, T7, C3, Cz, C4, T8, F7, F3, Fz, F4, F8, Fp1, Fp2, and an external electrode used for electrooculography (EOG)] were placed according to the international 10–20 System and referenced to electronically linked mastoid electrodes (Martín-Pascual et al., 2018).

2.4 Analysis

We used the open-source toolbox HERMES (Center for Biomedical Technology) (Niso et al., 2013) running on MATLAB R2013a (The Mathworks Inc.) under MacOS version 10.9.5 (Apple Inc.), for the functional and effective connectivity analysis. For cleaning the data, EEGLAB (Swartz Center for Computational Neuroscience, UC San Diego) was used (Delorme & Makeig, 2004). We band passed the EEG data with filter between 0.5 and 40 Hz. Then, we made epochs 500 ms before the cuts and 1000 ms after the cuts, removing the baseline. For rejecting artifacts, wrong data, and bad channels, we visually inspected the data and applied ADJUST plug-in (Mognon et al., 2011) for EEGLAB.

As mentioned, to determine functional connectivity, we chose analysis of the PLV, a phase synchronization index, while for the effective connectivity, Granger causality was studied. For the PLV analysis, we plotted the averaged connectivity of PLV in all the participants (\(N = 36\)) with 100 surrogates of the original data. We applied a Wilcoxon test with multiple comparisons with a false discovery rate of Type 1 (\(q = 0.1\)). For the Granger causality analysis, we also did a Wilcoxon test with 100 surrogates and a false discovery rate correction of type 1 (\(q = 0.1\)). More information about the implementation of these indices can be found in HERMES (Niso et al., 2013).

3 Results

3.1 Synchronization Associated with Cuts

To approach the PLV, we evaluated the instantaneous phase differences of the signals generated before and after the cut. The main idea was to evaluate whether those signals evolve together in time. We also analyzed differences between the styles of edition in which those cuts were inserted to detect if they affected functional connectivity. The PLV was analyzed in different bands: theta (4–8 Hz), alpha (8–12 Hz), beta 1 (12–20 Hz), beta 2 (20–28 Hz), and low gamma (28–40 Hz). We made those approaches for the activity before (from −500 to 0 ms) and after (from 0 to 500 ms) the cuts. We found statistically significant differences in synchronization between before and after the cut. More synchronization after the cuts was found in theta and beta 1 bands. A residual synchronization between electrodes P3 and Pz was also found in low gamma band, see Fig. 1.

Fig. 1
figure 1

Significant differences (\(p < 0.05\), Wilcoxon test) in theta, beta 1, and low gamma bands found after (from 0 to 500 ms) the cut in comparison with before the cut (from −500 to 0 ms), in PLV analysis

Full size image

We also approached synchronization associated with cuts by studying differences between the styles of edition in which cuts were inserted. Comparing the PLV after the cut (from 0 to 500 ms) in Hollywood-style movie with that in MTV-style movie, we did not find statistical differences between them. In a second approach, we compared the PLV in the whole epoch (from −500 to 1000 ms) in both styles of edition. In this case, we only found residual statistical differences in theta band between T7 and F7.

Overall, these results indicate that brain network activation after the cut in a movie is more intense than before the cut. However, the style of edition in which cuts are presented does not affect brain synchrony.

3.2 Granger Causality Associated to Cuts

Effective connectivity was approached by analyzing Granger causality. We compared that index between the before (from −500 to 0 ms) and after (from 0 to 500 ms) conditions. We found Granger causality before the cut was statistically significantly higher than after the cut, see Fig. 2. According to that result, causality in brain signal activity decreases as a consequence of the change of shots managed by cuts.

We also wanted to analyze whether the style of edition affected this index. For that, we first compared the before condition between the styles (Hollywood and MTV). We found only a very residual higher level of Granger causality in MTV-style movie than in Hollywood-style movie in left parietal and right frontal areas. Secondly, we compared the after condition between the styles. Again, we found only very residual significant connectivity with a higher level in the left parietal area in Hollywood style and a higher level in left parieto-frontal area in MTV style. As mentioned, these results were very residual and are not considered to be relevant for this investigation.

Fig. 2
figure 2

Granger causality in the cut. Green lines indicate average causal connectivity, while red lines indicate significant differences found (\(p < 0.05\), Wilcoxon test) when comparing connectivity before the cut (from −500 to 0 ms) with that after the cut (from 0 to 500 ms)

Full size image

4 Conclusions

There are not many studies about connectivity and causality exploring parameters of the communication process through the cerebral cortex. Most join emotional processes or their effects in cortical structures of deep regions of the brain when subjects are watching media content (Cha et al., 2015; Raz et al., 2016) or making aesthetical appreciations (Cela-Conde et al., 2013).

In this investigation, we approached viewers’ functional and effective connectivity related to a very specific variable: the cut. Previous studies have proven that despite viewers not always being aware of cuts (Smith & Henderson, 2008), these have an impact on their perception (Andreu-Sánchez et al., 2017a, b; Francuz & Zabielska-Mendyk, 2013). Our results suggest that the cut causes a synchronization effect. We approached this index through bands, since PLV seems to evaluate the synchronization better over a whole band (Bruña et al., 2018). We found a higher phase synchronization after the cut than before it in the theta and beta 1 bands. However, we found that Granger causality presents greater connectivity before than after the cuts. This result suggests that with regard to Granger causality, cuts interrupt connectivity in brain activity.

With the aim of studying differences in connectivity related to the styles of edition, we also analyzed the PLV and the Granger causality parameters between those styles. We found that the style of edition in which cuts are inserted seems not to affect connectivity in viewers’ brain activity.

This investigation brings new insights to learning how media content editing styles and audiovisual cuts affect viewers’ perception. It is also interesting to be aware that differences in editing of media contents could be used as attentional markers for creating new experimental approaches. This could be a line of research of interest for application in different areas in the near future.