Introduction

Movies consist of a rapid series of still images (called 'frames') being projected onto a screen creating the impression of motion [1]. While movies are historically and conventionally being recorded and displayed at rates of 24 frames per second (fps), there have been recent incidences of increasing the frame rates to 48 fps, 60 fps or even 120 fps, to increase the perception of smoothness. This has created a controversy among film advocates and spilled over into efforts to bring high frame rate technology to television [2]. The result has been a recent appeal to test experimentally the effects of different frame rates on observers’ perceptions and experiences. For example, investigating the effect of frame rate on perceived quality has shown that higher frame rates lead to higher perceived quality ratings [3,4,5], potentially due to reduction in visibility of motion artifacts [6]. Moreover, using stereoscopic 3D content, Wilcox and colleagues [7] evaluated the effect of motion smoothness and showed that individuals preferred the smoother motion of 48 fps and 60 fps over 24 fps. The same authors found similar results using expert viewers from the film industry [8]. However, most movies are not produced and/or displayed in 3D formats, with the majority shown on standard flat 2D televisions. This necessitates the need to investigate observers’ preferences for higher frame rates using 2D flat displays.

To the best of our knowledge, no study has investigated the effect of different frame rates on observers’ perception and preference using 2D regular cinematic content. This study aims to fill this gap in the research literature by asking individuals to report their preference for 24 fps versus 60 fps movie clips displayed on a standard flat 2D display. Furthermore, we use different movie categories (animation, montage, action and drama), which are different in their kinetic pacing, visual content, filmmaking styles and techniques, to assess if our findings generalize across different forms of content.

Eye tracking can provide objective evidence for an individual's visual attentional processes and patterns in relation to visual stimuli. Longer and more frequent fixations are indicators of higher interest, and individuals spend more time exploring and looking at the regions that are more appealing to them [9,10,11,12]. Therefore, employing eye tracking in addition to self-reports should enrich our understanding about the relation between attention and preference. It might sometimes be the case that what is visually appealing to individuals might not be the same thing that they say they prefer. Eye tracking has previously been used to investigate the visual path and quality of perception in relation to different fps [5, 13]; however, the results of those past studies are limited, as fixations and their dwell times were not included in the analyses and the frame rates used (i.e. 5, 15, and 25 fps) were much lower than those being used in movies. To fill the gap in the literature, in the current study, we explore how higher (60 fps) versus lower frame rates (24 fps) influence preference, using both self-reports and eye movements. We hypothesize that individuals will fixate more on the clips that is more appealing to them. We tested this prediction using a two-alternative forced choice eye tracking task, whereby participants selected which of two movies—identical in all respects save their frame rate—that they preferred.

Method

Participants

Thirty (8 male and 22 female) undergraduate students (M = 21.23, SD = 4.67) were recruited from the University of British Columbia. Students received course credit in return for their participation. All participants possessed normal or corrected-to-normal vision. Each gave written informed consent.

Stimuli

Twenty 30-s movie clips from four different categories (animation, montage, action, and drama) were used as the stimuli. Each category consisted of five segments from five different movies (see Table 1 for the list of movies). The original movie sequences were 24 fps and using Adobe Premier Pro were converted to 60 fps using optical flow method [14], providing two exact video sequences differing only in their frame rates. All the movie clips resolution was 1920 × 1080. Digital televisions use same motion-compensated frame interpolation to convert 24 fps to higher frame rates, such as 60 fps [15]. Each set of exact videos were located side by side on the screen and were compiled as one video clip, displaying the different frame rate video of the same sequence concurrently. In other words, same movie clip was presented side-by-side at different frame rates. The location of the videos on screen (left vs. right) were randomized, meaning half of the 24 fps videos were displayed on right and half were displayed on left side of the screen, while their twin 60 fps videos were displayed on the opposite sides.

Table 1 List of movies (release year) used as stimuli
Full size table

Eye Tracking Equipment and Procedure

Eye movements were monitored using a binocular infrared, remote, and eye tracker running at 250 Hz (RED250, SMI GmbH Germany) controlled with iView X software (v2.8). Stimuli were presented on a 22-inch LCD monitor (Dell P2210, 60 Hz, 1680 × 1050 pixels). Note that the monitor’s frame rate was 60 Hz and no smoothness adjustment was available for the screen. Initially, participants completed a five-point calibration procedure. Calibration was research controlled, and it was accepted if the mean spatial shift for four validation points was 0.5° of visual angle or less for vertical and horizontal deviations. Participants were seated, head free, at 70 cm from the monitor. Participants then viewed all twenty films in a random order, for 30 s each. After viewing each film clip, participants were asked to respond the question of “which film version did you prefer?” by choosing either left or right choices. At the end of the study, as a manipulation check, four of the previous movie pairs were randomly selected and presented to the participants and they were asked to select “Which film version appears to be the smoothest?” from three options: “Left”, “Right”, or “Both versions seem the same”. This task was to check if the participants are able to discriminate the quality of smoothness between the low and high frame rates.

Regions of Interest (ROI)

The films were divided into two regions of interest (ROI), one encompassing the left film and another ROI the right film. This analysis was performed using BeGaze software (v3.4).

Data Analyses

There were two sets of analyses. In the first set, we included all the participants (N = 30) and ran a binomial generalized mixed model to test the self-reported preference for either the 24 or 60 fps movies. We also ran a repeated measures analyses of variance (ANOVA) to compare the gaze preference for the different film pairs. Finally, we performed a Chi-square test to see if the participants were able to use smoothness to discriminate between the different frame rates. Those that could make the discrimination (N = 17) formed a subset set of participants who were analysed as above.

Results

Behavioural

A binomial generalized mixed model was conducted with participant (N = 30) as the random factor to test the likelihood of preferring films (left or right on the screen) depending on their frame rate (24 fps vs. 60 fps). Six-hundred data points were included in the analysis (30 participants × 20 films). Results did not show any difference in preference between 24 and 60 fps films, χ2 = 1.61, df = 1, p = .204 (see Table 2).

Table 2 Estimates of the binomial generalized mixed model (GLM) with standard errors (SE) and 95% confidence intervals
Full size table

Eye tracking

Table 3 shows the mean number of fixations as a function of fps, film categories, and films. A 4 (Film Category) × 5 (Film) × 2 (fps: 24 vs. 60) repeated measures ANOVA was performed, with Film Category, Film, and fps as within-subjects variables on fixation number. The main effect for fps was not significant, F(1,29) = 0.08, p = .774, partial ŋ2 < 0.01. Also, none of its interactions were significant: fps × Film Category, F(3,87) = 0.76, p = .519, partial ŋ2 = 0.02; fps × Film, F(4116) = 0.64, p = .632, partial ŋ2 = 0.02; fps × Film Category × Film, F(12,348) = 1.08, p = .372, partial ŋ2 = 0.03.

Table 3 Mean and SEM of fixation number for different fps (N = 30)
Full size table

Preferred Versus Nonpreferred Film Versions

For each of the two frame rates, we also compared the gaze behaviour for the self-reported preferred versus self-reported non-preferred film versions. The aim of this analysis was to investigate if participants looked more frequently to those films that they self-reported as preferring. Mixed models were conducted with participant (N = 30) as random factor to test the effect of different 24 fps films (preferred or not) on fixation counts. Results showed that participants had higher fixations on the preferred films (M = 36.43, SD = 14.18) than non-preferred ones (M = 20.77, SD = 13.53, t(29) = 8.19, p < .001). Also, for 60 fps, films participants had more fixations on the preferred films (M = 37.26, SD = 14.11) than non-preferred ones (M = 21.14, SD = 13.42, t(29) = 7.44, p < .001).

Frame Rate Discrimination Task

In this task, participants were asked to discriminate the frame rate that yields a smoother perceptual experience. One participant failed to complete the final task because of a difficulty with the eye tracker; hence, data from 29 participants are reported; and 116 data points are included in this analysis (29 participants × 4 films). The majority of responses (56.9%, 66 out of 116) were “both videos were the same”. For the remainder of the responses, 86% selected the 60 fps and 14% chose 24 fps as the smoothest film, χ2(1, N = 50) = 25.92, p < .001.

We also analysed fixations for the fps discrimination task, conducting a 4 (Film) × 2 (fps: 24 vs. 60) ANOVA, with Film and fps as within-subjects variables. The main effect for fixations was significant, F(1,28) = 6.28, p = .018, partial ŋ2 < 0.18. Participants had higher fixations on the 60 fps films (M = 31.47, SEM = 1.10) compared to 24 fps (M = 28.78, SEM = 0.97).

Subsample Analyses

We ran further analyses on those participants that were able to discriminate between the smoothness of the two frame rates (N = 17). A binomial generalized mixed model was conducted with participant as a random factor to test the likelihood of preferring films based on their frame rate. 340 data points were included in the analysis (17 participants × 20 films). Results showed a significant difference between the two film rates, χ2 = 4.76, df = 1, p = .029, (see Table 4), with participants preferring the lower frame rate over the higher frame rate (189 vs. 151 observations).

Table 4 Estimates of the binomial generalized mixed model (GLM) with standard errors (SE) and 95% confidence intervals
Full size table

We also analysed the subsample's fixations during the fps discrimination task, and found again that there were more fixations on the 60 fps films (M = 32.39, SEM = 1.40) compared to 24 fps (M = 27.33, SEM = 1.17), F(1,16) = 17.48, p = .001, partial ŋ2 < 0.52.

Discussion

In the current study, using self-reports and eye tracking, we investigated whether a film shown at a higher (60 fps) versus lower frame rate (24 fps) is preferred. This study was conducted to expand our understanding of viewers' preference when observing films at different frame rates on standard flat 2D displays. We used a two-alternative forced choice task, and asked participants to select their preferred movie version among the identical clips differing only in frame rate. Results of this study indicates that not all individuals are able to discriminate between films shown at 24 and 60 fps. Moreover, overall, they do not have any particular preference over 24 versus 60 fps films, nor does their gaze behaviour reveal any preferential bias towards 24 or 60 fps films. However, our results showed that regardless of the frame rate (24 fps or 60 fps), individuals fixated more on their self-reported preferred films.

Analyses of those individuals who were able to distinguish between the different frame rates (i.e. our 'subsample') revealed that 24 fps films were preferred over 60 fps films. Interestingly, when discriminating between these two frame rates, the subsample's eye tracking results showed more fixations on 60 fps films than 24 fps films. This mirrored the results we observed for the full sample of participants. Collectively, these data indicate that when trying to discriminate between two different frame rates, the eyes are drawn to films with higher frame rates regardless of whether participants are able to consciously discriminate between the rates. This suggest that the eyes are being drawn to the higher rates by low-level bottom-up (automatic) visual processes rather than top-down (volitional) processes [16].

In general, our result do not support the findings from 3D stereoscopic investigations that report individuals prefer higher fps films (i.e. 48 fps and 60 fps) than 24 fps films [7, 8]. This discrepancy in the results suggests that the display format (2D vs. 3D) might influence the appeal for higher fps films, a finding that will need systematic research scrutiny in the future.

Conclusion, Suggestions and Limitations

The results of the present study indicate that not all individuals are able to distinguish between 24 and 60 fps films, though when trying to make this discrimination, the eyes tend to be drawn to the higher frame rates automatically. We suggest that future studies collect information on individual differences, such as an individual's knowledge and expertise about movies, with the goal of discovering why some individuals are able to distinguish between the smoothness of a film's different frame rates, while others cannot. Moreover, we suggest that future research consider how individual differences, such as alertness, interest, and potential sex differences, combine with movie genre to affect perception and preference (e.g. do women prefer higher rate drama movies?). While no specific gaze behaviour was observed for preferring 24 fps versus 60 fps films, individuals looked more at the film versions they preferred, and those who could discriminate between the two rates preferred the films at the lower frame rate. As such, the results of the present study provide empirical support to those who argue that the viewing quality of films at higher frame rates is compromised on 2D displays.