Keywords

1 Introduction

Virtual reality technologies have evolved notably over the last few decades, and exponentially in recent years. This has become a medium with its own characteristics and interaction mechanisms that has given rise to the development of contents of the most heterogeneous nature and purpose, and whose potential for production still has a long path to travel, whose end we cannot even begin to glimpse.

One of the booming formats, among many others, within the complexity of what we would call << immersive >>, is 360° video. Due to the proliferation of increasingly accessible filming technologies and the possibility offered by different platforms (such as Youtube or Facebook) for its access and reproduction, it is a medium that is in the process of consolidation, although there are still many technical issues that differ from traditional video production that have yet to be resolved.

360° video has its own peculiarities and features with respect to other formats within the category and the virtual reality medium, whether these are linear or interactive media. Moreover, as we have mentioned, it has its own peculiarities with respect to video production that does not fall into the 360-degree category. It also has certain advantages, as well as certain limitations, to which visual production has not been subject until the proliferation of these technologies.

Although this paper does not deal with 360 video filming as such, this, with its particularities, would be the starting point for establishing the guidelines that lead to the model proposed in this paper, which would focus on a technology that has been consolidated in recent years thanks to the evolution of digital technologies and means of production, which is already within the reach of many users. We are talking about the so-called motion graphics, which will be dealt with in this publication, based on the production of 360-degree videos, and the extrapolation of monoscopic video projections filmed on a map for the creation of visual content with the 2D motion graphic technique (although it will later be extrapolated to a 360° environment).

In this way, just as filmed cinema had elements that could later serve as a reference in the field of animation (although both also had their own creation mechanisms), the structure and projection of 360° videos can serve as a basis for the creation of animation content with motion graphics techniques based on projections, primarily 2D motion graphic techniques.

2 Literature Review

Recent scientific and academic literature related to immersive virtual reality has focused, in a particularly relevant way, and in recent years, on the applications of its different manifestations in different fields, to the extent that it has been revealed as a technology with significant potential for implementation in a broad spectrum of scientific and technological domains, or those related to the cultural and entertainment industries, in addition to education, dissemination and communication.

These studies in the field of different formats expressed in the medium of virtual reality are clearly reflected in applications in fields such as teaching and learning (Pan et al., 2006) [1] (Kavanagh et al., 2017) [2] (Parong and Mayer, 2018) [3], or in areas related to science, whether in the learning of science itself, or in the representation of the information it offers (Pottle, 2019) [4]. Also of spatial interest is the application of virtual reality technology in different fields of scientific production (Li et al., 2017) [5] (Ayoub and Pulijala, 2019) [6], or, in a reverse process, the use of science itself to study our behaviour in an immersive environment (Clay et al., 2019) [7], either by measuring factors such as presence in that immersive environment (Slater, 2018) [8], or, on the other hand, by studying cognitive factors such as empathy (Schutte, 2017) [9].

However, 360 video could be considered, to a certain extent, a subcategory within virtual reality, with a series of specific characteristics, derived, in turn, from the video format itself. This is manifested, primarily, through a linear narrative, where events follow one after the other and the spectator has no control over them, nor the ability to move around the environmentFootnote 1. The 360° video presents, however, the freedom to set the field of view or FOV in any section of the 360° that make up the scene, always taking into account the ergonomic limitations of the human body itself, such as, quite simply, the turning of the neck.

This feature of 360 video is of great interest for content development, because sometimes the nature of the content or the experience developed does not require that the user/viewer does not interact with the scene. And, just as other media such as literature, cinema, radio or theatre, to name the most classic ones, continue to exist in this context of ongoing media revolution, the emergence of new media of this category and, therefore, new communicative possibilities, offers us an unprecedented opportunity to develop content in media which, a priori, present less interactivity with the environ-ment/technology/history, and a lesser degree of agencyFootnote 2.

That is why 360° videos, by presenting this specific feature, on the one hand, fit into the video category. However, they also fall into a subcategory of virtual reality, albeit, as we have already seen, with a limited level of interaction. Another key feature is that they do not require immersion (although they also offer that possibility) in order to be operational and efficient in their functions. Thus, they can be viewed with a device that does not allow immersion, such as a tablet or a mobile device without adaptation to vision, although, in these cases, the experience offered differs from the immersive one.

In reference to the studies on 360-degree video that can be found in the academic literature, these focus on different areas, both in terms of technical issues related to the filming of content, production and post-production, as well as on issues related to the high-quality reproduction of the content. Thus, we find recent studies such as Xu et al. (2020) [11] which address the issue of image processing in the medium (which requires a large amount of resources, due to the large amount of data that a 360 video has compared to a “traditional” video). Other authors such as David et al. (2017) [12] focus their research on the visual attention factor in 360-degree videos, reinforcing the importance of its study to gather information on our behaviour with our vision in a 360-degree environment, which in turn allows us to optimize the development of experiences and design them taking into account the variable of the focus on which we place our attention during the viewing experience. Along the same lines are the studies by Fan et al. (2017) [13], focusing on the prediction of the fields of vision (FoV) of users of virtual reality glasses or HMDs. Also, in the same year, Tran et al. (2017) [14], focused on the issue of quality metrics for 360 videos, proposing the development of parameters to establish these metrics that enable us to assess the quality of these videos. In addition to this, we can find publications that focus on aspects such as the importance of mobile devices for viewing (Broeck et al., 2017) [15] or mobile networks for accessing content through such devices (Mangiante et al., 2017) [16], stressing the constant need to optimize them, due to the amount of information that 360-degree videos handle.

As for the specific applications of the 360-degree video format within the field of virtual reality, these present an enormous degree of heterogeneity. This factor will be an element that will also take into account the possibilities offered to date by 360 video formats that are not necessarily immersive. Thus, the variety of fields of application of 360-degree video range from immersive 360 journalism (Van Damme et al., 2019) [17] to the evaluation of the effectiveness of learning in controlled experiments in areas such as health science education (Ulrich et al., 2019) [18], and the use of this 360-degree video technology for learning foreign languages (Repetto et al., 2021) [19], to give but a few examples.

Again, however, with regard to motion graphics produced with the same technology and projection base – although with a procedure that differs in some aspects, as we will see below – it is difficult to find literature focused on this modality. However, we have, for not so long, had relevant exponents of 360-degree videos created with animation and motion graphics techniques, such as TAS – The Canyon 360 4k VR (Tas Visuals, 2016) [20], the video clip Show it 2 Me, directed by Carter and Brooks (2016) [21] for the music project Night Club -and created in part with Tilt Brush-, or the already well-known Pearl (Google Spotlight Stories, 2016) [22], as well as other pieces that also have an informative vision, as is the case of Dreams of Dali, produced by The Dali Museum (2016) [23].

In this way, we can see that the technologies that already exist to produce 360º videos, such as 360-degree stereoscopic cameras, or 3D computational media, which make it possible to create immersive, rendered environments, can serve as a reference for working with other animation media based on 2D projection. This exponentially increases the possibilities for content development and, in turn, enables layered animation work, the use of more creative formats and, last but not least, the optimization of resources.

3 360 Video: Immersive and Non-immersive Typologies

The definition of 360-degree video is structured, however, by a series of characteristics and features that highlight the fact that we are dealing with a medium that incorporates the question of immersiveness, on the one hand, and the notion of belonging, on the other, taxonomically, to the category of virtual reality. But it also has some characteristic features which mean it has certain differences that make it a subcategory with respect to other manifestations of the virtual reality medium itself. This is true both when it comes to visualizing it and when it comes to producing it.

As far as the question of visualization is concerned, the following key differences can be found in relation to other formats that fall into the category of virtual reality. Among the differences we have found in this study are the following:

  • In many cases, the use of immersive reality glasses or HMD (head-mounted display) is not essential. This is due to the very nature of the video itself, as there is no high level of interaction beyond the ability of the user/viewer to see within their field of vision what they want to see within the 360-degree angle.

  • Preferably, the visualization of the story is always linear. A greater degree of interaction with the medium on the part of the viewer, beyond the occasional activation elements of other actions, would make the viewing experience more difficult, precisely because of the nature of the format. These two characteristics are interrelated, since the greater the degree of interaction, the greater the degree of user control necessary, and therefore the greater the degree of immersion requiredFootnote 3. This correlation does not necessarily run in the opposite direction. Thus, a 360-degree video can have a high degree of immersion, as the user employs HMDs, without having to have a higher degree of interaction than the freedom of the user to view the part of the scenario he/she wants. On the other hand, a higher level of interaction would mean that the content could no longer be classified in the video category, because it has perfectly defined features.

On the other hand, among the fundamental differences found at the production level with other virtual reality formats are the following:

  • The production does not require game engines. In fact, this option is normally used for other virtual reality formats, which require a higher degree of interactivity than video, but whose production processes are also more complex.

  • The 360-degree video camera models are designed for 360-degree video production, but there are other techniques that do not necessarily require the use of a camera. Nor, as mentioned above, is it necessary to use game engines in the case of video. It is also possible to use 2D animation techniques.

  • Another differentiating factor is that, although production layers can be superimposed, they are merged, with today’s technology, into a single image. In other words, 360-degree camera filming technologies can be used, and then video effects (as in so-called conventional videos), 2D and 3D animation techniques can be applied in this order, although the latter two can be combined to varying degrees, depending on the needs for content development on the one hand, and the features of the techniques applied on the other.

The following image (Fig. 1) shows the subcategories of viewing and creation within the field of virtual reality, taking into account the interaction and the forms of production of virtual reality.

Fig. 1.
figure 1

Source: Own elaboration. The authors

Subcategories of virtual reality, including 360-degree video, according to their production processes and the interaction capabilities they offer.

Full size image

Thus, knowledge of the specific features of 360-degree videos and the different layers of information they may contain is a vital tool for the division of video (combining filmed and animated elements) into production categories, in order to help optimize the content production process in this format.

4 Structural and Design Model of 360 Immersive and Non-immersive Video

The current model proposed is based on the projection presented by 360 photographs taken with cameras that make this function possible. In this way, the still image becomes the reference landscape for the creation of animations in immersive 360-degree videos. This is due to the type of projection it offers, usually referred to as equirectangular, or ERP, which would already have been addressed in studies such as those contributed by researchers such as Ray et al. (2018) [24], referring to the projection of a 360-degree physical world onto a flat rectangular image with a 2:1 aspect ratio. This implies, like other projections, a deformation in the position of objects. As can be seen in Fig. 2.

Fig. 2.
figure 2

Source: Taken on 4–08-2021 by one of the authors. Own elaboration.

360º photography of the Church of San Pedro Apóstol, in the Polvoranca Park in the Community of Madrid, after some post-production adjustments.

Full size image

It is also important to differentiate the elements that make up a 360-degree image projection. This, with today’s filming devices, will normally be represented as a 2:1 aspect ratio image, in 3840x1920 pixels, but it also allows other values, such as 1920x960, 4096x2048, or 8192x4096 (8K). In fact, the diversity of filming devices existing today will make an enormous amount of formats possible, with an increasingly higher image resolution. The elements that make up the scene are spread throughout the 360 images, and we can only see, at any given moment, those that are within our FOV or field of vision, which is normally 120 degrees on the device, although you can find devices that present 180 degrees, such as the HR VR Headset from XTAL, or even 200, such as the Pimax Vision 8K Plus VR Headset. These concepts are different from the human field of vision in physical reality, which is close to 180 degrees in binocular vision.

The correct levelling of the camera will allow the horizon, if there are no relevant geographical features, to be in the horizontal half of the recording, which would be called the Horizon. In contrast to traditional film cameras, the use of dipping and panning is counter-productive to the experience, as it distorts the natural interaction of the viewer/user with the virtual environment. However, it is possible, with the right knowledge and experience, and to ensure that the end result does not produce a counter-productive user experience for the viewer/user, that a certain degree of experimentation with respect to the potential camera position may take place.

The elements of the 360 experience should, in order to optimize it, also be structured in a certain way, taking into account that half of the image from the horizon downwards represents the ground, and half of the image from the horizon upwards, the elements that are related to the sky.

It is also important to establish a main target, which would be the most important element of the scene. This is already determined by studies such as those of Durán Fonseca et al. (2021) [10], in the so-called Levels of Narrative Interest, with Level A corresponding to the greatest importance of an element for the plot, Level B corresponding to the secondary elements, and Level C corresponding to the virtual landscape, which forms the basis of the plot. Thus, examples of the Main Target, the Horizon, and an approximation to the FoV, can be seen in Fig. 3.

Fig. 3.
figure 3

Source: Taken on 4–08-2021 by one of the authors. Own elaboration.

360º photograph of the Church of San Pedro Apostol, in the Polvoranca Park in the Community of Madrid, in which the main target, the Horizon, and the FoV are reflected.

Full size image

Other fundamental elements also appear, such as the equivalent of the poles, here called Pole A (up) or Pole B (ground or down), which are those that present a greater degree of distortion. This distortion is equivalent to that which relates to the equalization of the dimension of a point (the pole) with that of the equator, so that this image can be represented in a plane (Fig. 4). It is also important, and related to this, to note that what appears behind the user in the 360-degree experience, in the equirectangular projection, is on both sides (indicated by the back icon, Image 4). Also shown are the scene targets, corresponding to Level B within the Levels of Narrative Interest developed by Durán Fonseca et al. (2021) [10].

Fig. 4.
figure 4

Source: Taken on 4–08-2021 by one of the authors. Own elaboration.

360º photograph of the Church of San Pedro Apostol, in the Polvoranca Park in the Community of Madrid, including secondary references.

Full size image

In the subsequent image (Fig. 5), this increase in distortion towards the poles is also shown in the previous paragraph. This is shown in the orange gradient, where a higher degree of orange illustrates a higher degree of horizontal length distortion, being more pronounced at the poles. Also shown - in green - are estimates of the approximate potential vanishing points of the image, estimates and calculations addressed by studies such as those of Youjin et al. (2018) [25] or Oh and Jung (2012) [26], which were among those that contributed to determining the estimates of vanishing points in equirectangular images taken with 360 filming devices, and which vary depending on the position in which it was taken, and the elements of the scene.

Fig. 5.
figure 5

Source: Taken on 4–08-2021 by one of the authors. Own elaboration.

Structure of the projection of components in a 360-degree image.

Full size image

Another fundamental component, which must also be taken into account when creating 360 videos with motion graphics techniques, is equivalence, which has already been mentioned in previous paragraphs. Although some 2D motion graphics editing software allows a view of the corresponding part of the user’s field of vision, when making the equirectangular projection on a flat image, it would be necessary to take into consideration the correspondence of the elements in which this equivalence occurs, and which are cut off on the left and right in the equirectangular projection (Fig. 6).

Fig. 6.
figure 6

Source: Taken on 4–08-2021 by one of the authors. Own elaboration.

Structure of the projection of the components in a 360 image, showing the horizontal equivalences.

Full size image

It is also important to take into account the layers of information that can be included in a 360-degree video creation, which would include the possibility of producing 360-degree videos partly with 2D and 3D motion graphic techniques.

These layers of information would basically be three, although it is possible to include as many layers of information as desired or necessary, always within these typologies. They can also have single-layer typology.

These layers correspond to the following, as can also be seen in Fig. 7:

  • Layer A. This represents the part of the physical world filmed, the video. Any effects added to the video would also be included in this layer.

  • Layer B. This represents a virtual world designed with 3D computational tools, or with single elements (characters, objects) that would be embedded in that virtual world (which would be a layer B frequency) or embedded in the physical world (which would be an insertion of a layer B within a main layer A).

  • Layer C: This represents computer graphics created with 2D design tools. The creation process differs significantly from that of the elements belonging to layer C, since in this case it does not involve “modelling” with 2D tools. However, it is a projection of artificial two-dimensional elements onto a simulated three-dimensional world (360 video), but which is actually a projection (the equirectangular one). Another fundamental feature of Layer C is that it can contain elements that have been previously designed in Layer A (from reality) and Layer B (from modelling), these 2D editing techniques being a kind of “final art” of the different elements that are going to be projected on a plane that will simulate a 3D experience.

Fig. 7.
figure 7

Source: Own elaboration.

Layers of information that can be included in a 360-degree video, composed of filming (Layer A), 3D computer graphics development (Layer B) and 2D computer graphics development (Layer C).

Full size image

So, for example, a 360-degree video shot on film would represent a single layer (usually) within the Layer A category. A 360-degree motion graphics video created with 2D editing technologies would consist of a succession of C Layers, as these elements have been created from computer graphics with a limited level of reference to reality. A motion graphics video may also have been made with 3D computer graphics software, which would include a Layer B, if the rendering process has been done with such software, or a succession of Layers B, if it has been rendered in separate 3D computer environments. If this 3D environment generated by computational means with 3D creation software had a subsequent process of post-production in part, or the inclusion of graphics with other computational means, it would also include a Layer or succession of B Layers.

In this way, this whole procedure helps us to structure the information that must be present in a medium that is constituted as a 360-degree video, whose possibilities of representation are of great relevance, always taking into account its linear narrative features and the limitation in the interaction of the user/viewer with the virtual environment in comparison with other sub-categories of virtual reality.

5 Factors Related with User Experience in 360 Videos: What Do Spectators/Users Watch and Interact with?

One of the main questions relating to 360 video production is how the users interact with information and pictures shown in them. Factors such as agency or grounding are not variables to be taken into account. Nevertheless, 360 videos are an ERP projection of a position of the spectator/user in relation to the environment, which includes the time factor (events are occurring).

This ERP projection is going to allow the user, as explained, to look and pay attention to events or actions chosen by him/her. But what will the events or actions that are going to draw the attention of users be? And what will the facts which are going to help create atmosphere in the production be? Which factors should be adapted from the 2D cinema, and which ones from disciplines such as user experience?

One of the advantages of motion graphics is that it can also be produced by computational technologies, and, for that reason, it is not necessary to prepare the environment in the same way that we prepare to film a real environment with a 360 camera (with all the difficulties involved). However, this real-world filmed environment may also be a ground for researching the way users interact with real-world situations, and how that may be adapted to research in virtual reality, including 360 videos.

There has always been a need for user experience research in the field of virtual reality since this has been conceived as a medium. As the medium has evolved and became more accessible to users in the last decade, the 2010s, research into user experience in this area has increased dramatically. Rebelo et al. (2012) [27] identify the methods for evaluating user experience in the context of a virtual reality experience, as well as how research into UX may also take advantage of a virtual reality-based research methodology. Kuliga et al. (2015) [28] approach the medium of virtual reality as an empirical research tool in user experience, interaction design and human-computer interaction, by identifying some variables to be applied in research (environment, environmental appraisal, etc.). Meanwhile, researchers such as Rubio-Tamayo & Gertrudix (2016) [29] have also managed to design a taxonomy of levels of interaction between users and virtual reality environments, keeping in mind narrative factors and focusing on the interactive features of the medium in those levels. Kim et al. (2020) [30] identifies the diversity of types of interaction between users and devices in VR systems (which includes 360 videos), arguing, at the same time, that there is a significant lack of research on the taxonomy that may recognize the main features of virtual reality systems related with the factors that define the user experience. Shott & Marshall (2021) [31] explore, more recently, the effectiveness of virtual reality in fields such as education, by applying in their research a theoretical user experience framework. Wienrich et al. (2018) [32], on the other hand, undertake research giving relevant insights into relations between general aspects of user experience and virtual reality-specific ones. Those insights were obtained by applying the following four main research and analysis methods: the analysis of the evaluation requirements for a large-scale multi-user case, the relationship between evaluation concepts and features from the research fields of virtual reality, on the one hand, and user experience research, on the other. The other research and analysis methods applied by Wienrich et al. (2018) [32] are the subsequent testing of the relation between user experience and virtual reality and the integration of measurements and standards from both research fields and, finally, the discussion of implications for a holistic evaluation framework in both domains.

User experience and interaction design have therefore been applied to the subcategory of virtual reality, represented by 360 videos and films. Authors such as Keskinen et al. (2019) [33] focus their research on exploring the spectator/user’s viewing experience and determining that this experience may be affected by the user/spectator’s ergonomic position itself, the proximity of actions happening in the story, and the camera height, whose optimal position is 150 cm above the ground. Fan et al. (2022) [34] focus their research on developing a model for watching 360 videos by aiming to quantify the named Quality of Experience (QoE), and identifying various factors, features and variables in the context of QoE. Broeck et al. (2017) [35] conduct a comparative study focused on user experience in 360 videos on mobile devices by using different interaction techniques between user and the virtual information shown via the device. Those interaction techniques, in contrast with those observed in interactive and immersive virtual reality (which does not include 360 videos) are focused on the spectator/user’s view and points of interest for him/her, as those users do not have such a complete interaction as with information in 360 videos. On the other hand, other researchers such as Somrak et al. (2019) [36] conduct a study focusing on VR sickness by measuring the levels of discomfort in a virtual reality environment (based on 360 videos) and the user experience applied to those environments, showing a negative correlation between those factors: the increase of the user experience factor decreases the VR discomfort in an immersive experience.

Could all those factors determine what the optimal way to design motion graphics for 360 videos is? Of course, those factors should be kept in mind by content developers in creating experiences in 360 videos, and, also by researchers in getting users to interact more efficiently and comfortably with virtual environments and improving the factors determining the research into user experience.

In 360 videos, one of the main actions of the users/spectators is to look around the virtual environments while the events are happening. The user/spectator takes an active role, by looking around, and a passive role, as he/she cannot influence events or move freely around the virtual environment. And user experience for 360 videos should have specific features and take into account specific factors designed for user experience in virtual reality.

The user experience applied to 360 videos, in this case, as in interactive virtual reality, should also take into account different levels of interaction, by focusing on two main approaches: the user experience of the technology itself (related to the interaction between user and devices) and the user experience of the virtual environment (how information is represented by computational means and how users interact with it) (Fig. 8).

Fig. 8.
figure 8

Source: Own elaboration.

Factors to take into count for user experience and interaction design when designing a virtual environment in the 360 video format. A model proposed by the authors.

Full size image

6 Conclusions

360-degree videos are a format within virtual reality with its own characteristics, among which the linearity of the narrative and the possibility of viewing with or without complete audio-visual immersion stand out. Furthermore, the two senses involved, unlike other subcategories within virtual reality, are sight and hearing, although this does not mean that other senses cannot be added to the experience. On the other hand, there is the lower level of interaction with the medium with respect to other subcategories within the aforementioned virtual reality, and the aforementioned limitations of movement through virtual space (which are limited to a fixed position or a directed linear displacement, although the field of vision or FoV can be fixed in any desired position).

On the other hand, we have a wide range of creative and content development possibilities, taking into account the structural limitations of the video format. We also have a medium in which the potential creation of content, knowing the operating parameters, is relatively simpler than in other sub-categories of virtual reality, since it does not require such a sophisticated design of the potential interactions with the medium. Nevertheless, it is advisable, as this research shows, to be familiar with the operating structure and some of the elements in order to optimize the development of such content.

Just like the conventional video we have known so far, 360-degree video has a long way to go in terms of configuring itself as a medium with its own production, functioning and interaction dynamics with the user/spectator.