Keywords

1 Introduction

3D interactive technologies in the form of extended reality (XR), such as augmented and virtual reality (AR/VR), are reestablishing themselves as powerful tools for multimedia content creation [6]. With the recent reinvigoration of XR (AR/VR) [2], the potential of this technology to influence the next generation of users and disrupt current content-creation pipelines is potentially vast [22]. In the last decade alone, AR and VR technology have exploded back onto the media technology scene in a way that will permanently disrupt home media consumption and current production workflows. With this latest wave of growth, interest has been expressed in introducing new real-time 3D pipelines into multiple areas of the media industry. For example, 2D visual media and real-time 3D interactive tools can be seen and applied across various emergent “metaverse” applications. When combined with 2D visual media, real-time 3D interactive tools can find application across various broadcast-industry sectors, from content creation to studio set design.

At the same time, the modern classroom is beginning to employ these novel technologies to educate and inform tomorrow’s industry leaders and provide further training and support for the existing workforce [24]. In higher education, industry partnerships are fundamental for creating a pipeline of skilled operators for future studio productions and maintaining a suitably educated workforce. Thus, with the latest XR technologies, interest has been expressed in introducing emergent real-time 3D pipelines into multiple areas of the media training sector. However, how advanced and soundly these tools fit within current industry workflows is still debatable.

Human-computer interaction (HCI) and XR technology need further combined discussions to provide more informed guidelines for creative practices via 3D technology. While the potential of IVEs for co-creation is quickly becoming established, user-focused presentations via XR technologies fail to account for user perceptions of this technology’s past, present, and future in learning and industry practices. Therefore, we seek to explore the current differences observed between XR users and emergent 3D content creation tools, focusing on user-type-specific experiences when undertaking tasks with 3D animation and modeling software. The motivation for collecting this data was to provide the XR industry with an enhanced understanding of the potential future of this technology. Moreover, this data collection approach gave our XR application creators and developers a more empathetic understanding of the people they are designing for and their attitudes toward using this technology.

In the presented study, we aim to discover more about the experiences and familiarity of 3D technology by users in the field, gather information about the current requirements of contemporary 3D visual artists, and report on the potential future of 3D content creation from a broadcast industry perspective “in the wild” [16]. It is expected that this cohort will provide insight to inform current and future XR practices. Moreover, this paper explores real-time XR immersive content creation tools as an alternative to traditional desktop hardware and software for future 3D digital artists and industry creatives, revealing the nascent opinions of this cohort of new users towards this budding technology. As such, we present our findings from the standpoint of different users – novices, end-users, and advanced users – and report on their perceived impact on the current and future media industry trajectory.

2 Background

Immersive VR technology has been around since the early 1980s [6] and presents users with digital computer-generated 3D environments that simulate the physical world via audio, visual, and even haptic stimuli [1]. This technology has been observed in educational contexts [19], gamified medical experiences [20], empathy building [23] and aircraft training [21]. Milgram and Kishino’s virtuality continuum [10] presents a seminal field taxonomy that outlines mixed reality (MR) technology classification. In most modern XR systems, the user wears a head-mounted display (HMD) and holds a hand controller that facilitates interactions with 3D content in an immersive virtual environment (IVEs). The synchronicity between the IVE and the users’ movements is critical in creating immersion and a sense of “being there” [15].

Currently, shared IVEs facilitated via AR/VR technology allow users to interact, collaborate, create, and manipulate virtual objects using familiar hand-based apparatuses [4]. The flexibility of such systems allows them to be easily used for new and existing design applications within standing media industries. Applying simple, instinctive interfaces makes these applications easy to use and facilitates constructivist and exploratory learning by building knowledge and understanding through applied practices [24]. Currently, XR enables the creation of artistic content (e.g., Tilt Brush, Quill, Medium, Blocks, etc.) and aids many new forms of creativity and interaction. Although artistic creation via XR can be similar to classical arts practices such as painting and sculpting, others diverge to produce new and unique interactive features. Today, AR and VR are both accepted novel mediums for artistic expression that can effectively deliver intangible cultural heritage content [13, 14, 25].

3D creation tools in XR often rely on intuition and are designed for both novices and advanced users in animation and modeling. This low-entry approach enables creative storytellers without 3D animation software experience to prototype their concepts quickly. Furthermore, XR also involves complete body movements stimulating and embodying physical experiences, allowing artists to “step” into their creations and completely immerse themselves in the creative practice [3]. However, immersive devices also present specific constraints concerning user interface design [5, 18]. For example, the user must recognize the utility of a 3D tool and how to carry out a task with it. Furthermore, locomotion in an immersive environment must be carefully considered to prevent motion sickness [7, 9]. These factors present many novel opportunities and challenges to the realtime-3D interactive content creation field.

3 Methodology

An inquiry process is presented here that involved engaging with different users applying new 3D tools “in the wild” to better understand their experiences in an emergent application context. Thus, a methodology is presented that targets both novice and advanced users with 3D animation and modeling pipelines using XR in various practices. We explore these users’ attitudes and previous experiences regarding animation, 3D modeling, and motion capture involving observing, engaging, and empathizing with creative technologists to gather data on their experiences and motivations to use new tools and build a personal understanding of their concerns, requirements, and challenges when using XR technology in real-world practices. Thus, a heuristic experiment is presented that exposed both novices and advanced digital technology users to innovative animation, 3D modeling, and motion capture technologies designed for traditional screen-based media and XR.

3.1 Participants

Recruitment took place in the Republic of Ireland. A general call for participation was made via the project website, the university network, and an institute of technology. Volunteers were sought across a broad spectrum of potential user types; therefore, this call specifically targeted a 3D modeling and animation group. In total, 33 participants responded to contribute to this research.

Participants were first invited to report demographic information via an online questionnaire. This data included age, gender, education level, and employment sector. The participants were then asked to identify on 5-point Likert scales their general competencies with digital technologies (1 = Unskilled to 5 = Excellent); their familiarity with 3D modeling and animation (1 = Unfamiliar to 5 = Extremely Familiar); and their expertise using 3D modeling and animation technologies (1 = Novice to 5 = Expert). Professional employment status was polled to ascertain at which point in their career these users were currently situated. Participants were then categorized as “Novices”, “End-users”, and “Advanced Users” by the authors, as described by Nielsen [11]. Depending on user-type profiling, job title, and industry experience, participants were asked to attend a day-long practical workshop on using 3D modeling, animation, and motion capture (MOCAP) in an applied classroom workshop or to attend the RTÉ studios in Donnybrook, Dublin (Ireland’s national broadcast headquarters) for exploratory discussions around tech in the industry.

In total, 11 participants attended the classroom workshop, identifying as 6 Males, 3 Females, and 2 Non-binary users with an average age of 21.08 (SD = 1.83). According to the Irish National Framework of Qualifications (NFQ), the education profile of the cohort consisted of levels 6 (n = 5) and 5 (n = 6). All members of this cohort were currently enrolled at an undergraduate level and contributed to the study. In total, 22 participants attended the professional studio session. Eleven contributed to the survey (response rate = 50%), identifying as 9 Males and 2 Females with an average age of 36.73 (SD = 8.81). According to the Irish National Framework of Qualifications (NFQ), the education profile of this cohort consisted of levels 10 (n = 5), 9 (n = 4), and 8 (n = 2). According to the professional Nomenclature of Economic Activities (NACE), the group’s employment sectors were Education (n = 7) and Scientific and Technical Activities (n = 4); this included researchers, Ph.D. students, lecturers, directors, consultants, graphic designers, and tech company CEOs.

3.2 Apparatus and Material

Both sessions were in-person, pre-scheduled, and implemented strict Covid-19 social distancing and hygiene protocols. The novice cohort was invited to workshop XR technology in a classroom setting. Participants were given 3D modeling, animation, and MOCAP tutorials to complete that were delivered via prerecorded videos. They were shown first-hand how to use MOCAP technology for real-time animation using Unreal Engine. The cohort was then asked to model a 3D apple and animate the apple falling down stairs using VR. A lecturer was present at all times to demonstrate and provide additional support if needed.

The industry cohort was invited to experience animation, 3D modeling, and motion capture technology during a group visit to the RTÉ studios. A MOCAP device was demonstrated to the group, including real-time rendering and animation on a large-screen LCD wall display. Participants were encouraged to interact with the demo and ask questions. Following this, participants were debriefed and allowed to ask further questions. Two VR stations (Meta Quest 2 Link) were also set up for experiencing animation and 3D modeling. For these purposes, Tvori and Masterpiece Studio were used as exemplar applications. More information about the core technologies explored is presented below.

Perception Neuron (www.neuronmocap.com): The Perception Neuron Studio is a motion capture solution from Noitom. It boasts industry-leading sensor technology and an innovative sensor-processing algorithm, giving users the potential to explore new approaches to motion capture. This system includes inertial body sensors for full-body tracking, a transceiver, and gloves containing six 9-DOF IMU sensors. The system is self-contained with inertial trackers that provide a precise MOCAP experience.

Masterpiece Studio (www.masterpiecestudio.com): Masterpiece Studio Pro is a fully immersive 3D creation pipeline. It provides a suite of professional, intuitive, and easy-to-use immersive tools. Users can create high-fidelity 3D models and animations using easy-to-use and intuitive tools. The application allows users to create 3D concepts within VR, sculpt 3D assets, and prepare meshes for many different professional workflows.

Tvori (www.tvori.co/tvori): Tvori is an animation and prototyping software for VR and AR that can be used for previsualizations, animatics, or VR films. The application makes designing for both AR and VR simple. This software package can be used collaboratively to prototype interfaces, products, and experiences for XR. It is easy to learn, provides animations for design transitions and user interactions, and can be used collaboratively by remote teams.

After each session and to reduce face-to-face time during the pandemic, the different cohorts were asked to complete an online questionnaire to capture their impressions of these technologies in real-world practical situations. Qualitative data was recorded to add depth of knowledge and explore previous knowledge and experiences, potential benefits and problems of the technology, and future industry-focussed needs. Quantitative data were also recorded to highlight usability issues after the task-based exercises in the classroom.

The following questionnaires operationalized these inquiries. The UMUX-Lite was used to identify usability issues in task-based situations; open-ended questions were then used to expand upon these data: “What previous knowledge or experiences have you had with the animation, 3D modeling, and motion capture technologies you have experienced today?”; “What benefits or problems do you see arising from using animation, 3D modeling, and motion capture technology in this way?”; and “What do you think you would need from future animation, 3D modeling, and motion capture technology like this?”.

Fig. 1.
figure 1

Studio visit workshop and discussion

Full size image
Fig. 2.
figure 2

Students collectively experiencing VR

Full size image

4 Results

Empirical data was collected and analyzed. Quantitative data was used descriptively to report on the cohort demographic. Qualitative data were coded and used to enrich and add depth of knowledge to our aims and objectives. The analyses of open-ended questions took a thematic approach guided by the frequency and fundamentality of the issues raised by the participants [8, 12].

Fig. 3.
figure 3

User types: Workshop participants (left); Professional studio session (right)

Full size image

4.1 Novice Workshop Results

All participants (n = 11) self-reported having a “Good” ability to use digital technology (M = 3.92; SD = 0.79). Data relating to their familiarity or knowledge of 3D animation and modeling (M = 3.00; SD = 1.13) and their expertise or experience in using 3D animation and modeling technologies (M = 2.58; SD = 1.08) were captured to identify their specific user types [11]. The distribution of user types was weighted towards Novice users (n = 6), with three potential End-users and two Advanced users (see Fig. 3 - left).

All members of the group completed the individual tasks (see Fig. 2. The cohort described the usability of the 3D modeling and animation software in practice as follows. The group reported a mean UMUX-Lite score of 78.47 (SD = 21.75). This result was converted to a raw System Usability Score (SUS) to help benchmark the collected data (M = 73.91; SD = 14.14). The percentile rank for the raw SUS score was calculated as 60%, or “above average”, where a percentile above 50% is, by definition, above average [17]. ToT data calculated the average task completion time for 3D modeling and animation as M = 29 min (SD = 10.92).

Table 1. What are the novices’ opinions on using 3D animation and modeling technology in the classroom?
Full size table

Previous Knowledge and Experience: An overall breakdown of the qualitative data analysis for the workshop cohort can be seen in Table 1. Members of this group identified themselves as having little or no practical previous experience with 3D modeling software. Those with some experience listed Blender, Mudbox, Solidworks, 3DsMax, Auto CAD, Procreate, and Dragon Frame on the PC and generally had a mix of 3rd Year, 3D Design, Model-Making, and Digital Art level of expertise. None had previously used VR for these purposes.

Good and Bad Features: The group highlighted some of the benefits of using VR were that it was easy to use, a lot of fun, and they could see additional details that would not translate easily on traditional 2D PC modeling platforms. The ease of use that was experienced enhanced the accessibility of the medium and could potentially attract more people to the industry. It was recognized that the industry climate was very digital-orientated and tech-heavy, so the group felt it needed to know how to use new software and hardware for future employment. Although a lot of time would be required to learn how to use the software efficiently, the visualization of 3D data was thought to be much more enhanced and showed a lot more detail on the final output. The additional use of projection mapping and internal lighting spotlights for the 3D modeling object was considered advantageous for learning as a “visual experience”. This advantage allowed our novices to model and contextualize their work much more quickly than with traditional 2D PC software.

Additionally, it was highlighted how physical modeling and animation projects could easily be damaged in the real world; therefore, a 3D digital model was more mobile and easier to transport between different locations and digital platforms. Ultimately, the group thought that VR for 3D modeling and animation could potentially be a cost-efficient, fast, safe, and an easy way to save their learning progress. While the group offered several advantageous scenarios for the use of XR in the classroom, it was also noted that there might be other disadvantages as well. As one participant commented:

“It’s easy to use and could allow a wide range of people to get into it, but it could potentially lead to less student-to-student feedback and collaboration”.

The problems listed by the workshop group highlighted typical issues that novice users might face when learning to use such software. User-type-specific issues included how some learners might suffer from VR sickness and “get headaches or feel unwell” when using VR for the first time. For some, the user interface or “layout of buttons” was also highlighted as problematic for ease of use. Furthermore, the task was particularly challenging for those inexperienced with such programs. These issues were thought to be easily overcome with accessible tutorials and a more-knowledgeable-other present to help novice users better understand the task scenarios. Interestingly, it was believed that this technology could potentially lead to reduced face-to-face collaboration.

Table 2. What are the experts’ opinions on using contemporary animation, 3D modeling, and motion capture technology in practice?
Full size table

New Features and Requirements: For VR to be more prominent in the future, our workshop users highlighted that more VR headsets would have to be made available for general use. This suggestion also included designated VR areas to be included in classroom spaces. Additional technology integration was also recommended, such as ZBrush and Screen Tablets. Some students highlighted usability problems and that future software would require enhanced “ease of use” features for novices, where the user interface would be scalable in difficulty for different user types. For example, a simple interface for beginners and a more complex one for advanced users—“There should be more straightforward layout selections, for example, cut lines, extrude, etc.” For more streamlined interactions with other classroom software, it was suggested that future iterations have a layout that matches other programs for cross-platform use and “hot-key” shortcuts for fluid switching between platforms. For example:

“A better layout structure that matches other software, nearly cross-platform in terms of use, and hot-key shortcuts to be more fluid when switching between modeling software”.

Although, it was suggested that this could be avoided if software selection existed. It was also recommended that tutorials be delivered inside the HMD and made more straightforward and accessible for entry-level users. Although, this was contradicted as some students preferred to be made “aware of their surroundings” and alerted if someone was approaching them while wearing an HMD. This suggests that “mixed reality” would be an attractive feature for the classroom - a feature released by Meta for the Quest Pro after the workshops had taken place.

4.2 Industry Expert Results

All participants (n = 11) self-reported having an “Excellent” ability to use digital technology (M = 4.73; SD = 0.47) and attended the full session (see Fig. 1). Data relating to their familiarity or knowledge of animation, 3D modeling, and motion capture (M = 3.45; SD = 1.04) and their expertise or experience in using animation, 3D modeling, and motion capture technologies (M = 3.18; SD = 1.33) can be seen in Fig. 3 (Right). The distribution of user types was weighted toward End-users (n = 2) and Advanced users (n = 6) of animation, 3D modeling, and motion capture technologies, with three novices in the cohort.

Previous Knowledge and Experience: A breakdown of the qualitative data analysis can be seen in Table 2. Expectedly, this cohort reported much more industry, lab-based, and educational experiences in 3D modeling and animation using Maya, Cinema 4D, 3D Studio Max, and Unity 3D, and familiarity with more bespoke content creation approaches such as volumetric capture and other 3D reconstruction techniques. Most cohort members had previously experienced different versions of the presented technologies and platforms at exhibitions and conferences or had applied them in some way in their place of work. They also conveyed some real-world experiences with motion capture, although it was the first time some had seen a real-time motion capture and rendering pipeline firsthand. Traditional platforms for content creation were described, PCs, laptops, etc., and some group members were intimately familiar with VR development pipelines. This cohort was knowledgeable of the gaming and movie industry and was more familiar with the technology, terminology, and leading companies involved in the field than the previous group. The more advanced users within the group were skilled in 3D reconstruction algorithms, markerless motion capture, content creation experience in VR, and knew of the Perception Neuron Kickstarter.

Advantages and Disadvantages: One of the significant benefits identified by the industry user group was the use of natural movement for animation and the technology’s overall ease of use; for example, “As part of a complete pipeline, it could allow for quicker pre-visual and broadcast-quality animation”. Each platform experienced and discussed was also advantageous for three-dimensional thinking and creating throughout a project’s lifecycle. This approach to a 3D rationale was expected to benefit the creation of new content that could be produced quicker and more realistic representations of human movement. After experiencing 3D modeling in VR, some participants thought that creating 3D models in a fully immersive 3D environment was more intuitive and ergonomic for the artist than building via traditional 2D screen-based desktop media. As one participant described:

“It feels more natural, like drawing or physically sculpting with physical materials (e.g., stone or wood), than drawing with a mouse. I also like that it encouraged me to stand up while working, which I feel is healthier than constantly sitting at a desk”.

Low entry usability and “comfort-of-use” were considered an advantage for novice industry users, making the business more accessible for more “casual” creatives, for example, more agile and mobile motion capture approaches instead of stricter fixed MOCAP suites. The Perception Neuron Studio captured data from multiple sensors, ensuring precise motion tracking in line with other studio-based operations. The technology pipeline’s quick capture and display capabilities enhanced this feature: “Which means you can solve many problems in real-time”.

The featured technology was also considered beneficial from a research perspective as it facilitates faster and more accessible data collection, allowing users to create more extensive and diverse datasets, which are essential to current machine learning technologies. Another benefit was the production speed, where making television shows with a real-time virtual host could be realized quickly. As one participant reported:

“Integrating the 3D model with Unreal or Unity and the motion capture suit appears very easy and will lead to a more rapid workflow. The motion capture suit is more portable than traditional approaches that use many cameras to track the actor”.

Disadvantages were also acknowledged. Notably, the requirement is to have someone trained and familiar with the technology to guide and troubleshoot potential issues at every production stage. Another drawback was the requirement for high-end computer systems to process data efficiently. Some participants also believed that VR modeling would not be a suitable replacement for working in a traditional 2D environment and would be more useful as a viewing tool or for making minor adjustments to preexisting content. As a product of the Covid-19 pandemic, it was also highlighted that virus transmission could occur when using communal VR equipment and the constant need to sanitize equipment before and after use.

It was also reported that VR and MOCAP technology may still be too expensive. The initial costs to train, set up, and operate and the technical and hardware requirements would continue to make it difficult for general day-to-day use. Specific to the complete 3D modeling, MOCAP, and reconstruction pipeline, participants thought problems might arise concerning the accuracy of tracking live actors within a studio with multiple wireless technologies running simultaneously. In this case, signal quality may suffer due to the lack of external camera tracking and the inherent noise of the different wireless sensors being used. However, it was acknowledged that this potential problem might be addressed through training and familiarity with the system in question.

The Future of Technology: Fundamentally, all participant group members commented on financial costs, a general lack of technology-specific expertise, and the requirement of effective teaching materials for continued growth and acceptance within the industry—“A good set of tutorials as well as someone with expertise that can troubleshoot”. Moreover, although the cohort reported that the technology they experienced looked promising, the rise of virtual production tools will need further development concerning the availability of future skilled operators. On the one hand, a financial investment would be required to replace established traditional studios with next-generation real-time sensors and immersive virtual environments. On the other hand, operating such systems would also require industry-specific training. Therefore, a streamlined course should be created for advanced users to become fully accustomed to 3D control panels and the various available tools.

The group recognized that future growth in this area would require increased access to 3D modeling and motion capture pipelines to be integrated into the classroom to train the next incoming group of studio operators. Thus, the technology would need to be learnable, easily calibrated, and provide a robust data capture process reflective of the era’s industrial applications. The pipeline would also require data to be captured in an open and streamable format to create and drive virtual 3D characters across multiple platforms. However, it was expressed that this technology’s usability, affordability, and accessibility should not be detrimental to its accuracy. By achieving these goals, artists and other enthusiasts would have powerful 3D modeling, animation, and MOCAP capabilities. Future animation, 3D modeling, and motion capture technology (including mobile and wearable devices) will also facilitate and drive creativity and develop new ideas for using them effectively.

5 Discussion and Future Work

Firstly, the study participants could accurately describe and group themselves using a user cube approach [11]. In this way, we could provide a practical workshop for those with no substantial industry or education background in real-time 3D animation. Likewise, we could offer an informative breakdown of current and emergent technologies to the more advanced participants without having to infringe upon their working time with a task-based analysis. The novice cohort of learners reported a favorable usability score and sufficient ToT times.

On the one hand, the novice group had limited previous experience and knowledge of the VR platform they used during the workshop. Conversely, the advanced users had extensive professional experiences with the presented technology. This finding highlights a potential disconnect between the 3D software and media development pipeline between novices and advanced users. A more informed education and industry partnership should be formed to address this lacuna to ensure that new graduates, as novice users, have the correct skills to enter the professional domain.

The most notable similarity between cohorts was the identification of positive usability and user experiences of real-time visualization and creating media in 3D. This finding included complementary factors of learnability, comfort in use, and the naturalness of the creative process as a 3D-from-conception approach to creativity. At the same time, the technology was fun and novel for learners, and the speed and accuracy of the 3D data created were also considered professional standards. However, the disconnect between the novices’ desire for industry-specific skills and advanced users’ also identified a lack of training in new practices. Furthermore, issues relating to cyber-sickness and hygiene were commented upon by both groups.

Likewise, further improvements need to be made to increase accessibility for future integration of this technology in both contexts. This future requirements alignment was apparent from an awareness of both cohorts on the potential ubiquity of the technology and the need for more use cases. For this to happen effectively, there needs to be more work done to integrate existing 3D technologies into new design and creation workflows. Admittedly for both parties, this will require upskilling and financial investment from the industry and new learning materials in the educational domain. By addressing this future requirement, a classroom-to-stage education pipeline will be provided.

6 Conclusion

Within this paper, we explored the application of 3D media technologies by industry creatives and XR developers to develop an empathic understanding of the people these technologies are designed for and their attitudes toward using them in practice. Interests in AR and VR have disrupted the 3D media technology scene in a way that might permanently alter industry-recognized production workflows. In higher education, industry partnerships are fundamental for creating a pipeline of suitably skilled operators for future studio productions. It is the finding of this analysis that there currently exists a disconnect between the educational experiences of novice users and industry professionals concerning the use of this emergent technology. However, recommendations should be garnered regarding how this goal can be achieved moving forward.