Keywords

1 Introduction

Our experience of the surroundings and the environment we live in is as passive observers limiting the engagement potential to understand sustainable processes and how specific transformations and behaviors may impact our everyday lives. AR offers an interface between reality and the perception of reality, bridging the gap between the real and virtual worlds. AR provides experiences enabling end-users to move from observation to immersion which is often associated with the encouragement they experience in the digitally enhanced setting [1]. AR’s immersive nature helps the audience see details, believe in actions, and connect events in a virtual story and their own lives.

For a seamless interaction between the physical and virtual environment of the end-user, a virtual agent can be employed for performing tasks supported by AR but also improve social interactions and motivation [2]. AR typically involves providing augmented capabilities related to objects and locations recognized in images or videos of the real world captured by an AR device. The presence of an agent can significantly improve the tasks involved with these processes. At the same time, the thoughts, feelings, and behaviors of end-users may be positively influenced. Related to user engagement, embodied agents can achieve higher social presence (as measured by self-report or behavioral data) by successfully integrating virtual content with the real world [3].

However good AR may be, adoption from the end-users is one of the most common issues this technology faces. People have demonstrated attraction to new ways of interaction as they like to communicate and share information as freely as possible. Still, at the same time, they judge harshly projects not well-executed, with poor usability, and not interesting content [4, 5].

The proposed mobile AR application fosters an environment of understanding and addressing the specific needs of each end-user, providing alternative representations and meta-data, fostering interactivity, and offering personalized information and guidelines. Attraction to content, acceptance of the technology, as well as usability and novelty of the proposed solution have been validated in Katika et al. [4, 6]. The present study emphasizes the adoption of the virtual agent supporting the AR functionalities and is organized as follows. First, work related to AR and virtual agents is presented. Then, the system design, the most dominant features (i.e., virtual agent), and the evaluation methodology are described. Before concluding and elaborating on the future research efforts, we present the results from our analysis.

2 Related Work

AR is currently being utilized across disciplines for end-user engagement and has been described to amplify educational success and motivation. Related to end-user engagement that is often associated with these parameters, mobile AR’s adaptable nature can comfort the limitations that other engagement tools face. Immersive technologies have demonstrated a significant effect in educating end-users and offering an inclusive environment for people having a wide range of specificities [6].

Many researchers have elaborated on the use of AR in citizen participatory practices emphasizing the design and technical features but failing to communicate the adoption of their tools (e.g., [7, 8]). To bridge this gap, Katika et al. [4, 6] investigated the acceptance of AR as a citizen engagement tool aiming to empower and educate towards sustainability and circular economy practices. The results were promising, demonstrating that external variables, such as the end-user’s tech-savviness, age, education level, and gender, did not impact the acceptance of AR technology.

To contribute to the research activity performed in end-user engagement utilizing AR technology and further define its acceptance, we propose the enhancement of gamification and educational mechanisms to improve interaction with AR. In this course of action, building a system with a virtual agent can influence how end-users view and use a system as the system can take on a more social role [9]. The agent’s role is to influence users’ perceptions and interactions as they treat the system more as another person and less as an impersonal tool. At the same time, the presence of an agent can leverage both verbal and non-verbal communication to achieve higher engagement levels and change users’ perceptions and behaviors.

Barakonyi and Schmalstieg, [10], claimed that agents in mobile applications enrich interactions in AR. The authors detailed the design and implementation of agents in AR but did not elaborate on understanding users’ reactions to them. Studying the user agent interaction, Kim and Welch, [11], described how an AR agent is situated in, and interacts with, the world affecting user’s experience increasing a user’s sense of presence.

Therefore, it is apparent that one of the main properties that have been proven to enable the adoption of virtual agents is the feeling of being with another person (co-presence). As Bevacqua et al., [12], reported this element should always be evaluated using a combination of objective and subjective measures (such as questionnaires or physiological responses, respectively).

3 Methodology

In this study, we present the design and development efforts of an AR agent as well as the results from its demonstration to investigate its adoption and effect on end-user engagement and motivation towards sustainability practices. We built a generic model for end-user engagement and customized the augmented content to sustainability and circular economy practices to make it more relatable to the participants involved. The evaluation process took place through active demonstrations and web surveys.

The mobile AR application presented in this section supports triggered AR content in three ways [13]. The supported marker-based function requires a marker (QR) to activate an augmentation related to media. Second, the location-based AR functionality uses the device’s GPS location as a trigger to pair dynamic location with Points of Interest to augment relevant content. Finally, object recognition is a dynamic augmentation process based on trained Deep Learning Models.

Fig. 1.
figure 1

The overall architecture of the mobile AR application.

Full size image

Figure 1 shows the architecture of the mobile AR application (CirculAR) that contains three major components, including input, database, and an output engine. The app accesses the location of the device. In marker and object-based augmentations, both location and images are imported into the Content Management System to compare with the database. Then the output engine outputs the matching content on the screen of mobile devices.

3.1 Main Functionality and Use

The developed AR app acts as an engagement tool aiming to engage citizens in sustainability principles and practices, while empowering a sense of action towards this adoption. The main goal is to motivate and educate end-users around the notions of a living model supporting sustainability. Virtual content, consisting of 2D and 3D items, textual information, graphs, and quizzes, targets a wide range of end-users, aiming to motivate and engage them while improving their self-efficacy.

So far, the AR app has been used in Katika et al., [4, 6] to inform and engage the end-users on integrated sustainability frameworks that can be applied to our daily lives, in order for them to be able to: (a) identify their role in the water and waste value chain, (b) reflect on their part in the general picture of integrated water and waste management; (c) and to promote the idea of sustainability interventions as part of a broader awareness policy to ensure efficient and long-term viability of the resources.

3.2 System Design

The overall system architecture shown in Fig. 2 is designed to support two different users: the AR Content Management System (AR CMS) administrators and the mobile application users (CirculAR). The administrators produce and add content in the platform to create meaningful and educational experiences, so-called AR campaigns, that will be later enhanced with various gamification aspects and visualized by the mobile application end-users [4, 6].

The mobile AR application has undergone further improvements to support GPS localization and QR scanning for content attachment, but also has enabled object and image recognition functionalities ensuring a more in-depth penetration of the end-user to their environment. The virtual agent (ARis) is present in most functionalities and offers helpful tips and comments upon execution.

Fig.  2.
figure 2

The system design of the AR app supporting three means of interaction of the end-user with their environment through the virtual agent shown to the right side of the image.

Full size image

Figure 3 summarizes the main functionalities of the mobile AR app, starting from registration, following with tutorial and help functions, selection of content, navigation to the nearest AR experience, interaction with the environment through the camera and object manipulation, object recognition, and the gamification and educative features [4, 6]. The functional features, describe the main services that the mobile application offers to the end-users as well as the main AR capabilities of our design. The gamification features, describe the features aiming to improve the user engagement, learning effectiveness and the educative character of the AR app.

3.3 Virtual Agent in AR

The virtual agent is introduced upon logging in to the mobile AR app where the agent presents the mission to the end-user. A tutorial page describes his name (ARis) and the places in the app where he is expected to guide the end-user to ensure that they benefit from his presence. Figure 4 shows four UIs where ARis provides helpful tips to the end-users, urging them to interact with their environment in three ways (GPS tracking, QR scanning, and object recognition).

The virtual agent is activated along with the AR features of the mobile application (e.g., camera activation for content visualization and manipulation). ARis uses textual communication to communicate with the end-user. Upon introducing ARis, it becomes apparent to the end-user that even if the interaction with the app becomes ambiguous, interaction with ARis will enable further steps. The virtual agent is programmed to react differently in all three user-environment interactions (GPS tracking, QR scanning, and object recognition), shedding light on follow-up activities to unlock the attached media.

4 Evaluation

There are challenges in performing usability evaluations on AR systems considering that they cannot rely on design guidelines for traditional UIs [14]. Evaluation is heavily dependent on the characteristics of the end-users (such as, novices, experts, casual users, frequent users, children, adults, elderly, etc.). We chose to pursue one of the proposed AR user evaluations to assess the effectiveness of the presence of the virtual agent [14]. Following the recruitment and demonstration from Katika et al., [4], we assessed how the presence of a virtual agent affected several engagement attributes of the participants involved. Information regarding the demographics and validation of measures can be found in the same paper. The AR virtual agent survey consisted of 6 items presented in Table 1. The Likert Scale was used to demonstrate the agreement of the participant with statements from “strongly agree” to “strongly disagree”. At the same time, we assessed how the exposure of the participants to similar AR tools and mobile games, affected the virtual agent attributes (Table 2).

Fig.  3.
figure 3

The functional and gamification features of the AR engaging tool demonstrated through the user interfaces (UIs). a) the registration page (also available through Facebook), b) the profile editing (avatars are provided for further customization), c) the tutorial explaining all functionalities, d) the available campaigns with AR experiences and the distance from the end-user (the same UI displays the badges earned for each campaign as well as the overall score earned), e) the in-app map that navigates the end-user to the content of their selection (the camera button is activated as soon as the end-user reaches the AR experience), f) the camera activation from a GPS-enabled AR experience, g) the camera activation from a QR-enabled AR experience, h) the camera activation for object recognition, i) content visualization and j) content manipulation, k) a quiz and l) the user’s leaderboard setting.

Full size image
Fig.  4.
figure 4

a) The agent after login, and b) its functionality shown through tutorial. c) Prompting the end-user to assess an AR experience through QR scanning, and d) object recognition.

Full size image

The participants were exposed to two functionalities of the virtual agent: a) helpful tips during GPS-enabled experiences, and b) guidance to scan a QR code.

The demographics and other characteristics of the participants, as well as the recruitment and survey process, can be found in Katika et al., [6]. Based on the feedback of 127 participants, we assessed that the engagement attributes under investigation were positively affected by the presence of ARis during the entire AR demonstration. Figure 5 shows all the trends in the engagement attributes.

Table 1. Survey items assessing how the virtual agent (ARis) affected the AR experience.
Full size table
Table 2. Summary of survey items assessing the tech-savviness and the educational level of the participants.
Full size table
Fig. 5.
figure 5

The effects of the presence of ARis on the a) enjoyment and b) dedication to AR related to the user engagement attributes.

Full size image

As demonstrated in Fig. 5, end-users scoring high on the engagement attributes (mentioned at the y-axis of each graph) declared that ARis made the AR experience more enjoyable and improved their dedication. End-users who scored low on the engagement attributes did not demonstrate the same acceptance of the virtual agent. The most dominant fluctuations are noticeable in terms of usability and topic interest. For end-users failing to understand the usability of the AR app and the sustainability practices presented, ARis did not improve their experience and dedication. Considering the novelty of the AR tool aiming to improve engagement in sustainable and circular practices, its acceptance has been identified before this study and reached more than 80% during the demonstration [4]. Therefore, it is apparent that evaluating the acceptance of such a novel technology prior to building upon more features and reaching a broader audience is crucial.

Fig. 6.
figure 6

The effects of AR and mobile game familiarity on the attributes of the virtual agent.

Full size image

We proceeded by further showcasing the impact that familiarity with AR and mobile games had on the perception of the virtual agent. From the analysis presented in Fig. 6, it is shown that end-users more confident in both AR and mobile gaming were affected by the gender of the male-like virtual agent but also demonstrated higher dedication and enjoyment due to his presence. The preference towards a more human-like virtual agent did not show any significant fluctuation.

5 Conclusions and Future Studies

In this work, we reported designing and developing a user engagement mobile AR application coupled with a virtual agent supporting AR functionalities. End-user evaluation on the virtual agent reported an increase in the dedication and enjoyment of the AR app and demonstrated an increase in engagement attributes.

The contextually relevant tasks are recommended from the virtual agent to the user as an augmented message via text. Further implementation is required to enhance accessibility and inclusiveness in this field, as audio or other interfaces may allow the end-user to experience more virtual agent capabilities. Additionally, further research is required to understand whether the gender of the agent and the physical characteristics influence performance, as has been reported in VR from Makransky et al. [15].

Finally, leveraging AI functionalities and semantic understanding technology, the agent can transform into a virtual assistant to better understand user input, locate the required resource, and respond in an accurate, timely manner.