Keywords

1 Introduction

The novel technology Augmented Reality allows the extension of the physical world by virtual information and is already used in many application areas. AR has an enormous potential and enables novel interaction possibilities, but faces challenges in the development process, including the complex modeling of 3D content and interactions [17]. Furthermore, the relevance of the physical world requires AR user interfaces to comprise virtual and physical artifacts, resulting in a particularly complex development process [1]. Since no standards for AR UI engineering have yet been established [18, 20], each UI concept has to be considered individually. Thus, Ashtari et al. [3] name the lack of concrete design guidelines as one of the key barriers to AR development.

Current AR usability research often focusses on the adaption of established UI engineering methods to AR requirements. For example, sketching has been applied for the conception of AR UIs, especially regarding virtual objects [13, 16] or as a foundation for interaction prototyping [12]. Prototyping itself is applied to iterate and evaluate interaction concepts in order to foster design decisions [14], even using related technologies like Virtual Reality [6].

Only few approaches have focused on the core objective, the formulation of usability guidelines for the creation of AR UIs. Here, most approaches formulate best practices by performing meta analyses of AR applications. In a few cases, this results in usability principles [15], pre-patterns [22] or design heuristics [11], which are often focused on a specific domain, such as educational video games [7], kindergarten applications [21] or industry 4.0 [2]. However, it remains unclear whether a joint set of principles and pre-patterns is actually accompanied by a positive usability and, in addition, which of the defined patterns play a particularly crucial role for the usability. As a first step, this paper evaluates the implementation of a joint pattern model as well as the role of of individual UI patterns for the usability of an AR application.

In Sect. 2, related work of current AR usability research is highlighted. A derived design pattern model is defined in Sect. 3, followed by the presentation of an AR application implementing this model (see Sect. 4). The user-based study evaluating the AR application in Sect. 5 contributes to AR usability research by providing recommendations for AR UI engineering in Sect. 6.

2 Related Work

AR usability research is often concerned with adapting existing design heuristics to AR requirements. These design heuristics are usually a collection of best practices [9] that simplify recurring problems [4]. For instance, Dünser et al. [10] linked user-centered design principles to AR requirements and derived challenges to be considered by researchers. This results in recommendations for AR UI design, like the reduction of cognitive overhead. Nevertheless, the design principles are limited to a small set and remain rather general. As Dünser et al. state, their work should be seen more as a research encouragement and less as a holistic pattern model.

Tuli and Mantri [21] conducted a promising meta-analysis of existing usability guidelines through research exploration, expert evaluation, and the derivation of mobile AR design principles for kindergarten children. The resulting patterns include aspects of cognition, orientation, design, and support. However, this research has a strong focus on a specific target group, so that a transfer of findings to other domains needs to be evaluated.

Our research especially builds on the work of Ko et al. [15] and Xu et al. [22] (see Sect. 3). Ko et al. [15] define AR usability principles by analyzing existing research on mobile applications, tangible UIs and heuristic evaluation methods, resulting in new guidelines to solve AR usability problems. Based on this research, 22 AR usability principles, such as Enjoyment or Learnability, were defined and classified. Xu et al. [22] analyzed academic and commercial AR games and generated best practices for AR UI engineering. Based on this, Xu et al. formulate nine design pre-patterns, such as World Consistency or Landmarks.

Although research regarding the definition of AR design principles exists, approaches are often focused on individual design artifacts or application domains. Consequently, the definition and evaluation of a joint pattern model is an essential first step towards applicable AR usability standards.

3 AR Design Pattern Model

In order to investigate the role of of existing design patterns for the usability of AR systems, design principles were identified and then transformed into a joint model. As shown in Table 1, design principle categories formulated by Ko et al. [15] were adopted. We also distinguish between Usage, dealing with the application operation and environment, Interaction, comprising interaction possibilities within the virtual and physical world, Information, covering the configuration, structure and visibility of information, Cognition, affecting users’ cognitive abilities, as well as Support, including patterns that support the application usage.

Table 1. AR design pattern model
Full size table

Specific patterns were collated from Xu et al. [22] and Ko et al. [15], then reviewed and consolidated into shared patterns based on their core objectives. Here, the patterns Learnability and Predictability were merged, since they both essentially concern the explicitness of UI artifacts. The joint pattern Hierarchy, Navigation & Availability was defined, which determines the structuring of and navigation through information. The pattern Multimodality & Hidden Information was consolidated since the transmission of hidden information is usually tied to the consideration of multiple modalities. Furthermore, User Control and Responsiveness were merged, since they relate to the system’s uninterrupted response to user input and the resulting sense of control for the user. In addition, only those patterns from Xu et al. [22] were extracted which were evaluated as transferable from their AR game origin to general AR applications. All extracted and merged patterns can be found in Table 1, where their origin from Ko et al. (1) or Xu et al. (2) is highlightedFootnote 1. The pattern implementation listed in Table 1 is explained in the following chapter.

4 AR Scribble

In order to evaluate the defined pattern model, the smartphone application AR Scribble was developed, which imitates a physical spray can, allowing users to virtually paint within the real environment (see Fig. 1). Since the feasibility of design patterns depends on the application use case, not all, but as many patterns as possible were implemented (15 out of 21, see Table 1).

Fig. 1.
figure 1

AR scribble

Full size image

A consistent appealing UI design was implemented by following Apple’s corporate design, allowing to reuse empirically evaluated artifacts throughout the application. Within the UI, a centered marker constantly visualizes the currently configured line width and color. By default, the color and line width are set to appropriate values (a well visible yellow tone in a medium line width), but can be configured by the user. Although touch input serves as the primary interaction form, color changing can be performed using a voice command as well. Besides mentioned functionality, only two additional UI buttons were implemented (speech recognition, help menu), ensuring a well organized and non-overloading design. As shown in Fig. 1, the help menu offers further functionality explanations. To simulate the use of a real spray can, the virtual fill level decreases while painting and users are prompted to shake the smartphone to refill the can. Prompts are visually displayed as well as haptically transmitted by vibration and disappear as soon as the paint is refilled. Since this frequently occurs while painting, users develop a feeling for when to refill the can. Besides haptic signals, acoustic output has been integrated, which imitates the sound of painting with a real spray can. As soon as the painting button is released, the acoustic signal stops. Furthermore, painted virtual artifacts are illuminated through adopting ambient light from the real world. The whole application was tested extensively in several iterations, resulting in a stable application.

5 Empirical Study

The application presented in Sect. 4 was analyzed within a user-based study, evaluating the overall usability as well as the role of individual patterns for the usability. In this chapter, we first introduce our research questions, experimental design, and sample, followed by the presentation and discussion of results.

5.1 Research Questions, Experimental Design, and Sample

As stated in Sect. 2, current AR usability research often focusses specific domains or individual UI artifacts. In our study, we wanted to evaluate whether a bundling of design patterns and their implementation are related to a positive usability. Thus, the following research question was formulated: Does the consideration of joint AR design patterns correlate with a positive usability (RQ1)?

In order to identify particularly influencing patterns and thus derive explicit recommendations for AR UI engineering, the role of individual patterns for the overall usability needs to be considered. This results in the following research question: Which of the design patterns play a particularly crucial role for the overall usability (RQ2)?

To investigate these research questions, an empirical study was conducted with \(N = 18\) participants of which 13 were male and 5 female. Participants were on average \(M = 29\) (\(SD = 7.27\)) years old. Within our study, several tasks were performed that guided participants through the full feature range of AR Scribble to ensure that all implemented patterns were noticed. First, participants were asked to paint their initials into the real environment with free choice of color and line width. Next, participants outlined their painted initials with a thick red and a thin blue line. These tasks were performed under controlled conditions, within the same physical room under identical lighting conditions using the same device.

Afterwards, the System Usability Scale (SUS, see [5]) was surveyed to evaluate the overall usability of the application, rating statements like “I think that I would like to use this system frequently." on a 5-point Likert scale ranging from 1 (“Strongly disagree") to 5 (“Strongly agree"), resulting in a cumulative SUS score on a scale of \(0-100\). To evaluate the role of individual patterns for the SUS, the individual pattern implementation was evaluated (e.g., “I think the pattern ’Default’ in AR Scribble is well implemented (initial color and line width)") on a 5-point Likert scale ranging from 1 (“Strongly disagree") to 5 (“Strongly agree"). Finally, the prior AR experience was surveyed (“How much experience do you have in operating AR applications?") on a 5-point Likert scale ranging from 1 (“None") to 5 (“Very much") in order to consider an influence of previous AR experience on the usability evaluation.

5.2 Results

The overall usability was calculated and results in a total SUS score of \(M = 80.56\) (\(SD = 11.26\)), with a minimum single score of 47.5 and a maximum of 97.5. The relationship between individual patterns and the overall SUS score was evaluated through correlation analyses, based on the individual implementation ratings. As shown in Table 2, the patterns Enjoyment (\(M = 4.50, SD = 0.51\)), Consistency (\(M = 4.50, SD = 0.62\)) and User Control & Responsiveness (\(M = 4.50, SD = 0.62\)) were rated as particularly well implemented, whereas Physical Effort (\(M = 3.83, SD = 1.09\)), Device Metaphors (\(M = 3.94, SD = 1.06\)) and World Consistency (\(M = 3.94, SD = 0.99\)) were rated slightly lower. The correlation values indicate, that Control Mapping (\(r = .497, p = .036\)) and User Control & Responsiveness (\(r = .486, p = .041\)) show a significant positive correlation with a small effect sizeFootnote 2 with the SUS. The patterns Enjoyment (\(r = .558, p = .016\)), Hierarchy, Navigation & Availability (\(r = .521, p = .022\)) and Multimodality & Hidden Information (\(r = .521, p = .027\)) show a significant positive correlation with a medium effect size. Finally, the pattern Consistency (\(r = .718, p = .001\)) shows a highly significant positive correlation and a large effect size.

Table 2. Correlation results regarding pattern implementation and SUS
Full size table

In order to assess a possible influence of prior AR experience on the usability evaluation, the AR experience was correlated with the SUS score. Participants reported a prior AR experience of \(M = 2.54\) (\(SD = 1.25\)) and the correlation of the SUS score with the prior AR experience showed no significant result (\(r = -.199\), \(p = .429\)).

5.3 Discussion

The usability evaluation of AR Scribble resulted in a SUS score of 80.56. Since meta studies found that a SUS score between 78.9 and 80.7 is considered as a “grade A-" usability (based on the american school grading system, see [19]), the usability of AR Scribble was rated as particularly good. Thus, underlying assumptions of RQ1 were confirmed, since the consideration of a joint model of current design patterns was positively related to a positive usability.

Analyses regarding individual patterns showed a strong significant correlation between Consistency and the overall usability. It is conceivable that users seek consistency in order to master an application, especially when familiarizing with novel technologies. This assumption is supported by the significant correlation coefficients for the patterns Hierarchy, Navigation & Availability, Control Mapping and User Control & Responsiveness, since their objectives also promote clear structures, well-organized interfaces and uninterrupted use. The significant effects for the pattern Enjoyment strengthens this assumption, since an appealing design thrives on clear structures and well organized information. Additionally, the results implicate that AR UIs should integrate multiple modalities, since the pattern Multimodality & Hidden Information was significantly positive related to the SUS score, likely facilitated by the increased freedom of use. Thus, the underlying assumptions of RQ2 were confirmed by revealing particularly crucial patterns.

Nevertheless, some limitations of the empirical study need to be mentioned. Although it can be assumed that the implementation of individual patterns had a positive influence on the SUS, the causality could also be reversed. Future research should investigate causality effects by means of long-term studies or more complex experimental designs. Additionally, the design pattern model focused on two main sources and was evaluated through a single smartphone application. Subsequent studies should extend the pattern model with further research (e.g., [2, 11, 21]) and evaluate additional AR interfaces to exclude influencing effects as well as to investigate non-evaluated patterns.

6 Conclusion

We explored design patterns for AR systems by evaluating a joint design pattern model, implemented through an AR application. A conducted user-based study indicates that the implementation of these design patterns lead to a positive usability. The examination of individual patterns revealed that patterns concerning the consistency, structure and organization of UI artifacts were particularly influential, emphasizing users’ need for clear structures and interactions in new technologies like AR. Our results further indicate that users seek freedom in their choice of interaction methods.

For future work, several interesting research options remain. We plan on extending the defined pattern model through including further AR usability publications. Additional AR applications will be developed to evaluate our findings. Finally, other related research questions will be addressed, such as the transfer of results to other device groups, as well as the consideration of UI complexity and application domains as influencing factors.