Keywords

1 Introduction

Active mobile devices have by now outnumbered the global population with the vast majority being smart phones [1]. Utilizing mobile phones is increasingly becoming a common way of life, as mobile phones are being utilized for both personal and professional purposes. Although smart phones can make the work easier, using mobile phones at work may also distract users from concentrating on tasks and result in less productivity [2,3,4]. This situation is becoming more serious given that mobile phones are carried and utilized by individuals anywhere and anytime, their usage is subjected to various physical and social contexts, the amount of contextual information from smart phone services has been growing, and mobile phone users are living in an increasingly active information environment [5, 6]. Continually keeping awareness of and reacting to the changing contextual information (e.g., the surrounding light) manually is tedious and time consuming. Mobile context-aware systems and services could help mobile phone users maintain awareness of updated contextual information by mitigating the information overload [5, 7, 8].

Although mobile phones with context-awareness are technically capable of capturing contextual information, predicting users’ intention and executing actions automatically without evoking users’ awareness, involving varied levels of users’ awareness in the loop instead of taking awareness away from users have touted as necessary ingredients for ensuring the quality of UX [9,10,11,12]. In the field of automation system, numerous studies have discussed human performance problems (e.g., including vigilance decrements, complacency and loss of situation awareness) in complex, automated systems control due to human out-of-the-loop [13, 14]. Human performance problems may because by over-reliance on automation, human’s role in interaction with automation system as passive monitor rather than active processor of contextual information, and a loss of feedback from the automation system. We assume that providing the passive awareness of contextual information (e.g., via notifications) and maintaining the active awareness of mobile phone users in parallel, are crucial to the success of context-aware mobile services. Consequently, the interplay between mobile phones’ and users’ awareness constitutes a Hybrid Awareness System (HAS) that dictates the behavior of human-computer interaction systems as a collective [11, 12, 15].

This study conceives awareness as a property of both the mobile phone and its user. In a real sense, this study contributes to the working conference on Human Work Interaction Design by dealing both with the analysis of the human work, learning, play, leisure activities and with the design of the mobile interaction [16,17,18]. Our conceptualization of awareness hence departs from contemporary definitions that are tailored specifically for other fields of study. For example, in the field of Computer-Supported Cooperative Work awareness has been defined from the human perspective as “the amount of knowledge that a person has about a topic in particular” [19]. Conversely, within the technically-oriented context-aware community, awareness has been defined as “the ability of the system to leverage the context to provide the appropriate response to the users” [12]. In contrast, our HAS approach strives for the middle ground by theorizing awareness as the encapsulation of both mobile phones’ and users’ Awareness properties. In this sense, HAS should entail: (1) perception or sense for acquiring contextual information (i.e., physical, social, user, and computing contexts) to be utilized subsequently by the mobile phone or user to anticipate and determine appropriate actions; (2) comprehension for interpreting the significance of acquired information, as well as; (3) action or execution for behaving appropriately in accordance with the outcomes of comprehension.

A good UX of HAS demands seamless collaboration between mobile phones’ and users’ awareness by balancing both mobile phones’ and users’ active and passive awareness. Although mobile phones’ awareness promises to reduce the mental workload on users’ awareness by improving the usability of mobile phones and rendering users’ interactions with these devices to be more efficient or less effortful [20], the broad range of contextual awareness afforded by mobile phones could culminate in issues like information overload, loss of control, mental distress, privacy violation, untimely distraction and unwanted interruption [21, 22]. Despite that tremendous effort has been expended to enhance and maintain passive users’ awareness by pushing notifications to mobile phone users during opportune moments [23,24,25], studies have shown that users tend to engage in proactive “checking habits” by performing brief and repeated inspection of dynamic content which can be quickly accessed from the device [7]. Certain users have even developed obsessive-compulsive inclinations by being actively aware of their mobile phones to the extent that they interrupt their ongoing tasks to inspect dynamic content on these devices even when notifications were turned off [8, 26, 27]. Our suggestion is that users maintain a degree of active self-awareness, expecting mobile phones’ awareness to complement and extend the users’ awareness only when the latter is ineffective. The mobile phones’ awareness should not be competing with the users’ awareness by being over-sensitive to contextual information or even wrestling control away from users [9].

Although extensive research into context-aware services (i.e., conceptual algorithms, network infrastructure, middleware, and applications), little attention has been paid to the interplay between mobile phones and users in being aware of diverse contexts from a human-centric perspective [9, 11, 21, 28, 29]. We thus collected data on (a) ‘Turning Screen On’ events to examine the Human Active-Computer Passive mode; (b) ‘Light change’ and ‘Auto-brightness’ events to examine the Computer Active-Human Passive mode, as well as; (c) conducted surveys to ascertain users’ subjective assessment of select context-aware applications. Building on our earlier work [30, 31], where we put forth a preliminary framework to explicate the UX of mobile context-aware systems and services from the users’ perspective based on focus group discussions, this study embraces a mixed-methods approach to unravel UX considerations related to such systems [32]. Consequently, this study contributes to a comprehensive understanding of UX shaped by the interplay between mobile phones’ and users’ awareness, which in turn will inform the quality design of mobile context-aware services.

2 Methodology

2.1 Method

A mixed method was adopted to collect data and Grounded Theory Method was used to develop the conceptual model by this study. A log study and a diary study, along with post-hoc semi-structured interviews, were conducted to understand the phenomenon of the interplay mode between the awareness of the user and the awareness of the mobile phone. The post-hoc interviews and open-ended questions from the diary study were analyzed with an axial coding strategy [33] to explore the causal conditions of the interplay, the strategies of managing awareness, and the consequence of the interplay between awareness of user and mobile phone on UX.

2.2 Participants

The study was conducted with 15 Danish and 17 Chinese Android Phone users, ranging from 19 to 28 years of age. The participants were students, with 15 of 17 Chinese participants from a design department of a technical university in Beijing, and 15 Danish participants from a variety of higher education studies including law, medicine and design. All participants had owned an Android mobile phone for no less than half a year. Each participant was compensated with 350 DKK or RMB for their participation. The research tool was developed based on the Android studio software, and allowed us to capture the contextual information with open APIs.

2.3 Research Phases

Prior to consent, participants were notified of the purpose of the study and the types of data to be collected. The entire research lasted for approximately four months and was carried out in four phases consecutively.

First Phase: Collecting log data of usage behavior about application, sensor data of contextual events and diary data of UX with two tailored tools (MOCCA.Capture and MOCCA.Diary) developed for this study. This phase lasted for a week for each participant (Table 1).

Table 1. Sample of Log Data
Full size table

Second Phase: Analysis of usage behavior, contextual events and diary data. The sensor and log data was analyzed individually by a tool developed with MS function to illuminate the frequency of application usage, physical contextual events and computing contextual events that transpired during the 7 days in the first phase. Secondly, participants documented their usage experience with 30 separate mobile context-aware applications that had been sampled by us from the participant pools mobile phones current configurations, rated their satisfaction in interacting with these applications, and articulated the reasons behind their rating.

Third Phase: Questionnaire survey and post-hoc interviews were conducted to understand the phenomenon unveiled by user behavior, contextual events and diary study. Users were asked to rate their feeling of 30 context-aware services if they having experience of using them.

Fourth Phase: Analysis of survey data. The Grounded Theory method [33] was used to guide the coding of raw survey data to develop our conceptual model, and to revisit the interview transcripts to discover what interactional strategies were used by user to manage and handle the interplay between awareness of user and mobile phone in the mobile and dynamic contexts. Finally, the influence of the interplay mode between awareness of user and awareness of mobile phone on UX were analyzed.

3 Results

3.1 Active Human and Passive Computer Awareness

Turning the screen on by a user represents a typical type of interaction event initiated through active users’ awareness (human awareness). We analyzed the relationship between preceding and follow-up usage sessions of the ‘Turning Screen On’ event with the aim of uncovering how the interaction initiated by active users’ awareness disturbed the ongoing status of users. Results indicate that 519 (49.7%) of the 1046 usage sessions following Turning screen on is ‘Turning screen off’, 52% of the time from Turning screen on to Turning screen off is 5–15 s (Fig. 2), followed by less than 5 s (19%), 15–30 s (11%), longer than 70 s (10%) and 30–70 s (8%). By comparison, 36% of the other 526 (50.3%) events, which transpired after Turning screen on, were completed within 5 s, followed by 5–15 s (34%), 30–70 s (14%), 15–30 s (9%) and longer than 70 s (8%) (χ2 = 11.56, p < .05). These results imply that users, after turning on the mobile phone and checking the content for 15 s, display awareness in turning off the screen for around 35% of the situations. Further analysis of the preceding and follow-up usage sessions (104 of 1042 events with zero-time duration between preceding usage session and Turning screen on were excluded) of Turning screen on events illustrated that 461 (49%) of these events encompass similar behavioral patterns. Notably, it is easier for users to revert to their original status when the preceding usage status is Screen Off (64%) (χ2 = 21.835, p < .01) (Fig. 1).

Fig. 1.
figure 1

User behavior related to active awareness

Full size image
Fig. 2.
figure 2

Time duration from turning-screen-on to follow-up usage session

Full size image

3.2 Active Computer and Human Passive Awareness

The level of surrounding light usually changes dynamically and an Auto-brightness feature on a mobile phone can sense the level of lighting in real-time and adjust the brightness of the screen both automatically and implicitly. Sensitivity to light events thus capture the awareness capability of Auto-brightness feature to light events as illustrated in Fig. 3. Results indicate that the temporal distribution of light events (higher than 5000 lx or lower than 200 lx) and adjustment of screen brightness are highly correlated (r = 0.76). Nevertheless, the Sensitivity to light events of the Auto-brightness feature is only 2% (Mean = 2%; Max = 18%; Min = 0), thereby implying that the Auto-brightness feature system only reacted to 2% of the light events defined by this study on average. The temporal distribution of Sensitivity to light events revealed that the Auto-brightness feature functions better in the morning and deteriorates drastically in performance at around 8 am when users began to utilize applications extensively.

Fig. 3.
figure 3

Temporal distribution of contextual events

Full size image

Results from the survey questionnaire show that users were generally satisfied with the Auto-brightness feature. Only six out of 24 participants (one didn’t utilize this feature and seven didn’t realize that this feature existed during the research) were somewhat or very dissatisfied with the Auto-brightness feature.

3.3 Formation of UX

Services which feature context-aware functions are ubiquitous in current mobile phone applications. We reviewed the context-aware services running on our participants’ Android mobile phones and found a total of 30 context-aware services that were implemented in our participants’ mobile phones’ current applications (see Sect. 2.3 for details). In Fig. 4 we present the ratings of those of the 30 context-aware services that were utilized by more than half of our participants. The results indicate that applications, which provide services that exhibit awareness of personal and social contexts (e.g., browsing habits, hobbies, location, shopping history, and social network) were least preferred. Conversely, applications, which exhibit awareness of computing and physical contexts (e.g., internet connection, system language, and time zone) were most preferred. These results suggest that users’ preference for a context-aware applications are influenced by the type(s) of context afforded by the application. User would like to keep the awareness of personal contexts in their own hands and let mobile phone sense impersonal contexts and take execution automatically.

Fig. 4.
figure 4

Ratings of most common context-aware services found on participants’ phones.

Full size image

The Open Coding of survey results produced 23 codes and 5 categories were identified in further with Axial Coding by analyzing the major UX concerns, conditions of producing UX concerns, actions/interactions of users and context-aware services produced in response to the UX concerns and resulting consequences. One of the core finding is that the UX constructs of mobile context-aware services, including instrumental value and meaning, were indeed formed by the interplay between mobile phones’ awareness and users’ awareness. Additionally, users also concerned about the interactive manner and implementation of mobile context-aware services. This is exemplified through a conceptual model in Fig. 5 and quotes from the interview as showed below.

Fig. 5.
figure 5

Framework of UX constructs.

Full size image

Collaboration Between Users’ and Mobile Phones’ Awareness.

It’s in line with our assumption that collaborations between users’ and mobile phones’ awareness plays tremendous role in forming the UX of mobile context-aware services. Basically, mobile phones’ awareness was expected to be capable of complementing human ineptitude in allocating attention to changing contexts while in some cases it might impair the human awareness by cultivating over-reliance on mobile phones’ awareness. Additionally, including user in the loop of interaction, allowing for manual correction by user in case that mobile phones’ awareness malfunctions and offer visible feedback to users about the result of mobile context-aware are also considered crucial to the quality of UX. It’s noteworthy that awareness of users’ personal value and hobby are mentioned by Danish user which might indicate that personal value and hobby, as parts of users’ context, should be took into account in designing context-aware services.

[Allowing for manual correction by human] “Saving battery. But if I was outside and it dims suddenly it might affect my usage and I have to adjust it manually” [P26].

[Complementing human ineptitude] “It’s normal that I forgot to adjust the brightness of screen or night mode, so reminding me to night mode at certain time can help push me to do that.” [P31].

[Visibility of executing status] “Intelligent and saving time. But sometimes I might not be aware of that it has already adapted to the current input type, so I changed it manually and later on realized that I have to change it to the correct one” [P26].

[Adaptation to personal value and hobby] “Primarily based on what friends on social media are attending, and to my own preferences…” [P8].

[User in the loop] “I’m not sure whether I used this functionality, it’s probably that I didn’t pay attention to whether the screen is locked when I’m in a call” [P21].

[Impairing human awareness] “I might miss some events if the volume of alarm clock to be muted when the context is quiet” [P31].

Awaretiquette.

What distinguishes context-aware system from traditional system is that context-aware system is capable of initiating interaction with proactive action based on understanding of context. The proactive role of context-aware system introduces new concerns which don’t exist in traditional system with reactive and passive action. The proactive actions initiated by context-aware system should be comfortable, acceptable and desirable to users, just like humans behaving in good manners in communicating with each other.

[Protecting privacy] “… it wants to know everything and it invades privacy that they can get, a little annoying for me personally” [P15].

[Polite proactive awareness] “Avoid the rudeness and embarrassment of bursting into music in quiet situation” [P31].

[Interruptability] “It gets very invading in some kind because It’s constantly asking what are you thinking… so that’s where it gets, annoying or too much invades” [P15].

[Way of presenting information] “It’s very rude (not friendly) to remind me with the words “please continue to download if you are upstart wealthy”, it’s uncomfortable” [P26].

[Trust] “Usually the sequence of recommending stores is based on the amount of expense of each store on ads, so I don’t trust the recommending ones. I tend to trust what I searched by myself, it seems that this kind of recommendations are ads to me. I do t want my phone to keep track on my behavior. It can be abused by third parties” [P31].

Implementation of Mobile Context-Aware Services.

When active awareness was adopted, mobile context-awareness services should let the related tasks works in flow without interrupting or annoying users by taking sensitive, accurate and intelligent execution and letting users control the execution anytime they want to.

[Sensitivity] “But if I was outside and it dims suddenly it might affect my usage and I have to adjust it manually” [P26].

[Accuracy], “The recommendations of Zhihu is really accurate” [P21].

[Intelligence] “… [mobile context-aware systems and services service] enables a lot of the smart functionality is that when you walk, close to a metro station it will give you, the next departure times something even though i don’t use it. So all of that isn’t able by like the location right like using google maps as well location data” [P6].

[Controllability] “I’m not actually pretty sure because maybe I will feel like a little lose control. If, maybe at eleven o’clock is turned the light more down because sometimes I might not want it because sometimes I might be out, in the weekend, so, I’m not pretty sure actually what I want” [P9].

[Innovation] “I find the app very innovation but at the same time a bit unnecessary. Some of the things are very smart but some other not very necessary for me” [P9].

Instrumental Value.

Not surprisingly, efficiency quality such as reducing the operations was considered valuable. However, users’ feedbacks also indicated that the value of context-aware mobile services might be discounted with the lack of intelligibility and failure to show users how to function.

[Efficiency] “It help reduce the operations to switch between keyboards” [P27].

[Intelligibility] “Not satisfied. I have no idea of how to do anything. I would like a better interface” [P8].

[Utility] “It’s smart, can’t image how it will be used; It did not actually once remind me of anything, I guess the conditions were too specific, hence they actually never met” [P15].

Meaning.

The value of context-aware mobile services went far beyond saving time, it might also bring about health, finance and even social benefits to users.

[Health Value] “To protect eyes and healthy when staring at the phone screen” [P28].

[Financial benefit] “It’s good that it can help save data” [P26].

[Social benefit] “Avoiding the rudeness and embarrassment of bursting into music in quiet situation” [P31].

4 Discussion

4.1 A ‘Hybrid Awareness’ Experience?

This study sought to illustrate the interplay between users’ awareness and mobile phones’ awareness by examining the UX of a sample of 30 different context-aware services. We expected that UX concerns about mobile context-aware services would be different from those of mobile services in general and rather associated with the interplay between users’ awareness and mobile phones’ awareness.

In line with these predictions, we observed a different mode of usage behavior initiated by mobile phones’ awareness and users’ awareness, and by potential challenges to users’ awareness resulting from malfunction of mobile phones’ awareness. We also found that the UX concerns of mobile context-aware services were mainly associated with the interplay between users’ awareness and mobile context-aware services. The results of usage analysis showed that interplay between awareness of user and mobile phone shaped the different mode of usage and users had high level of awareness about the contexts which mediated the usage of mobile phone. Data of applications with active awareness (implicit interaction) and passive awareness (explicit interaction) showed that human awareness can not be excluded from the loop of interaction in active awareness of mobile context-aware systems and servicesS and intelligibility/utility issues were major concerns in passive mobile context-aware systems and services. Finally, questionnaire survey showed that user attempted keep awareness of personal contexts in their own hands and let mobile context-aware systems and servicesS to deal with the impersonal contexts. Taking all results together, a conceptual mode illustrating how awareness of user and awareness of mobile phone interplay with each other with quality of user experience, as illustrated by the following quote from participant P14:

“it is difficult for me to answer, because I have to be aware of what’s happening on my mobile phone, when I’m using it i have to take care of what’s happening on my telephone when it’s just lying in my pocket do i have to be aware someone’s calling me, i have to be aware what it’s doing. So there are passive and active or positive use scenarios” [P14].

Awareness of user and awareness of mobile phone form an awareness entity. The mobile context-aware systems and services should take awareness of user and contextual factors into consideration in capturing contexts, predicting the intention of user and executing actions. The awareness of user should be considered a type of context in designing context awareness system in addition to the temporal, spatial, social and computing. Only few studies explored how to initiate interaction with mobile phone according to contexts of user, e.g., [8]. This study showed that user’s awareness of using/not using mobile phone is highly context-dependent and, furthermore, that detecting the awareness of user in certain contexts and decision of using awareness strategies should be based on collaborations with user’s awareness. Users had different types of awareness which should be considered, too.

A quotation from a participant may shed light on the constituents of a hybrid awareness system:

“…I have to be aware of what’s happening on my mobile phone. When I’m using it, I have to take care of what’s happening on my telephone and when it’s just lying in my pocket I do have to be aware someone’s calling me, I have to be aware what it’s doing. So there are passive and active or positive use scenarios…” [P14].

To conceptualize the varying UX of the interplay between users’ awareness and mobile phones’ awareness, we propose the Hybrid Awareness System (HAS) model shown in Fig. 6.

Fig. 6.
figure 6

The Hybrid Awareness System (HAS) models the interplay between mobile phones’ and users’ active and passive awareness as depending on the types of context. Varied forms of interaction are required to address UX concerns across different interplaying modes.

Full size image

The conceptual model in Fig. 6. highlights the four key factors that governs the interplay between users’ awareness and mobile phones’ awareness. These factors include awareness, interaction, context types and UX. In this model, mobile phones’ and users’ awareness constitute a hybrid awareness system in which the two types of awareness collaborate with each other in a dynamical fashion. When faced with information on personal and social contexts, users’ awareness takes precedence over that of mobile phones in analyzing the information and taking appropriate actions. Conversely, when faced with information on computing and physical contexts, mobile phones’ awareness assumes the prominent role in analyzing the information before executing actions both automatically and implicitly. Nonetheless, keeping users in the loop of interaction is still necessary even when the interaction is automatic and implicit.

4.2 Implications for Design

Context-aware systems and services are ubiquitous thanks to the advances in sensor technology and computing capability of mobile phones. The results of this study showed that design of mobile context-aware services should take mobile phones’ and users’ awareness into account as an hybrid awareness entity. Compared with the traditional way of interaction featuring human-proactive and computer-reactive functions, computers in context-aware system are becoming more proactive, as they are able to detect contextual events and initiate interaction proactively based on an understanding of human intention.

From the perspective of human-computer interaction, the success of context-aware systems and services does not only depend on the efficiency and effectiveness of fullfilling the tasks delegated by humans, which is measured by usability. The success of context-aware systems and services relies on their (a) understanding the state of users’ awareness, (b) adopting explicit or implicit interaction according to the dynamic state both of mobile phones’ and users’ awareness, and (c) predicting the intention of user and initiating interaction at opportune moments. Furthermore, the implementation of mobile phones’ awareness should avoid impairing the users’ awareness, and instead offer complementary solutions; designer should avoid cases where mobile phones’ awareness deteriorates or even stops working while users have developed an over-reliance on such systems and services.

Second, the findings of this study call for attention to the issue of what we call ‘awaretiquette’ in the design of mobile context-aware services. To that end, efforts to be made not only include the traditional issues, e.g., privacy and trust, but also the emerging issues with the advent of proactive interactions should keep with the contexts. In the context of chinese culture, ‘awaretiquette’ refers to initiating appropriate interactive actions according to occasions (分场合) and to speaking different languages to human and ghost (见人说人话,见鬼说鬼话). The results of this study indicated that being aware of physical contexts are more preferable than that of social contexts out of considerations about controversial issues related to control of social relationships. Consequently, designing social-aware services needs to pay more attention to the issue of ‘awaretiquette’ by using appropriate interactions.

Finally, the design of mobile context-aware services needs to consider the contribution of context-aware systems and services to UX going beyond the instrumental values, e.g., to the meaning to life. Automation and implicit interactions of context-aware services lead to efficiency and low workload which maximizes the instrumental values. However, it also results in side effects, e.g., being out of control, out of the loop, and to invisibility of system status. The design of context-aware services should make full use of the strengths of context-awareness and avoid the deleterious effects.

4.3 Limitations and Future Work

This study was carried out with a small sample size to explore the UX issues arising from interplays between users’ awareness and mobile phones’ awareness. We identified the constructs and sub-dimensions of UX that is worth of further investigation in the future with a bigger sample size and a study devoted to more specific issues.

Literature showed that humans easily generalize etiquettes for human-human interaction to human-computer interaction [34] and require computers to have appropriate degree of etiquette when interacting with humans. Although the etiquette issue has attracted a wealth of attention in the field of human-robot interaction [35,36,37], sparse attention have been paid to the interaction between users’ awareness and mobile phones’ awareness [38]. As what revealed in this study, mobile phone users have experienced the problems of etiquette caused by mobile context-aware services. Research in the future can be devoted on the special issues of etiquette associated to proactive and implicit interaction, interruption for switching among users’ and mobile phones’ awareness, and the cultural factors.

In this study, we developed a conceptual model interpreting findings of UX concerns and interplay between users’ and mobile phones’ awareness. The model call on further and dedicated research with bigger sample size to investigate the specific issues such as the UX dimensions that distinguish context-aware services from context-unaware services, the interplay modes between users’ and mobile phones’ awareness in a specific context-aware services.

Finally, a majority of the participant in this study were students. Although learning can be considered a type of work for students, future research should investigate the potential differences between students and employees, and between classrooms and workplaces, for the influence of the interplay between mobile phones’ and users’ awareness on the user experience of using mobile phones.

5 Conclusion

In conclusion, our findings and proposed conceptual model support a dynamic view of UX shaped by the interplay between mobile phones’ and users’ awareness. We showed (a) how over time the usage behavior of mobile phone users are influenced by the interplay between mobile phones’ and users’ awareness, (b) how the malfunction of mobile context-aware service might risk the UX by fostering over-reliance on mobile phones’ awareness, and (c) how four factors identified in this study appears to govern the interplay between users’ awareness and mobile phones’ awareness. We propose a ‘Hybrid Awareness System’ (HAS) that models the interplay between mobile phones’ and users’ active and passive awareness as depending on the types of context. Furthermore, we suggest that our new term ‘awaretiquette’ may be used to refer to initiating appropriate interactive actions according to occasions (分场合) and to speaking different languages to human and to ghost (见人说人话,见鬼说鬼话).