Keywords

1 Introduction

As a common task situation in daily life, multitasking is typically characterized by individuals performing or switching between two or more tasks at the same time [1]. It includes multiple types with varying lengths of time spans. In different multitasking scenarios, user needs, behaviors, human-computer interaction experiences, and task performance are different [2, 3].

Task relevance refers to the extent to which tasks achieve related or similar goals [4]. Based on the resource theory of information processing, some researchers argue that humans have a limited cognitive capacity for information, and when two concurrent tasks compete for limited cognitive resources, people may experience cognitive overload and task performance may be reduced [5, 6]. Multiple resource theory (MRT) states that humans have several separate pools of cognitive resources, so that individuals can use these resources to perform tasks simultaneously [7]. When the content of tasks is related, they can create synergy with each other, thereby facilitating task performance and improving task performance [8].

Interruptions occur when the users decide to stop their current activities and move to perform a different task [9]. Internal interruptions are determined by people’s self-willingness, while external interruptions are triggered by external environmental [10]. Some studies show that external interruptions are somewhat nested, when people are interrupted by one thing they do not immediately return to their original task [11]. While internally interrupted work takes longer to resume and may be more disruptive [12]. Self-regulation theory (SRC) states that when unable to immerse themselves in task performance, individuals will self-regulate their behaviors to restore the balance between task demands and skills [11]. Later research find that people are more inclined to multitask when they are in a negative state, such as feeling frustrated, stressed or mentally exhausted [13]. Although multitasking interruptions have a negative impact on task performance, they can help individuals relax and relieve stress [14].

In summary, the present study aims to design a multitasking scenario in which university students interact with a service-oriented intelligent assistant, to investigate the effects of task relevance and interruption mode on users’ cognitive load, need gratification, flow and task performance.

2 Method

2.1 Participants

Thirty participants were recruited to participate in the experiment (15 males and 15 females, 18–25 years old). All the participants were with normal or corrected visual acuity and had English proficiency at cet-6 level. Additionally, in terms of a priori knowledge, all participants had experience with multitasking behaviors in their daily lives (mean normalized multitasking proficiency of 0.59 as measured by the MMT-R scale [15, 16]) and all had experience with intelligent assistants.

2.2 Design

This experiment adopted a 2 × 2 within-subject design. The independent variables were task relevance (task-related vs. task-independent) and interruption mode (internal interruption vs. external interruption). Participants were randomly assigned to different English reading topics and completed four sets of control variable experiments formed by different combinations of task relevance and task interruption mode, and one set of experiments in which no interaction with the intelligent assistant occurred as a control group. The dependent variables were task performance (percentage of correct English reading completed) and subjective feelings (cognitive load, needs gratification, flow).

Cognitive load was measured by the NASA-TLX, which includes six dimensions, each rated on a scale from 0 to 100 [17]. The scale for needs gratification was adapted from Jeong et al. [18, 19], and flow was adapted from the widely used flow scale [20], using a seven-point Likert scale for each statement (“1” = “strongly disagree”, “7” = “strongly agree”). All the measurement items used here have been widely used and validated in prior studies.

2.3 Procedure

The experiment was completed offline in a one-to-one session. Each participant was equipped with one Bluetooth headset and one tablet computer. The Bluetooth headset was connected to the experiment computer and transmits the voice of the voice assistant. The participant used the tablet (iPad Air3) to complete the English reading and post-experiment subjective evaluation. The voice assistant was achieved with the read-aloud plug-in in the experiment computer, which simulates the voice interaction between the participant and the intelligent assistant.

Each participant completed five sessions in sequence under the guidance (Table 1). Each group of participants had different voice interactions with the intelligent assistant. Completing a word query that affected the comprehension of the English reading content was a relevant task, and completing an interfering email check and reply was an irrelevant task. The participant initiated the query command was an internal interruption, and the participant responded to the interaction initiated by the voice assistant was an external interruption.

The voice of the intelligent assistant is female. All instructions are based on secondary tasks. In internal interruption experiments, the intelligent assistant responds directly to queries about the meaning of words or whether an email has been received. During external interruption experiments, the intelligent assistant prompts “Have you encountered any problems” or “You have received an email “.

Table 1. Experimental tasks
Full size table

Analysis of Variance (ANOVA) was used to analyze the effects of task relevance and interruption mode on task performance and subjective perceptions. Bonferroni was used for difference comparisons. Effect sizes were assessed by the η2p. All data were analyzed using SPSS 26.0 software and the significance level was set at 0.05.

3 Results

3.1 Task Performance

According to the results of the analysis (Fig. 1), there was a significant effect of whether to perform multiple tasks on correct task completion (F (1, 29) = 15.09, p < 0.01, η2p = 0.34). Furthermore, there was no significant effect of task relevance, interruption mode on correct task completion (Fig. 2) and no significant effect of the interactions on correct task completion (F (1, 29) = 1.54, p = 0.23, η2p = 0.05). Post-hoc tests showed that the control group, which did not perform multiple tasks, had a higher rate of correct task completion compared to the experimental group (t (29) = 3.88, p < 0.01).

Fig. 1.
figure 1

Effects of multiple tasks performing on task correctness (error line is ± 1 standard error)

Full size image
Fig. 2.
figure 2

Effects of task relevance and interruption mode on task correctness (error line is ± 1 standard error)

Full size image

3.2 Subjective Feelings

Figure 3 presents the cognitive load of users with different task relevance and interruptions. The analysis showed that whether multitasking was performed had a significant effect on cognitive load (F (1, 29) = 16.78, p < 0.0001, η2p = 0.37) (Fig. 4). However, the main effects of task relevance and interruption mode on cognitive load were not significant, and the interactions had no significant effect on cognitive load (F (1, 29) = 0.28, p = 0.60, η2p = 0.01). Post-hoc tests showed that the control group that did not perform the multitask had lower cognitive load compared to the experimental group (t (29) = 4.10, p < 0.0001).

For need satisfaction, the presence or absence of multitasking had a significant effect on subjective perceived need satisfaction (F (1, 29) = 4.45, p < 0.05, η2p = 0.13). Although the main effects of task relevance and interruption mode on need satisfaction were not significant (Fig. 5), the interactions had a significant effect on need satisfaction (F (1, 29) = 6.64, p < 0.05, η2p = 0.19) (Fig. 6). Post hoc tests showed that the control group that did not multitask had lower need satisfaction compared to the experimental group (t (29) = 2.11, p < 0.05). While for the main effect of task relevance, there was no significant difference in needs gratification between relevant and irrelevant multitasking (t (29) = 0.02, p = 0.99). For the main effect of interruption mode, there was also no significant difference in needs gratification between internal and external interruptions as well, (t (29) = 0.99, p = 0.33). Further simple effects analysis revealed that, in the case of performing related multitask, needs gratification was significantly higher for internal interrupts (M = 4.49, SD = 0.88) than for external interrupts (M = 4.13, SD = 1.11, t (29) = 2.29, p < 0.05).

For flow, there was no significant effect of whether perform multiple tasks (F (1, 29) = 2.40, p = 0.13, η2p = 0.08). While the main effects of task relevance and interruption mode on flow were not significant (Fig. 5), the interaction effects of both were borderline significant (F (1, 29) = 4.18, p = 0.05, η2p = 0.13) (Fig. 6). Post hoc tests showed that for the main effect of task relevance on flow, there was no significant difference between relevant and irrelevant multitasking (t (29) = 1.95, p = 0.06). For the main effect of interruption mode on flow, there was also no significant difference between internal and external interruptions, (t (29) = 1.10, p = 0.28). Further simple effects analysis revealed that, in the case of performing the relevant multitask, the flow of internal interrupts (M = 4.66, SD = 0.75) was significantly higher than that of external interrupts (M = 4.33, SD = 0.85, t (29) = 2.11, p < 0.05). In the case of internal interruptions, the flow level for performing relevant multitasking (M = 4.66, SD = 0.75) was significantly higher than for performing irrelevant multitasking (M = 4.28, SD = 0.76, t (29) = 2.65, p < 0.05). Overall, in the multitasking experimental setting, the execution of relevant multitasking with internal interruptions performed best in terms of flow.

Fig. 3.
figure 3

Effects of task relevance and interruption mode on cognitive load (error line is ± 1 standard error)

Full size image
Fig. 4.
figure 4

Effects of multiple tasks performing on cognitive load (error line is ±1 standard error)

Full size image
Fig. 5.
figure 5

Effects of task relevance and interruption mode on needs gratification and flow (error line is ±1 standard error)

Full size image
Fig. 6.
figure 6

Interaction effects of task relevance and interruption mode on needs gratification and flow (error line is ±1 standard error)

Full size image

4 Discussion

The experiment results suggest that multitasking impairs task performance and increases cognitive load and need satisfaction. This result is in line with many previous studies, such as Kirschner and Karpinski [21] who found that Facebook users had lower task performance in relevant research activities compared to non-users, and Jacobsen and Forste [22] who assessed college students’ media use behavior on their academic performance, they obtained similar results. This study suggests that multitasking in learning situations, whether performing related or independent tasks, has a negative impact on task performance, which supported by some scholars who think “once students use smart devices during their studies, their academic performance decreases”.

However, the main effect of task relevance on both objective performance and subjective perceptions was not significant in this experiment. It is inconsistent with previous experiments by Hembrooke [23] et al. in which they found that for students of similar levels, groups dealing with relevant multitasking outperformed groups dealing with irrelevant multitasking. The non-significant main effect between the two levels of task relevance may be since the secondary task completed during multitasking was less difficult than the main task. The secondary task did not take up much cognitive resources, which in turn did not have a significant effect on objective performance. Additionally, in previous studies related to multitasking, Zhang [24] and others found that, in work-related multitasking, instrumental demands were mainly satisfied, whereas in media multitasking interaction, emotional demands were mainly satisfied. Based on this, the main and related tasks taken in this experiment satisfy users’ instrumental needs, while the unrelated tasks satisfy users’ affective needs. Although the sources of the needs satisfied are different, they both have a positive impact on users’ needs satisfaction. As the main task itself have a high satisfaction of the participants’ overall needs (normalized mean value of need satisfaction for the control group >0.7), the needs for emotional experience, social interaction and self-identity arising outside the main task were low, and the impact caused by dealing with related and unrelated multitasks was not significant. Similarly, for flow, the user's high level of experience in completing the entire task, which measures the level of concentration and engagement with the task during the task. Wang et al.’s study [25] showed that, learning tasks are inherently more immersive as they require more task engagement than other tasks. The participant effect on objective performance and subjective perception, probably for internal and external interruptions in the experiment triggered less difficulty in multitasking. The information brought about by partial interruptions to multitasking was likely to be completed by the participant through parallel tasks without task interruptions, reducing the cognitive cost due to task switching, affecting the user’s subjective perception as well.

The interaction effects of task relevance and interruption mode on need satisfaction and flow were significant. Flow Theory and Self-regulation Theory [26] suggest that self-interruption occurs when individuals are unable to achieve a state of flow through the ongoing task. Whereas internal drivers of multitasking behavior are mainly due to the generation of psychological, emotional or social demands [27]. Based on Motivated Cognition Theory (MCT), internal interruptions are individuals themselves in a position where needs gratification and higher levels of flow occur. Therefore, internal interruptions lead to higher needs gratification and flow.

In summary, future research could be investigated in the following ways. First, the range of participants could be increased to expand the experimental coverage group. Previous multitasking studies have shown that there are differences in multitasking ability and cognitive load among participants of different ages [28, 29]. Multitasking performance and perceived levels also vary according to individual multitasking ability and individual characteristics [30]. The results lacked significant differences in terms of need gratification and flow, as the participants’ perceptions of flow and task satisfaction fluctuated less under different experimental settings. Secondly, secondary tasks with quantifiable task difficulty can be used to balance the difficulty and time spent on the primary and secondary tasks. And the impact of differences in secondary task difficulty on the users’ experience could be explored. In this study, the secondary tasks were set at a relatively low level of difficulty and occurred for a shorter period than the main task, making the differences in participants’ subjective feelings with respect to different task relevance and task interruption modes relatively small.