Keywords

1 Introduction

The ability to self-manage time is one of the key challenges for knowledge workers [e.g., 21]. Time management (TM) means activities like assessing, planning and monitoring time use with the goal to organize time use in a productive and healthy manner [e.g., 5]. Commercial activity logging tools like ManicTime, RescueTime or SLife already claim to support time management. Scientifically however, the usefulness of activity logging tools for (learning about in the sense of improving) time management is surprisingly unexplored. Automatic activity logging has often been used to measure the behavior of study subjects, investigating for instance the cost of task interruption [4] or work patterns dependent on daytime, weekday or location [1]. Only more recently, time-stamped data have been investigated also as means to support time management. The authors of [7] explored which visualizations and analyses of calendar data would be helpful for time management (21 participants). In study described in [12], we found that people were consistently surprised about their actual time use as automatically logged, in particular about the extent of work time fragmentation (7 participants). In both settings, participants were looking not only for insights on data, but for “actionable analytics”, i.e. analyses and interpretations of data that would help them decide on future behavior.

Activity logging as computer support for time management is related to the wider research on personal informatics and quantified self. Under these names [3, 9, 10] people measure (quantify) their own behavior. Popular areas of interest are behavioral patterns related to health or sustainable living, and typical goals are to gain insights and to change (improve) the behavior under scrutiny [3, 10]. Manually or automatically created activity logs in this context serve the purpose to provide a data basis beyond memory for re-evaluating experiences [3, 8–10]. Such reconstruction and re-evaluation with the goal to learn for the future can then be understood as reflective practice, where learning could mean new insights, change in behavior as well as a change in perception (e.g., [2, 13]).

2 User Study

2.1 Method

We conducted two field studies of 6 weeks each. Each was designed to show the effects of an intervention with respect to improving time management.

The studies took place in two companies with similar characteristics: Both are medium-sized IT consulting and development companies in Germany. The intervention was different though in each company. At one company, only self-tracking was used (ST-setting), while in the other, additional coaching was provided (C-setting). Note that both studies took place in parallel to otherwise regular work.

All participants (i.e. in both settings) were instructed to have an activity logging tool (AL tool) running in the background throughout the study and to manually label time whenever they found this useful (self-tracking). The used AL tools were KnowSelfFootnote 1 (used by all participants in the ST setting, and half of participants in the C-setting) and ManicTimeFootnote 2 (used by half of participants in the C-setting). Participants were further instructed to review their time use in the AL tool daily and to write down insights and plans for change (reflective practice). The suggested daily time was ten minutes at most for the reflective practice. In the self-tracking setting (ST-setting), participants received no further information or instruction with respect to time management.

In the coaching setting (C-setting), participants received bilateral coaching in addition. All participants in the C-setting had a weekly 1-hour bilateral coaching meeting with the same coach for everyone. The overall goal of coaching was predefined: Coaching was intended to support the improvement of time management. The coaching was paid for by the company.

2.2 Evaluation Tools

TM skills and behavior were measured with a 12-item scale (TM scale - by Hansen, taken from [14]). The TM scale asks for compliance with typical time management practices. Additionally, we asked for gained insights (learning), changes in TM and the perceived benefit of AL tools on a 17-item scale (TM Learning and Change). Example items are “I gained a deeper understanding of my TM behavior” or “I made a conscious decision to change my behavior”. Measurements were self-assessments before and after the study with both open and closed items (5-point Likert scales: 1 = strongly disagree; 5 = strongly agree). Bilateral interviews were conducted after the study with all 20 participants plus the coach from the C-setting.

2.3 Participants

All participants were volunteers. Overall, 20 study participants completed the study (2 drop-outs in the ST setting filled only the pre-questionnaire). 5 participants were female, 15 male. One was younger than 19, eight were aged between 20 and 29, ten between 30 and 39, and three between 40 and 49. At the time of the study a majority of study participants were employed full-time in their respective company (19 out of 20). A majority of participants were experienced and skilled knowledge workers, in terms of their job description (18 out of 20). These were team managers or experts required to essentially work on their own towards a specified goal; typically they would be given only a project or a customer and the overall goal such as “to acquire a new contract with a potential customer” or “to implement a strategy for a customer”.

3 Results

Questionnaire and interview data were analyzed with respect to learning and behavior change, as well as with respect to the contribution of activity logging tools.

In both settings, participants were positive that learning in terms of gaining insights with regard to TM had occurred. There was no statistically significant difference between the ST-setting and the C-setting in any item related to learning (TM Learning and Behavior scale).

There was a single item in the TM Learning and Change scale that showed a significant, namely “I made a conscious decision to change my TM behavior” (independent samples t-test t(18) = 2.52 p = 0.022). Participants of the ST-setting reported significantly lower values for this conscious decision to change (M = 3.3; SD = 1.059) than participants in the C-setting (M = 4.3; SD = 0.675).

Additionally, we had measured compliance with TM best practices before and after the study with the TM scale (see Table 1). At the beginning of the study, participants in the ST-setting rated their TM skills and behavior significantly better than participants in the C-setting (independent samples t-test: t(18) = 2.32, p = 0.032). However, participants in the C-setting reported significant improvements over time (repeated measures t-test: t(9) = 3.4, p = 0.007), while participants in the ST-setting reported essentially no changes. A mixed-design ANOVA (within subjects: pre/post; between subjects: ST-setting/C-setting) shows as interaction effect F(1,18) = 4.34, p = 0.0518, η 2p  = 0.194. This closely misses the p < 0.05 level of significance, but shows a large effect size despite a relatively small number of participants. We therefore assume the different patterns of change originate in the different interventions given to the two groups (self-tracking vs. coaching).

The usefulness of activity logging (automatic plus manual labeling) was stated to lie in providing accurate information about time use with very little extra effort (manual recording tasks if necessary/required) by six participants in the self-tracking setting (out of 7 interviewed) and seven in the coaching setting. This is in line with the key claim underlying research in the quantified self and personal informatics community.

In both settings, participants missed peer support. While participants were of course free to discuss their TM efforts with colleagues informally (in each study, all participants knew of each other), we had not instructed participants to do so, and neither AL tool had social software features like sharing and comparing one’s data with peer’s data, messaging, or discussion forums. One participant in the ST-setting, for example, suggested some sort of sharing one’s plans and progress for improving TM in order to make visible to others whether “you are holding up your part of the deal or not”. Note that we did not specifically ask about social software features but rather generically inquired about what had been missed, what could be improved, etc.

4 Discussion and Conclusion

4.1 Limitations of Study Methodology

The two studies we compare within this paper are field studies – they are by nature more messy than lab studies. In an endeavor to allow readers to judge for themselves on the validity of results, we discuss the shortcomings of our methodology.

Firstly, we are reasonably sure about the quality of interventions in relation to typical similar interventions: The activity logging tools did not differ significantly in terms of used evaluation tools. However, the study is not a comparative study of two different tools. A second point of critique could be that only subjective data about time management behaviour are available. Although KnowSelf would have allowed us to collect anonymised data about work activities (only application names and timestamps in cleartext; file names and full-text like note would have been hashed), most study participants did not share their data with us. Finally, we used the two studies in this work also in a comparative manner (mixed-design ANOVA). We did this, because the two companies are comparable with respect to the kind of work (IT consulting and development) and the culture (both in the same area in Germany). Differences found between the two settings (ST-setting, C-setting) could therefore come not only in the different intervention but also in the different organizational culture.

4.2 Insights

The value of self-tracking lies in gaining self knowledge: Participants in both studies gained insights on their personal time management, decided on and implemented reasonable changes in their time management strategies with no significant difference between the two settings. Mentioned TM strategies were for instance “saying no”, “planning more time buffers”, “create priorities for tasks” or “focus on priorities” (translated quotes from German). This result is interesting and relevant, as it confirms what marketing of commercial time tracking tools already claims, i.e. they support time. This result is also novel, as nothing similar is to date available in literature on improving time management aided by activity logging tools. In addition, this result supports the assumption and claim of the quantified self and personal informatics community, namely that self-tracking can indeed foster learning and ultimately behavior change. This result also indicates that there might be quite enough to gain from the cheaper solution of self-directed learning aided with self-tracking when compared to the definitely more expensive solution of paying a coach to give additional support.

The added value of coaching lies in supporting behavior change: Participants in the C-setting decided on changing their behavior more consciously than participants in the ST-setting and rated their compliance with predefined TM best practices (TM scale) significantly better after the intervention study, while participants in the ST-setting essentially did not change their respective self-assessment. As underlying mechanisms we draw out the following from background literature and qualitative data (from interviews): Firstly, interpretation of activity logging data requires background knowledge, in our case of time management. In the C-setting the participants were supported in this task by the coach, in particular via explicitly setting goals, and experimenting with concrete solution approaches. Indeed, self-tracking approaches often experience data interpretation as a key hurdle [3, 9]. This has also been expressed as design challenge for software to provide actionable knowledge (see [4, 13] in the case of time management). Secondly, there is a social norm to keep promises. The underlying psychological mechanism is emotional commitment and/or guilt aversion (which it is, is under discussion [5, 6]). In the C-setting, the participants had agreed with the coach to review their time use and achievement of goals on a daily basis, as well as to spend significant time reflecting on their progress in the weekly coaching meeting.

Social software features to the rescue: Social software features in activity logging tools for improving time management could address the latter, in that they could provide a platform for making “promises”. In addition, they could provide the means to identify systemic time management issues within an organization.