Keywords

1 Introduction

Modern technology plays a great role in the ongoing shift from a closed, clinician-driven health care system, towards a collaborative environment where patients are actively involved in their own treatment. Over the last years, dozens of traditional treatments have been transformed into digital interventions, allowing people to access therapy and treatment on their mobile phones, at any time. This allows therapists to treat more patients simultaneously, while using less resources, increasing efficiency [3, 6].

Self-report questionnaires, which are increasingly delivered through mobile applications, are one of the most used assessment tools in clinical psychology, in practice and in research settings [9]. They are widely used in combination with traditional face-to-face therapy as well as Internet Delivered Psychological Treatment (IDPT) and form the basis for Experience Sampling Methods (ESM).

Self-report questionnaires gather information, for instance, on the well-being of patients, and are of great value to researchers and practitioners alike, as they are designed to get a specific piece of information that is necessary to assess the situation of the patient and to decide the next step in a treatment plan.

When self-report questionnaires are delivered online, they are usually provided in the same reliable format that has been proven to function offline. However, there are a number of open questions that arise when inquiring through online self-report questionnaires as compared to the classical format. This research focuses on ways of increasing adherence through NOW Interactions; in fact, there is a significant correlation between the time spent on IDPT and clinical effects [5, 10, 17], and high response rates in ESM give a more complete picture of the studied phenomena [30].

From a design perspective, adherence is negatively affected by 1) questions being asked retrospectively, 2) long questionnaires, and 3) sub-optimal designs for mobile interfaces. First, asking information retrospectively requires extra effort from patients, which may result in a feeling of exhaustion and may make the collected data prone to low validity. Second, many questions in a single questionnaire can negatively impact completion rate, for instance, by not having enough time or concentration at hand. Third, online questionnaires are often similarly constructed as their paper-pencil twins, and do not take advantage of new presentation and interaction formats available on mobile interfaces, giving a better user experience.

NOW Interactions is based on principles from interaction design [11], microinteractions [12], and persuasive design [7]. In the e-health use case, they aim to help people accomplish the task of providing information through self-report questionnaires. Timely and smooth interactions are ensured by dividing complex information requests into sequences of small steps presented as microinteractions. The user interface is designed to function as a means to facilitate motivation by sending effective triggers that cater to the ability of the user, by being timed to a fitting moment, and by allowing information to be provided instantly without opening an app. This makes it easier, or just as easy, to provide the information as it is to dismiss the request, see Fig. 1 for an example of NOW Interactions.

Fig. 1.
figure 1

An example of NOW Interactions containing a question about activation (l) and valence (r), in Norwegian.

Full size image

2 Background and Related Work

2.1 Self Report Questionnaires

Self-report techniques are used to let respondents report on their own behaviour, feelings, or intentions without interference [20]. These techniques are used in many different fields and situations, for example to measure public opinion, carry-out research studies, assess psychological health and other medical issues, and to support behavioural studies. Advantages of using self-report include low development costs and the possibility to reach a large sample group with relatively little effort. The main disadvantage to using self-report might be the possibility of providing invalid answers, as the answers given by respondents cannot be fact-checked but are taken for granted at face-value [9].

In clinical psychology, standardised self-report questionnaires are one of the most used assessment tools in practice and in research settings [9]. Respondents are asked to read questions and select their response. The questions in standardised self-report questionnaires are most often retrospective; they for instance ask the respondent how often something has occurred over the last two weeks. In clinical psychology, such questionnaires are widely used in combination with traditional face-to-face therapy as well as Internet Delivered Psychological Treatment (IDPT). The questionnaires are used to gather information, for example, on the well-being of patients, and are of great value to researchers and practitioners, as they are designed to get a specific piece of information that is necessary to assess the situation of the patient and to decide the next step in a treatment plan.

Self-report is also the base for the Experience Sampling Method (ESM) and Ecological Momentary Assessment (EMA), where people are asked to self-report in real time on subjective experiences from their lives. These techniques have been applied in numerous clinical studies in order to, for example, evaluate indicators of substance abuse or to gain better insight in social Interaction or mental health. The collected data from such studies helps to advance both science and practice.

2.2 Adherence and Response Rate

Adherence is seen as a very important factor in clinical psychology. There is a significant correlation between the time spent on IDPT and clinical effects [5, 10, 17]. Thus, finding methods to uphold communication with participants and to increase adherence could contribute to the health of participants.

Similarly, response rate in ESM (also ‘compliance rate’ in medical literature) is seen as very important. The response rate describes the number of answered notifications divided by total amount of notifications sent in the sample. A high response rate gives a more complete picture of the studied phenomena [30]. Literature suggests several methods to increase response rates including incentives, providing feedback [15, 28], and incorporating gamification elements [13, 14, 19, 31]. There is an expressed need to investigate how creative input methods on mobiles can help to reduce participant burden and increase response rates [30].

2.3 Responding to Questionnaires

Questionnaires are traditionally considered a communication process between the questionnaire initiator and the respondent [11, 16]. This is important to keep in mind when constructing appropriate questions and integrating questionnaire logic, but also a reason to consider how design can facilitate this communication.

Responding to questionnaires is also seen as an iterative process where each question represents an iteration that includes several steps of complex information processing [2, 29]. The first step in this process is understanding the question, the second step includes the retrieval of information from memory, the third step involves a judgement process related to answering truthful or not, and the fourth step is to match the fabricated answer to the provided response options [21], see Table 1.

At any point during this process there is a chance the respondent stops answering. Insight into this process can be valuable for understanding how and where design can support this communication process, and thus maximise the chance of the respondent fulfilling the questionnaire. Table 1 shows how the steps in the cognitive process connect with design factors that could influence these steps.

Table 1. A simplified version of the steps in the cognitive process of responding to questionnaires
Full size table

2.4 Questionnaires on Mobile Phones

Usually, the respondent receives a notification on their phone with a suggestion to answer the questionnaire; the respondent taps the notification, which opens the self-report questionnaire in a browser window or an app where the respondent answers the questions and hopefully fulfils the whole questionnaire.

When self-report questionnaires are delivered online, they are usually provided in the same reliable format that has been proven to function offline. Validated paper-based questionnaires are often used as the basis for self-report questionnaires, they have proven to measure what they aim to measure, which benefits researchers and clinicians as it makes it easier to compare the collected data to prior research.

Unfortunately, the questions are often asked retrospectively. This requires extra effort from respondents, which may result in a feeling of exhaustion and may make the collected data prone to low validity. Step 2 and 3 from Table 1 might be in jeopardy here.

Second, many questions in a single questionnaire can negatively impact completion rate, for instance, by not having enough time or concentration at hand. Mobile users are likely to be interrupted at any time due to the nature of the device, so this is something to take into account.

Questionnaires that are originally developed for the paper medium do not take advantage of new presentation and interaction formats available for mobile interfaces. Long, complex questions may even be directly unsuitable to present to mobile users, as readers are known to have more difficulties to comprehend digital text [8]. Also, questions that include many answer options may become problematic due to space issues on small screens. Related to this, it should be considered that when the medium on which one presents self-report questionnaires has (great) influence on the results of the questionnaire, it becomes part of the methodology. Researchers might need to rethink their idea of standardised validated self-report questionnaires, update them to a new standard, and make them mobile compatible and thereby more user-friendly.

Thus, from a design perspective, adherence and response rates are negatively affected by 1) questions being asked retrospectively, 2) long questionnaires, and 3) sub-optimal designs for mobile interfaces.

2.5 Questionnaire Redesigns

Research has been carried out about questionnaire design and how they can be optimised for mobile use [4, 7]. Literature suggests survey designers should optimise the lay-out of their questionnaires for mobile phone use. Suggestions include simplifying questions to include suitable answer input types for mobile [4].

There have also been solutions introduced that use the unlocking mechanism of the phone to collect answers to a simple question; this is called ‘unlock journaling’ [34].

NOW Interactions goes beyond this and aims to improve the whole interaction by maximising the amount of information collected while having as few interactions as possible, thus taking as little as possible energy and time from the respondents.

3 NOW Interactions

NOW Interactions facilitates a communication that benefits both respondents and questionnaire initiators. The user interface of NOW Interactions is designed to persuade people to provide bits of information in a moment, at the right time, with minimal interruption of their current activity. This makes it easier for users to respond than to dismiss an information request, without even needing to open an application. The overarching idea is to maximise the amount of information collected, while having as few and as small interactions as possible; thus taking as little as possible energy and time from respondents. This reduces participant burden and increases adherence, which is the ultimate goal.

3.1 Design Foundation

The overarching principle of NOW Interactions, is based on behaviour theory [12]. By making the task (answering a question) as low effort as possible, the chance of the respondent actually performing the task increases. According to the theory of persuasive design, when performing an action, such as providing health related information by answering questions, people need motivation, ability, and triggers; if one of these is missing, the action will likely not be performed [12]. The theory also shows an interplay between motivation and ability; to perform a difficult task, one needs higher motivation than to perform a simple task that takes little effort. NOW Interactions embodies this concept by employing the user interface to facilitate motivation and cater to ability, with good timing, to have effective triggers. This concept also aligns with equity theory, which suggests that people are more likely to provide input if the cost/benefit ratio is in their favour [1]. NOW Interactions reduce the time and energy people need to use on responding to a question or questionnaire, thus increases the chance that the task will be performed.

Interaction design focuses on designing interactive products to support people in their everyday and working lives [24] and helps people achieve their goals in the best way possible. Questionnaires can be seen as two-folded, on one side, questionnaire initiators really want answers to their questions, yet they depend on respondents to take the effort to answer. By focusing on respondents and the whole process respondents go through when answering questionnaires, designers can better adjust to their needs. NOW Interactions connects the steps within the cognitive process of answering questionnaires to motivate design choices. Table 2 is an updated version of Table 1 with NOW Interactions design choices.

Mobile self-reporting implies the context situation where respondents are likely to be interrupted, either by other notifications on their phone, or real-life happenings. NOW Interactions anticipates this situation by presenting questions as microinteractions. Microinteractions are interactions that focus around a single use case, such as answering a single question, and making this use case as pleasant and convenient as possible [27]. Dividing complex tasks into smaller sub-tasks, makes it easier for the respondent to stay concentrated on each individual task, as well as to pick up where they left off when they get interrupted. This makes the task at hand more manageable and less overwhelming.

Mobile notifications have been shown to play a key role in getting users attention [23]. NOW Interactions shifts the use of such notifications from being used as reminders towards being the interaction itself. The question functions as a reminder, which can be answered straight away, without extra steps, taps, or waiting time such as needing to open an app. This is makes NOW Interactions timely and very suitable for in situ self-report questionnaires, as it reduces the time between the reading of the question and the provision of the answer.

Table 2. A simplified version of the steps in the cognitive process of responding to questionnaires related to the design of NOW Interactions
Full size table

3.2 Interaction Model

A single NOW Interaction relies on one notification, which includes both a question and the accompanying answer possibilities. After an answer has been chosen, the notification disappears.

This single NOW Interaction is envisioned to be part of a larger model that relies on different data sources to make a substantiated decision on whether or not a NOW Interaction is required; and if so, when it should happen and what it should contain. Different sources feed data to a User Profile; this could include sensor data, mobile phone use data, data from wearables, background information, user interactions, and direct input from the user, etc.

The user profile allows NOW Interactions to adapt in real-time to the personal and current situation of the user. A decision process, based on an analysis of the user profile, determines the details of the NOW Interaction. This way, it is ensured that the interactions are adapted and personalised to the needs of the user. For example, no requests will be sent while the user is sleeping or at the gym, and the questions depend on which health data should be provided.

NOW Interactions collect data throughout the day in moments that fit the user, by sending short inquiries. This releases people from having to answer many questions at once and the information that they need to provide is still fresh in their memory. The collected data feeds back into the user profile, which decides the next step; this could be, for example, another NOW Interaction, an in-app questionnaire, a guided exercise, or nothing (in the case that all information for this moment has been collected).

4 Prototype

To evaluate the potential of NOW Interactions to improve information collection from self-report questionnaires, to test the feasibility of this method, as well as gauge user acceptance, a functioning prototype was designed and developed and eventually tested in a small pilot study.

4.1 Prototype Development

The prototype was developed in two phases. The first phase included idea development by carrying out desk research and semi-structured interviews with two patients that had gone through an IDPT to improve their mental health and a psychologist specialised in working with IDPTs. The focus of these interviews included mapping how patients use IDPTs and what type of elements they believe could help them adhere to such programs. The results of these interviews included suggestions about how IDTPs and patients can benefit from redesign. Both patient interviews revealed that a low-threshold way to stay in touch with the programme was missing. This resulted in the idea of NOW Interactions and an accompanying non-functional digital prototype. Figure 2 shows an example from one of the first digital designs.

Fig. 2.
figure 2

One of the first digital designs for NOW Interactions

Full size image

In the second phase, over several iterations, the non-functional prototype was remodelled to a functional prototype. During this process, there were many sessions where designers and software developers came together to discuss the progress of finding the best approach. This was a challenging process as there were no off the shelve solutions available, so the technology to run NOW Interactions needed to be built from scratch; this went not without snags. Androids were chosen for the prototype due to the possibility for interactive push notifications, however, limitations of the platform and the required functionality resulted into concessions, both from the development team and the design team. An example is that Android’s notification platform is not very flexible with the design space for notifications, i.e., there are limits to how much space is available to fit both questions and answer options. Figure 3 shows how the available space eventually was utilised for an individual NOW Interaction. Note that the focus of this prototype was not to test different layouts of these notifications, but rather test the interaction.

The result of this phase was a working prototype that sends two predefined NOW Interactions three times a day, to collect information, without connecting it to a user profile. The first NOW Interaction in the pair is set to be send at a certain time, while the second NOW Interaction launches after the answer to the first notification has been received by the server.

Before testing the prototype in the pilot study, it was pretested on a simulator and ten Android devices with different screen sizes and settings; this revealed some issues such as when dark mode was enabled on the device.

Fig. 3.
figure 3

The design space for Android notifications & how it was utilised for NOW Interactions

Full size image

4.2 Area of Application

To make a prototype with meaningful content, we decided to make use of an adapted version of the Swedish short self-report measure of core affect [32, 33], often used in psychological evaluations. Core affect presents how we are feeling and comprises a combination of two dimensions: valence (pleasant to unpleasant) and activation (deactivated to activated) [22, 26].

Traditionally, core affect has been measured by using a number of self-report rating scales with adjective end-points [25]. The Swedish short self-report measure of core affect demonstrated that it is possible to greatly reduce the number of scales necessary to get an accurate measurement of core affect. They also showed there is no significant difference between different answering scales [32]. For this prototype we decided to use one valence and one activation question for each pair of NOW Interactions. Three times a day, at dedicated times, a notification was sent with a question directed to measure activation, see Fig. 1 (left). After an answer has been selected, the notification disappears, and the second question is prompted, this time directed to measure valence, see Fig. 1 (right). The three question sets comprise different questions, but uses the same response format as the other sets, as seen in Table 3.

Table 3. The timing and content for the NOW Interactions in the pilot study
Full size table

4.3 Pilot Study

The pilot consisted of three parts: 1) a workshop including a focus group discussion, a demonstration, and a user test with the aim to reveal any complications with the chosen icons or problems with the concept, 2) a hands on test of the prototype over two days, and 3) a follow-up interview and a TWEETS evaluation [18] to reveal any hidden issues that may have occurred during the hands on test and to add final thoughts.

Two high school classes with twelve students of approximately 18 years of age agreed to participate in the first part of the study. They were asked to design icon sets to fit the questions as shown in Fig. 1, and to perform a user test on each other, where they would scope for mindset around the idea of answering questions this way.

As a preparation for the second part of the pilot study, it was explained to the students that this tool is meant to keep track of how people that may experience difficulties in their lives are feeling, and that this may or may not apply to them personally. It was also explained that the app would anonymously collect their answers together with a time stamp, but that this data would not be used to do any psychological modelling and only be used to improve the performance of the app.

Due to lack of Android mobile phones amongst the students, only three students were able to test the NOW Interactions on their own phone and participate in the hands-on testing of the prototype and the interviews. Restrictions due to Covid-19 greatly limited the access to find and follow up students. (To compensate for the possible lack of finding technical errors, we set up an additional informal user-test with five participants during an academical workshop, which revealed only a few minor technical problems.)

Table 4 shows an overview of the categories of collected data, note that in some cases individuals have provided several answers in the same category.

Table 4. An overview of the type of collected data during the pilot study
Full size table

4.4 Lessons Learned

The user test gave some answers that address the aims of this pilot study. The prototype with the NOW Interactions worked and seemed suitable for collecting information. It was possible to employ NOW Interactions for use in a health domain, and the user test showed the users were curious about the future possibilities of NOW Interactions.

From the data that we collected from the students through NOW Interactions, we noticed that the time between sending the notification and the answer was reduced from the second set of questions on. This made us wonder if they took the time to actually read the questions. In the follow up interview, the participants confirmed that they did not realise that the three question sets during the day asked different questions, though with similar icons.

All participants were positive about not having to open an application, but still being able to have a meaningful interaction. They believed that this would greatly increase the chance of them answering questions. Two participants mentioned that they did not feel that the NOW Interactions were interrupting in their activities at hand, even though they received two questions in a sequence.

5 Discussion

Based on the feedback from the students in the pilot study, NOW Interactions seem worthy of investigation, however, there are still many aspects that need further thought, design, development, and research.

We managed to ensure smooth interactions by dividing complex information into sequences of small steps, and present them to the users as microinteractions. However, even though the technique is working, and seems promising, the actual design and layout of NOW Interactions still needs more work. We need to find out why the test participants did not read the questions and find solutions to make sure that in future versions, they will. Different layouts need to be tested, as well as alternative response formats. We also have to research if personalising icons to individual user preferences can be can be beneficial.

There are also technical limitations to solve. At the moment, NOW Interactions is only available for Android devices, as IOS does not support this technique. Even though the Android mobile platform allows for this type of communication, it is not a widely used feature, and will require careful planning, as not to be abused. An overflow of such notification questions would probably be overwhelming for users.

Although this small test group did not seem to find these interactions intrusive, we have to do extensive research about how people that are not feeling well will receive this technology. Possibly, receiving notifications related to one’s illness could influence this perspective, as it handles about personal medical data. This is also a technological challenge, as to how sensitive data is handled.

NOW Interactions aim to not just rethink the design of self-report questionnaires, but to rethink the whole interaction. Related to this should be considered that when the medium on which one presents self-report questionnaires has (great) influence on the results of the questionnaire, it becomes part of the methodology. Researchers might need to rethink their idea of standardised validated self-report questionnaires, update them to a new standard, and make them mobile compatible.

Once the new designs are ready, a next step for research on NOW Interactions is to test them over a longer period of time, possibly in combination with an IDPT- or medical tracking app such as menstrual cycle tracking, water intake measurement, or headache diaries. It is also necessary to test how the notification questions themselves should be improved to fit the users own preferences and needs, toggling the use of colours, different icons, and different lay-outs.

Furthermore, we have an aim to test the interplay between NOW Interactions, wearable devices, and activity data. Activity data could be used to time the notifications, as well as to design a meaningful time frame in which interactions should be completed.

Even though the results of this pilot-study seems promising, there are still many dots to connect and steps to take in furthering the research, and making NOW Interactions ready for use.

6 Conclusion

In this paper we have argued that it is time to take advantage of the presentation and interaction features mobile technology has to offer.

On a more general, but very important note, this requires a discussion between clinicians and technicians about how standardised self-report questionnaires can be updated while keeping their great, established value, but also making use of the possibilities of modern technology and the needs of respondents.