1 Introduction

Everyday life today is being computerised with increasing speed. Science fiction writers and scientists have visions of computerised life of the future that will have effects on humans’ lives in good and bad ways [1–5]. Mark Weiser also had a vision of a computerised future but in a way that people would not even notice that their life is supported with ubiquitous computing (i.e. technology which is at once pervasive, yet invisible) [6]. In their article, Coming age of calm technology, Weiser and Brown brought out the need for design principles and methods that enable users to exploit topical information all of the time because “calm technology engages both the center and the periphery of our attention, and in fact moves back and forth between the two... As we learn to design calm technology, we will enrich not only our space of artifacts, but also our opportunities for being with other people. When our world is filled with interconnected, embedded computers, calm technology will play a central role in a more humanly empowered twenty-first century” [7].

Naturally, not all computing applications can, and will, be calm. Some applications are designed to attract the user’s attention (e.g. games in virtual reality) and others might be adjusted to comfort the human need to be able to control the computerised environments in a more conventional way when preferred. In any case, it is nowadays widely recognised that intelligent compound systems, where many users and devices communicate simultaneously, clearly require a new approach to system design and evaluation [8, 9].

Fig. 1
figure 1

Evolution of computing (drawing by Pertti Jarla)

Full size image

Shifting focus from “off the desktop” computing to ubiquitous computing applications (that should be available everywhere and all the time) means that we have to understand and support a very heterogeneous group of users’ everyday practices [10]. Besides, if we try to create the user experience as pleasurable as possible in this kind of computerised environment, we also have to develop new interfaces. Interfaces that are transparent and support different user action and goals seamlessly ought to be, in many cases, gesture- and/or voice-based. Interfaces may even be emotionally adaptive (affective interfaces). The challenges of emerging new technologies and services, especially in the field of mobile and ubiquitous computing, also call for new approaches to build systems and interfaces that naturally support users’ goals and actions (Fig. 1).

VTT (Technical Research Centre of Finland) has studied and developed ubiquitous computing in several projects from concept design up to working artefacts and environments [11, 12]. In these projects, new methodological challenges have been recognised, and particularly, in the early stages of the product development process. Most of all, the early concept definition phase, including different participants (and stakeholders), has been recognised to be the most important factor, determining the success of the whole development process. In VTT’s internal methodology development project, Käykse (“Is it OK?”), we moved from ethnography (e.g. user observation and interviews) to methods in which parties sketch together an intelligent environment that suits them all. Because technical building and prototyping of intelligent environments is expensive, illustrations and modelling are needed to allocate resources effectively.

The goal of the Käykse project has been to recognise the problems in designing and evaluating intelligent environments, and to evolve the research frame. The intention was to increase the dialogue between experts (e.g. technology developers, system designers) and potential users. From now on, when we mention experts, we mean all of the different professionals related to product development. The project aimed to develop design methods that help in adjusting user needs and technical possibilities to each other. We have sought to understand the user experience of tomorrow’s services and products as early as possible, and to create an innovative, inspiring, pleasurable and entertaining user experience and concept definition session to all stakeholders.

The structure of this paper is chronological. Firstly, in Sect. 2.1, we present the groundwork of the approach: human–computer interaction (HCI) related participatory design. In Sect. 2.2, we introduce recent research on adapting novel methods to the concept definition phase. In the next section, we present our own cases and, finally, in Sects. 4 and 5, we give some recommendations based on our experiences and introduce some of our future development ideas (Fig. 2).

Fig. 2
figure 2

Presenting new interfaces for potential users (drawing by Pertti Jarla)

Full size image

2 Participatory design and recent novel methods for early concept definition phase

2.1 Participatory design in HCI

Early user involvement in the product development process is nowadays generally accepted and standardised [13]. However, the methods and techniques for accomplishing this may vary widely. It is also uncertain whether a product development team will actually commit themselves to employing potential users in designing the forthcoming product or service as early as possible. Reasons to this arise from the increasing cost of the early product development phase and the difficulties in finding and understanding correctly the available methods. The most often used methods for early design phases are interviewing or focus groups. In addition to these so-called traditional or conventional methods, new methods have been adapted from other areas (e.g. ethnography, applied anthropology and participatory design) to better understand the actual usage situation of potential users with a new technological application [14–17].

The foundations of HCI-related participatory design can be traced to 1970s Scandinavia, from where it spread to other parts of the western world. User-centred design (UCD) and participatory design have shared many ideas, techniques and methods, but, in a way, UCD has in the past been more technology-driven, focussing on laboratory testing and finding so-called interface-related usability problems. Participatory design, on the other hand, has emphasised some ideological arguments (e.g. democracy) and given a more holistic view to system development [18, 19]. For the past ten years, and surely partly because of the launching of ubiquitous computing, the HCI community and UCD methodology have shifted towards a more holistic view of involving users in the product development process. Besides the conventional methods of user involvement, innovative user participation and user-driven innovation sessions have also been introduced.

2.2 Recent novel methods for early concept definition phase

There have been several recent attempts to renew the role of participants in designing solutions for future environments. Active participating roles for the potential end users have been achieved by utilising e.g. drama, props and role-playing. One of the latest examples of this is the work done by Howard et al. [20] in which “scenarios are ‘acted out’ by actors and/or candidate users during participatory design sessions, rather than being ‘walked through’ by designers and users.”

Role-playing sessions have been made more vivid by different means. In role-playing games, the users have participated in creating use scenarios and product concepts by playing in a mise-en-scène made out of toys. Users have then envisioned and tried out product ideas according to a given situation and a set of rules [21]. In another case, called situated and participative enactment of scenarios (SPES), users were provided with a simple mock-up of the device and they were followed during their daily routines. The mock-up helped the users to envision the use scenarios as situations arose during the day [22].

In a Dynabook project, the designers visited users’ homes and observed the users as they performed the given scenarios of possible uses for an electronic book. The project indicated that scenarios created by the users in an actual usage environment should complement the scenarios created by the designers [23]. Similarly, in a method called “bodystorming”, the design sessions were carried out with real users in the original context instead of the office [24].

In experience prototyping, the designers investigated user needs as actors role-playing users in a real user environment. The aim was to find solutions for a new rail service, and the design team first took train journeys and role-played users in a staged user environment, and then in a real use environment. The technique helps the designers to understand the users’ point of view when designing future devices [25]. In the Smart Tool project, the users were involved in envisioning the future design concept with the use of drama. The designers played out scenarios based on the environment of the user. The users were also invited to a workshop that was conducted in a staged environment created by the designers [23]. Real users as actors and improvisation methods have recently been tested mainly for the design of working environments [26–28]. There has also been attempts to evolve the drama approach for product development and to apply drama methods more profoundly to the technology development process [29].

3 The interactive scenario

3.1 Starting point

We developed the interactive scenario method to increase the participation of potential users in the early stages of concept design. We sought flexible methods so that they can be utilised in various projects dealing with ubiquitous computing. We started with role-playing methods and found those very useful and rather light to put into use. However, we wanted to evolve methods that involve physical participation. Improvised acting and scenario playing contain many of the elements we sought, so we decided to base our methods on them.

In testing the methods, we concentrated on the cases of a smart home and a future ski resort, because at that moment, there were projects going on that needed new ideas for designing concepts for these environments. We found that it is essential to know the theme of development. The aim of testing was to develop the methods further, find new ideas, discover methodological problems and try to solve the found problems. According to the observations, findings and feedback, the upcoming sessions were prepared and the methods developed further.

To evaluate improvisation, we videotaped all the sessions. We also observed and participated. After each session, we sent the participants a questionnaire where they were asked to comment the method, their feelings during the session and opinions about their own role in the session. We describe the sessions in the next section.

We held three improvisation sessions, each with different participants. The first session consisted of experienced improvisation actors, experts and our research team. Our aim was to test the method and improve it, based on our results. Potential users attended the second session in the audience, influencing the acting and, in the last session, the improvisation actors were left out and the users were encouraged to act out scenes with the research team. In order to gain results of first-time experiences, we had different actors, experts and potential users present in all three sessions.

3.2 Improvisation acting in design

We recruited the improvisation actors for our research from a local improvisation theatre group. One of our team’s researchers is a member of the group and acted as the director of the sessions. The group has several years of experience in improvisation acting based on the guidelines of Keith Johnstone. Johnstone teaches a style of improvisation which encourages the use of certain restrictions for the actors. These restrictions function as guidelines, which makes it easier for the participants to improvise. These restrictions can include physical or social features of played characters, or themes and contexts where the acting takes place [30]. After a consultation with the group, we decided to adopt several of the performance techniques they use as the basis of our first two sessions.

The first technique we used was what the actors called “piece of paper” improvisation. During their performances, they had asked audience members to write sentences of speech on pieces of paper, one sentence on each. These pieces of paper were placed on the stage so that they could be randomly picked. During a scene, the players would occasionally pick a piece of paper and read the sentence on it aloud as a line, which will then be incorporated into the play. The actors don’t read any of the pieces beforehand. We decided to write sentences describing certain ubiquitous technologies on these pieces of paper and see how the actors reacted to these.

The second technique took advantage of the improvisation actors’ experience in acting out non-human roles. We assigned each actor the role of a household appliance or area. The items and areas were able to communicate with each other, as well as perform functions, which the actors found useful. We occasionally presented different situations to this artificial home, and then observed how the actors reacted.

The final technique was a variation of the second technique. One actor took the role of a human, while the other took the role of a smart home. The human phoned home and the ensuing conversation was observed.

In all of the techniques, the audience was free to interrupt and ask questions, as well as guide the situation towards a different direction. Interaction between the observers and actors was encouraged, and the situations seen were discussed after the scenes.

In the last session, our aim was to get the research team and the users to act out scenes together. Techniques as such were only used as an aid to planning the session. We decided to approach the situation by starting a normal conversation of the technologies and situations we wanted to address, and then act out scenes from our conversation.

3.3 First session: improvisation actors and the research team

The first session took place between our research team—four experts in ubiquitous environments—and four improvisation actors. The session took place in our virtual reality laboratory, which was organised in a traditional theatre style with a stage and an auditorium. The actors were not briefed before the session of the ubiquitous technologies covered in the session. All of the three techniques were used.

We used the “piece of paper” technique in two different scenarios (Fig. 3). In the first one, two actors were playing the roles of a couple living in a smart home. They did their dishes, vacuumed and performed other household tasks. The occasional reading of the prepared sentences by the actors brought new technologies into the scene. Occasionally, the scene was put on hold and the situation discussed with the audience. The audience gave input on how realistic the situations seemed, suggested alternative ways of doing things and thus affected the outcome of the scene. The second “piece of paper” scenario consisted of two actors in a skiing resort with modern technologies available. This was acted in the same way as the first scenario.

Fig. 3
figure 3

“Piece of paper” act in the first session

Full size image

In the machine-to-machine communication of the second technique, we asked the audience to provide the roles of the actors (e.g. vacuum cleaner, kitchen, living room, refrigerator, porch). Again, active communication between the actors and the audience was encouraged.

The third technique presented one actor in the role of a businessman returning to a futuristic smart home from a business trip. The businessman phoned home and gave it tasks to do before he got home. The home responded and their dialogue was observed.

3.4 Second session: improvisation actors, research team and the users

The second session was organised in the same way as the first. We used the same laboratory organised in the same layout. The largest difference to the first session was the presence of potential users in our audience. The users were selected according to design-for-all principles. The four users present were aged 70, 67, 21 and 21. All of them had experience with computers, and all but one used a mobile phone. The “piece of paper” technique was again used in two different scenarios (i.e. smart home and skiing resort) in the same way as during the first session. This was done in order to get the audience acquainted with the idea of improvisation acting. After the “piece of paper” technique, we presented ubiquitous technology ideas to the audience and discussed possible usage situations for them. The actors then played scenes portraying these situations. After each scene was over, we discussed the events with the audience to find out their reactions to what they had seen.

The last technique was quite similar to the machine-to-machine technique used in the first session (Fig. 4). We asked the audience to name a few household appliances or areas of a home. In addition to this, we asked for events to occur during the acting. The actors incorporated these events into their improvised scene. In order to better guide the scene, this time we interrupted the acting regularly with a “pause” command, upon which the actors stopped acting while we discussed the situation and gave more directions from the audience.

Fig. 4
figure 4

Robotic appliances and discussion after scene

Full size image

3.5 Third session: research team and users

The third session was very different from the first two. Our aim was to study the possibility of acting without the help of professional actors, with the research team and users doing the acting themselves. Our design team and three potential users attended the session. Feedback from the second session indicated that people would be more inclined to act while in familiar company, so we chose three users who were familiar with each other and with us from previous projects. The users were all roughly 20 years of age.

We began the session with a very light discussion to make the users feel at ease in the situation. After this, we discussed different situations in ubiquitous environments, and, based on these discussions, two members of our research team acted out a scene (Fig. 5). Our aim was to familiarise the users with the idea of acting, and to encourage them to act themselves. Next, we discussed new situations with the users and constructed a scenario where they were attending a sports competition in the Finnish town of Kuopio. None of the users were enthusiastic about acting, so we ended up just talking the scenario through. However, the users did take up the intended roles during this discussion, so only the physical aspects of acting were lacking.

Fig. 5
figure 5

Potential users acting at the sofa (using an intelligent media appliance) and research team member acting a scene

Full size image

We ended the session with one of our team acting as a household appliance in a home that the users inhabited. The users chose the appliance to be a futuristic television and got so carried away by the scene that they ended up acting some situations after all.

4 Results

4.1 Feedback from the users and the experts

After each session, we sent the participants a follow-up questionnaire. According to the feedback, both the users and the experts found the atmosphere in the sessions relaxed, although a common “warm-up” in the beginning would be useful to get to know each other and ease the boundaries between users and experts. The experts found the method to be fresh and open. The emergence of ideas in the sessions is immediate and helps the researchers to understand the users’ point of view. Also, according to the researchers’ feedback, the usage situations based on improvisation are intuitive and concrete. The sessions should be carefully planned, but at the same time, have time for the ideas to emerge. The experts also pointed out the challenge of transcribing the material collected from the sessions.

The potential end users appreciated the way we collected their opinions in the early phases of the design process. They pointed out the importance of briefing in the beginning of the sessions so that participants know what is expected of them. The end users also stressed the controllability of the sessions because the scenes can easily deviate from the actual theme. According to the feedback, the participation of end users is desirable but the present audience was somewhat hesitant to act in the sessions. The majority of the participants found the method very entertaining.

4.2 Main findings

The method works at its best in the ideation phase when designing large complex entities, e.g. ubiquitous environments. The technical features cannot be discussed in detail using this technique. Improvisation is especially useful for testing ideas, specifying existing scenarios and designing concepts. The method also works when illustrating ideas, concepts or usage situations for end users. Services, especially everyday life systems related to spaces, operating sequences and stages that differ from each other, can be designed by means of improvisation. In addition, improvising services and applications in working environments would presumably be very useful with professional users.

In the future, the method should be developed more towards drama with a more specified script of the actions. At the beginning of the session, there should be a common discussion to orientate participants to the methods and learn the aims of the session. That way, there would be more interactions between the actors and the audience.

Our aim was to generate ideas and specify scenarios in a participative way. It was easy for the actors to understand the spatial and bodily dimensions of the scenarios. Hence, we found that the method was effective when designing spatial interfaces and ubiquitous computing environments. Also in the evaluation of ubiquitous computing environments (e.g. in laboratory set-ups), it would be fruitful to act the usage situations.

Both the audience and the improvisation actors invented completely novel usage situations. For example, when improvising the sending of MMS messages, they found means that are much more usable than the existing ones. We also found that the method is not that suitable for analysing detailed technical solutions. As a result of using the method, it is possible to elicit usage-context-centred ideas for both functional (user-related) and non-functional (technical) requirements.

Actors

Improvisation actors are experienced in generating new ideas in a short time. Therefore, it was easy for them to identify with the given usage situations. The possibility for the other participants to comment and participate has to be planned carefully because the actors led the situations quite enthusiastically and independently.

During the sessions, generating ideas was sometimes rather lively and, therefore, it is essential to invest time in analysing the results from video tapes afterwards. In the sessions, the ideation should be as free as possible and no criticism should be allowed. Consequently, the new ideas which, at first sight may seem ridiculous, appear to be the most feasible.

Product developers

Since the experts (researchers, designers etc.) are familiar with the technologies and themes, they can inconspicuously guide the sessions with a technical view, wishes and limitations. Also, the business partners can join the sessions and give their input to the scenes by, for example, writing cards that include technologies and contexts. Thereby, they have the possibility to influence the development process according to their special interests.

Potential end users

When the end users join the sessions as actors, much more effort should be put in to create a comfortable and relaxed atmosphere. We found that the atmosphere was more relaxed when both the actors and the audience consisted of potential end users. Also, in the beginning of the session, it is important to introduce the technique and the session to the participants in a suitable way according to the goals of the session and the composition of the participants. The users should know what is expected of them. Also, certain directions concerning acting will help them to orientate themselves when e.g. acting a given situation. We found it challenging at the same time to give limitations and still maintain a free atmosphere to generate novel, and even wild, ideas.

In addition, it should be noted that the group ideation methods are not suitable for everybody, and acting may not be easy for all participants. Overall, acting should be based on volunteering and the participation to improvisation and scenario playing could be arranged so that participants have a clear choice of whether to act or otherwise participate in the session. The significance of a skilful director is emphasised when potential end users join the session. Particularly with large groups, it is important that the director makes sure that all participants have the possibility to express their opinions.

5 Conclusion

The interactive scenario method including improvisation and user acting seems to be very suitable for early-phase concept definition of complex systems that require “off the desktop” kind of activity (i.e. ubiquitous computing especially). The method is not an easy one and it takes much less effort to start with simple role-playing sessions than to plan, carry out, analyse and report on an interactive scenario session. However, the results clearly show that, where issues such as spatial and user’s physical interaction with futuristic interfaces are concerned, physical acting of scenarios brings out aspects (ideas, innovations and problems) that would not have been recognised so obviously in any other way. Even though the method requires more work (e.g. preparation) and practice, one of the most important rewarding aspects has been that all of the participants, including the research team, had a very cheerful time in these sessions. This speaks strongly for utilising the method in future projects.