Keywords

1 Introduction

Sustainability Living Lab (SLL) offers a socio-technical infrastructure for sustainable innovations to emerge, be implemented and tested with and by potential users (Liedtke et al. 2012a). Three elements characterize SLL as a user-centric process:

  • The design is situated in real-life (e.g. existing homes) and realistic settings (e.g. home labs)

  • The focus is on behaviours and experiences of daily life practices

  • The approach addresses the technical, social and temporal dimensions of practices in large scale and longitudinal setups.

Sustainability Living Lab supports a user-centric and contextualized innovation process (Schuurman et al. 2012) in the context of living and working practices. It facilitates the implementation of technical and behavioural interventions in real (-istic) contexts of use (Keyson et al. 2013).

As discussed by Krogstie (2012), Living Lab serves as a platform to combine design research with innovation praxis in which knowledge is generated through the building and deployment of designed artefacts. Sustainability Living Lab combines social, engineering and behavioural sciences with design research to unleash and manipulate the factors that sway experiences around behaviour and technology. As a user-centred process, SLL relies on future users’ participation to understand practices in the presence of designed artefacts. However, existing methods fall short in supporting users in the process of identifying and articulating relevant practices and their impact when discussing the experience around designed artefacts (Krogstie et al. 2013). As practices are adopted and become part of people’s routine, users need to engage in cognitive efforts to bring them to the foreground, resulting in a demanding and biased data collection process (Mulder et al. 2005).

The first generation of SLL innovations falls in two patterns: solutions designed around user behaviour (e.g. home automation) and solutions that aim to control behaviour (e.g. pervasive technologies). These solutions are characterized by a technology-centric approach failing to address the complexity of daily life practices. They assume that behaviours and needs are static elements and do not interact with other elements in social life (Scott et al. 2009). A second generation is emerging addressing the adaptability (Pallot et al. 2010) of these technologies so users can appropriate them in the complexity of their own contexts (Schwartz et al. 2014; Budweg et al. 2011). This view extends the goal of SLL, as stated by Scott et al. (2009) “beyond improving environmental product performance toward shifting lifestyles in more sustainable directions”.

A prospect rises to implement in-situ and integrated design research methods that support users to capture frequent information of their daily practices integrating aspects around users’ needs and values as well as sustainability impact. The knowledge generated provides an integral and contextualized view on daily life practices, encompassing:

  • Description of practices (how are they implemented, what influences them, who and what is involved),

  • Explanation of practices (why do they exists: what is the expected impact on people’s needs, desires and experiences)

  • Assessment of practices (what is the perceived and measurable impact).

The approach is based on in-situ and mixed methods to systemize the integration of objective and subjective aspects of daily life practices at different stages of the innovation process. Integration techniques are implemented at two levels: quantitative and qualitative user-centred methods are integrated to connect daily life practices, technology and user experience; and the objective and subjective aspects of practices are integrated to contextualize users’ experiences and provide links to objective impacts.

In this chapter the aforementioned SLL integrated approach is presented (in Sect. 4.1: In-situ and mixed designs interventions, the in-situ tools and integrated techniques are described). The chapter starts with a brief state-of-the-art review of Living Lab’s methods, the challenges and related approaches. Next, the approach is presented, illustrating three possible integration scenarios. The scenarios target different needs, resources and skills coming from stakeholders, technical facilities, design researchers and future users involved in a Sustainability Living Lab project. The chapter concludes by addressing challenges in the design and implementation of in-situ and integrated methods regarding technology, research, and participation.

2 User-Centric Living Lab Methods

The differentiating aspect of Living Lab Methods compared to other user-centric methods pertains to the active involvement of the users in the R&D process, entailing a collaborator role in creating new solutions (Pallot and Pawar 2012; Eriksson et al. 2005; Niitamo et al. 2006; Schuurman et al. 2012; Krogstie 2012). Users are seen as key actors in bringing the ecosystem of their everyday life central in the process of ideation, experimentation and evaluation of technological artefacts.

From the second generation of Sustainability Living Lab a shift in focus is observed, moving innovations away from addressing what technology can do to achieve sustainable outcomes, to what people can do with technology to develop sustainable practices (de Jong et al. 2008; Scott et al. 2009; Liedtke et al. 2012a; Krogstie et al. 2013; Schwartz et al. 2014). Underestimating user-bound factors like compatibility with lifestyles, aesthetics, and comfort has resulted in developing solutions that have had little to no impact on sustainability when introduced in people’s life context (Scott et al. 2009; Liedtke et al. 2012b).

Therefore research is needed to develop methods and tools that encompass the complex interactions between users, technology and practices in real life context to design for the process of users’ appropriation of technologies and its impact on daily life practices and sustainability. Technology appropriation is a user process of adopting and adapting technology so it fits into their living and working practices. Users may adapt the intended technology use and/or the technology itself to fit users’ lifestyle (Dourish 2003). As consequence, the practices around a technology usage may be altered or new practices may emerge. This in reality may result in users developing new forms of using technology and appliances in the house, as for example when turning on the oven to allocate heat on a painful knee.

Two elements characterize a new generation of user-centred methods to embrace these issues:

  • In-situ methods to capture the temporal and contextual nuances of users’ practices (Mulder et al. 2005; de Moor et al. 2010; Hess and Ogonowski 2010).

  • Mixed methods to capture the technical and social aspects of practices in a qualitative and quantitative manner (de Moor et al. 2010; Schuurman et al. 2012; Scott et al. 2009; Schwartz et al. 2014).

In-situ methods aim to capture an ecological overview of daily life practices, generating knowledge that is bounded to temporal and contextual factors. In-situ methods in Living Lab setups have been implemented as technical and non-technical instruments addressing the need for gathering insights about social practices and social networks (Hess and Ogonowski 2010), measuring user behaviour and experience (Mulder et al. 2005) and measuring quality of experience (Moor et al. 2010). As users need to engage in reporting and reflective activities, challenges related to the implementation of in-situ methods address issues on interruptibility, cognitive demand, boredom and intrusiveness (de Jong et al. 2008; Scott et al. 2009; Rek et al. 2013; Ogonowski et al. 2013). Approaches and techniques have been developed to lower burden by providing a simple structure for describing practices (Scott et al. 2009), to lower interruption by estimating appropriate times for feedback (Vastenburg and Romero 2010; de Moor et al. 2010), to provide benefit through suggestions and social support (Karaseva et al. 2015; Schwartz et al. 2014; Scott et al. 2009; Pallot et al. 2010) and by building trust, transparency and empowerment (Ogonowski et al. 2013; Rek et al. 2013).

Mixed methods extend the descriptive knowledge of practices gathered from monitoring techniques to integrate subjective aspects from a user perspective. Quantitative techniques are valuable to capture large set of objective and subjective data at a relatively low cost, that can be make easily accessible to an open network. Aggregated data provides accurate knowledge on observable behaviours (Veeckman and van der Graaf 2015). However, in the context of Living Lab quantitative methods fall short in two aspects: (a) understanding appropriation of technologies and adoption of new practices; and (b) involving user experience in ideation and evaluation of technologies. Efforts in developing mixed methods for Living Lab are still in their initial phases of conceptualization (de Moor et al. 2010; Schuurman et al. 2012; Karaseva et al. 2015; Pallot and Pawar 2012) or are presented as trials not yet formalized (Schwartz et al. 2014; Scott et al. 2009). These efforts implement integration techniques by collecting data from qualitative and quantitative sources, however they are not addressing other stages of integration and no discussion is provided on how to systemize its implementation. It is expected that a full integration in all stages of a design research project will result in adaptive innovations that are responsive to the interconnections between people's practices, their experiences and related sustainability impact.

User involvement is an ongoing challenge in the implementation of Living Lab’s methods. In addition to the challenges of in-situ methods stated above, Living Lab brings other challenges that exclude user groups from participating. For instance, participation requires users to replace mature technologies with unstable or not fully functional ones, which can drastically affect practices that are well established in people’s daily life (Budweg et al. 2011; Ogonowski et al. 2013). This real cost is only matched by potential benefits of user participation in contributing to innovation. These benefits in most cases fail to address the interests and needs users have when participating (Mensink et al. 2010).

From a research perspective, Living Lab poses another challenge to support large-scale and cross-national projects. On the one hand, this entails collecting data efficiently as well as ensuring consistency across cases. On the other hand, this requires flexible methods to address different needs, resources and skills from the parties involved.

3 Emerging Methods

In-situ methods have been proposed as a promising strategy to characterize practices from a user perspective and at different time frames. This enables comprehending practices within the complex ecosystem of users’ experiences and lifestyles. State-of-the art implementations in Living Lab (de Moor et al. 2010; Mulder et al. 2005; Romero et al. 2013) refer to Experience Sampling Method (ESM) as an appropriate approach to connect user experience and practices to real contexts and for long periods of time. Daily Reconstruction Method (DRM) is an alternative in-situ strategy that characterizes practices of one day through a systematic reconstruction process on the following day.

Sensor networks are also discussed as relevant techniques to contextualize daily practices. The advantage of these two prominent strategies in Living Lab settings increases when they are integrated. Whereas integration has been mostly implemented at data analysis, integration at other stages of the design process opens up opportunities to facilitate in depth and focus insights and exploration of practices. Mixed Method Research (MMR) addresses the need for integration at different stages in a research process defining several mixed method designs that support different integration strategies.

In the following sections a brief introduction of the Mixed Methods Research, Experience Sampling Method and Daily Reconstruction Method is provided. Wireless sensor networks are out of the scope of this chapter, as they do not directly involve researcher, designer and users. For detailed information about wireless sensor networks, please refer to NRC (2001).

3.1 Mixed Methods Research

Mixed Methods Research (MMR) refers to the integration of qualitative and quantitative approaches to answer research questions (Creswell and Piano 2011). Methods are integrated at different stages in the research process including data collection, data analysis and data interpretation. Qualitative and quantitative data can be mixed in three different ways: by connecting, having one data source build on or follow up on the other; by merging, to compare or relate results from both data; or by embedding, to explain one data result by the other (see Fig. 2.1).

Fig. 2.1
figure 1

Three ways of mixing data. Notation: a predominant method is symbolized in capitals; in the absence of a predominant method both approaches are equally represented in the results (Creswell and Piano 2011)

Full size image

MMR offers a pragmatic orientation to address “practical” issues related to a research problem. For example, when dealing with the complexity of a situation, when knowledge needs to be contextualized, when individuals with different methodological orientations need to work together, when the expected impact cannot be obtained with only one type of data, or when there is an explicit need to do qualitative research.

3.2 Experience Sampling and Daily Reconstruction Methods

Measuring user experiences contributes to the assessment of technology appropriation. User experiences assess the interconnections between user, daily life practices and technology. It characterizes the interaction with products in different time span of usage (Roto et al. 2011): anticipated experience (before usage), momentary experience (during usage), episodic experience (after usage) and cumulative experience (over time). While all stages are relevant to the process of appropriation, momentary experience deserves special attention as it encapsulates the dynamics of the adaptation and adoption processes. In-situ self-reporting methods are used to capture momentary experience.

From socio-psychology research, the Experience Sampling Method (ESM) was developed in the late 60s in respond to the appearance of a technology (the pager) that could prompt people on the move allowing researchers to ask at random times questions to capture people’s feelings in the moment (Barret and Barret 2001; Larson and Csikszentmihalyi 1983). ESM has evolved in the last decades, including context-aware capabilities to expand the sampling strategy from random, to time-based and to event-based. A context-aware ESM tool combines sensor networks with self-reporting techniques providing a good platform to link instances of technology use and self-reported experiences (Consolvo et al. 2006; Intille et al. 2003).

There are important considerations in the design of ESM studies. As noted by Myin-Germeys et al. (2009) and Hektner et al. (2007) ESM designs should take care of the frequency, time-demanding and cognitive effort of participants to self-report. On the long term, participants often lose their motivation to provide information every time they receive a prompt. Issues related to repetitive interruptions arise (Christensen et al. 2003), creating barriers for long-term participation, such as annoyance, burden and boredom (Scollon et al. 2003). Adaptive sampling rates aim to avoid undesired interruptions (Vastenburg and Romero 2010) while engaging strategies such as empathy, personal benefit, fun and control could keep user to self-report for longer periods (Rek et al. 2013).

Daily Reconstruction Method is an alternative method that implements users’ data collection of the experience of a given day by a systematic reconstruction process conducted on the following day (Kahneman et al. 2004). Compare to ESM it reduces users’ burden and captures a more complete coverage of the day, however it increases memory bias. A combination of ESM and DRM has been proposed (Khan et al. 2008) where ESM works to capture short moments in the day that are later used as memory anchors for reconstructing the experiences and practices around them.

4 Mixed Approach for Sustainability Living Lab

The presented approach aims to systemize longitudinal, large scale and cross cultural SLL studies by implementing in-situ methods and integration techniques at different stages of the innovation process.

Figure 2.2 illustrates the three levels of integration proposed, using a three-ring metaphor of the top view of a funnel, starting from an extended surface representing the complexity of the context under study, moving deeper into more specific and narrow areas of practices, to finally touch upon specific sustainability and human aspects of practices.

Fig. 2.2
figure 2

Mixed approach for sustainability Living Lab

Full size image

The four axes showed in Fig. 2.2 represent different types of user involvement (Users), design research approaches (Design), research goals (Research) and innovation outcomes (Outcome).

  • Users: users’ involvement defines different roles of users as collaborators in the design research activities. The outer ring represents sporadic and passive involvement of users in collecting data. The middle ring indicates an active role in generating and interpreting insights. In the inner ring users are active collaborators in ideating, prototyping and evaluating solutions.

  • Design: design research approaches incorporate users’ needs by means of different activities that result in solutions addressing different levels of complexity. The outer ring provides solutions that address general user needs in context, by means of surveys, interviews and monitoring sensor networks. The middle ring offers solutions that involve deeper insights, which are generated together with users, by means of in-situ self-reporting tools. The inner ring brings solutions that are developed by users, by means of co-design and prototyping sessions.

  • Research: the depth and richness of knowledge that can be generated varies at the different levels of integration. The outer ring characterizes knowledge based on the validation and verification of current practices. Knowledge in the middle ring moves deeper into explaining and exploring existing and alternative practices; and the inner ring relates to knowledge generated by the experimentation and evaluation of sustainable practices.

  • Outcome: different levels of knowledge of practices can be obtained at the different levels of integration. The outer ring offers long-term analysis and description of existing situations and their impact on sustainability. The middle ring implements interventions exploring deeper into more specific and narrow areas of practices. The inner ring involves prototypes which touch upon complex and specific sustainability and human aspects of practices.

These axes form quartets of [User, Design, Research, and Outcome] setups to characterize each level of integration. For example:

  • The outer ring is characterized by a sporadic and passive user involvement in collecting data (User), where activities address generic users’ needs (Design), the generated knowledge describes the sustainability of current practices (Research), and innovation is informed by identifying directions and potential impact (Outcomes).

  • The middle ring is described by an intensive and active involvement of users in collecting and generating insights (User), activities involved deeper and complex users needs (Design), knowledge explores alternative sustainable practices (Research), and innovation develops contextual interventions on specific areas of practices (Outcomes).

  • The inner ring incorporates users as active collaborators in ideating, prototyping and evaluating solutions (User), activities materialize expectations and desires of users (Design), knowledge projects the impact of new practices (Research), and innovation develops prototypes with validated impact on sustainable practices (Outcomes).

4.1 Integration Techniques Based on Mixed Methods

The integration levels are implemented by integration techniques based on Mixed Methods Research (Creswell and Piano 2011). The overall integration gives priority to qualitative methods in understanding, experimenting and evaluating user appropriation of technologies and emergent practices. Quantitative methods are embedded offering an objective and subjective layer to measure impact. The interaction techniques support the development and application of mixed tools. In this section, the integration techniques and the tools for each level are described. In Sect. 4.1: In-situ and mixed designs interventions an implementation of these mixed tools is presented in the context of home energy use.

First level of integration (outer ring): This level is characterized by merging techniques for data analysis. Quantitative data from sensors and other objective data sources are merged to describe baseline impact on sustainability. Merging of quantitative data is also implemented with qualitative data from interviews conducted at the beginning and end of the study to describe sustainable and non-sustainable practices.

Second level of integration (middle ring): Connecting and merging techniques are applied in data collection, analysis and interpretation. In data collection, quantitative and qualitative self-report tools are connected to support users inform qualitative and reflectively on daily practices. For instance, by connecting quantitative reports as inputs for qualitative self-report tools, so the earlier work as memory anchors to facilitate reflections in the latter (ESM supporting DRM). For data analysis, quantitative data from sensors and quantitative and qualitative data from self-report tools are merged to describe the impact of a specific context on sustainability and on user experience. The outcomes are contextual users insights on daily practices and mixed probes: visualizations of the integration of objective and subjective data around practices.

For data interpretation, this level also supports user research design sessions (e.g. contextual interviews and user re-enactment). Mixed probes are connected to these sessions to get deeper and focus explanations of the phenomena described in the first level.

Third level of integration (inner ring): after analysis, connecting techniques are applied between the resulted mixed probes and co-design and co-prototyping sessions to interpret the results and generate requirements for the design of artefacts/prototypes.

An interactive setup of user experimentation and evaluation applies real-time merging of sensor networks and self-reporting data to enrich data collection. Through, in-situ interventions and in-situ experiments, merged data of sensors and self-reports is used to provoke reflective insights from users as well as to evoke experiences by guided interactions with artefacts, respectively.

4.2 Choosing the Appropriate Level of Integration

An ideal innovation process includes all levels of integration to address in its full extent the complexity of users’ appropriation of technology and adoption of sustainable practices. However, the approach offers alternative setups to address specific configuration in SLL projects. The criteria for selecting the appropriate level of integration consider:

  • The setup and scope of the project

  • Project resources (technical infrastructure)

  • Collaborators (researchers, designers and users) skills, experience and availability.

A SLL project may encompass several studies with different setups and scopes. Setups involving prominently quantitative methods and with a descriptive goal are placed in the outer ring—first level of interaction. For instance, the first level of integration supports the implementation of long-term monitoring studies and pre and post user interviews with the purpose to define a baseline of practices and their sustainable performance. When the outcome is aimed to go beyond a descriptive baseline, qualitative methods are needed to support the involvement of users and therefore a deeper level of integration is suggested.

Project resources also affect the integration depth in the study. The deeper the layer of integration the higher the need for a robust technical infrastructure to support the development and application of mixed tools.

Finally, the availability, experience and skills of collaborators define to what extend mixed tools and design research sessions can be developed and applied. On the one hand, the intensity and frequency of the sessions depends on the time availability of collaborators. On the other hand, skills are needed for richer and deeper use of the mixed tools. For instance, design researchers’ skills in ethnographic and co-design may result in better practices to incorporate mixed tools in the sessions. Similarly, cognitive skills are needed from users in generating and applying mixed probes as well as in participating in in-situ interventions and experiments. Therefore different users’ needs and abilities require different variations of in-situ self-reporting tools and integration techniques.

5 Conclusion and Challenges

This chapter introduces a methodological approach for Sustainability Living Lab that stages an innovation process based on user-driven in-situ methods, sensor networks and integration techniques. The integration techniques intends to empower collaborators in connecting and contextualizing daily life practices, technologies and user experiences in the process of developing sustainable innovations. By incorporating tools based on quantitative and qualitative methods and by mixing objective and subjective data, the integration techniques elicit and trigger descriptive and reflective insights at different stages of the innovation process.

The central and active role of users as collaborators is supported by means of mixed tools that are developed and applied by them at different stages. Different levels of integration are proposed by setting up different research activities and user involvement (see Fig. 2.2):

  • First level—outer ring offers verification of daily practices in context by means of low contact user research and monitoring tools. This level is implemented by merging integration techniques for data analysis.

  • Second level—middle ring explores and analyses opportunities for sustainable practices by means of design interventions. Merging and connecting techniques are implemented at all stages of data collection, analysis and interpretation.

  • Third level—inner ring supports users in the development and evaluation of their own solutions for sustainable behaviour by means of co-creation and self-experimenting sessions. Real-time merging and connecting techniques are implemented at all stages of collection, analysis and interpretation.

There are two main impacts on innovation that are expected by using this approach. First of all, the resulted innovations address the complexity of technology appropriation in daily life practices. Secondly, and as consequence, such innovations enable dynamic processes of adoption of sustainable practices.

The promising aspects of this approach still require further research to address issues with respect to technology dependency and methodological scope.

The integration techniques rely strongly on high-end and expensive technical infrastructure based on wireless sensor networks and big data analysis. Stability, reliability and scalability of this infrastructure are required to guarantee successful implementations in real life contexts, for long periods of time, and while capturing, analysing and visualizing continuous streams of contextual and behavioural data. When resources are not sufficient to ensure these requirements, cheaper alternatives will result in unstable, less reliable and less scalable setups and higher efforts from collaborators.

The implementation of large-scale and cross-national projects requires that the application of methods and techniques is replicable and comparable. Despite the effort to systematize the proposed approach, as reported earlier, the action of conducting the methods and techniques is vulnerable to contextual and subjective factors. This may result in knowledge generated by data gathered at different frequency and depth (quality). This limitation opens the discussion in the Sustainability Living Lab agenda with regard to the comparability of cases and a user-driven process.