Instrumental genesis in the design studio

Abstract

The theory of Instrumental Genesis (IG) accounts for the mutual evolution of artefacts and their uses, for specific purposes in specific environments. IG has been used in Computer-Supported Collaborative Learning (CSCL) to explain how instruments are generated through the interactions of learners, teachers and artefacts in ‘downstream’ classroom activities. This paper addresses the neglected ‘upstream’ activities of CSCL design, where teachers, educational designers and educational technologists use CSCL design artefacts in specific design-for-learning situations. The paper shows how the IG approach can be used to follow artefacts and ideas back and forth on the CSCL design and implementation pathway. It demonstrates ways of tracing dynamic relations between artefacts and their uses across the whole complex of instrument-mediated activity implicated in learning and design. This has implications for understanding the communicability of design ideas and informing the iterative improvement of designs and designing for CSCL.

Introduction: researching CSCL design

There are several complementary strategies available for research that is intended to improve the quality and outcomes of Computer-Supported Collaborative Learning (CSCL). For example, one can focus on CSCL itself, trying to understand the processes, relationships, practices, tools, and so on that are involved in successful and unsuccessful learning episodes. But one can also study the educational work involved in facilitating, orchestrating, and/or designing for CSCL. At first glance, research on how people design for CSCL may seem unnecessarily far away from where the important action takes place. However, design is a key locus for the incorporation of the outcomes of research into real-world educational work. We cannot learn for other people, but we can try to be, and to help others become, better and more knowledgeable teachers and designers. From this view, research into how people design for CSCL – and how they get better at designing – is closer to the real action (of improving educational opportunities), and more consequential, than might first appear. Our program of research has the practical goal of supporting and improving educational design activity, including through the development of methods, tools, and resources for educational designers. This motivates an accompanying scientific goal: we need to understand how educational designers – whether specially trained or self-taught – engage in design work, and how their design practices change, so that more effective design methods, tools, and resources may be developed and adopted. This is a dynamic space in which to work: design tools and practices co-evolve, so we need ways of conceptualising the phenomena we are investigating that help us track their joint movement.

In this paper, we adopt an approach based on the theory of Instrumental Genesis (Lonchamp 2012; Rabardel and Bourmaud 2003) to investigate CSCL design activity: focusing on the mutual evolution of artefacts and their use, within an educational design context. Our main aim is to understand the evolving role of artefacts (digital and material) in mediating CSCL design activity. We do this in a way that also acknowledges the distributed nature of design and of learning in CSCL. Design activity, in many educational situations, is distributed across people and time. For example, students often have to do non-trivial design work to customise a collaborative learning task they have been given, and/or to agree on an appropriate set of tools and resources to use. Similarly, designers can learn from the successes and failures of work they have done, and so can those of us who are trying to research and improve CSCL design tools and methods. Thus, a second aim of our paper is to track and illustrate some of the ways these design and learning processes unfold, and artefacts move, back and forth along the CSCL design and implementation pathway.

The first part of the paper introduces the problem area and the theoretical framing, and provides a brief review of related literature focusing Instrumental Genesis (IG) and its relevance for CSCL research and practice. We build upon the work of Lonchamp (2012), who first introduced IG to CSCL. We then describe our research aims, setting, design, and methods. After that, the paper provides illustrative examples of how the IG approach can be used to follow artefacts and ideas back and forth on the CSCL design pathway. The discussion in the last section of the paper complements and extends the analysis of IG in CSCL offered by Lonchamp (2012) and others, showing how CSCL research, design, and learning activities can be productively regarded as a single, distributed, dynamic system.

Framing the problem: educational design for CSCL and instrumental genesis

Researching design

Studying how teachers (learn to) orchestrate and facilitate CSCL is well established as a research area (Song and Looi 2012; Looi and Song 2013; Leeuwen 2015). Studying and improving educational design – otherwise known as ‘design for learning’ - can also enhance opportunities for knowledgeable action in educational practice (Goodyear 2015; Laurillard 2012; Mor and Mogilevsky 2013). Usually, there is more time to consult, and consider the applicability of, research evidence from the learning sciences when designing tasks and materials, and there is less time for this in ‘live’ classroom teaching. However, effective participation in research-informed design activity depends upon educational practitioners being equipped with ideas and tools to support their engagement in productive practices of design. This places the understanding of practices related to the use of technology in educational design squarely on the research agenda.

As White (2008) points out, within educational design research, artefacts are created with the dual purpose of supporting and investigating learning. Our research extends this idea: providing insights into the role of artefacts in the collaborative learning of educational designers and of educational design researchers.¹ On this view, educational designers are not only designing for other people’s learning; they are themselves learning - about students’ needs and learning contexts, new design tools and methods, ways of solving emergent problems in their designs, etc. Design researchers like ourselves are also learning from our empirical research, including from our observations of, and reflections on, CSCL designers in action. Design and design research activity involve mixtures of intentional and incidental learning which also inform modifications to the environments in which future design and learning activities take place (Damşa et al. 2010; Illeris 2009; Kali et al. 2011).

Design theorists have offered some sharply contrasting ways of understanding and describing design. Advancing a ‘technical-rational’ view, Simon (1995) described design as “inherently computational – a matter of computing the implications of initial assumptions and combinations about them” (Simon 1995, p. 247). By contrast, Schön (1987) saw design as a form of reflective practice, involving the application of personal knowledge and experience to each unique set of circumstances. Debates about whether design is a ‘science’ or an ‘art’ abound in the design literature and in studies examining how designers learn and practice design (Adams et al. 2003; Carvalho et al. 2009; Dorst and Dijkhuis 1995; Papanek 2001; Schön 1987; Simon 1996). Although competing views may influence how designers engage in collaborative design activity, this specific discussion is beyond the scope of the present paper. For the purposes of the present study, we refer to ‘design’ as an intentional activity of transforming ideas and knowledge into an artefact, product, or service. Our specific focus is on ‘educational design activity’, where designers are creating and (re)configuring artefacts, products, or services with the goal of facilitating and supporting other people’s learning.

It is essential to note that the design of artefacts, including tools, often continues into the period of the artefact’s use. Design does not stop with the (professional/specialist) designer: ‘users’ also design, and this continuity is important (Folcher 2003; Manzini 2015). This has been acknowledged in a number of commentaries on design. For example, Krippendorff (2005) talks of a ‘semantic turn’ in design: such that the designer’s primary goal should be to create artefacts that have meaning in the lives of their users, enabling users to move the artefacts onwards through various kinds and stages of usage. “No artefact can be realized within a culture without being meaningful to those who can move it through its various definitions” according to Krippendorff (2005, p. 186). Gatt and Ingold (2013) also remind us that designers design things in a world that is constantly under construction, the world is changing as it responds to the activities of its inhabitants. Designers’ outputs need to mesh with the ongoing activities of other people (Goodyear and Dimitriadis 2013).

Such insights should make it clear that the proper evaluation of a new design tool or educational artefact is no simple matter. We cannot assume that tools and other artefacts have entirely fixed properties or that their users have fixed preferences or fixed methods of use. To understand what is happening in the examples of computer-supported collaborative design for CSCL presented later in this article, we need a theoretical framework that can bring together the evolving nature of tools and practices.

Frameworks for understanding relations between human activity and technological artefacts

One of the broad challenges tackled by the CSCL community has been to find ways of conceptualising how collaborative activity unfolds through interactions with technological artefacts. In so doing, the community has tested and adapted ideas from Cognitive Science, Science and Technology Studies, Human Computer Interaction, Applied Psychology, and research on learning and change in complex workplaces.

The notion that cognition is best understood as distributed across individual minds and devices is now commonplace in CSCL (Salomon 1993; Hutchins 1995; Hollan et al. 2000; Strijbos et al. 2004). Contemporary theoretical accounts that link multiple people and devices fall broadly under two headings: those inspired by Actor-Network Theory (ANT) – see e.g. Latour (1996), Fenwick and Edwards (2010) and those based in Activity Theory. From our perspective, ANT’s insistence on elegant symmetries between human and non-human actors makes it unnecessarily hard to take into account distinctively human capacities for intention and interpretation. In contrast, Activity Theory (Nardi 1996; Engeström 1999; Kaptelinin 2005) gives a central place to intention. Indeed, activities are identified through their orientation to accomplishing a particular goal: they are object-oriented. Activity Theory foregrounds the relationships between acting subjects, their objects and mediating artefacts the subjects use to accomplish their objects.

Instrumental genesis

The theory of Instrumental Genesis (IG) derives from work by French ergonomists, building on Activity Theory and especially the notion of instrument-mediated activity (Béguin 2003; Rabardel and Béguin 2005; Rabardel and Bourmaud 2003; Verillon and Rabardel 1995). It focuses on the “integration of artefacts into the structure of human activity and provides perhaps the most elaborate conceptual account of such integration” (Kaptelinin and Nardi 2006, p.110). Critically important for us, IG explicitly focuses on how design is continued in use and hence distributed between designers and users through mediating artefacts and instruments.

Lonchamp (2012) and others have argued that IG offers a helpful corrective to accounts in the educational technology literature which position either computers or people as the prime sources of change. It offers sharper ways of theorising human-computer relations, while avoiding deterministic thinking (Oliver 2011, 2013; Orlikowski 2007;Overdijk et al. 2012, 2014; Ritella and Hakkarainen 2012; White 2008).

Instrumental Genesis (IG) is concerned with the mutual evolution of an artefact and its uses for a specific purpose within a given environment (Lonchamp 2012; Rabardel and Bourmaud 2003). This approach has served to explain how people’s activity progressively develops, as well as how they adapt technological artefacts to the conditions of usage. Given the critical role that aspects of human-computer interaction have in CSCL settings, the IG approach has attracted the attention of some members of this research community (Lonchamp 2012; Ritella and Hakkarainen 2012), though it is still not widely known. From an IG perspective, every human activity is directed towards a goal of some kind - an object that lends direction and structure to the activity: “[...] the activity does not have a direction and does not really start until the object of activity is defined” (Kaptelinin 2005, p. 16).

People (acting subjects) work on and towards the objects of their activity in ways that are usually mediated, rather than direct. Rabardel and Bourmaud (2003) distinguish four kinds of mediation, reflecting different relationships between subject and object, self and others (Table 1). The first two kinds of mediation concern the subject’s actions on the object.

Table 1 Four kinds of mediation according to the Theory of Instrumental Genesis

The four kinds of mediation summarised in Table 1 are illustrated in our analysis later in the paper. All these forms of mediation can occur in a variety of ways, but here we foreground the role of instruments as mediators. How then do artefacts shape activity? This is, of course, a core issue for the field of ergonomics, in which notions of Instrumental Genesis first emerged. By extension, it should also be a fundamental issue for research and development in educational technology, including CSCL. But it is surprisingly neglected (Oliver 2011, 2013).²

The hybrid nature of mediating instruments

A mediating instrument is typically both technical and psychological in nature (Béguin and Rabardel 2000). For Rabardel and Béguin (2005), an instrument is:

a composite entity made up of an artefact component (which can be understood as an artefact, the fraction of an artefact or a set of artefacts) and a scheme component (one or more utilization schemes, often linked to more general action schemes) (2005, p. 442).

Rabardel and Béguin (2005, p. 436) distinguish between the effects of:

• properties that are intrinsic to the artefact, such as its size, weight, hardness, or structural complexity and

• utilisation schemes - in rough terms, methods for using the artefact.

Utilisation schemes are extrinsic to the artefact: they attach to the subject who is using the artefact and are behaviour organisers, or a means for organising activity – among other things, they enable people to assign goals to actions, and meanings to experiences. Utilisation schemes have both private and social dimensions: the ways in which people make use of an artefact are sometimes private inventions, sometimes learned from other people, and sometimes a mixture of the two.

Instruments, instrumentalisation and instrumentation

If an instrument can be understood as a hybrid of an artefact and utilisation schemes, then instrument-mediated activity is shaped by properties of the artefact and the evolving utilisation schemes of the artefact’s user(s). This perspective turns out to be particularly helpful for situations in which artefacts and their uses co-evolve. This is especially relevant for our research, as we are not only interested in people’s use of artefacts but also in the changes that take place as people adjust artefacts to their needs and through their actions, and how artefacts and ideas move back and forth on the CSCL design pathway.

Instrumental Genesis, therefore, involves both the artefact, with which a person associates an action to perform a task, as well as the utilisation schemes, with which a person sees an instrument as a functional component.

Instrumental Genesis entails two sub-processes: one that is artefact-oriented (which Rabardel called ‘instrumentalisation’) and the other is subject-oriented (‘instrumentation’).³ Both processes are dynamic, and while instrumentalisation is oriented towards the evolution of the artefact side of the instrument, instrumentation relates to evolution on the human side of the instrument. Instrumentalisation involves enriching the properties of an artefact, or (temporarily or permanently) modifying its structure or its functioning. Instrumentation is characterised by an ‘evolution of the person’, and is closely connected to their utilisation schemes (Table 2).

Table 2 Evolution of the artefact and the person/utilisation schemes

The assimilation of new artefacts to schemes happens when a person realises that a new interpretation for the use of an artefact is possible. In other words, a person may discover that an artefact might also be useful to perform a new function, for example using an email inbox as a to-do list. The assimilation of schemes happens when they are applied to other artefacts, for example when email is no longer seen just as a communication tool, but also as a tool for organising one’s work. Alternatively, schemes can also accommodate as an adaptation response to changes in the environment. When this happens, the same artefact can gradually be used in accomplishing other tasks that were not in the repertoire of the design intentions. The concepts of instrumentation and instrumentalisation are illustrated in our analysis of the activity of educational designers and educational design researchers later in the paper.

In practical terms, instruments are never isolated, they commonly intertwine with each other whilst peoples’ activity unfolds (Rabardel and Bourmaud 2003). These groups of structurally linked and/or loosely coupled instruments comprise an instrument system (Guin and Trouche 2002). Instrument systems allow people to operationalise a number of quite heterogeneous artefacts and instruments with the purpose of accomplishing specific tasks or for performing continuing activity of a certain type (Vidal-Gomel and Samurçay 2002). The concept of instrument system is critical for the analysis of human activity, since people rarely limit themselves to using unitary tools. Instead, they interact with an ecology of tools and artefacts through numerous interrelated instruments.

Related studies

Previous studies reporting on CSCL as instrument-mediated activity

Drawing on Rabardel’s ideas, Lonchamp (2012) described the mediating role of CSCL systems, and characterised educational settings as constituted by: (i) the subjects – the people involved in the activity, e.g. teachers, learners; (ii) the instrument-mediated activity – with its object of knowledge and competence development; and (iii) the learning instruments – which mediate relations between subjects and objects, subjects and subjects and the reflections of each subject on themselves and their activity. Lonchamp noted that, in educational settings, instrumental mediations may happen in the ‘preparation phase’ and the ‘use phase’. In the preparation phase, the subjects are typically teachers and educational technology specialists, and the object of the instrument-mediated activity is designing for learning in a CSCL system. In the use phase, the subjects are the students and tutors and the object of the instrument-mediated activity is collaboratively developing students’ knowledge through interactions within a CSCL system. Lonchamp (2012) explained how instruments are generated between learners and artefacts and also the roles of educators and other learners in CSCL activity. He referred to this as the ‘downstream’ activity of learners and teachers using CSCL artefacts in specific learning situations. He explicitly chose not to focus on the ‘upstream’ activities of design for learning.

Overdijk et al. (2012) offered a theoretical account for the “agent-artefact connection” and reviewed Instrumental Genesis as one of the lenses to examine the “potential for action” of technical artefacts. The authors spoke of a “mutual shaping” of agent and artefact, where the artefact shapes the learner’s behaviour and the learner shapes the technical artefact. In their account, Overdijik and colleagues (Overdijk et al. 2012) recognised that the “design of technological settings can only be indirect, in the sense that technological settings establish preconditions for educational opportunities but do not causally determine those activities” (p.194). In later work, Overdijk et al. (2014) offered a descriptive account of the introduction and use of a technical artefact within a classroom context, as a way of understanding learners’ connections to artefacts, their interactions with them, and how artefacts shape classroom activity.

Previous work using IG to understand CSCL has focused on the use phase rather than the preparation phase. Our current study addresses this gap.

Educational design patterns from an IG perspective

According to Béguin and Rabardel (2000), utilisation schemes can be transmitted both informally and formally (e.g. through manuals or instructions), which may or may not be part of the artefact itself. This opens up a connection between IG and design patterns (which have been central to our practical work in enhancing educational design practice). Design patterns were introduced by Christopher Alexander et al. (1977) in architecture as a means for sharing design experience. A design pattern involves the pairing of a problem statement and a potential solution described within a broader context. It also includes a rationale for the solution, grounded in research findings, theory, or experiences (Goodyear and Retalis 2010). Higher level patterns are kept at a level of abstraction that renders the solution applicable in a wide range of contexts. Details are left to be worked out (or embellished) by other lower level patterns. By connecting lower and higher-level design patterns in sets, one may create a pattern language for a particular class of complex problem/solution. Although design patterns and pattern languages were originally developed for work with the built environment, other disciplinary areas such as software engineering and education have successfully applied this approach to sharing re-usable design ideas. A notable example of design patterns work within the CSCL community is COLLAGE (Hernández-Leo et al. 2006a). COLLAGE is a web-based tool offering educational designers support for structuring collaborative learning sessions through scripting, such as through patterns like Jigsaw, Pyramid or Think-Pair-Share (Prieto et al. 2011). For example, a Jigsaw design pattern includes a description of the learning objectives associated with this type of task design, the type of problem and the complexity of the collaborative learning flow, as well as information about the context in which the pattern can be applied (Hernández-Leo et al. 2006b). The connection between patterns and designing in CSCL can also be found in the extensive literature on CSCL scripting (Fischer et al. 2007). Scripts can be used to micro-manage dialogue between students (e.g. Weinberger et al. 2005), or to model a sequence of collaborative tasks at a macro level (e.g. Tchounikine 2008). Whilst some CSCL methods are sometimes interchangeably referred to in the literature as both patterns and scripts (e.g. see Jigsaw, Pyramid, Think-Pair-Share in Conole et al. 2010; Dillenbourg and Hong 2008), design patterns can be seen as conceptual models that help in describing or materialising CSCL scripts (Hernández Leo et al. 2010).

From an IG perspective, a design pattern, and its subsequent instantiated script, learning task, learning artefact, etc., can be seen as artefacts inscribed with their own collectively-defined utilisation schemes. (In other words, suggestions about the way or ways an artefact can be used are ‘written in’ to the artefact itself.)

The closest commonality between patterns and utilisation schemes may be seen in Verillon and Rabardel’s definition of a utilisation scheme (Verillon and Rabardel 1995, p. 10) as an enabler for subjects to associate artefacts with their actions by providing a set of repeatable and generalisable characteristics of artefacts’ utilisation. An approximation of this notion was also suggested by Folcher (2003), who indicated that a knowledge-sharing information system can be seen as an instrument system that provides individual users with generic or specific forms to support individual activity. In educational contexts, Corcoran (2011) hinted that an IG perspective can be taken to explain how teachers re-use and adapt off-the-shelf resources created by others, for use in their own design practice. In general, Rabardel and Bourmaud (2003) explained that instruments can be dynamically and collectively shaped by the community. As a result, instruments can gradually become inscribed with, or carry information about, some of that community’s shared knowledge. In this way, educational design patterns (such as ‘Jigsaw’) can be seen as artefacts which have gradually and collectively evolved with the aim of improving effectiveness and adoption by a wider community of educational practitioners. Through this evolution, educational design patterns become explicitly inscribed with their own utilisation schemes: which we can now recognise as a mark of what Rabardel and Bourmaud (2003) refer to as shared (local or extended) community heritage.

Consequently, the final part of the motivation for the current study is to explore the ways in which CSCL artefacts come to be inscribed with their own utilisation schemes.

Research aims

Our research addresses what Lonchamp (2012) called the (neglected) ‘upstream’ or ‘preparation phase’ in CSCL. It helps the exposition of our research plan to distinguish between three sets of people involved in the CSCL design research pathway. We argue that from time to time, people in each of these sets design and learn. To avoid ambiguity, we refer to the sets as follows:

• Set 1: At the ‘downstream’ end are people engaged in intentional CSCL - the people who are normally labelled ‘students’ or ‘learners’.

• Set 2: Just ‘upstream’ there is a set of people who engage in design for these students’ learning. These people are usually a mix of teachers, educational designers and educational technologists. In our program of research, the work of these people is extensively supported through the use of educational design patterns.

• Set 3: Further ‘upstream’ again is a set of design researchers – such as ourselves – who create and test new methods and tools to help the people in Set 2 do a better job of designing for the CSCL needs of Set 1.

Each of these sets usually involves collaborative teams, who are using computer technology to develop new insights, understandings, skills and working practices, and so the whole phenomenon can be characterised as CSCL. Each set has somewhat different purposes, and rather different clusters of artefacts are used and produced. That said, the overall process characteristically involves artefacts – and schemes for their use – passing back and forth between the sets of people, over different periods of time. From our perspective, as educational design researchers, the whole of this is best seen as a single complex system, imbued with CSCL (cf. Glanville 2015; Jonas 2014; Sweeting 2016).

In this paper, our focus is on the activity of educational designers (Set 2) and educational design researchers (Set 3). We use the term ‘design pathway’ as a way of signalling that design activity and artefacts are constantly evolving and that artefacts and utilisation schemes move back and forth between people in Sets 1, 2 and 3. Educational designers and educational design researchers usually develop creations to connect to, and/or to facilitate, the ongoing activities of others. As they do so, they are also themselves constantly drawing on ‘new learning’, which feeds into their future design activities.

Focusing the research

Our focus for analysis is on activity in the ‘preparation phase’, observing the activities of educational designers (Set 2) in a purpose-built Design Studio. We concentrate on showing how instrument-mediated activity can be followed:

• ‘downstream’ (to Set 1), with the resulting design of the CSCL tasks and scripts, aimed at promoting rich interactions, being proposed to and continued by the students; and

• ‘further upstream’ (to Set 3), with the resulting re-design of the Design Studio (as an instrument system) continued in use, prompting changes in the design artefacts (instrumentalisation) and/or in the CSCL design practices (instrumentation).

In the next part of the paper, we provide illustrations of how IG has sharpened our analysis of the role of artefacts and their use in collaborative design activity. We follow artefacts downstream – articulating how some of the artefacts brought to the Design Studio by Set 3 shaped the discussion and production of artefacts by Set 2, which in turn would influence the instrument-mediated activity of Set 1. We show how utilisation schemes move further upstream, or in other words, how something produced by Set 2 impacts on the instrument-mediated activity of Set 3. Instrumental Genesis contributes by offering a ‘language’ that expresses the complexity of the relationships in a CSCL system, always looking at (at least) subject-instrument-object triples at the different levels in which the actors (learners, teachers, designers, researchers etc) interact with and (re)design CSCL artefacts.

Empirical study: instrumental genesis in the CSCL design studio

The research setting: the design studio and its users

Because our research program has the practical goal of improving the work of educational designers, most of our empirical research has been located in a purpose-built design studio, created to explore how people design when their intention is to help other people engage in worthwhile CSCL activities. Design studios have been characterised as spaces for creative exploration, where designers and artisans individually or collectively engage in conceiving, designing and/or crafting new products, artefacts and services (Cennamo and Brandt 2012; Goldschmidt et al. 2014; Salama 1995; Schön 1987). Although design studios are commonly used in disciplines such as architecture, product and graphic design, the use of design studios by those who design for other people’s learning is still rather rare in education. We have found design studios to be productive sites for future-oriented research into educational design activities, design knowledge and prototype design tools. To this end, we constructed an Educational Design Research Studio (EDRS) – to be both a site for collaborative design and a means for researching studio-based design.⁴ Before offering a more comprehensive description of the EDRS, we need to identify the objects of the two sets of participants involved in the research, since these have a bearing on our specific research design:

• A set of people (‘designers’ - Set 2) creating a complex artefact – a course design – intended to benefit another set of people (‘students’ - Set 1) by providing structure and resources for their learning activity. (For example, some of their attention was on selecting and customising CSCL collaboration scripts.)

• Another set of people (‘design researchers’ - Set 3) observing the use, by Set 2, of the artefacts that collectively constitute the Design Studio. These observations had a dual goal – intended to understand the activities of the designers (Set 2) and also to inform future improvements to the Design Studio, including the design tools, methods and divisions of labour used within it.

A classical approach to evaluating the design work of Set 2, and/or the facilities of the Design Studio, would be to ask whether outcomes have improved. Are the students in Set 1 showing significant learning gains? Are Set 2’s designs judged to be better now than they were before? Drawing on Instrumental Genesis, we argue that such connections are not so simple. Instrumental Genesis shows that it is possible to sharpen the analysis of what is occurring and to use the insights thus derived to do a better job of supporting design work in the future: that is, to improve the work of people in Set 3 for the proximal benefit of people in Set 2 and the distal benefit of people in Set 1.

White (2008) argues for the use of Instrumental Genesis to inform the design of innovative learning tools, highlighting the dialectic nature of the processes enacted by the designers and the learners who use those tools. This dual character of a designed artefact, can be seen (i) as part of instrumentalisation, when taking the perspective of the learner’s instrumented activity, and (ii) as part of instrumentation, when considering the designer’s evolving understanding of the learner, via the learners’ engagement with the designed CSCL artefact. Figure 1 depicts the mediating role of this designed artefact (see central circle) in what Lonchamp (2012) called the CSCL ‘use phase’. The instrumental genesis process can be described as follows. The subjects in Set 2 (‘designers’) represent their instructional objectives to subjects in Set 1 (‘students’) through the designed artefact. As students engage in the learning activity, they both instrumentalise and are instrumented by the designed artefact. As a result, the views that subjects in Set 2 have about the learners and the design artefact evolve, also resulting in instrumentation and instrumentalisation processes on the designers’ side. Arrows in Fig. 1 represent either direct connections (arrows that directly connect the circles) or instrument-mediated activity (the two arrows that touch the edge of the designed CSCL artefact circle).

Fig. 1

The mediating role of a designed artefact in the CSCL ‘use phase’ depicted by White (2008)

Focusing now on the ‘preparation phase’, and building on White’s model, we suggest a representation that includes similar dialectic processes, but which also brings Set 3 (i.e. the observing researchers who are working on supporting the CSCL design activity) into the picture. Figure 2 shows how the Design Studio, seen as an instrument system, also has a dual character. (i) From the designers’ perspective (Set 2), the Design Studio plays a mediating role for designing the CSCL artefact or task that will be proposed to the students for their use. (ii) From Set 3’s perspective (which can include the research team and also the wider CSCL design community and its accumulated knowledge), the Design Studio has an evolving role in mediating the collaborative design activity of Set 2 people. In this case, the designed CSCL artefact (in the form of a course design and/or CSCL scripts) can be considered as a shared object imbued with its utilisation schemes and instructional meaning, which are expressed by the design patterns that compose it. As educational designers engage in a collaborative learning design activity in the Design Studio, they both instrumentalise and are instrumented by the numerous artefacts contained in the Design Studio to produce the actual learning design(s). As a result, the views that subjects in Set 3 have about the designers, the learning designs and the Design Studio, evolve and produce similar instrumentation and instrumentalisation processes on Set 3’s side.

Fig. 2

The mediating role of the Design Studio as an instrument system in the CSCL ‘preparation phase’

Understanding the design studio in action

The Design Studio at the University of Sydney is a specialist research facility equipped with a range of digital and physical tools and interactive surfaces, two large writeable walls, an interactive whiteboard (IWB), a data projector, iPads, various items of furniture, paper, pencils, coloured markers, etc.: all intended to support the collaborative activity of small design teams (see Fig. 3). Because it is a research facility, our Design Studio also has a built-in audio-visual recording infrastructure that allows the capture of research data. During design sessions, multiple radio lapel microphones and ceiling-mounted video and high definition time-lapse cameras record the activity, including conversations, gestures and movement in the physical space, for analysis by our research team after the event.

Fig. 3

The (digital and non-digital) artefacts in the physical space of our Design Studio

The studies we have carried out in our Design Studio typically involve three to eight people working together on a design task, over a period of two to four hours. The design sessions have covered a wide range of disciplines (e.g. product, project or learning design); levels of authenticity and duration formats (e.g. single versus multiple design sessions). For example, in some studies, the designers are ‘intact’ groups who are already working together on a real design task: they ask to use our Design Studio as a collaborative workspace and to have their design activity recorded (e.g. designing an app or an educational game). In other studies, we set the design task, and/or the design methods, and/or intended roles for each member of a design team, as well as providing the tools and other resources of the Design Studio. These more artificial studies normally involve volunteer participants – such as postgraduate learning sciences students, tutors, learning designers and teachers – with varying levels of expertise in educational design. The present research study used one of these more controlled scenarios, whose primary practical purpose was to get deeper insights into the collaborative design activity of participants designing for CSCL activities using an interactive, multi-touch tabletop display (also known as a collaborative design table).

Figure 3 shows the area of the Design Studio in which we located this collaborative design table, near an interactive whiteboard. This configuration was chosen because it allows the designers to: i) use the design table as the main working device, mirroring the view on the interactive whiteboard, or ii) split the task so different team members could build two candidate designs in parallel, on the design table and the interactive whiteboard or iii) compare two different designs, each showing on a different device. A software design tool called CoCoDes (Martinez-Maldonado et al. 2016, 2018) was deployed on the design table and on the interactive whiteboard. CoCoDes provides a multi-touch user interface, customised to support collaborative high level conceptual design work on tertiary education courses. Figure 4 (left) shows its main user interface. This offers a configurable timeline where teams can define all the learning tasks for students for a full study period. A pattern language (PL) is pre-loaded, containing patterns we have crafted for student tasks, learning spaces, learning resources, etc.

Fig. 4

Close up of CoCoDes interface. Left: the main design interface. Right (above): result of ‘double-tapping’ the Jigsaw instance to reveal textual elaborations of the learning activity it represents. Right (below): a designer manipulating the sequence of tasks within a Jigsaw pattern

CoCoDes provides digital icons that represent each of these patterns. They can be quickly manipulated by direct touch, allowing bimanual input and fluid interaction with the visual representations of the design patterns. Figure 4 (left) shows some instances of these patterns on the design timeline (e.g. see the coloured squares labelled as Lecture and Module in red, Individual assignment in green, Jigsaw in blue, etc). This allows the (Set 2) users to rapidly create, and also alter, a sequence of learning activities – where icons are used to represent each activity. Each of these representations can be ‘double-tapped’ to reveal textual explanations and elaborations of the learning activity they represent (e.g. Fig. 4-right, above). So one of the ways that the social – as distinct from the private – part of the utilisation scheme can be disseminated is through textual annotation of the artefact itself. An example of this is an icon representing a Jigsaw pattern. This digital artefact can be simply and directly manipulated to place it into a sequence of learning activities. This is particularly straightforward for people who have a working knowledge of what Jigsaw is/does. But also, tapping to access the text of the pattern reveals (part of) the artefact’s utilisation scheme. Thus, in IG terms, a participant (from Set 2) may combine knowledge that constitutes their private utilisation scheme for this artefact with the shared utilisation scheme inscribed in the artefact, to bring into being an instrument for their object of course design. More specifically, the designer (from Set 2) will use this combined knowledge to add a particular kind of collaboration script to a specific point in the sequence of learning tasks that will be tackled by Set 1 at learn-time. For example, Fig. 4 (right, below) shows how a designer could add a preliminary step (called background activity) to be enacted before the ‘regular’ sequence of subtasks.

Overview of the studies, participants and tasks

The overall research design involved two connected studies – Study 1 and Study 2. In what follows, we decribe specificities of these two studies in turn, showing how findings from Study 1 influenced decisions about the participants and tasks in Study 2. We then introduce the common features of both studies.

Study 1 involved four teams, each of two designers. Designers were recruited via an email invitation to participate in the project. All participants were postgraduate students in educational technology. Participants were allocated into dyads according to their time/day availability. The goal set for participants in Study 1 was to produce one candidate high-level design for a real university course: a one semester course in introductory computer science. Participants were not directly involved in the teaching or instructional design of this particular course. The participants worked on this design task for 1–1.5 h in the Design Studio. All the artefacts shown in Table 3 were made available for the participants, except for the Learning Design Dashboard.

Table 3 Artefacts comprising the Instrument System in our Design Studio and the kinds of mediations envisaged by the EDRS designers: epistemic (E), pragmatic (P), reflexive (R) and inter-personal (I)

The course to be re-designed by participants was an undergraduate engineering and computer science subject called “Human-Computer Interaction” which is commonly offered each term to second and third year students at the School of Information Technologies of the University of Sydney. The course is usually delivered during 12 weeks. This course was in part selected because any course re-designs resulting from our studies could be passed to the course coordinator to consider the implementation of design changes. Secondly, this course was also valuable in terms of CSCL research as it commonly involves a wide range of collaborative learning activities in the classroom and online, featuring multiple group configurations, group tasks and with a final assessed group project. One of the practical aims of Study 1 was to gain some formative feedback on the prototype CoCoDes system – a normal part of the development process for new design tools and methods (Masterman 2015; Murray 2016).

In addition, we also wanted to find ways of incorporating the use of design patterns in the process of redesigning this course. However, as will be discussed in the research outcomes section, the design patterns and pattern language available in Study 1 were hardly ever consulted – participants did not use any of the CSCL scripts (e.g. Pyramid, Jigsaw, Think-Pair-Share) in their designs, instead focusing on basic group formations and sequenced tasks. There were no specific roles assigned, and so all participants acted as learning designers. Analysis of post-hoc interviews indicated that the design task in Study 1 was too open and that participants did not perceive a need to discuss, search information, or integrate scripts into their designs. The lack of interaction with the design patterns was also a contributing factor in re-structuring the design of our second study, in ways that would encourage participants to engage in discussion and exploration of scripts.

Study 2 was a follow up study, conducted six months later, for deeper scientific purposes. Study 2 also involved four teams, each of three designers – see Fig. 5. Six participants from Study 1 were distributed across these four teams. Again, an email invitation to participate in the study was sent out, and all recruited participants were postgraduate students in educational technology, with no prior involvement in the course they were asked to redesign. Triads were formed according to availability, but also to ensure that each group had at least one participant from Study 1 who had experience in using the artefacts of the Design Studio: in order to investigate their role in the collaborative design activity. The goal set for participants in Study 2 was to produce two candidate high-level designs of the same university course, resolving some competing design goals. In increasing both the number of participants in each group, and the designs that participants needed to create, we aimed to forge better opportunities for discussion and resolution of conflicting goals, through a task that more closely simulated those in real world design teams. Each team member was assigned one of three possible roles (Lecturer (Instructor), Learning Designer and Quality Assurance Officer). Participants assigned to the role of Learning Designer had some previous knowledge of design patterns, with the other roles being randomly allocated. According to their role, each team member had specific information about the course and goals. Some of the goals provided to the participants were complementary to the goals given to other participants, and some were conflicting (e.g. they had to build two of three possible course modalities: a lecture-based, a blended learning or a fully online course). Thus, the task was purposefully designed to involve the resolution of conflicting information and goals, agreement about the different design versions to be built, compliance with institutional metrics (e.g. a minimum of face-to-face contact between students and instructors), and the construction of designs using the CoCoDes tool. This was different from Study 1, in which there was no social scaffolding of this kind: no assigned roles.

Fig. 5

Two teams of designers generating candidate designs for a university course in our Design Studio

In both studies, before engaging in the collaborative design activity, participants completed a pre-task that consisted of a short tutorial to generate rapid learning design prototypes. They also had opportunities to ask questions and become familiar with the CSCL design patterns that were available in the catalogue provided. After the collaborative design activity, each participant completed a questionnaire about how they used the artefacts and the space of the Design Studio. Then, a 20-min semi-structured interview was conducted with each team of designers. Two researchers were also present in the Design Studio while participants engaged in the collaborative task. Their main role was to observe and take notes, and ask questions in the de-briefing of the experiment, but the researchers could also provide assistance with the equipment, if required. In addition, audio and video recordings were made.

In both Study 1 and Study 2, all participants were given the following paper materials:

• a design brief (indicating the requirements and constraints of the course design),

• a catalogue of pedagogical design patterns (a simple pattern language describing relevant patterns for the course).

Each participant was also provided with a tablet device that included:

• digital copies of the design brief and the pattern language, and

• access to the official online system, run by the University of Sydney, that provides detailed descriptions of university courses, their requirements and intended learning outcomes from previous editions of the same course.

Table 3 presents an overview of the digital (hardware/software), non-digital, and furniture artefacts that were used by design team members in the Design Studio. The table also depicts the mediating design intentions of Set 3 for each of the artefacts and tools made available in the Design Studio (epistemic, pragmatic, reflexive and inter-personal mediations). For instance, locating a multi-touch interactive tabletop in the middle of the studio was intended to facilitate pragmatic mediation. Similarly, the software of the interactive whiteboard served as a pragmatic mediator between the designing team and the learning designs (via touch interaction) but it was also intended to serve as an epistemic mediator by extending the perceptual fields of the design team when used as an alternative display. The dashboard had an intended reflexive mediation role because it featured analytics about the designs being developed. The paper-based documents and the information shown on the tablet were intended to serve as epistemic mediators (e.g. as the main sources of the course and design patterns information). As most of the inter-personal interactions were face-to-face, all tools, but particularly those that allowed scribbling, could be used as inter-personal mediators (e.g. the writeable wall, which was intended to be used by teams to externalise their agreements, for planning their task, etc).

In sum, Fig. 6 depicts how things changed from Study 1 to 2, including people, social scaffolding, design goals, and tools.

Fig. 6

Evolving re-design conducted by people of Set 3 from Study 1 to Study 2

Data collection and analysis

Data collection for the two studies took place over a period of seven months, with Study 1 taking place six months prior to Study 2. Both studies included multiple participants allocated into small design groups. Data gathering included observation notes, short individual paper questionnaires completed just after each design session, debriefing group interviews for each session, artefacts produced (captured via digital images), and video recordings of participants’ activity.

The unit of analysis included the groups of educational designers and the artefacts they produced as our aim was understanding subject-instrument-object triples. This type of unit of analysis builds on work from Activity Theory (Engeström 1987; Engeström 1999; Kaptelinin 2005) where the entire activity system is seen as the unit of analysis. We examined people’s speech exchanges, their actions, the objects they manipulated, and their interactions with and (re)design of CSCL artefacts. Two researchers were involved in data analysis. The researchers discussed initial impressions of observational data captured at each session, and then each researcher independently screened video data to gather moments of interest for further analysis. A selection of video passages were independently watched and each passage was thematically analysed using IG concepts, by both researchers. The researchers discussed their two independent analyses to reach an agreement. Video data was transcribed and video images were processed for the purposes of reporting the selected illustrative examples.

In qualitative studies, notions of generalisability and reliability are usually replaced by validity and trustworthiness, rigor and attention to the quality in the research process (Creswell 2013). We employed multiple qualitative techniques described in the literature as best practices for studies using such a logic of inquiry. For example, we used data triangulation (various design sessions in two studies), triangulation of sources of evidence (individual questionnaires, group interviews, artefacts, video data, observation notes), and analysis triangulation (two researchers in the analysis process). We also used thick description: a strategy in qualitative research to offer detailed information so data collection and explanations are replicable (Denzin and Lincoln 2000).

Research outcomes

In this section, we provide illustrations of how the Instrumental Genesis perspective helped us conceptualise the instrument-mediated activity in the ‘preparation phase’ of the CSCL design pathway. In particular, we discuss how artefacts, instruments and utilisation schemes move ‘downstream’ and ‘upstream’.

Artefacts moving downstream

Example 1 Design patterns flowing downstream From Study 2. Design patterns proposed by Set 3 mediated the design discussions and production of a collaboration script by Set 2 that would be enacted at learn-time by Set 1.

Artefacts such as design patterns incorporated into the course design by members of Set 2 move downstream to eventually become artefacts in the run-time/learn-time environment for learners in Set 1. In Lonchamp’s terms, they move from the ‘preparation phase’ to the ‘use phase’. There may be some substantial transformational and/or translational processes involved in this. But, for most design(ed) artefacts, it is possible to follow them downstream to see them re-emerge at learn-time. A design pattern might emerge in the form of a scaffold for a learning activity. It may result in a package/sequence of step-by-step instructions given to subgroups of Set 1 students, suggesting how they should organise aspects of their groupwork. For example, in the pattern language used in Study 2 there were a number of design patterns purposely tailored to support the formation of different social arrangements (e.g. Group formation: led by the teacher, by the students, alphabetically) or the enactment of a collaboration method (e.g. Small group work, Whole class discussion, Jigsaw, Pyramid, Think-Pair-Share). A resulting collaboration script of this kind may take a number of forms – a short PDF document, or a sequence of prompts in a groupware system, for example. The degree of ‘prescriptiveness’ in the scaffold produced using a design pattern, however, will vary. Patterns suggest problem-solution pairs, and the detail can always be modified, customised and adapted by designers. Whatever the physical form, it emerges as an instrument (or, at least, an artefact inscribed with its intended utilisation scheme) for the subjects in Set 1 to work on their object(s) of coming to understand multiple views on a topic and/or of carrying out group self-regulation.

We illustrate the above through an excerpt involving members of Set 2 (Triad C) who incorporated the instances of some patterns into their course designs. The design patterns were made available to all members of Set 2 by the research team (Set 3) who carefully selected patterns that are commonly used in university teaching and learning practices. The design patterns were offered as ‘design proposals’ that members of Set 2 could use or adapt according to their instructional intentions. As a result of the design discussions, the Jigsaw and Think-Pair-Share (TPS) patterns were embedded into one of the group’s designs. Each instance of a pattern conveyed some explicit meaning in terms of the steps that need to be enacted by the learners (e.g. a three-step collaborative script with the following tasks/group arrangements: an individual task, a Jigsaw group task and an expert group task). The designers decided on the physical position of the instances of the patterns on the interactive surface across a timeline of 12 weeks. In so doing, Set 2 needed to discuss and decide how the instance of a pattern would fit with other critical expected events or planned tasks during the semester.

The episode that follows captures the efforts of the Learning Designer (LD), the Lecturer (L) and the Quality Assurance Officer (QAO) in coming up with an agreed design. This example focuses on their discussion - about adding a design element that draws on an artefact (pattern) imbued with their learning design thoughts and utilisation schemes about how to deploy the design. The episode also illustrates the instrumentation and instrumentalisation processes that took place with one design team (Triad C) after the Lecturer suggested to “talk about the online group activities of the subject”:

Episode 1: Think-Pair-Share as a Group Formation strategy

• 1 1 L: OK, so, let’s talk about the online group activities. Let’s add “Group Formation” (pattern “Group Formation” added to thetabletopdesign by L).

• 2 LD: Yes, one of the first activities I would highly recommend for forming the groups is this activity called “Think-Pair-Share” (LD speaks while consulting the digital version of thepattern languageon thetablet). This basically allows students to think about the problem to then turn to their partner or their group to share what they think about the activity. Then, they can come to a plenary session at the end or you can choose not to. That way you may elicit positive attitudes from your students and they will get to know each other.

• 3 L: OK.

• 4 QAO: And the discussion can be on a topic unrelated to the discipline so they don’t get tripped off in deep discussion related to the discipline just yet.

• 5 LD: Yeah.

• 6 QAO: The activity could be about some trendy topic, an environmental issue or anything.

• 7 LD: Yeah, something that I have done before to just get people talking and knowing each other is asking them to describe chocolate or something that everyone has an opinion about […].

• 8 L: Yeah, but these are engineering students, I would be inclined to trust them a bit more. We can give them simple tasks within the framework of the course. We can ask them to discuss an engineering concept that is valuable for the course but the hidden intention is to have them form groups.

• 9 QAO: Yeah, sounds great.

• 10 LD: Ok, I am just a bit concerned that the group formation is very late in the semester (pointing at the “Group formation” pattern on the tabletop. L moves the Group formation pattern physically).

• 11 LD: I would highly recommend moving the “Group Formation” to the second week.

• 12 L: Ok (L moves the“Group Formation”pattern to a week earlier in the semester).

This episode illustrates the process of instrumentation. The Learning Designer first proposes to include a pattern (Think-Pair-Share) as a solution to the group formation problem formulated by the Lecturer (see line 2). As the Learning Designer reads about a pattern and makes a suggestion – the ‘evolution of the person’ or the instrumentation has not yet happened – but this seems the ‘launching moment’. He reads and proposes the pattern language representation to others but also, at the same time, to confirm these ideas to himself. This may be considered as his individual instrumentation – as we confirmed in an interview performed afterwards (described below). The Lecturer immediately agrees, saying ‘OK’ (line 3). Then the three team members suggest different ways of deploying the pattern for different purposes. The passage described in lines 4–9, shows the Learning Designer elaborating on the Quality Assurance Officer’s idea (example with the chocolate) and the Lecturer disagreeing (mentioning the engineering example). This passage seems to confirm a group movement towards instrumenting the pattern – as previously discussed patterns already have imbued into them a suggested utilisation scheme. The subsequent passage (lines 10–12), includes the design team’s final decision on the timing of the group formation in the timeline of the course. This passage illustrates the instrumentation process to collectively assimilate the new artefact (Think-Pair-Share) by giving it a new meaning. The Lecturer discovered that Think-Pair-Share might also be useful to perform a new function - as an icebreaker to facilitate group formation. In the next part of the episode we confirm this, and another process:

Episode 1 (continuation): Jigsaw to complement the Think-Pair-Share activity

• 13 LD: The most advanced activity I would like them to do before hitting the project is called the “Jigsaw” method. I think they are going to [use it for their own work] quite frequently if they are familiar with it. Basically, you give them a problem and they all investigate different aspects of that problem. Then they come back to the group and they shed some light about that problem from different perspectives. A very practical way to do that is to give them an article that is 20 pages long. If there are four students, then they read 5 pages each. Then they come back in one group and shed light on those pages. Then in the project, where you know, the workload can be less manageable, it may be important for them to know how to interrogate each other effectively.

• 14 14QAO: And they could think about these processes, for example as they work on the “Think-Pair-Share” activity and then they walk through the “Jigsaw”, it can help them think in terms of methodologies they could use in their groups. I think they are quite connected.

• 15 LD: Yeah, and I would try to implement the “Jigsaw” a bit further in the semester.

• 16 QAO: I think the “Think-Pair-Share” activity is a nice scaffolding to the “Jigsaw”. They both may help students to have a good outcome from their projects. Is that the logic?

• 17 LD: That’s the logic behind this. That the “Jigsaw” comes a bit later in the semester once groups are more comfortable with each other.

• 18 L: OK let’s do that. So, the group formation happens in week two and groups stay formed for 6 weeks (pointing at the 6 weeks of the semester). “Think-Pair-Share” and “Jigsaw” activities happen during this period.

• 19 QAO: That’s got to be

• 20 LD. Yeah, and there should be a facilitator in the picture there (LD checks the Pattern language again).

• 21 L: So, if some of that happens in the laboratory…?

• 22 LD: Yeah, sure, there is room for that. But you can do some of this online. Also, the laboratory or a computer lab would work perfectly for that.

In this case, besides the instrumentation of a new pattern (Jigsaw) in a similar way to what the team did previously (see lines 13–15), the episode also illustrates the start of the pattern instrumentalisation. Team members start to consider how they will modify (or apply) and pair both patterns to suit their specific purpose (lines 16–19). They comment on how their ideas match the original utilisation scheme (e.g. whether there is a facilitator involved in the execution – line 20) and in which learning space the collaborative activity will be executed (e.g. in the laboratory or online – lines 21–22). Importantly, as the design team moves along, discussing when it would be the optimal time to have this as a “learning activity”, their speech changes, moving from consulting “patterns” to referring to these as “activities” – so the passage illustrates the evolution of the artefacts which, in this case, already had a utilisation scheme which also evolved temporarily.

Overall, this example shows the epistemic mediation role of the patterns as artefacts inscribed with their own utilisation schemes. (Refer back to Table 1 for the four kinds of mediation.) It also illustrates the collective instrumentation process of the design team in appropriating and giving new meanings to the patterns, and the instrumentalisation of those patterns to serve particular design intentions. However, the IG framing also suggests the continuous evolution of each individual. This was confirmed in a post-hoc interview when a video recording of the session was shown to the Learning Designer. The Learning Designer explained the thoughts and motivations he had during the episode illustrated above, saying:

I was very focused on my role (Learning Designer) so I was also looking at the pattern language and the different activities. Although I knew about the activities from work, because I use them a lot in the school, I just needed a recap and was interesting to see the pattern descriptions of the Jigsaw and the Think-Pair-Share activities. So, I was just reading up on that and trying to sell it to the others [team members] making sure they were implemented in the design as well. (Study 2, Triad C, Participant D7)

The above passage illustrates how an individual team member “evolved” as part of his own instrumentation process. In this case, the Learning Designer already has some understanding of what Jigsaw and Think-Pair-Share patterns are. But the utilisation schemes of these patterns are further shaped (a) by the description of the patterns in the pattern language provided (individual instrumentation) and (b) as a result of the collective implementation of the pattern in the course design (group instrumentation).

Example 2 The design dashboard mediating reflection in the ‘ preparation phase ’ From Study 2: The learning design dashboard created by Set 3 mediates the design discussions and production work of Set 2, by offering visualisations about students’ learning tasks (Set 1).

The design dashboard artefact created by members of the research team (Set 3) is offered to the educational designers (Set 2) as a means to support reflection upon metrics automatically calculated from their learning designs on-the-fly. The design decisions taken by designers in Set 2 move downstream to make changes that affect the learning tasks designed for learners (Set 1). In Rabardel’s view (2003), something like the learning design dashboard can serve as a reminder to designers of important information to be considered during the design session, and trigger design changes. The reflexive mediating role of the dashboard can be seen in light of Béguin’s (2003) view of design as a reflexive conversation between designers and the object of design: the designers strive to reach their goals, but the object of design can ‘reply’ with unexpected ‘resistances’ offering a pause and prompting designers to reflect and learn. In the CSCL Design Studio, this reflexive activity takes place in a dialogue among the Learning Designers and the other actors, with artefacts such as the design dashboard triggering some of these discussions or supporting reflection.

In Study 2, there were episodes where the design dashboard played a key role in mediating reflexive design discussions. In some cases, the educational designers (Set 2) adjusted learning tasks (for learners – Set 1) based on their interpretation of the representations in the dashboard, and how well these aligned with the specific goals defined by members of the research team (Set 3). For example, the Quality Assurance Officer had the specific goal of trying to optimise the number of hours in the course dedicated to face-to-face collaborative learning activities. We illustrate the ‘resistances’ triggered by the dashboard, through an excerpt from the work of the educational designers (Set 2, Triad D). Triad D used the dashboard to swiftly identify potential problems with the balance of collaborative activities in their course designs and to compare trade-offs between two candidate designs. The episode below unfolded towards the end of the session, when all actors were physically working on the candidate design ‘B’ at the interactive whiteboard (IWB), after completing the candidate design ‘A’ at the tabletop.

Episode 2: Mediating design discussions via a design dashboard

• 1 1 QAO: What does the dashboard say in terms of the learning activities? (walking towards the tabletop area to face thedesign dashboard,see Fig. 7a).

• 2 L: Not sure

• 3 QAO: (after looking at the dashboard and at the IWB for around 10 s she says) So, in the design ‘A’ we got 16% of learning activities that we haven’t associated to an online or face-to-face learning space (both the L and the LD walk up towards the QAO to face the dashboard too).

• 4 L: Are they here? (L points at the tabletop, see Fig. 7b)

• 5 QAO: Yes.

• 6 LD: Ok, we need to define what students will do.

• 7 QAO: Let me first just fix something (the QAO finds that the timing of one large activity in design B is not right after looking at thedashboardand walks to theIWBto fix it).

• 8 L: So, 16% (the L and LD keep trying to identify the learning tasks that are not defined in thetabletop).

• 9 LD: Here we need to add some of the learning tasks we added in the other design (‘B’).

• 10 L: Perhaps we can add some details to the “Project” (both L and LD start making changes to the candidate design ‘A’ again in thetabletop, see Fig. 7c)

• 11 LD: We can add some of these patterns such as the “Jigsaw” or a “Pyramid”.

• 12 L: Yes.

• 13 LD: Maybe we can add an Idea generation task to the “Project” too.

• 14 L: OK (L starts to implement the changes while the QAO returns to the tabletop area to see what they have been doing).

• 15 QAO: Is there any activity towards the end?

• 16 L: Yes (stares at the timeline visualisation of thedashboardand points at one pattern representation in thetabletopcorresponding to a task belonging to the Project pattern in the last week of the course). Where does this number come from? (pointing at one of the visualisations in thedashboardthat represents the balance between face-to-face and online activities).

• 17 QAO: This should be alright (also pointing at thedashboard, see Fig. 7d), this is design B (staring at the IWB). But for design ‘A’ we should only have around 70% of ‘Lecture time’ (looking at thewriteable wallwhere they wrote their agreed design goals, see Fig. 7e) but we have more. Where do you think this is coming from?

• 18 LD: That’s what I have been adding. I can show you what I have been doing. I have been defining some Idea generation tasks at the beginning of the course Project (all stare at the tabletop while the LD explains the changes she made while the others were reflecting on thedashboard, see Fig. 7f).

Fig. 7

Members of Triad D comparing their candidate designs and using the different displays while reflecting on the learning activities to be enacted in learn-time by Set 1

In addition to the instrumentation and instrumentalisation processes occurring in this session, we also see evidence of the four kinds of mediation roles shared across the different digital and non-digital displays and surfaces, and people in the Design Studio. The reflection is prompted first by the Quality Assurance Officer who focuses the team’s attention on the design dashboard (see lines 1–2 in Episode 2). The dashboard was designed by the research team (Set 3) as a “reflexive tool” – because it is supposed to show visual representations of the tasks that educational designers (Set 2) are designing, with the intention of inducing them to reflect. After a closer inspection of the dashboard and the interactive whiteboard, the Quality Assurance Officer leaves the other two members because she finds something in one design that needs fixing (lines 3–7). The reflection is continued by the Lecturer and the Learning Designer who start making changes in the original design, displayed on the tabletop, according to the indicators shown in the dashboard (lines 8–14). Then, based on the dashboard visualisations, the Quality Assurance Officer and the Lecturer try to make sense of why their goals are not being accomplished, by looking at the tabletop, the interactive whiteboard and writeable wall (see lines 15–17), to later discover that this issue was being introduced by the Learning Designer as they were speaking (line 18).

Overall, this example shows the reflexive and inter-personal mediating role of the dashboard (see Table 1), along with the other displays and surfaces, in triggering design changes downstream. The mediation of the instrument system is also pragmatic as the team members are manipulating the object of design (action on the object) and epistemic as they are trying to figure out how the dashboard can help them gain a better understanding of the object of design – lines 1–3 and again line 16. The role of the research team (Set 3) in designing the dashboard as a tool to promote reflection is also critical, since the visualisations and indicators in the tool shape the kinds of dialogue that emerge between the educational designers (Set 2) during the design sessions. In short, this episode illustrates the complexity of the Design Studio as an instrument system, where some tools and artefacts that are mainly intended to serve as epistemic and pragmatic mediators (e.g. the CoCoDes app in the interactive whiteboard and the tabletop) can also play a role as reflexive and inter-personal mediators (e.g. when used in conjunction with the dashboard to mediate reflection, self-management and dialogue).

Utilisation schemes moving upstream

As illustrated above, in using (as) an instrument such a thing as the CoCoDes, educational designers (Set 2) go through processes of instrumentation and instrumentalisation. Instrumentation here happens when a new meaning is given to (something in) the Design Studio or when participants discover how to use (something in) the Design Studio to perform a new function that was not envisaged by the research team (Set 3). Instrumentalisation is about enriching the properties of the Design Studio or particular tools (e.g. the CoCoDes application), when participants might modify or adapt its structure or its functioning to better suit their work. Observing how educational designers (Set 2) do this is one way in which the research team (Set 3) can enrich their utilisation scheme(s) for the Design Studio in terms of both collaborative design practice and toolsets. Moreover, new Set 3 knowledge of this kind – new ways of using the Design Studio and its constituent artefacts – can be passed on to subsequent cohorts of educational designers (Set 2), through explicit training and/or through inscriptions in new versions of the artefacts. Here, the Design Studio becomes an instrument for the subjects in the research team (Set 3) to work towards their objective of improving the practices of design for learning. Instrumentation at a Set 3 level happens when the research team re-designs the Design Studio, or modifies CoCoDes as a tool, to better support design work – based on knowledge gained from previous experiments (e.g. from Study 1 to Study 2) or from earlier iterations of this experiment – modifying artefacts for the groups of people in Set 2 in triads B, C and D based on observations of Set 2 in triads A, B and C, respectively.

Other artefacts produced by the research team (Set 3) to mediate the design conversations of educational designers (Set 2) include: specific scaffolds about how to tackle the task, role differentiation and regulation tools, which were added to the Design Studio as a result of previous observations (e.g. the dashboard in Study 2). Cycles of instrumentalisation occurred as the research team (Set 3) added small modifications to some of the artefacts with which Set 2 participants interacted in the Design Studio. New knowledge gained from observations of educational designers (Set 2 Triad A’s) instrument-mediated activity encouraged the research team (Set 3) to modify task artefacts offered to other groups of educational designers (Set 2 Triad B), and so on, in each iteration of the experiment. Next, we provide an example scenario that illustrates how utilisation schemes can move upstream.

Example 3 Epistemic scaffolding triggered by knowledge gained from previous sets of design sessions Non-use of CSCL scripts (by members of Set 2) in Study 1 led to modifications in task artefacts provided by the R&D team (Set 3) in Study 2

Although the same design patterns and pattern language were made available in both Studies 1 and 2, participants in Study 1 barely consulted the pattern language and did not consider incorporating the CSCL scripts (e.g. Pyramid, Jigsaw, Think-Pair-Share) into their designs. They focused instead on basic group formations and sequenced tasks. In all the post-hoc interviews, participants indicated that the design task was quite open and they did not feel a need to discuss, find more information about, or integrate ‘innovative’ CSCL scripts into the design. An explicit request for a much more scaffolded design context was made by one designer in Study 1 (Dyad D) as follows:

If the instructions of the trial would have asked for adding some specific scripts then we would have had a look to the pattern language, wouldn’t we? Maybe if we had to choose any of these patterns in terms of the learning outcomes of our design, then it would be interesting to have a better look at how to include more complex scripts. This way we could wonder what may happen if we apply these patterns, how students would react and how they could be put into practice Study 1, Dyad D, Participant D7.

This motivated the research team (Set 3) to make changes in the epistemic and social scaffolds. Our first study – with educational designers (Set 2) – lacked detail on specific roles, only asking participants to produce two designs. In the collaborative task in Study 1, the educational designers (Set 2) disregarded the patterns, alerting the research team (Set 3) that the task description would need some adjustments. Specific roles – characterised by competing goals – were created and introduced in the experiment with educational designers (Set 2) in Study 2. In particular, the Learning Designer role had the goal of actively trying to integrate CSCL scripts into the team’s learning designs. The aim was to encourage participants to engage in negotiation to reconcile competing demands in the design work, as well as to use the design patterns. By introducing explicit role differentiation, and adding a requirement for the use of the design patterns, the research team (Set 3) was then able to observe the educational designers’ (Set 2) instrument-mediated activity in relation to the design patterns, goals and roles. This modification between Study 1 and 2 resulted in all the teams in Study 2 integrating and structuring their discussions around explicit design patterns (e.g. see the first example in the previous section). In the post-hoc interviews, two groups of participants in Study 2 described the impact of this newly introduced epistemic scaffold, as follows:

We did try to include complex patterns as it was one of my responsibilities, so I was trying to do it. Well, that's the interesting part for me. You know, I don't have much experience as a lecturer, but I've read a lot about design patterns and I know they're useful. They really have a higher purpose in terms of how they can be deployed in the classroom, which is really cool. Unfortunately, we never got to talk deeply about why they work, but we would if we had more time in a real scenario. Study 2, Triad D, Participant LD.

The [pattern language] was really helpful to keep in mind that we may want to include some kinds of collaborative tasks into the design, and justify why, how and when in the course it would be better [to use them]. Study 2, Triad D, Participant L.

I think having the pattern language at hand was useful for me. Without it, I couldn’t have known what all the tasks would mean. I had an idea, as I mentioned before, of the activities that were part of the task because of my educational background. If I would have been the actual lecturer of this subject, I think I would like to use the patterns in my daily life, I could probably use them, not just pick one of them just because it's part of a task, but actually properly use them according to the knowledge of the subject. Study 2, Triad B, Participant L.

These examples illustrate the instrumentation of the Design Studio for the research team (Set 3). The Design Studio becomes an instrument system that is operationalised by the research team (Set 3) for generating understanding about the critical role of an explicit design pattern language in mediating the learning design activity of the educational designers (Set 2). At the same time, the example shows instrumentalisation of the Design Studio: the utilisation scheme of the pattern language (lack of integration of CSCL scripts) by the educational designers (Set 2) in Study 1 led to (permanent) modifications in task artefacts provided by the research team (Set 3) in Study 2. The scaffolding in Study 2 triggered productive conversations and reflection involving the explicit use of CSCL scripts and other design patterns. This was completely absent in Study 1. As shown in the first and third quotes above, participants tried to explain how the pattern language could have an impact in a more realistic scenario. In other words, the provision of specific roles served as an epistemic and reflexive mediator in the learning design activity. This suggests that, for realistic scenarios, teacher-designers may need to be encouraged, scaffolded, guided or provoked to integrate innovative CSCL scripts or design patterns into their design work.

Discussion

The overarching practical aim of our research is to find better ways to support educational design activity, and so understanding how designers engage in CSCL design work, how tools and resources shape their collaboration, and the role artefacts play in their design activity, are crucial. The main contribution of this article arises from an approach based on the theory of Instrumental Genesis, to conceptualize the evolving role of artefacts (digital and material) in mediating CSCL design activity, in ways that acknowledge the distributed nature of both, design and learning in CSCL. The IG approach has allowed us to conceptualise design activity in terms of the mutual evolution of artefacts and their use, within the educational design context of our empirical research. IG offers analytical tools to discern between different forms of instrument mediation – epistemic, pragmatic, reflexive, and inter-personal mediations (see Table 1). Being sensitised to these different kinds of mediated activity offered us ways of theorizing and identifying the ‘design intentions’ embedded in the tools the educational design research team made available to the educational designers using the Design Studio. It also sharpened our analysis of the educational designers’ interactions with these tools. Understanding the role of artefacts and the evolutionary nature of CSCL design activity is crucial for the communicability of design ideas and for the future improvements of CSCL (re)designs.

Building on Lonchamp’s work (Lonchamp 2012), our analysis illustrated the ways that instruments generated through interactions between designers and artefacts move ‘upstream’ and ‘downstream’ on the CSCL design and implementation pathway. In essence, we were able to identify how artefacts and their use by educational designers shaped the ensuing activity of these designers, and how this in turn, is carried forward into the learning designs they produced. In addition, we also showed how the educational designers’ activity fed into the work of the research team studying and supporting them.

The point of departure for our research was the ‘preparation phase’, looking closely at the activity of educational designers. However, as Rabardel and Béguin (2005) have argued, Instrumental Genesis goes beyond explaining instrument-mediated activity by helping to generate understanding of design as a continuously evolving activity. Design and learning, in our view, unfold as distributed and collaborative processes in which people who are commonly given differentiating labels – as users, students, learners, teachers, designers, learning technologists and educational researchers – all participate, though with different objects in mind. So our focus is not so much on particularities of singular CSCL activities, but on the distribution of activities across the different actors involved in the CSCL design pathway (Sets 1, 2 and 3; Fig. 8). As our examples illustrate, CSCL design unfolds through complex inter-related layers of instrument-mediated activity involving members of these different sets of people. Our focus in this paper was to show how artefacts and their use shape the collaborative design activity, and the role of artefacts in the collaborative learning of educational designers and educational design researchers.

Fig. 8

Our contribution

To this end, we argue that both the ‘users’ and ‘designers’ go through learning processes, which from a user’s perspective may involve learning how to use, adapt or interact with what has been designed, and engagement in instrumentation and instrumentalisation processes. In our scenario, the ‘users’ in Set 1 (the ‘learners’, see Set 1 in Fig. 8) would explicitly be in a learning role – as the educational designers (Set 2) are designing a course or specific learning artefacts to be used in a CSCL task. However, moving a little ‘upstream’ – we shift our focus onto the educational designers (Set 2) as the ‘users’ in the Design Studio (the ‘designers’ in Fig. 8). What these designers learn through their design activity is often implemented in their future design activity. As the process of producing and reworking a CSCL artefact unfolded, designers (Set 2) were constantly experiencing new learning, which in turn, changed the dynamic of the object being designed (the CSCL tasks and the course design as a whole). This was seen in Example 2 when the Quality Assurance Officer, after noticing a specific occurrence on the dashboard, asked others to “hold on” while something needed to be “fixed” in their previous design. Similarly, Set 3 (in general, the CSCL research community, or specifically our research team – see Set 3 in Fig. 8) experienced “new learning” through observations of the design processes and the instrument-mediated activity of the educational designers (Set 2) – in our case, across each session of Studies 1 and 2. This also resulted in modifications by Set 3 of artefacts in the Design Studio (e.g. see Example 3). Design in this case is best understood as a distributed, cyclical process “where the result of one person’s activity constitutes a source for the activity of another” (Rabardel and Béguin 2005, p. 451).

Instrumental Genesis helped us form a picture of how this distributed activity is in constant movement, but also connected, as if in a web of elements – or a system. This is reflected in the dialectic nature and the dynamic co-evolution of the design activity we observed and our own activity in re-designing artefacts for use in the Design Studio. Thus, the Design Studio, with its range of artefacts – e.g. tabletop, design patterns, dashboard, etc. – functions as an instrument system. The artefacts in the Design Studio can be seen as brought into the design context by the research team (Set 3) to facilitate the instrument-mediated CSCL design activity of the educational designers (Set 2) who are also thinking about the needs of imagined students (Set 1). Instrumental Genesis allowed us to map and establish correspondences between how the tools made available to the educational designers became part of their design conversations and influenced their design decisions, which is illustrated in the examples presented, offering a detailed account of the richness in types of mediation we identified.

Certain artefacts may be more ‘adaptable’ or ‘modifiable’ and thus more likely to enable instrumentalisation based on the designers’ intentions (Rabardel and Béguin (2005). For example, as we discussed, design patterns are artefacts already inscribed with their own utilisation schemes, which are intended to be appropriated and included as part of a sequence of learning tasks by the educational designers (Set 2). However, there were different uses of design patterns in Study 1 and Study 2. (See Example 3). In the ‘preparation phase’ in the Design Studio, educational designers (Set 2) could choose to include one or more from a range of different patterns, as part of their design. Designers could also customise the patterns, adding or deleting elements. The presence of patterns among the artefacts available to educational designers (Set 2) in the Design Studio prompted conversations about when – in the timeline of the course – they could be used, how to use them, why teachers use these elements, etc. Some interesting insights emerged in the discussion – as in Example 1, where the educational designers discussed the use of group formation as a strategy to help students get to know each other, and the importance of establishing grounds for conviviality at an early stage in the course. In the ‘use phase’, patterns would be likely to be adapted at learntime, when teachers interact with students (Set 1), as they are likely to (re)configure their designs and adapt to the emergent activity. In addition, artefacts such as the tabletop and the dashboard can also be seen as ‘modifiable’, as the educational designers (Set 2) aggregate different components to produce each design and the research team (Set 3) adjusts interface elements across sessions.

In sum, the pivoting object which is at the centre of the whole dialectic process is the actual CSCL object of design. In the ‘use phase’, the learners and teachers get instrumented by and instrumentalise the (complex) CSCL artefact designed for them in order to accomplish the learning task. In the ‘preparation phase’ the educational designers (teachers and technology specialists) get instrumented by and instrumentalise the instrument system intended to support their CSCL design (in our case, the Design Studio) and the educational design researchers feed back into the system what they learn in observing the work of educational designers.

Concluding comments

This paper complements and extends previous work that has explored IG in CSCL (Lonchamp 2012; Ritella and Hakkarainen 2012). Specifically, it investigates the neglected ‘upstream’ activities involved in CSCL design. It illustrates how the IG approach can be used to follow artefacts and ideas back and forth on the CSCL design pathway. We frame research, design, teaching, and learning in CSCL as a distributed activities, and show how the use of CSCL artefacts in specific design situations resonates both ‘downstream’ and ‘upstream’. We believe that this account has made the following important point – that CSCL design is best seen as a dynamic communicative process, in which the various ‘users’ of designs play a substantial role in the evolution of that which has been designed. In order to develop useful methods, tools and resources for supporting the work of educational designers, one needs to examine how these come together in complex activity. It does not make much sense to try to freeze an artefact and come to some global, context-free, estimation of its worth, or to promulgate a universal principle based on such an evaluation. Rather, one needs to understand artefacts and ways of using them as dynamically coupled in instrument-mediated activity.