Developing an Online Authoring Tool to Support Teachers in Designing 21st Century Design Based Education in Primary School

Abstract

Design Based learning (DBL) as an educational approach which is emerging in primary education. Because of the limited availability of prescribed teaching materials for DR, learning activities are often developed by instructors themselves. However, it is often difficult for teachers to develop DBL activities. The paper investigates how primary school teachers can be supported in developing successful DR learning activities, in which pupils can develop both core curriculum objectives and 21^st century skills. The research questions are: How can teachers be supported in the design of DBL activities aimed at concrete learning objectives? Are teachers able to apply the DBL creation tool as intended; to follow the design strategy offered, to reflect on the DBL activities using the tool and to improve the design iteratively? Are teachers able to successfully develop DBL learning activities using the tool?

The paper describes the development of a tool that supports primary school teachers in creating DBL teaching materials. A web-based tool has been realized iteratively by means of design research. The resulting supports the teachers in developing 21^st century education and encourages the teacher to reflect, even if it does not yet produce complete teaching materials for the classroom. However, in order to be effective in promoting DBL, there is a need to integrate the underlying concepts of DBL (such as incorporating design generations into the course material) even more extensively into the tool and to supplement it with an explicit pedagogical strategy and concrete assessment procedures.

1 Introduction

In order to be carefully prepared for the future, children today are increasingly encouraged to learn 21^st century skills such as working together and solving problems. Teachers are faced with the challenge of integrating these skills into their lessons, in addition to achieving the regular core curriculum objectives such as arithmetic, writing and subject knowledge. Interest in Design Based Learning (DBL) [1, 2] is growing among primary school teachers. They see it as helpful in teaching children 21^st century skills. DBL (Design Based Learning) is an educational approach in which pupils learn by developing (‘designing’) solutions for (open) social challenges in cooperation with each other through design-methodology. During DBL the pupils work on authentic challenges, which often lead to intrinsic motivation, and provide insight into how the learned knowledge and skills can be applied in practice.

Despite the fact that DBL is seen as an appropriate approach to learning 21^st century skills [3], there are practical obstacles, which often prevent primary school teachers from (successfully) implementing DBL in class. There are no fully developed textbooks, the teacher has to take on a coaching role as opposed to the traditional role of teacher [4] and the teaching materials often have to be developed by the teachers themselves [5]. The development of suitable DBL teaching materials is also challenging and time-consuming; teachers are often unfamiliar with the concept of DBL, unable to determine the appropriate level of openness in the assignment for the pupils, and experience difficulties in linking DBL activities to the development of crucial basic skills (such as numeracy and language skills).

The project was carried out in very close cooperation with three schools of PlatOOlab [24]. PlatOOlab is an organisation of schools in the south of the Netherlands that is interested in educational innovation and seeks to implement the teaching of 21^st century skills.

This chapter presents the development of an online tool that supports primary school teachers in creating DBL learning activities. This DBL creation tool seeks to help teachers to integrate 21^st century learning goals into DBL activities. In this, it also seeks to encouraging teachers to reflect on the DBL activities they have developed, thus strengthening the teachers’ professional agency. The DBL creation tool has been developed in three iterative cycles, in collaboration with teachers and the directors of the three participating schools. The design of the tool is based on the design in the master thesis of van der Sanden [30].

2 Related Work

2.1 Design Based Learning and 21^st Century Skills

Europe is facing an “innovation crisis”: research needs to be better translated into new and better products and services in order to improve quality of life in Europe and to remain competitive in the global marketplace. To counter this crisis, Europe needs people with an entrepreneurial and creative way of thinking. However, current education largely concentrates on knowledge acquisition, automated skills, and little emphasis is put on understanding, critical thinking, the ability to apply knowledge in new situations, creativity and collaborative skills. As a result, teachers, entrepreneurs, academics and public authorities are increasingly calling for the implementation of the so-called 21^st century skills [3], that particularly focus on the less traditional academic skills needed for the future.

Trilling and Fadel [6] have divided 21^st century skills into three categories (see Fig. 1):

• Learning and innovation skills, including critical thinking, Problem-solving thinking, Communication, Collaboration, Creativity, Innovation and entrepreneurial skills;

• Digital literacy skills, including information literacy, media literacy and information and communication technology (ICT) literacy;

• Career and life skills, including flexibility and adaptability, initiative and self-management, social and multicultural interaction, productivity and responsibility.

Fig. 1.

A model of learning 21^st century skills [6].

As a teaching/learning strategy, Design Based Learning facilitates various of these 21^st century skills. Since DBL focusses on solving complex and poorly defined design problems, this would challenge pupils critical thinking, creativity and self-directed learning ability [7].

The Design Based Learning process follow consists of a series of steps, such as: exploring the problem, developing ideas, building and testing prototypes and evaluating the outcome. In projects, some of these steps may be performed more than once, since during the process new temporary designs become available to be tested, new perspectives may be discovered instigating re-testing of re-design, and new aspects to the problem may become visible pointing for solutions in new directions that are to be explored. However, a DBL project will typically start with a divergent, chaotic phase and then gradually move forward towards a solution; a process in which the emphasis gradually shifts towards convergent thinking (Fig. 2).

Fig. 2.

How the process of divergence and convergence proceeds to a clear final design [34].

Teaching using DBL is attracting increasing interest [7]. However, it has been little applied in schools, mainly due to practical problems such as a lack of Design Based teaching materials and un acquaintance of the teachers with DBL. Moreover, examination syllabi may get in the way, putting pressure on classroom practice preventing teacher to explore with new forms of education, even those aiming for the learning of the desired 21^st century skills.

Nevertheless, DBL has been identified as an innovation with the potential to bring about a major change in learning and can contribute to a new pedagogy, which may transform education [7].

Teachers are the key to the successful educational change and the implementation of DBL. In order to bring a change, teachers must understand the principles of DBL, integrate these into their professional conception of teaching, and need an overview of means and techniques to practically implement DBL in their classrooms. To do this, teachers need resources, examples and professional development opportunities [8].

Such implementation will very much involve the teacher in the role of ‘learning activity/material designer’. For this, a variety of educational frameworks is available, such as Van den Akker’s spider web (Fig. 3) [9]. Such frameworks can help to structure educational processes and their design. In this study, the ‘spider web framework’ was used to develop a practical tool for primary school teachers to design their own DBL learning activities.

Fig. 3.

The curricular ‘spider web’ by Van den Akker [9].

The various learning elements in the spider web al interact in establishing effective education. The spider web gives an overview of the learning elements that are to be orchestrated in order to arrive at a coherent lesson or lesson series. We used it as a basis for the lay-out of our educational design support tool. In this, we downplayed the learning elements ‘forms of collaboration’ and ‘sources and materials’ but put extra emphasis on the element ‘role of the pupil’. The results form a solid foundation for our design support tool that will assure that the teachers will pay adequate attention to all elements.

2.2 Tools for Teachers

No ‘plug and plan’ teacher design support tools are available. Existing educational materials that focus on promoting Design Based learning in education, such as Stanford ‘d.school’ [10], focus on the Design Based learning materials, rather than on supporting the instructor to design the learning materials himself. A toolkit that helps teachers to integrate Design Based learning into their lessons, an approach that has much in common with DBL, has been developed by the IDEO [11]. This toolkit uses Design Based learning to support teachers in developing new teaching materials. But this not necessary is Design Based Learning Material. Unlike these toolkits, our approach is specifically focused on supporting teachers in creating DBL in which students learn 21^st century skills.

To support designing teachers, it is crucial to reduce the ‘design space’. A first component is structuring the design process. This takes away much of the interrelatedness of the various design choices which is difficult to understand for the novice teacher/designer. Structuring can be less strict for more proficient teachers/designers that already can understand the ‘when and why’ of various choices [35]. A second component is to make design options transparent, and to reduce their number through pre-selection.

To make websites that present complex processes or situations in an intuitive and predictable way, these should be structured according to users’ mental model(s) of these processes or situations. An example is the lay-out of the railway station manager’s dashboard. Hence the online design tool should be organized according to the teachers’ perception of the process of ‘planning education’. This model can be found by examining how a representative group of teachers views this [36, 37].

Structuring the design process and pre-selecting design options alone would ‘robotize’ the teachers’ design process, frustrate teachers as autonomous professionals and hinder their development as reflective practitioners [27]. Hence the online support system should give room for teachers’ professional autonomy and professional decisions. Moreover, it should stimulate reflection while designing education. Even more so for designing DBL since reflection is in the heart of the design process itself [2].

Hence, structuring and pre-selection of design choices should be minimized, but must also be rigorous enough to prevent cognitive overload. Scaffolding and guidance to support teacher in taking the remaining design-decisions is key. An adaptive strategy was developed [38]: Guided Learner Adaptable Scaffolding (GLAS). They distinguish 3 types of scaffolds: supportive (e.g. advise, highlighting options), reflective (e.g. clarifying alternatives and criteria, asking for deliberations), intrinsic (e.g. relating to the design process as a whole).

2.3 Properties of DBL Activities

In developing a tool that supports teacher in the creation of DBL learning activities, we need to start from a clear description of DBL and its characteristics. Here only a short description can be given - a more detailed description based on various resources [12,13,14] can be found in [2]. Design Based learning activities consist of the following six components:

• 1. Design Elements and a Design Process

The design process should move through various steps and the focus should gradually shift from divergence (e.g. generating directions and ideas) and to convergence (e.g. deciding on the size of the challenge and selecting ideas). In DBL, the design problem that students work on should be valuable for real life. The project should aim for adding value for the user, subject and possibly for other stakeholders.

• 2. Collaborative Learning and Reflection Process

Reflection plays a crucial role in the design processes [39]. Through reflection, questions are articulated concerning the aim, design or current prototype. Seeking answers can involve theoretical exploration, a shift in view-point, information search of testing. At the same time, reflection plays an important role in the learning process since it contributes to understanding and personal internalization of results. Hence, it directly contributes to 21^st century learning. In DBL, the pupils learn by iteratively switching between doing and reflecting. Group work and collaborating promotes this process.

• 3. Assessment and Learning Objectives

Learning objectives of DBL activities should comprise both skills (e.g. design skills, 21^st century skills) and the ‘classical’ learning goals such as knowledge (e.g. mother tongue) or domain-specific skills (e.g. arithmetic).

• 4. Project Properties and Design Assignment

The design assignment, i.e. the (social) challenge for which the pupils are asked to come up with a solution, should be open and should provide the pupils with the opportunity to work on relevant learning objectives. The design project must be recognizable and motivating for the pupils. In addition, it should match the level of the students. Based on the similarities between learning from inquiry and learning from DBL, the level scheme of Ireland, Watters, Lunn Brownlee and Lupton [22] and with the addition of the degree of self-determination by the pupils, can be elaborated into a level scheme that indicates at which levels in DBL can be worked. Levels are: experienced, getting acquainted, challenging, guided design, targeted design, coached design.

• 5. Teacher and Pupil Role

In DBL, it is important that the instructor properly fulfils his/her coaching role. Pupils should increasingly take responsibility for the design process, products and learning outcomes. They themselves are the driving force behind their design and learning process. Teachers must give room for this but should offer just as much support that the tasks for the individual students are manageable, challenging and motivating [6].

• 6. Teaching Materials and Learning Environment

Just like the design assignments, the learning environment is open to be decided on by the teachers. This means that the local school environment and/or parents are principally involved in the project. This gives external confirmation and value to the entire effort. It creates meaning, relevance and motivation in the pupils.

These six components serve as a beacon in developing the DBL support tool.

3 Methodology

3.1 Research Questions

The central research question is: Can an online tool be developed that supports primary school teachers creating 21st century lessons that include ‘classical learning goals’ using DBL?

This question is structured on the basis of the following sub-research questions:

• (1) Are primary school teachers able to apply the DBL creation tool as intended; in particular, to follow the design strategy offered, to reflect on the DBL learning activities using the tool and to improve the design from there iteratively?

• (2) Are primary school teachers able to successfully develop DBL learning activities by means of the DBL creation tool?;

  • (a) Are the DBL learning activities, which the instructors develop by means of the tool, generally practical and educationally sound (can be implemented)?

  • (b) Do the DR learning activities, which teachers develop by means of the tool, have the intended characteristics (authentic, open, limited problem space, core objectives, etc.)?

  • (c) Do the DR learning activities, which teachers develop by means of the tool, enable pupils to achieve the desired learning objectives of both types?

3.2 Research Setup

The development of the DBL creation tool was done using the methodology of Design Research [15,16,17]. Design research has many similarities with e.g. a design approach used for developing computer-based products [18]. In this educational methodology research and design move forward simultaneously. After an initial design, ‘experimental phases and evaluation’ and ‘redesign’ alternate. The entire process can consist of several rounds.

The first evaluation round usually deals with basic issues occurring in the designed material such as missing communications, smaller structural inconsistencies, technical flaws of textural errors that hamper classroom process. Hence open data-collection methods are preferred, while additional information is usually collected, for example by expert reviews. The first round usually only produces indicative answers to the research questions. In the second and following round, the full focus is on answering the research questions.

Evaluation is often carried out using a multi method approach in which the results of e.g. material-analysis, classroom observations and interview or questionnaires are combined [19]. Data from these different sources are combined by triangulation providing internal confirmation [e.g. 19].

The various data sources principally address the education to be evaluated at different curriculum levels [20]. Interviews with teachers – for example - address the ‘perceived curriculum’, whereas classroom observations address the ‘observed curriculum’. Hence, the data from the various sources supplement each other. Alignment and discrepancies point out the level to which the ‘intended curriculum’ was actually transformed in ‘classroom reality’.

Data collection and analysis was organized according to the evaluation scheme shown in Table 1). Five ‘main variables’ are evaluated on 4 of the Goodlad - curriculum levels: Theoretical, Perceived, Material and Experienced curriculum. For each level particular data-sources are available (bottom row in Table 1). When used, the cell of the scheme fill, and comparison of the findings in one row over various columns reveals how the intentions underpinning the tool worked through in e.g. teacher perceptions, teaching materials created and/or classroom reality.

Table 1. General evaluation scheme [24].

The categories reflect the studies research questions. The first category is fundamental to evaluate the general effects off the tool on the created education and concern general aspects of learning environments according to [26]. The second employs the work of [21] on adoption of educational innovations and implementability. Key factors considered here are: understandability, usefulness, ‘easy to use’, and congruence with the teachers’ convictions.

The third category ensures the evaluation of the way the tool addresses the various DBL components (Table 1). The fourth category focusses on teacher support and the stimulation of reflection. The fifth category concerns the educational goals.

3.3 Participants and Data Collection

The project was performed in a team of 6 researchers including an ICT specialist, and one researcher who was not part of the designing team. An expert panel was formed that comprised 7 experts in the field of education and/or the use of ICT in education. The various versions of the tool were tested by 7 teachers. Three school directors were involved, making up stakeholder-board and feedback group.

Instruments.

Our multi-method [19] approach comprises: formative analysis of the tool by stakeholders and experts, interviews with teachers, observation of teachers who design lessons with the DBL tool, observations in the class that apply the course material made, and an analysis of the course material designed with the tool.

Workshops.

During the development of the tool, three co-creation sessions with stakeholders were organized, at which school leaders and teachers of the PlatOO schools were present. In addition, a number of educational researchers/teachers from and around the Eindhoven School of Education were asked to assess the tool formatively as an expert. In the first research round no format was not imposed beforehand [8] and the results of this were immediately incorporated into the first design. Questions that were raised in the co-creation sessions of the first round:

• How can a DBL activity be organized in practice?

• What are the characteristics of a DBL activity to be a good learning activity?

• How can a tool support the creation of DBL activities?

Expert Panel.

In sessions of 30–60 min, five educational experts were interviewed individually about the DBL properties that the tool weighs up and how reflection can best be stimulated among teachers during the design process. Furthermore, the tool was evaluated in terms of ‘practical ease of use for teachers’. The questions put central in this expert panel were:

• Background; (Experience, expertise)

• What is your prior knowledge of Design Based Learning?

• To what extent is DBL being stimulated by this tool?

• Does the DBL become clear in this tool for teachers who have no previous experience in this field?

• To what extent is this tool supportive for teachers?

• To what extent does the tool encourage the teacher to reflect on the composition of the lesson?

• And how could this best be facilitated?

Background Interview.

Prior to the experimental use of the tool by teachers, a ‘background interview’ was held with these teachers to get a clear picture of their background, previous knowledge, perspective on teaching, etc. The topics discussed were:

• Background; (experience, age, training, classes taught, …)

• How do you design your lessons?

• Proficiency concerning design and DBL: (knowledge, experience, tool seen previously, expectation, motivation, …)

• Perceived issues in current education that DBL could help to solve

• 21st century skills:

  • Knowledge

  • How do you already do this in your education?

  • Reflection by pupils in your education; importance? how? How many?

  • Planning by pupils in your education; importance? how? How many?

  • Practical: What is going to happen? Planning; (when in lesson, number of lessons, group, time, subject, …)

Observation of Designing Teachers.

In a 60-min session, the teachers were observed while using the tool to design a Design Based learning activity. During the design process, the actions carried out by the teachers and the discussions were recorded: by observation, audio-taping as well as ‘digitally’. A researcher was present to help the teachers in case they got stuck. After the design session there will be a short interview. Topics of the interview/observations are derived from the general analysis schedule, in particular: the ease of use of the tool; the workflow (are the design steps in the tool understandable and logically organized/sequenced in the experience of the teacher); the considerations/reflection that the tool provokes; comments from the designing teacher.

Observation in the Classroom.

Lessons were observed in which the teachers used their self-designed DBL-material in classroom. Special attention was paid to the points indicated in the last column of Table 1. In classroom the DBL-characteristics of the education in practice can be evaluated. Also, it can be seen to what extent the use of the tool has contributed to a lesson design that may or may not function properly. In the first round this usually is a ‘rough picture’ due to the practical and functional issues concerning the tool and its use (see above).

4 Developing the DBL Creation Tool – Round One

The development of the DBL creation tool presented in this paper happened in two iterative development rounds. This section reports on round one.

4.1 Starting Point and Paper Version of the DBL Creation Tool

The development of the DBL creation tool is based on the experiences gathered in earlier work by the authors [2] in which several design explorations investigates how digital means can support children and teachers in the Design Based Learning process. The focus was mainly on supporting the children. As part of this earlier work, a literature study was carried out to determine the important characteristics of Design Based learning activities, as described in the related work section. These activities have also led to a paper version of an ‘initial’ DBL creation tool (see Figs. 4 and 5) [30].

Fig. 4.

The paper based first iteration of the DBL creation tool.

Fig. 5.

The paper-based lesson planning made by the teachers.

In the first phase of the development of this paper-based tool, we looked at how the structure of a design process can be linked to topics and practice of today’s primary school class. Co-creation sessions were held with three teachers from two primary schools. During these sessions, DBL activities were created using the initial DBL-framework as shown in Fig. 4. The main insights of these activities were that teachers can develop learning activities on the basis of this framework and that a mind map activity can be used by teachers to brainstorm about how to link the learning objectives to a design assignment that is meaningful to their pupils [30]. The created learning activities also showed that dividing the design process into design phases, learning objectives and (suggested) design methods, was a useful structure for the teachers [30].

To give an example, one teacher made a DBL activity for 3 sessions in which the pupils had to develop a menu for the school restaurant that would appeal to the user group and that would be healthy. A mind map allowed the teacher to brainstorm on the topics that come up in the design assignment, such as user requirements, cooking and healthy eating considerations, the knowledge that should be provided to the students during the different phases of the process and the learning goals that could be assessed. Using this first paper prototype of the tool resulted in a paper-based lesson planning made by the teachers (see Fig. 5) [30].

4.2 Developing the First Digital Version of the DBL Creation Tool

To make the tool more dynamic and accessible, we aimed to further develop it into a digital DBL creation tool. To do so, two co-creation workshops were organized with teachers, school principals and design-researchers. The first workshop aimed to gather further requirements for a DBL creation tool and the second workshop aimed to elicit and verify the ‘building blocks’ that are needed to create successful DBL learning-activities.

The discussion during the first workshop confirmed that the workshop-participants considered the concept of DBL to be very well suited for the learning of 21^st century skills. The participants experienced that teachers are often uncertain about which regular core curriculum objectives can be addressed in DBL learning activities. A tool to create DBL activities should therefore help teachers, during the process of designing the activities, to get a clear picture of the possible core objectives that can be addressed. Additionally, the participants noted that not all teachers have the same level of experience in creating DBL activities, therefore the tool might support different experience levels of users (e.g. a more step-by-step approach for novices, and more freedom for experienced teachers). Practical requirements that surfaced during the workshop included the suggestion that the tool must deliver a very concrete product that can be stored and shared; the suggestion to make the tool suitable for different grades of primary school; and the suggestion that the tool should match with infrastructures already available in schools.

Based on the insights from the first workshop, a ‘workflow’ or step-by-step process was identified that teachers could go through when developing DBL learning activities. This workflow was further discussed in a second workshop. Participants in this workshop concluded that most teachers will use this workflow in a step-by-step fashion, starting with the decision on the general design assignment context and theme, and ending by deciding on the specific methods used. However, to accommodate more experienced teachers, the tool should also support a flexible workflow, in which teachers can choose the order in which they develop the DBL activities.

By discussing the workflow, the workshop participants also formulated a number of new requirements for the DBL creation tool. Firstly, the tool should provide help in selecting different learning goals (e.g. 21^st century goals and subject-specific goals). Secondly, it should support the practical planning of the design process. Thirdly, it should support an iterative and reflective approach to designing teaching materials, by stimulating teachers to regularly reflect on the materials they developed.

As a result of the two workshops, the workflow what translated into 7 concrete building-blocks that are required to design a DBL activity. These building-blocks, visualized in Fig. 6 form the basis of the DBL creation tool. The tool guides teachers through these building blocks and thus helps them to set up a Design Based learning activity.

Fig. 6.

The building blocks of the DBL creation tool.

As visualized in Fig. 6, teachers are recommended to start with the bottom three building blocks, ‘Authentic learning environment’, ‘Theme’ and ‘Class’, which together form the context in which the Design Based learning activity takes place. Following, in the ‘Assignment frame’ building block, the teacher records certain choices that together form a direction (assignment frame) for the assignment. The teacher can use this frame when writing the draft project brief. This brief describes the assignment frame for the students. The building block ‘Design process’ then helps to translate the project brief into concrete Design Based learning activities for the pupils.

While working on a Design Based learning activity, pupils go through a Design Based design process (learning process). This process consists of different phases that are determined by the teacher and/or pupils. Each phase consists of a design activity and one or more design methods. In each phase, the teacher can define what the pupils should do (design activity) and in what way (design methods).

4.3 Evaluation of the First Digital DBL Creation Tool

Evaluation Approach

To gain insights into the potential of the DBL creation tool, and to pinpoint points for improvement, it was evaluated with educational experts and teachers. Firstly, the tool was demonstrated to, and discussed with a panel of experts.

Secondly, 4 teachers used the tool individually or in pairs to create a Design Based learning activity of their choice during sessions of 60 min. During these sessions, a researcher was present to observe and offer explanations when needed, and the teachers were interviewed to gather their experiences. All participants were experiences teachers, but none of them were experienced with developing or teaching Design Based learning activities. See Fig. 7 for an impression.

Fig. 7.

Evaluation sessions with teachers.

Thirdly, one of the designed learning activities was tried out in class by the teacher who designed it, a researcher observed the lesson. See Fig. 8 for an impression.

Fig. 8.

Impression of evaluation in class.

Findings

Educational Design.

The evaluation of the first round identified a number of practical problems regarding the design of educational activities using the tool. It appeared that there was a need for a specification of the Design Based learning properties in the tool.

Furthermore, the tool lacked elements that are relevant for designing lessons, such as homework and testing. The experts argued that the tool was more structured along the design-process, rather than along the educational process. Interestingly, all elements of the educational design were considered by the teachers in the learning activities they developed, also those that were not included in the tool itself. This was also observed in the lessons. The tool led to more awareness on the complete package of educational design elements among the teachers.

Adoption and Implementation.

The workflow of the tool was not always clear for the teachers. Additionally, observing a lesson in which a designed activity was conducted with pupils revealed that the tool lacked concrete output materials that teachers can use while teaching. Teachers also felt that it yielded a lot of extra work in addition to the regular work when designing lessons, which meant that it was not experienced as time effective. All participants recognized that 21^st century skills are important but insufficiently supported in current materials, and that DBL offers an opportunity to solve this. However, since it was not yet possible with the first version of our DBL creation tool to link the activities to specific (21^st century) learning goals it was difficult for the teachers to experience it as a useful product for education.

Experts recognized that the tool forces teachers to make concrete decisions as part of the process of designing a lesson, which can stimulate reflection. This was confirmed by teachers who mentioned there were more reflective in their educational design when using the tool.

Characteristics of DBL.

The sessions in which teachers used the tool showed that the concrete specifications of what makes a good DBL activity were not yet clear, both for the teacher and the designers of the tool. The design process which pupils that conduct DBL go through in their projects was linearly incorporated in the tool, as a result of which it was not experienced by the teachers as intended by the designers. As a result, the teaching materials that were designed did contain the necessary DR specifications, but in the lesson the focus on these disappeared. It also appeared that the teachers needed to redesign the lessons during the lesson, because at that moment the needs and strengths of the pupils in relation to the teaching material became clear.

Educational Concerns.

The evaluation revealed that the cognitive workload of the teachers was not balanced, because the tool was not perceived as clear and comprehensible. Furthermore, the link between the didactic learning process of the pupils and the activities in the tool was lacking, as a result of which the role of knowledge in the designed lessons was missing for the teachers. It also turned out that the tool could steer more towards reflective design of lessons.

4.4 Improvements to the DBL Creation Tool

Based on interpretation of the results of the evaluation we have concluded the following improvements, which steered the development of the second digital version of the tool. The characteristics have been developed on the basis of the needs and feedback of the participating teachers and the observations of the researchers.

• The tool - and in particular the workflow – needs to be made more recognizable from the point of view of teachers: it must be more closely aligned with the workflow of educational design with which teachers are familiar. For example, by relying on ‘Van den Akker’s spider’s web’ [17].

• The desired workflow should be presented more clearly in the tool.

• It must be made clear which practical educational product the tool delivers.

• All the elements of lesson design [17] need a clear place in the tool - including homework and testing. For example, homework can be another ‘design phase’ or ‘design step’.

• The specific characteristics of DBL need to be made clearer and these must be included explicitly in the tool.

5 Developing the DBL Creation Tool – Round Two

5.1 Redesign of the DBL Creation Tool

In order to improve the DBL creation tool, a new tool was developed based on the results of the first evaluation round (see Fig. 9).

Fig. 9.

Screenshot of the second iteration of the DBL creation tool.

One aspect related to educational design has been added in the second DBL creation tool: an option for defining homework. This option supports teachers in linking different lessons of the DBL project. Furthermore, a conscious decision was made not to add the option for defining the assessment to this version of the tool yet, because it requires its own specification. This would be a next step for the further development of the tool.

The workflow of the tool has been clarified, to make the adoption and practical use easier for to teachers. The theory of Van den Akker [17], which is closer to educational design, was used to make the design process of the lessons more explicit for instructors.

The iterative process of designing lessons by instructors will also be better facilitated in the new tool, in which there is a clear distinction between making fundamental choices for the basis of the DR project and the elaboration of each of the activities. The space for redesigning the elaboration is built into the tool.

Furthermore, the lesson description that that can be saved, after all aspects have been completed, is tailored more to be practically applicable by teachers and students.

5.2 Evaluation of the DBL Creation Tool

For the second iteration, the evaluation of the DBL creation tool, consisted of different methods. First, observations were conducted of teachers using the tool, in which the teachers designed a series of lessons. The tool was used by the teachers to create a Design Based learning activity in 60 min (see Fig. 10). These observations were followed by a short interview of the teacher about the experiences of using the tool.

Fig. 10.

An impression of the observation session with the teachers.

The second method was an evaluation of the tool using expert interviews. Five educational experts were interviewed, while conducting a walk-through of the tool. The experts gave feedback on the interface of the tool and on the educational and Design Based learning aspects of the tool. The results of the methods were incorporated into the Evaluation matrix (Table 2).

Table 2. Evaluation matrix of design round 2 [24].

5.3 Results

Educational Design General.

One aspect that was missing from the tool was more details with regard to the lesson preparation. This includes lesson planning with “start”, “core” and “finish”, which for the teachers was an essential part of the lesson planning. The terminology used for the (design) activity lists needed further explanation. Also a link was lacking in the tool between the learning objectives and the activities, which can be explained by the lack of explanation for each activity. A positive aspect of the activities was that the teachers gained a lot of inspiration from the many possibilities in choosing activities and thus started to think more clearly about the design of their lessons. There was also a lack support for the assessment of lessons. However, this was deliberately chosen by the designers of the tool since the support of testing for teachers did not fit in the scope of this project.

Adoption and Implementation.

The teachers experienced the tool as clear and logical. The tool led to depth and awareness of the lessons of teachers. The teachers also felt that the tool could lead to modern education due to the clear link with 21^st century skills in the designed lessons. However, the terminology from the design domain that was used in the tool still needs to be explained better and tailored to terminology of teachers.

Characteristics of DBL.

The sequential structure of the tool fits well with the teachers, however, it does not yet support the complex structure of iterative thinking of Design Based Learning. This can limit the iterative thinking of instructors to a certain level. Teachers also indicated that you can develop other learning activities with the tool, not specifically DBL learning activities. This indicated that the management of the DR characteristics was lacking in the tool to a certain extent or was unclear to the teachers.

Educational Perspective.

The cognitive load of the new tool on the teachers is well balanced, the teachers understand what is expected. The missing aspect of the tool, which was defined in the goals of this project, was the reflection of the designing teachers during the development of lessons. The teachers also experienced this as lacking, some teachers indicated that they would like to have an inbuilt reflection moment per lesson and overall. There was also a lack of freedom of movement for some teachers in the tool, although this is built into the tool, so we need to look at how this can be made clearer in the tool.

5.4 Recommendations

Based on the results of the user tests, we have drawn up certain recommendations for the development of the latest digital version of the tool. The most important adjustments for the latest version of the online tool are listed below.

• Reflection triggers should be embedded in the tool for interim reflection by the teacher. These reflections could best take place after the conclusion of a “phase”. The reflection triggers can best be presented in the form of questions, which are not too guiding and stimulate reflection; not focusing on the teachers accountability.

• The evaluation showed that the teachers do not see the tool as specific to DR learning activities. In order to clarify this and to give teachers more guidance to develop real DR learning activities, certain DR triggers need to be added to the tool. One of these is a feedforward system comprising a tick-box where teachers can choose whether the lessons should follow an open (diversifying) or more converging strategy, that would filter the learning activities suggested by the tool during the rest of the design process. An option must also be added for each lesson in which the teacher can indicate which iteration the pupils should work on per lesson. Furthermore, in the first “phase” of the tool, more questions need to be asked of the teacher who focuses on Design Based learning, such as “how many iterations do you want the students to go through in this lesson package? and “Do you want the students to evaluate their ideas themselves and reflect on them?”.

• The list of teaching activities should be described more clearly and clustered. First of all, the teaching activities need to be worked out in detail, with regard to the role of the pupils and of the teacher, and which learning objectives these activities fulfil. Second, the list of teaching activities should be clustered by design phase, in order to provide more guidance to the teachers in developing a design process.

• An improvement must be made in the terminology of the tool, in order to attune this more to the target group primary school teachers.

• The core objectives and 21^st century skills should already be chosen by the teachers in the first phase of the tool. When selecting specific activities, it must be possible to select the sub-learning objectives.

5.5 Final Version of the DBL Creation Tool

In the latest version of the DBL creation tool, several improvements have been made compared to the previous tool. First of all, in order to improve the workflow, the phases are visually more clearly indicated by means of color gradation, an adapter bar and numbering. This leads to clearer navigation and a better overview. Furthermore, some functionalities have been moved in order to improve the workflow: for example, it can be indicated earlier in the process which core objectives are to be worked on.

In order to better support inexperienced teachers in the design of DBL learning activities, the DBL characteristics have been added in the form of tips. E.g. an option to indicate if this lesson is about converging or diverging and in which iteration the students are. It is now possible to add or remove lessons and move them back and forth.

Screenshots are presented of the project overview screen (Fig. 11), the fields to define the project, fields to fill in the lesson and the final reflection suggestions (Fig. 12).

Fig. 11.

Screenshot of the project overview screen.

Fig. 12.

Screenshot of the field to define the project with the reflection suggestions.

6 Discussion and Conclusion

This report describes the development of a tool that supports primary school teachers in creating teaching materials for teaching 21^st century skills, mainly by DBL. After defining initial conditions, the study progressed as “design research” consisting of several iterative design rounds. From this study, a detailed DBL creation tool has been developed as an end product and conclusions have been drawn for improvements that require further research.

6.1 Progress of the Project

The first design round showed that an in-depth redesign was still needed for the DBL creation tool, which led to 2 evaluation rounds. This was demonstrated by the fact that the second evaluation was more focused on the practical aspects of the tool. Because of this need for evaluation and time problems due to the duration of the second redesign, an extensive test of the tool in the classroom was not done. This time shortfall is also due to the planning in schools, time for building the second tool, and that turned out during the construction of the second tool that a sharper definition of DBL properties was needed. Because of this, evaluation could hardly involve the fourth level of Goodlad.

Throughout the project, it became apparent that there was a need to shift and sharpen the concept of Design Based Learning. This is a result of this research, but also a factor that has influenced the course of the project. The didactics of Design Based Learning have also been better defined through this project. This was done by elaborating on the basis of the theory of Ireland [22] about how DBL is offered to the teacher, which has consequences for the design space in the tool. This scheme supports teachers in understanding and selecting classroom learning activities based on the didactic level and direction of the activity.

The DBL creation tool wants to support teachers in applying didactics when creating DBL activities. Because it is a relatively unknown approach in education, it is important to ensure that the didactics of the learning activities fit in well with the level of the pupils. Students should not be over-questioned but challenged at the right level. In order to support the instructor in these choices, the tool provides guidance and regulation for the preparation of a project with various learning activities.

6.2 Conclusion

The central research question was:

How can PO instructors be supported in the design of DBL learning activities aimed at concrete learning objectives?

This question was examined on the basis of the following sub-research questions:

• (1) Are PO teachers able to apply the DBL creation tool as intended; in particular, to follow the design strategy offered, to reflect on the DBL learning activities using the tool and to improve the design from there iteratively?

Both research rounds have shown that the teachers very clearly apply the DBL creation tool as intended. The evaluation shows that the DBL creation tool fits in well with their mental model of their vision on education. It has not been demonstrated in the evaluation moments that the teachers improve the designs step by step over a longer period of time. Instead they are continuously improving there lessons. The tool therefore stimulates designing the lessons as an iterative process. Furthermore, during the first round, it appeared that many iterations on the elaborations of the design were devised by the teacher in (not during lesson preparations) the classroom, while the pupils carried out the learning activities.

• (2) Are PO teachers able to successfully develop DBL learning activities through the DBL creation tool?

• 2(a) Are the DBL learning activities, which the instructors develop by means of the tool, generally practical and educationally sound (can be implemented)?

The learning activities developed are practical and contribute to modern education. The learning activities do not necessarily meet the characteristics of Design Based learning. So, a clarification of the concept of Design Based Learning is required, in order to build it into the tool more clearly, so that the message also gets across better to the teachers. To this end, a first step has been taken, based on the results of the last evaluation round. The learning activities that have been developed have been recognized by the experts as being appropriate from an educational perspective. There are several aspects built into the tool that create this educational accountability.

• 2(b) Do the DR learning activities developed by the instructors using the tool have the intended characteristics (authentic, open, limited problem space, core objectives, etc.)?

After the first evaluation in this project, there appeared to be a need for a new definition of the characteristics of Design Based Learning. As a result, the properties mentioned in this research question have also shifted, as the property of “limited problem space” is no longer considered essential. The DBL learning activities do, however, comply with the rest of the intended Design Based characteristics, as also mentioned in the Related Work section. This can be seen in the built-in aspects of the tool and in the teaching material that has been developed.

• 2(c) Do the DR learning activities, which the teachers develop by means of the tool, enable pupils to achieve the desired learning objectives of both types?

Whether the DBL learning activities developed enable pupils to achieve the desired learning objectives of both types cannot be substantiated on the basis of the results of the evaluations carried out. However, we have positive expectations about this, because the learning objectives are linked to the DBL activities in the tool and by the designing teachers. This was also reflected in the course material developed, so the expectations are that this goal will be met during the implementation. These expectations were also confirmed by the educational experts interviewed.

6.3 Adaptation and Implementation of the Tool

The final version of the DBL tool, called TEACH21, was delivered as a standalone web tool, which can be used free of charge via the website (http://platoolab.id.tue.nl). A paper version of the tool and a manual can also be downloaded from the website.

The contribution of this tool to modern education is clear. Although teachers are able to develop their own teaching materials, they often do not know enough about the characteristics of DBL to develop their own well-designed DBL-type learning materials. TEACH21 takes the instructor step-by-step through the course material development process. TEACH21 then helps the teacher to describe a project, define the context of the design problem, and then fill it in by working out the details of a flexible number of lessons. TEACH21 provides suggestions for design activities, learning objectives and reflection questions so that instructors think about the characteristics of DBL and the quality of the teaching material.