Keywords

1 Introduction

The aspirational goal behind the development of authoring tools for many years has been to enable users, and teachers in particular, with low technical expertise to create or modify the content and ideally the adaptivity of AIED system, according to their preferred pedagogical strategies [2, 3, 9]. However, the usability of such tools and particularly the time required to invest in learning them, are factors that affect teachers’ adoption and engagement in the design process of authoring [6]. It is important to understand that teachers have different expertise, needs and motivations and authoring tools should aim to meet those. In this paper, we present our approach to better appropriate authoring tools for teachers through participatory co-design activities.

Our case study is on AuthELO [5] that has been specifically designed for authoring intelligent support for Exploratory Learning Objects (ELOs) i.e. open-ended environments such as simulators, microworlds and other inquiry learning environments [8]. AuthELO’s design is inspired by the example-tracing approach [1] that encourages authors to develop feedback by executing the activity like a student and the FRAME approach [4] that requires separating the different concerns of feedback, reasoning, analysis and raw data from the model/events. This provides the author with data in a log window that represent the various states of the student interaction throughout the learning activity. Based on this evidence, the author can then perform analysis to derive the facts that drive feedback decisions in real time. Then the variables that correspond to those facts are used to set up rules for the generation of formative and summative feedback (see Fig. 1 and our previous work [4, 5] for more details).

This paper reflects on a participatory design study aiming to inform further development of AuthELO towards lowering the entry threshold for teachers who have little or no programming expertise.

Fig. 1.
figure 1

Parts of the AuthELO interface. After configuring the ‘logging’ in the corresponding tab, authors can start doing the activity like a student. That will immediately start generating data in the log that can used for analysis and authoring feedback rules as in this example. The owl is the chosen feedback agent and here it is being tested with a 3D Logo activity.

Full size image

2 Participatory Design for the Authoring UI

For the purposes of this study participation of teachers in the design process was of paramount importance since the main aim is to lower the entry barrier. This is a clear case for participatory design, a well-accepted method for attempting to solve a complicated design problem with the active involvement of people from different backgrounds and different expertise [10].

Based on a non-random sampling strategy and a design-thinking approach, we carefully selected 6 newly qualified teachers who were studying at the UCL masters in Education and Technology and had a range of expertise in using technology in pre-school and primary education settings, but no programming background (non-programmers group). The main goal was to provide rich and in-depth data and so the participants were further divided into two focus groups that were facilitated by one of the authors (EP) going through ideation, sketching brainstorming, and thinking aloud around the interaction with a prototype.

We also selected 3 more experienced computing teachers with enough programming background to teach computing but not necessarily professional programmers to develop applications. They are all skilled in basic JavaScript (novice group). They were supported to develop 15 different activities in a 3D Logo environment called MALT [7], and the corresponding support in AuthELO. We recorded the support that one of the authors (SK) had to provide. The objective was to see how the authoring is used and identify difficulties, commonalities and patterns in their solutions that can provide the basis of a higher-level language.

3 Key Findings and Discussion

Due to space limitations, we focus only on two key themes that emerged.

The Influence of Block Coding. One of the participants of the first group (familiar with block coding user interfaces) spontaneously proposed to introduce blocks of code with pre-defined “variables already written on, so we can drag and drop the blocks and connect them”. Building on this idea, another participant drew a sketch with custom select lists “from where you can click on to see all the variables and choose one”. Ensuing a conversation and brainstorming, the group sketched their final idea that involved use four custom select lists as shown in Fig. 2. They named the first list ‘condition’, and from this list, they could pick the words ‘if’, ‘then’ and ‘others. The second one was named ‘situation’, and when clicked all the previously set variables would appear on the list so they are able to pick the one that they need. The participants named the third list ‘action’, and with this list, they set what the variable should do, e.g. ‘display’, ‘do’, ‘play sound’. The rest of the discussion was pragmatic and involved including a list that the participants named ‘type’ to choose from the list of resources that should be displayed and other aspects such as a delete button. The key contribution here was the idea that the interaction would result with the constructed rule clearly visibly below that would be added in the list of rules.

Fig. 2.
figure 2

Low fidelity paper prototype proposed by the non-programmers group

Full size image

The second focus group, led by a one member of the group who volunteered to sketch their thoughts, steered towards an idea of ‘board’ with a standard structure where the words ‘if’, ‘then’, ‘else’, ‘or’ were written i.e. a custom select list for the various conditional statements (called ‘conditions’). This idea became the centrepiece around which two other boards were proposed (‘actions’ and ‘reactions’) as well as a ‘construction’ area for dragging and dropping the various choices to make the feedback rules.

Situation and Actions as Predicates. Analysing the brainstorming of the non-programmer groups, a dominant theme was their concern on how to trigger feedback—something that they came up with unprompted. Without any prior knowledge in authoring rules or functional programming, they referred to ‘situations’ or ‘actions’ in a similar way to predicates in programming. Generalising even from a simple task they had, the participants referred to “all previously set variables to appear on the list so they are able to pick the one that they need”.

Analysing the support we had to provide to the novice users, the majority also revolved around the type of actions logged. Observing and generalising the solutions, we managed to reduce the code into a very small set of functions that seem to be a common requirement in all the activities. In particular, some of the high level constructs that were used in analysis and feedback and seem general to other situations are: (1) Number of actions since a particular trigger point (e.g. the start of the activity, or a particular event such as enabling the camera view), (2) The number of actions of a particular type (e.g. the number of times a button was clicked), (3) A list of actions of a particular type and references mostly to the first and last of those. Furthermore, other high-level constructs were the time elapsed from the beginning of the activity, a pre-defined expected duration of the activity and a way to refer to the potential feedback messages and their types directly in a simplified way.

4 Conclusions

In this paper, we described our approach to potentially lowering the threshold for intelligent support authoring. We have incorporated the high-level constructs that emerged in the design of AuthELO to ease the authoring declarative statements. The work described here also paves the way for a new user interface that takes advantage of the prevalence of block coding among computing teachers. Of course, it remains to be seen whether busy teachers with low technical expertise would be inclined to engage deeply with an authoring system even if it has a lower threshold. Our sample is not the most representative of teachers but the participatory design process indicated that while full development of an intelligent system would not necessarily be of interest to our participants, they value the possibility to easily modify the pre-designed feedback and occasionally add to suit their needs. Although, the results are from a small sample, we think that the designs proposed can reduce the amount of initial knowledge required from an author, increase readability, testability and maintainability of the code generated, and allow the analysis to be communicated easier between authors in collaborative projects.