Abstract
One of the fundamental aims of authoring tools is to provide teachers with opportunities to configure, modify and generally appropriate the content and pedagogical strategies of intelligent systems. Despite some progress in the field, there is still a need for tools that have low thresholds in terms of the users’ technical expertise. Here, we demonstrate that designing systems with lower entry barrier can potentially be achieved through co-design activities with non-programmers and carefully observing novices. Following an iterative participatory co-design cycle with teachers who have little or no programming expertise, we reflect on their proposed enhancements. Our investigations focus on Authelo, an authoring tool that has been designed primarily for Exploratory Learning Objects, but we conclude the paper by providing transferable lessons, particularly the strong preference for visual interfaces and high-level pedagogical predicates for authoring analysis and feedback rules.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
The aspirational goal behind the development of authoring tools for many years has been to enable users, and teachers in particular, with low technical expertise to create or modify the content and ideally the adaptivity of AIED system, according to their preferred pedagogical strategies [2, 3, 9]. However, the usability of such tools and particularly the time required to invest in learning them, are factors that affect teachers’ adoption and engagement in the design process of authoring [6]. It is important to understand that teachers have different expertise, needs and motivations and authoring tools should aim to meet those. In this paper, we present our approach to better appropriate authoring tools for teachers through participatory co-design activities.
Our case study is on AuthELO [5] that has been specifically designed for authoring intelligent support for Exploratory Learning Objects (ELOs) i.e. open-ended environments such as simulators, microworlds and other inquiry learning environments [8]. AuthELO’s design is inspired by the example-tracing approach [1] that encourages authors to develop feedback by executing the activity like a student and the FRAME approach [4] that requires separating the different concerns of feedback, reasoning, analysis and raw data from the model/events. This provides the author with data in a log window that represent the various states of the student interaction throughout the learning activity. Based on this evidence, the author can then perform analysis to derive the facts that drive feedback decisions in real time. Then the variables that correspond to those facts are used to set up rules for the generation of formative and summative feedback (see Fig. 1 and our previous work [4, 5] for more details).
This paper reflects on a participatory design study aiming to inform further development of AuthELO towards lowering the entry threshold for teachers who have little or no programming expertise.
2 Participatory Design for the Authoring UI
For the purposes of this study participation of teachers in the design process was of paramount importance since the main aim is to lower the entry barrier. This is a clear case for participatory design, a well-accepted method for attempting to solve a complicated design problem with the active involvement of people from different backgrounds and different expertise [10].
Based on a non-random sampling strategy and a design-thinking approach, we carefully selected 6 newly qualified teachers who were studying at the UCL masters in Education and Technology and had a range of expertise in using technology in pre-school and primary education settings, but no programming background (non-programmers group). The main goal was to provide rich and in-depth data and so the participants were further divided into two focus groups that were facilitated by one of the authors (EP) going through ideation, sketching brainstorming, and thinking aloud around the interaction with a prototype.
We also selected 3 more experienced computing teachers with enough programming background to teach computing but not necessarily professional programmers to develop applications. They are all skilled in basic JavaScript (novice group). They were supported to develop 15 different activities in a 3D Logo environment called MALT [7], and the corresponding support in AuthELO. We recorded the support that one of the authors (SK) had to provide. The objective was to see how the authoring is used and identify difficulties, commonalities and patterns in their solutions that can provide the basis of a higher-level language.
3 Key Findings and Discussion
Due to space limitations, we focus only on two key themes that emerged.
The Influence of Block Coding. One of the participants of the first group (familiar with block coding user interfaces) spontaneously proposed to introduce blocks of code with pre-defined “variables already written on, so we can drag and drop the blocks and connect them”. Building on this idea, another participant drew a sketch with custom select lists “from where you can click on to see all the variables and choose one”. Ensuing a conversation and brainstorming, the group sketched their final idea that involved use four custom select lists as shown in Fig. 2. They named the first list ‘condition’, and from this list, they could pick the words ‘if’, ‘then’ and ‘others. The second one was named ‘situation’, and when clicked all the previously set variables would appear on the list so they are able to pick the one that they need. The participants named the third list ‘action’, and with this list, they set what the variable should do, e.g. ‘display’, ‘do’, ‘play sound’. The rest of the discussion was pragmatic and involved including a list that the participants named ‘type’ to choose from the list of resources that should be displayed and other aspects such as a delete button. The key contribution here was the idea that the interaction would result with the constructed rule clearly visibly below that would be added in the list of rules.
The second focus group, led by a one member of the group who volunteered to sketch their thoughts, steered towards an idea of ‘board’ with a standard structure where the words ‘if’, ‘then’, ‘else’, ‘or’ were written i.e. a custom select list for the various conditional statements (called ‘conditions’). This idea became the centrepiece around which two other boards were proposed (‘actions’ and ‘reactions’) as well as a ‘construction’ area for dragging and dropping the various choices to make the feedback rules.
Situation and Actions as Predicates. Analysing the brainstorming of the non-programmer groups, a dominant theme was their concern on how to trigger feedback—something that they came up with unprompted. Without any prior knowledge in authoring rules or functional programming, they referred to ‘situations’ or ‘actions’ in a similar way to predicates in programming. Generalising even from a simple task they had, the participants referred to “all previously set variables to appear on the list so they are able to pick the one that they need”.
Analysing the support we had to provide to the novice users, the majority also revolved around the type of actions logged. Observing and generalising the solutions, we managed to reduce the code into a very small set of functions that seem to be a common requirement in all the activities. In particular, some of the high level constructs that were used in analysis and feedback and seem general to other situations are: (1) Number of actions since a particular trigger point (e.g. the start of the activity, or a particular event such as enabling the camera view), (2) The number of actions of a particular type (e.g. the number of times a button was clicked), (3) A list of actions of a particular type and references mostly to the first and last of those. Furthermore, other high-level constructs were the time elapsed from the beginning of the activity, a pre-defined expected duration of the activity and a way to refer to the potential feedback messages and their types directly in a simplified way.
4 Conclusions
In this paper, we described our approach to potentially lowering the threshold for intelligent support authoring. We have incorporated the high-level constructs that emerged in the design of AuthELO to ease the authoring declarative statements. The work described here also paves the way for a new user interface that takes advantage of the prevalence of block coding among computing teachers. Of course, it remains to be seen whether busy teachers with low technical expertise would be inclined to engage deeply with an authoring system even if it has a lower threshold. Our sample is not the most representative of teachers but the participatory design process indicated that while full development of an intelligent system would not necessarily be of interest to our participants, they value the possibility to easily modify the pre-designed feedback and occasionally add to suit their needs. Although, the results are from a small sample, we think that the designs proposed can reduce the amount of initial knowledge required from an author, increase readability, testability and maintainability of the code generated, and allow the analysis to be communicated easier between authors in collaborative projects.
References
Aleven, V., et al.: Example-tracing tutors: intelligent tutor development for non-programmers. Int. J. Artif. Intell. Educ. 26(1), 224–269 (2016). https://doi.org/10.1007/s40593-015-0088-2
Dağ, F., Durdu, L., Gerdan, S.: Evaluation of educational authoring tools for teachers stressing of perceived usability features. Procedia - Soc. Behav. Sci. 116, 888–901 (2014). https://doi.org/10.1016/j.sbspro.2014.01.316. http://www.sciencedirect.com/science/article/pii/S1877042814003334
Gaffney, C., Wade, V.P., Dagger, D.: Authoring and delivering personalised simulations an innovative approach to adaptive eLearning for soft skills (2010). http://www.tara.tcd.ie/handle/2262/67212
Gutierrez-Santos, S., Mavrikis, M., Magoulas, G.D.: A separation of concerns for engineering intelligent support for exploratory learning environments. J. Res. Pract. Inf. Technol., 103–116 (2012). http://www.acs.org.au/__data/assets/pdf_file/0020/16706/JRPIT44.3.347.pdf
Karkalas, S., Mavrikis, M.: Feedback authoring for exploratory learning objects: Authelo. In: CSEDU 2016 - Proceedings of the 8th International Conference on Computer Supported Education, Volume 1, Rome, Italy, 21–23 April 2016, pp. 144–153 (2016). https://doi.org/10.5220/0005810701440153
Karoui, A., Marfisi-Schottman, I., George, S.: Mobile learning game authoring tools: assessment, synthesis and proposals. In: Bottino, R., Jeuring, J., Veltkamp, R.C. (eds.) GALA 2016. LNCS, vol. 10056, pp. 281–291. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-50182-6_25
Kynigos, C., Grizioti, M.: Programming approaches to computational thinking: integrating turtle geometry, dynamic manipulation and 3D space. Inform. Educ. 17(2), 321–340 (2018). https://eric.ed.gov/?id=EJ1195612
Mavrikis, M., Gutierrez-Santos, S., Geraniou, E., Noss, R.: Design requirements, student perception indicators and validation metrics for intelligent exploratory learning environments. Pers. Ubiquitous Comput. 17(8), 1605–1620 (2013). https://doi.org/10.1007/s00779-012-0524-3
Murray, T.: Coordinating the complexity of tools, tasks, and users: on theory-based approaches to authoring tool usability. Int. J. Artif. Intell. Educ. 26(1), 37–71 (2016). https://doi.org/10.1007/s40593-015-0076-6
Vines, J., Clarke, R., Wright, P., McCarthy, J., Olivier, P.: Configuring participation: on how we involve people in design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2013, pp. 429–438. ACM, New York (2013). https://doi.org/10.1145/2470654.2470716
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Mavrikis, M., Karkalas, S., Cukurova, M., Papapesiou, E. (2019). Participatory Design to Lower the Threshold for Intelligent Support Authoring. In: Isotani, S., Millán, E., Ogan, A., Hastings, P., McLaren, B., Luckin, R. (eds) Artificial Intelligence in Education. AIED 2019. Lecture Notes in Computer Science(), vol 11626. Springer, Cham. https://doi.org/10.1007/978-3-030-23207-8_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-23207-8_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23206-1
Online ISBN: 978-3-030-23207-8
eBook Packages: Computer ScienceComputer Science (R0)