Open Learner Models as Drivers for Metacognitive Processes

Abstract

Maintaining a model of the learner’s understanding as they interact with an e-learning environment allows adaptation to the learner’s educational needs. An Open Learner Model makes this machine’s representation of the learner available to them. Typically, the state of the learner’s knowledge is presented in some form, ranging from a simple overall mastery score, to a detailed display of how much and what the learner appears to know, their misconceptions and their progress through a course. This means that an Open Learner Model provides a suitable interface onto the learner model for use by the learner, and in some cases for others who support their learning, including peers, parents and teachers. This chapter considers some of the similarities between the goals of supporting and encouraging metacognition in intelligent tutoring systems and learning in general, and the benefits of opening the learner model to the user. We provide examples of two important classes of open learner models: those within a particular teaching system and those that are first-class citizens with value independently of a teaching system. The chapter provides a foundation for understanding the range of ways that Open Learner Models have already been used to support learning as well as directions yet to be explored, with reference to encouraging metacognitive activity and self-directed learning.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

The type of learning technology addressed in this chapter is adaptive learning environments, or intelligent tutoring systems (ITS). This technology typically has three main components: a model of the domain or subject of study (e.g. a model of topics, concepts and interrelationships between concepts); a learner model, capturing the individual user’s understanding of the domain, as inferred during their interaction (e.g. from navigation choices, answers to questions, problem-solving attempts, time on task); and a pedagogical model to allow personalisation of the teaching or guidance, for the learner. In this chapter we focus on the learner model and promoting metacognitive activity by providing the learner with access to the model of their knowledge.

Metacognition has been defined in many ways, but it is generally considered to involve higher-order thinking about cognition, for example, involving knowledge about cognition. Importantly, it relates to regulation or ­monitoring of cognition, with the associated aspects of learner control over their own learning processes. (See, e.g. Georghiades, 2004; Schraw, 1998; Veenman, Van Hout-Wolters, & Afferbach, 2006.) Much of the work refers back to Flavell’s introduction of “metacognition and cognitive monitoring”, presented through discussion of metacognitive knowledge (comprising knowledge of person, task and strategy variables) and metacognitive experiences (Flavell, 1979). The importance of enhancing metacognitive awareness in learners has often been argued (e.g. Schoenfeld, 1987; Schraw, 1998), including the use of computer-based metacognitive support, such as for training general learning ability (Derry & Murphy, 1986); tutoring help-seeking strategies (Roll, Aleven, McLaren, & Koedinger, 2007); developing self-awareness through learning by teaching (Wagster, Tan, Biswas, & Schwartz, 2007); a reflection assistant for problem-solving (Gama, 2004); and encouraging learners to develop greater awareness of cognitive and metacognitive learning strategies (Bull, 1997).

Although metacognition is often described as requiring conscious processing and application, it has also been suggested that some lower levels of consciousness in processing may still be metacognitive, for example, through habitual regulatory behaviour (Veenman et al., 2006). It is this latter view that we adopt in this chapter: we acknowledge both the benefits of explicit metacognitive instruction or support and the potential to support metacognitive activity in a less explicit manner. We discuss these issues with reference to open learner models.

As stated above, modelling a learner’s understanding (e.g. from questioning, tasks, help or hints requested) allows an ITS to adapt the interaction to suit the student. Open learner models (OLM) are learner models that are accessible, or “open” to the learner they represent. (See Bull & Kay, 2007; Dimitrova, McCalla, & Bull, 2007 for recent overviews of open learner modelling.) There are many reasons for making a learner model open to the learner, and we discuss these in the next section, noting the links and the relevance of many of these goals for metacognition. In the following section, we explore metacognition in relation to two types of open learner model: those embedded in a tutoring system and those used independently of the larger tutoring environment. We explain these ideas with carefully chosen examples which illustrate some of the breadth of possibilities explored in research into open learner modelling. We conclude with a discussion of the links between learner control of their learning and open learner modelling and the essential role that OLMs can play in supporting metacognition and metacognitive development.

Metacognition in Open Learner Modelling

The SMILI:) (Student Models that Invite the Learner In) Open Learner Modelling Framework (Bull & Kay, 2007) provides a method of describing and analysing existing OLMs, and it offers a set of guidelines for the designer of an OLM to consider. The framework aims to improve understanding of the nature of OLMs and their potential roles. Its elements can facilitate comparisons between OLMs and systems that use OLMs.

SMILI:) identifies various purposes for opening the model. We now summarise these, italicising those that are particularly relevant for metacognition:

• Improving learner model accuracy by allowing the learner to make contributions to their learner model

• Promoting learner reflection through confronting students with representations of their understanding

• Facilitating planning and/or monitoring of learning

• Facilitating collaboration amongst learners

• Facilitating competition amongst learners

• Supporting navigation

• The right of access to information stored about oneself

• Learner control over and responsibility for their learning

• Trust in the learner model content

• Formative assessment

• Summative assessment

While some of the above points have not been specifically identified as means to support metacognition, it is clear that this might also apply in such cases. For example, allowing learners to provide information directly for their learner model, to help increase its accuracy, can have the effect of prompting learners to think about their knowledge and understanding more precisely. Similarly, an OLM that facilitates navigation to other parts of a system through some kind of highlighting of links may also help learners to more deeply consider the structure and prerequisites within a domain.

Most OLMs are embedded in an ITS, and so designing the open learner model involves design decisions and compromises. It is necessary to ensure that the OLM does not compromise the effectiveness of the main teaching interface. So, design for externalisation of the learner model requires decisions about integrating viewing of the model into the larger interaction. We provide examples of OLMs in ITSs in Sect. 2.1, with a focus on how the OLMs aim to support metacognition.

Independent OLMs exist independently of any single system or ITS (Bull et al., 2008). Learner modelling occurs in the usual manner, but the primary purpose of the independent OLM is to help learners to recognise any problematic issues themselves, through inspection of their learner model, and then independently carry out ­appropriate work to overcome difficulties identified. This approach has links with the goals of enhancing metacognitive behaviours, with a focus on encouraging learner independence. We consider independent OLMs in Sect. 2.2.

Supporting Metacognition with Open Learner Models in Intelligent Tutoring Systems

Learner models are the core drivers of personalisation in an ITS. They may well be the defining component of an ITS, since there is such diversity in the other elements that may be needed for any particular tutoring system. Learner models can take many forms. The most appropriate depends on many factors, including pragmatics, such as the system’s knowledge representation and reasoning approach for the domain knowledge and the teaching expertise. Others relate to the needs of the particular user, for example, their age and goals.

The dominant form of learner model reported in the ITS literature appears to be an overlay of the domain expertise. This means that the ease with which a model may be made available and understandable to a learner depends upon the representation of the domain. When that domain expertise is large or complex, it may be very difficult to make it usefully open to the learner. A natural approach to this problem is to define a part of the learner model that summarises the key elements that are meaningful and helpful for a learner.

One excellent example of this is in the SQL-Tutor (Mitrovic & Martin, 2002). This is a constraint-based tutor which makes use of a hundreds of constraints. It would be quite difficult to create a meaningful interface onto these. Instead, it ­presents a summary of the aspects that make sense from a student’s perspective as illustrated in Fig. 23.1. It shows just six aspects of the learning domain, each a key element of SQL. For each of these, the learner can see their progress in terms of the demonstrated correct understanding (the leftmost green part of each bar), incorrect understanding (the central red part) and the remaining white part indicating course content the student has yet to cover. In the figure, we can see that this student is only about halfway through the content but has mainly demonstrated correct understanding so far. A comprehensive evaluation of this approach showed significant learning benefits, especially for weaker students, and positive attitudes to this high-level progress indicator (Mitrovic & Martin, 2007). Notably, the open learner model assisted students in making better choices about problems they should tackle, a metacognitive skill for managing their learning. This form of open learner model has also been used in cognitive tutors (Corbett & Anderson, 1994) which also have a large complex underlying learner model but present the learner with a simple interface that has a readily understood skill metre.

Fig. 23.1

Skill metres of the SQL-Tutor (Mitrovic & Martin, 2007)

A similar role for an OLM, as a starting point for the student to decide what to learn next, is found in the QuizGuide (Brusilovsky & Sosnovsky, 2005) adaptive educational hypermedia system (illustrated in Fig. 23.2 by the targets and arrows). Although this is for the same broad domain (SQL), the underlying system representation is quite different, being based on a coding of each available task with the concepts or learning objectives. In both cases, the key issue is that the information made available to learners facilitates their ability to determine how well they are progressing in different aspects of the domain, providing a support for reflection (e.g. encouraging them to think about their understanding, skills or level and think about their learning process). From this, the OLM facilitates learners’ control of their learning as it helps them decide what to learn and how to plan their learning, important metacognitive skills. Indeed, these interfaces also help learners monitor their progress, because they can monitor the effectiveness of their plan, in terms of the changes in the open learner model.

Fig. 23.2

Arrows in targets in QuizGuide (Brusilovsky & Sosnovsky, 2005)

A rather different approach to open learner modelling is illustrated in Simprac (Chesher, 2005; Chesher, Kay, & King, 2005), a tutor for medical students learning about the long-term management of chronic illness (Fig. 23.3). At the top left is one of the consultation interfaces; in this example, the interface enables the learner to examine parts of the simulated patient. The middle-right screen is presented to the learner at the end of each consultation with the simulated patient. It shows the learner each of their actions in the last consultation, and they are asked to reflect on these by assessing the importance of each question they asked the patient, as well as each aspect of the examination and tests ordered. The lower histogram shows the learner’s performance in terms of the issues they explored, compared against their cohort.

Fig. 23.3

Simprac OLM (Chesher, 2005; Chesher et al., 2005)

One of the challenges of this domain is that learners can easily become entrenched in one perspective of the problem and its management: in spite of evidence that a management plan is ineffective, doctors may fail to recognise that this is the case. Accordingly, this tutor was created with a reflective layer, a set of interface elements that were designed to encourage the learner to reflect on their actions in the last consultation. To do this, the interface calls on the learner to reflect on all elements of the series of simulated consultations with patients. Following Schön (1987), the tutor supports reflection at two levels. First, it supports reflection-on-action meaning that the learner pauses at the end of a consultation to reflect on the step in that consultation. It also supports reflection-on-reflection, as the learner is encouraged to reflect on the way that they did the reflection phase. These are all metacognitive actions. For the core goals of supporting ­metacognition, an important aspect of the design of this OLM is that it shows learners their own performance in relative terms at two levels. First, it shows their performance compared with the expectations of the author of the tutor, an approach that can ensure that the tutor fits in with the teaching approach of a course and programme. Second, it shows their skill compared with a relevant, matched group of learners. In Simprac, there are three groups: medical students, general practitioners and experts in the particular domain of the tutorial. This tutor deals with a very different class of task from the SQL of the systems above: notably, there is some disagreement between experts about the best practice. It may be unrealistic and discouraging to show a medical student their performance against an expert, especially as an expert may be able to use quite different strategies from those that are best for a medical student. There are open questions about how to design and present a learner model that can best support reflection and particularly how to do it in ways that facilitate learning of the domain and of metacognitive skills. However, one important issue involves ensuring that the learner can compare their own progress and performance against meaningful standards that fit into any broader learning context.

While the above examples make available a quite small (part of a) model, there may be cases where there is value in enabling a learner to gain an overview of a large model. This issue has been explored in SIV (Kay & Lum, 2005). The SIV visualisation enables a learner to see their progress over the hundreds of elements in a course in user interface design. The left part of the screen in Fig. 23.4 shows the learner’s knowledge of concepts by the size, colour and positioning of the concept labels. The ontology underlying SIV was critical for enabling learners to move up and down granularity levels, and it also enables learners to focus on sets of related concepts within the domain. Students use this to plan their study for final examinations, with the OLM showing areas where they have weakness. Notably, the evidence available for this OLM comes from sources of varying reliability (shown to the user as illustrated in the right of the screen in Fig. 23.4), and different learners interpreted that evidence ­differently, some valuing one source highly while other students did not. This raises the question of providing learners with control over the system’s interpretation of evidence that informs their learner model: without this, the individual learner will find the OLM less useful. This raises some additional issues for metacognition and learner control, particularly whether the learner is entitled to decide how to value the different sources of learner modelling evidence.

Fig. 23.4

The SIV overview (Kay & Lum, 2005)

SIV also provides a summary view of learning progress of the class, which is also invaluable for the teacher. To this point, we have focused on metacognition in relation to the learner. However, any ITS, learning management system (LMS) or similar tool that is used in the context of a course, with lectures, labs and other activities, has the potential to support metacognitive skills of the teacher. A suitable OLM can enable the teacher to assess the effectiveness of their own teaching or a particular innovation: the OLM can show the progress of the class and potentially this class compared with other relevant cohorts. Essentially, the teacher is a learner who is continuously learning how to teach. This metacognitive role for the OLM has broad significance. It has been shown to be effective in the context of a Logic Tutor (Merceron & Yacef, 2003) and has been explored in the context of a widely used LMS: CourseVis showed a high-level representation of a class activity on the LMS (Mazza & Dimitrova, 2004). While the classroom teacher has a different relationship to an ITS than that of a student, there is potential for important learning gains if the teacher’s metacognition is scaffolded by an OLM.

Independent Open Learner Models to Facilitate Metacognitive Activity

Unlike the examples in the previous section, we here consider OLMs as first-class citizens that have value on their own, independently of any particular teaching system and potentially making use of learning data from multiple teaching systems. We consider this class of OLM likely to become of increasing importance, as there are growing numbers of electronic learning support tools of various sort, including the ubiquitous LMS, conventional software tools that are used as part of the learning as well as the many online e-learning tools and ITSs.

These independent OLMs are designed for use independently of individualised teaching or guidance as is typically provided by an ITS. Such independent OLMs usually have, as their primary aim, the promotion of metacognitive activities, such as self-assessment, self-monitoring, reflection and planning (as in some of the above examples), but within an overall context of encouraging autonomous or independent learning outside the system. Students can use these OLMs to help them identify their knowledge and difficulties and plan where they need to invest effort to overcome any problems. The responsibility for determining and undertaking appropriate activities lies with the learner. Therefore, an independent OLM may or may not have a domain model: the domain may be as simple as an unstructured list of topics (we would not consider this to be a “model”) or may comprise complex relationships of some kind to support diagnosis for the learner model contents. Either way, the role of the independent OLM is not to teach domain content, rather to promote and support independent learning and decisions by the user.

Two independent OLMs displaying learner models at different levels of detail/structure have demonstrated the possibility to support students alongside lecture courses—that is, in real-use settings (Bull et al., 2008). Figure 23.5 illustrates the simple skill metre and a similar graphical overview of knowledge level in OLMlets (Bull, Quigley, & Mabbott, 2006), and the structure of map and tree views of the Flexi-OLM learner model (Mabbott & Bull, 2006). In each case, colour is used to represent the level of knowledge of a topic or concept, and short text statements of misconceptions can be viewed, designed to prompt learners into investigating their specific problems. For example, from OLMlets used in an adaptive learning environments course: “You may believe that whether students like a system is more important than whether they learn from it”; “You may believe that a system does not have to understand the learner model.” From an introductory mathematics course: “You may believe that denominators are added when adding fractions”; “You may believe that, when adding matrices, the individual terms within a matrix are added together.”

Fig. 23.5

Independent OLMs with simple displays (top OLMlets Bull et al., 2006) and structured displays (lower Flexi-OLM Mabbott & Bull, 2006)

OLMlets was designed specifically to promote formative assessment (i.e. assessment designed to provide feedback to support the learning process—rather than summative assessment that produces a grade or mark) and learner autonomy for independent use alongside a range of courses (Bull et al., 2006). Learners answer questions relating to the key concepts of a course and view a simple overview of their knowledge levels and statements of their misconceptions (top of Fig. 23.5), as a starting point for their independent work. The simplicity of the model presentation reflects the simplicity of the underlying learner model, as it is intended for easy introduction by instructors, into a variety of courses. Deployment of OLMlets throughout several ­university electronic, electrical and computer engineering modules showed that students will use an OLM such as this to support their learning and are able to do so in a manner that suits their learning preferences, and the structured tree and map views of Flexi-OLM (bottom of Fig. 23.5) were also used by many students taking the Cprogramming module for which it was designed (Bull et al., 2008). As no additional computer tutoring or metacognitive support was provided in either case, any usage of the OLMs suggests that learners were gaining some benefit simply from the availability of an independent OLM. Thus, although we do not have specific information about how students were using these OLMs (e.g. to recognise their knowledge state, to plan their learning, to reflect on their difficulties), the fact that they were using them suggests that some kind of metacognitive activity was taking place that students perceived as helpful.

A clear example of an independent OLM to prompt metacognition is the Notice OLM (Shahrour & Bull, 2008). Notice is based on the second-language acquisition literature on awareness and “noticing” language features in language learning (Rutherford & Sharwood Smith, 1985; Schmidt, 1990) and “noticing the gap” between one’s own language rules and the (correct) target language forms (Schmidt & Frota, 1986): issues that have much in common with the general metacognition literature. Notice uses salience/highlighting techniques [recommended for ­computer-assisted language learning (e.g. Chapelle, 1998)], to draw the learner’s attention to grammatical elements. Figure 23.6 shows the “comparison view”: coloured highlighting in the learner model (left) indicates the correctness of the student’s use of irregular plural nouns based on the learner model representations, next to native speaker or expert use (the system model: right). This is one method of encouraging learners to “notice the gap” between their language and the language to which they have been exposed (Schmidt & Frota, 1986), as mentioned above.

Fig. 23.6

The Notice OLM for language learning (Shahrour & Bull, 2008)

Notice was found to facilitate immediate noticing of language elements (irregular plural nouns and irregular simple past verbs) by adult second-language learners, and much of this knowledge was retained at a significant level, as demonstrated in a delayed post-test 1 week after the experimental session (where no teaching of the target features had taken place in the meantime) (Shahrour & Bull, 2008). While we do not know whether learners remembered the forms based on their interaction with the OLM or whether they subsequently actively tried to notice or find out about the forms (as is one of the key aims of an independent OLM), it does appear that this kind of approach can be useful to prompt noticing in language learning. It will be interesting to explore the extent to which this may also apply in other subjects.

Negotiated learner models are interactive OLMs that allow the student to negotiate the learner model contents with the system (Bull & Pain, 1995; Dimitrova, 2003; Kerly & Bull, 2008). If the student disagrees with any of the system-inferred learner model data, they can challenge the system in an attempt to change the model, with each party required to justify their beliefs about the student’s knowledge, through discussion. For example, the system may offer the learner the opportunity to compromise (i.e. agree on an ­intermediate representation of knowledge, if the system’s and the student’s confidence in the learner’s knowledge are sufficiently close to allow a compromise to be a reasonable action); the student may provide the information that they have forgotten certain concepts since their last interaction, indicating that the learner model has “slipped backwards” or that their understanding has increased due to a lecture or from reading undertaken between sessions with the system; and the system may require the student to convince it to change its model by taking a short test to demonstrate their knowledge (or lack of knowledge). The top of Fig. 23.7 shows an excerpt of the display of learner confidence in their knowledge placed alongside the system’s confidence in their knowledge, in order to highlight any differences to the learner; below is an excerpt from a student attempt to challenge the learner model in menu-based model negotiation in Mr. Collins (Bull & Pain, 1995). Such negotiation of the learner model is designed (1) to help improve the accuracy of the model by allowing the student to contribute information for consideration in the modelling process and (2) through the process of discussion of the learner’s knowledge, to prompt learners to reflect on their understanding and develop a greater awareness of their learning needs. This also places some of the responsibility for the learning interaction, with the learner. The latter point is particularly relevant for promoting metacognition.

Fig. 23.7

Negotiating the learner model in Mr. Collins (Bull & Pain, 1995)

In CALMsystem (Kerly & Bull, 2008), the learner’s level of knowledge of topics is displayed for comparison to the system’s inferences about their knowledge (left of Fig. 23.8). However, the model negotiation process is more flexible than in Mr. Collins, using natural language in discussion with a chatbot (right of Fig. 23.8). Statements such as the following to the chatbot (by 10–11-year-olds) are indicative of self-monitoring: “but I need more work on it”, “I am getting better”, “I have changed my mind about my beliefs”, and “can I change a belief [in the model] about separating solids and liquids please”. A study over two sessions with children aged 10–11 in a science class demonstrated significant improvements in self-assessment accuracy both in an inspectable-only condition (left of Fig. 23.8) and a full negotiated learner modelling approach (both parts of Fig. 23.8) and with significant improvements in the negotiated condition over the inspectable condition (Kerly & Bull, 2008). It appears, therefore, that use of a simple inspectable model for this age group can help learners, but the process of discussion of their knowledge can bring further benefits and so could be recommended where such an approach would integrate well with the aims and interactions with a system.

Fig. 23.8

Negotiating the learner model in CALMsystem (Kerly & Bull, 2008)

As with Simprac (Chesher, 2005; Chesher et al., 2005) in the previous ITS section and Notice (Shahrour & Bull, 2008) in this independent OLMs section, OLMlets (top of Fig. 23.5) allows students to compare their knowledge against a standard. Here instructors input the expected level of knowledge for each stage of the course (defined by week, day or lecture number, as appropriate), and students can view their own skill metres (or other representations) alongside the expected knowledge for the current stage of the course, displayed in the same form, to support their self-evaluations and planning in the context of present expectations (Bull et al., 2006). This allows students to, for example, note that although their current level of understanding of a concept may be quite low, it is nevertheless in line with expectations for that stage of the course. OLMlets also allows students to release their model data to their instructors, thus offering the benefits to teachers suggested above in ITS contexts, in the use of independent OLMs, and has been shown able to promote spontaneous (face-to-face) peer discussion and help-seeking amongst students when they choose to release their learner models to each other (Bull & Britland, 2007). This is therefore another common goal of metacognition researchers and open learner modelling researchers. Furthermore, an OLM designed to help parents help their children with fractions was found also to highlight to parents misconceptions that they themselves held about calculating fractions (Lee & Bull, 2008).

This section suggests that independent OLMs can in themselves enhance metacognitive behaviours related to the identification of knowledge, regulation of learning or planning of learning activities, and they can be used to prompt actions to facilitate learner independence.

Long-Term Open Learner Models

The examples above have all been associated with a rather limited context. In the life of the learner, we might build a quite comprehensive learner model that draws on the full range of evidence about the learner’s progress. This learner model could then support reflection on long-term learning, such as reading progress over the whole of primary school education or mathematics progress through the whole of school. A key value of such a model would be as an OLM for reflection by learners, perhaps in conjunction with their teachers and parents, to monitor progress; identify serious, long-term problems; and plan learning.

In Fig. 23.4, at the left of the SIV display, the user model visualisation tool is a generic learner model display for large user models (Apted, Kay, Lum, & Uther, 2003). We have used it in several contexts. For example, it was initially designed for use in a Graduate Medical Programme where it aimed to show students their progress on around 600 learning topics that span 2 years of study. In this case, the evidence for learning came from a system that students could use to do multiple-choice self-tests. Even the example of Fig. 23.4 involved a semester long course with two main sources of evidence about learning:

• Student grades, extracted from an LMS, where this provided marks from the weekly lab sessions and the marks on each of the questions of the final exam

• Evidence based on interaction with an online lecture delivery system where students ­listened to online audio that was associated with “slides” with the amount of time students spent on each slide matched against the known audio length, to infer which lecture slides the student appeared to have “attended”

This is an interesting example since it involves multiple sources of evidence and each is of quite different grain size (Kay & Lum, 2005). Importantly, the visualisation display can be used independently of any application, taking an arbitrary learner model in the required format and making it available to the learner for reflection on their progress. It enables a learner to identify areas that the learner model indicates they are weakest in. The display can be configured to allow the learner to define their own standard; for example, one learner may only want concepts treated as known if they have a current knowledge level of at least 80% while another learner may set this threshold at 60%.

Another example of a long-term learner model is shown in Fig. 23.9. This is one of several OLM created for use in conjunction with a project management tool used for educational purposes (Kay, Maisonneuve, Yacef, & Reimann, 2006; Reimann & Kay, 2010). It was used over a semester in a software engineering capstone project subject where students worked in teams to create software. The project management tool is widely used by programmers. One of the goals of this subject is that students develop their group work skills and this learner model assists them and their facilitators to see aspects of the team operation. The display in the figure shows the interaction between team members on the wiki, where an interaction was judged to occur when two people edited the same wiki page. The heavier the line, the more the interaction. In the actual interface, each coloured dot is labelled with the learner’s login ID (removed here to anonymise the display).

Fig. 23.9

Example of an interaction diagram

We can easily see that all team members are interacting with the exception of the person ­represented by the (green) dot near 6 o’clock. We can also see that some people interact more than others. We include this example to illustrate important possibilities offered by OLMs to support long-term learning:

• They can display some of the many sources of long-term data that is available as digital traces of activity.

• They can be integrated into arbitrary online tools, including those that were not explicitly designed for learning.

• The addition of an OLM creates new possibilities for people to learn, based upon reflecting on the OLM, potentially realising that they have strengths and weaknesses they were not previously aware of.

• It can support the learner in monitoring their progress as they aim to change those behaviours.

• It is particularly valuable when the individual learner can see themselves in relation to relevant peer groups so that they can assess the significance of their personal performance.

While this representation does not show the quality of contributions, it does provide other important information that may prompt metacognitive behaviours. Returning to the OLMlets independent OLM (top of Fig. 23.5), based on use across courses in a degree, students can follow their progress towards the range of more general learning outcomes required for a professional-accredited engineering degree (Bull & Gardner, 2010). Each learning outcome is listed with the courses contributing to this learning outcome included and level of understanding indicated by colour (two learning outcomes are given as examples in Fig. 23.10). Each course may contribute to several or many learning outcomes. The aim is to help students identify the general engineering skills required of a professional engineer and how their courses combine to help them achieve these skills, and their own progress is indicated as a focus for their attention.

Fig. 23.10

An example of an independent OLM linking elements across a degree

OLMs have the potential to support such metacognitive, long-term learning outcomes as they can be readily applied to arbitrary data collections by the learner as part of their long-term lifelong learning. Within more formal learning contexts, we need to make it easy for teachers to integrate arbitrary data sources into OLMs for use by their students, making use of suitable information that enables the individual to better assess their own achievements in relation to the standards that are relevant for their context.

Discussion: Links Between Research Directions in Metacognition and Open Learner Modelling

We have described a range of approaches to open learning modelling, in terms of the relationship to an ITS and some of the forms that OLMs have taken. We have also identified several issues that are important for an OLM to provide effective support for metacognitive activities of reflection, self-monitoring as well as planning and control of learning processes. If metacognitive skills were explicitly modelled by an ITS, an interactive OLM for these, too, could be the basis for a ­metacognitive activity and could provide an additional source of evidence about these skills in the learner’s self-perceptions. An interactive OLM, which allows the learner to provide evidence about their knowledge directly to the OLM, is in line with a philosophy that encourages the learner to take control over and responsibility for their learning.

There are many issues that we have touched on and which are important for future roles for OLMs to support metacognition. One of these relates to capturing, recording or extracting metacognitive aspects of students’ learning processes. The log of student actions and interactions with their OLM could provide a key source of evidence about metacognition. This suggests a role for OLMs that show these inferred models of metacognition. This may help learners become more aware of their own metacognitive processes. This leads to the issue of evaluating the effect of metacognitive feedback and interventions and poses a rather interesting new interface challenge for OLMs since it seems likely that a learner (and their human supporters, such as parents and teachers) may need new forms of interface that make it easy to see changes in the learner model in terms of such interventions. Designing tasks for metacognitive assessment is a potential role for OLMs. For example, a student can be asked to rate their own expertise and then be provided with the system’s corresponding assessment in the OLM.

Another key aspect relates to interpreting and assessing metacognitive behaviour, which is ­precisely what a teacher or facilitator does when discussing an OLM with students. There is potential for exploring support for explicit recording of these processes so that they can be revisited as part of long-term reflection on progress. The automation of this process is becoming increasingly feasible by exploiting educational data mining techniques (EDM).

Another important potential use for OLMs could be supported by better understanding how to design tasks for metacognitive assessment. This is completely congruous with OLM since the learner’s interaction with their OLM is often just such a task. Although there has not been much work on the explicit use of OLMs for displaying the parts of the learner model that represent metacognitive skills, this seems a promising direction to explore. It should lead to more generic OLM interfaces that might be available as an additional layer of support for reflection, beyond the domain-specific aspects that each demand different interfaces. We can even envisage that learners may expect every tool to provide them with such a metacognitive OLM interface or that data for it is stored in a way that enables the learner to explore it independently. We can envisage that this will support new OLMs and associated techniques for measuring and displaying metacognition over time or in changing contexts. Such generic tools create new possibilities for assessing metacognition in educational technologies compared to the classroom or the lab.

There is considerable potential for exploiting research on metacognition to inform work on OLMs as well as in the improved understanding of the ways that OLMs can support metacognitive processes and help develop metacognitive skills. We have distinguished two contexts for OLMs. When they are within an ITS, there is potential for careful design of the ITS and OLM, in terms of the interface and the underlying learning experiences so that there are immediate links between learning activities and the OLM. We have much to learn about the best ways to do this and how it may interact with many aspects, such as trust, gaming, exploration and toying with the ITS. We have also indicated some of the different possibilities and issues for a learner model that exists outside a particular ITS and the ways that its OLM interfaces might support and encourage metacognitive activities. In both of these roles, OLMs can serve several purposes, most being strongly linked to metacognitive activities of reflection, monitoring progress, planning both in the short and long term and aiding the learner in taking responsibility and control of their own learning and progress.