Keywords

1 Introduction

Adaptive instructional systems (AISs) provide machine-based instruction through technologies like Intelligent Tutoring Systems (ITSs) which interact with learners and make decisions about interventions based on the needs and preferences of each individual learner [1]. These interventions are based on a model of that learner or team and the conditions in the instructional environment or application. A simple model of instruction includes instructional elements (Fig. 1) to be considered during the AIS authoring process [2].

Fig. 1.
figure 1

Elements of an adaptive instructional model

Full size image

In most ITSs, learning objectives are defined by the author and outline the terminal goals of the instruction. For example, a tutor that guides/adapts instruction for a marksmanship task might have a learning objective, “understand how to maintain a steady position during weapons fire”. In a tutoring architecture, like the Generalized Intelligent Framework for Tutoring (GIFT) [3, 4], the objectives are set as concepts to be learned. This is usually the level at which the tutor tracks individual or team learning. Learning events represent content offered to the learner and focuses on how and what the learner experiences. Conditions and measures set standards against which the tutor determines the learner’s progress toward learning objectives. Finally, the tutor will react to changing learner attributes (e.g., change in competency level) and environmental/application conditions (e.g., place in the instruction) by intervening with the learner. This intervention could take the form of feedback, interaction with the learner (e.g., ask a question, prompt the learner for more information or provide a hint), or changes to the environment (e.g., increase the difficulty level of the task).

This model usually includes critical information about the learner and the instructional domain that informs a machine learning algorithm in the tutor and that algorithm is trained by consuming data involving both successful and unsuccessful decisions. Decision success/failure is based on their effect on learning outcomes which include: knowledge and skill development, retention, performance, and transfer of skills from instructional to operational (work) environments.

A basic adaptive instructional model (Fig. 2) involves learner actions and conditions, environmental conditions, instructional policies, and interactions (actions, observations, and assessments) capturing data between the tutor, the environment (sometimes referred to as the application), and the learner.

Fig. 2.
figure 2

Interaction within an instructional model

Full size image

The instructional model and its policies can be improved over time through a class of machine learning techniques called reinforcement learning (RL) algorithms. RL algorithms are often used to improve the accuracy and reliability of adaptive instructional decisions. However, this method requires usually large amounts of data to develop optimally effective instructional policies that drive tutor strategies and tactics. To reduce the development time of instructional policies, we advocate a mechanism to collaboratively develop instructional models for a variety of task domains. Before we discuss community models, it is appropriate to review how reinforcement learning works in practice.

2 Reinforcement Learning for Learner and Instructional Models

Reinforcement learning, a type of machine learning, enables software agents to automatically determine an optimal action within a specific context in order to maximize its own performance over time [5]. Reward feedback is all that is required for a software agent to learn or reinforce the selection of optimal behavior(s). The process of selecting optimal behaviors over time and many examples is known as a Markov Decision Process (MDP) [6].

Conditional systems within AISs are used to determine tutor behavior or more specifically their decisions and interactions with the learner and the environment. These interactions are in the form of instructional strategies and tactics in systems like GIFT [3, 4]. AISs consider instructional strategies and tactics which are bounded by constraints posed by policies and these policies are often framed in terms of Markov Decision Processes (MDPs) [7] which seek to reinforce maximum outcomes or rewards over time [6, 8]. To select an optimal policy or modify it based on new information, the MDP considers the current state and the value of any actions which might transition to successor states with respect to a desired outcome.

According to Mitchell [9], the optimal action, a, for a given state, s, is any action that maximizes an immediate reward, r(s, a) and the value, V, of the immediate successor state, s’. What does this means for AISs? First, based on our model in Fig. 2, states are much more complex in AISs than in most systems. States must capture conditions of the learner (e.g., performance, affect, competence) and the instructional environment (e.g., concept under instruction, concept map (hierarchical relationships between learning objectives), and recent content presented) that affect the learning experience. Actions, referred to as tactics in GIFT, are the set of instructional options available in the current state.

The reward function is tied directly to learning outcomes (e.g., knowledge and skill development, performance, retention, and/or transfer from instructional to operational or work environments) and the value is the anticipated performance in the next state. It is easy to see that the number of possible states can be very large in AISs and that the task of validating MDPs for all possible states could take a single organization a very long time. Hence the need for a process to divide the validation process into smaller discrete elements that can be processed by researchers in parallel, but to a similar standard.

The idea would be to have learner models grow quickly based on the individual and team instruction received by a learner in a variety of instructional environments in the learning landscape (e.g., formal education, training, reading, job-related tasks). Instructional models which might form the basis of widespread policies and strategies or tactics in specific domains could be evaluated across a variety of populations with emphasis on what works best for novices, moderates, and experts in any given domain. In this way it would be possible to generalize instructional techniques that work broadly and label them as domain-independent policies or strategies.

3 An Approach to the Development of a Community Model

Based on our goal to reduce the time to validate a complex instructional model, let’s simplify our model for adaptive instruction by dividing it into its three essential elements: the learner model, the instructional environment, and the instructor or tutor. The learner model consists of the attitudes and behaviors along with the cognitive states of the learner. This model could also include long term information that highlights trends, habits, preferences, interests, values, and other data that could influence learning.

The instructional environment consists of learning objectives (LOs; also known as concepts), a concept map (a hierarchical relationship of concepts to be learned as shown in Fig. 3), a set of learning activities which include content and directions on how the learner will interact with the content, a set of measures to determine learning and performance, and a set of available tutor strategies and tactics to respond to various learner attitudes, behaviors and cognitive states.

Fig. 3.
figure 3

Mastery policy in a concept map within an instructional model

Full size image

Examining the concept map in Fig. 3, we see that it illustrates prerequisite relationships among nine concepts (A-I) with A-D showing the learner mastered those concepts. Concepts E, G, H, and I have not been attempted, but F has and the results show that the learner has not yet attained mastery of this concept. This type of map helps the tutor understand the relationship of concepts, what the learner knows and does not know, and provides context for machine learning algorithms to select appropriate instructional strategies.

The tutor also consists of a set policies that drive its behavior and interaction with the learner and the instructional environment. The goal is for the policies to be updated regularly as the tutor interacts with more and more learners and finds new highs to override previous best practices. According to Chi and Wylie [10], learner activities vary from least effective to most effective are: passive (receiving), active (manipulating), constructive (generating), and interactive (dialoguing). As activities are selected and presented to the user, the tutor uses measures to assess progress toward learning and performance. For example, in a tutor that instructs learners to read, the tutor might engage the learner in a reflective dialogue (interactive activity) about a recently read passage to ascertain the learner’s comprehension of the concepts presented in that specific reading.

To further our approach, we might consider generalizing terms and measures (Table 1) in lieu of using specific measures. Reducing the number of discrete states also reduces the matrix for selecting the best possible response by the tutor to existing conditions.

Table 1. Instructional model element descriptors
Full size table

4 Discussion

By coming to consensus on a common set of terms and defining their relationships in an ontology [11], we might realize the degree of interoperability needed to develop community-based models for AISs. However, we also realize that their complexity [12] and the lack of interoperability [13] between various AISs may slow the progress of developing these models. The good news is that current events have highlighted significant opportunities to capture and share the data needed to grow community learner and instructional models.

Recently, the Institute of Electrical and Electronics Engineers (IEEE) Learning Technologies Steering Committee (LTSC) approved a study group to examine opportunities for standards to promote interoperability and reuse with this class of technologies known as AISs [14]. The modularity of systems may play a large part in motivating organizations to share data and resulting models. If successful, the IEEE LTSC’s initiative will likely result in a high degree of sharing among AIS components, tools, methods, and data.

In 2017, under the auspices of the North Atlantic Treaty Organization’s Human Factors and Medicine Panel (NATO-HFM), a research task group examining technologies and opportunities to exploit Intelligent Tutoring Systems for adaptive instruction completed its task and recommended “the development of standard learner model attributes which include both domain-independent (e.g., demographics) and domain-dependent (e.g., domain competency, past performance and achievements) fields which are populated from a learner record store (LRS) or long-term learner model. This will promote standard methods to populate real-time models during ITS-based learning experiences and allow for common open learner modeling approaches and transfer of competency models from one tutor to another” [15]. If adopted, this recommendation may be an impetus in creating a large, diverse community from which data for community learner models could be harvested.

While not a recent phenomenon, the advent of the educational data mining repositories DataShop [16] and its successor, LearnSphere [17] provide mechanisms for contributing and consuming experimental data related to learners interacting with instructional systems like AISs. “LearnSphere integrates existing and new educational data and analysis repositories to offer the world’s largest learning analytics infrastructure with methods, linked data, and portal access to relevant resources” [18].

5 Next Steps

There are some significant barriers to developing community models to enhance adaptive instruction. The first challenge is not technical… The rights to data are complex. The second challenge is developing models of teams is much more complex than developing models of individuals [19].

The first challenge is attracting organizations to contribute regularly to a database of instruction for the benefit of the community. DataShop and LearnSphere have a long history of incentivizing the learning science community to contribute. They have overcome the barrier of who owns the individual data and the rights to how it might be used by stripping out personally identifiable information (PII) so that no data can be associated with a specific individual. These sanitized databases form a basis for sharing without violating any individual rights. LearnSphere also has selective access in which it allows the collecting organization to determine who is allowed rights to use the data.

However, some of the individual details we mentioned previously (behaviors, trends, habits, preferences, and values) that might be useful in tailoring instruction, may not be shared in experimental databases today, but are part of our internet DNA. Websites like Amazon and Google collect information about us regularly to tailor our internet experiences (e.g., shopping, entertainment) and recommendations. Dino Wilkinson, an international attorney at Norton Rose Fulbright reported that “under English law, there are no property rights in data as such – although this has not necessarily prevented individuals and businesses from treating data as property. Markets exist for buying or selling data and individuals regularly disclose their personal data in exchange for goods and services. However, the value in these cases is created through the right to sell or use the data in a certain way rather than a legal right of ownership” [20]. Ultimately, it could be individuals who own their data, manage access to it, and license it for use by others. Until then, scientists who collect data have to be sensitive to its use. This will slow, but not stop the progress to model individuals for use in adaptive instructional systems.

The next horizon for AISs is to be easily applied in both individual and team instructional domains. The second challenge, team modeling, has some twists and turns that make their development much more complex than the development of individual models. The first twist is common sense: team tasks are more complex to assess than individual tasks since they involve individual members engaged in interdependent roles and responsibilities in pursuit of a goal or a set of goals. This means a team of X members has X sources of data, and many sets of interactions which may be important to measure and assess during machine-based team tutoring.

Teams are involved in the process of teamwork and the learning and maintenance of team skills required by the team taskwork. Teamwork involves “coordination, cooperation, and communication among individuals to achieve a shared goal” [21]. Teamwork is largely a domain-independent process and includes the social skills (e.g., tact and trust) needed to function as an effective team. The interaction of teamwork on team taskwork is prevalent in the literature [22,23,24,25], and the antecedent attitudes, behaviors and cognition has been recently analyzed and defined in structural equation models derived from a major meta-analysis of the team and tutoring literature [26].

A large part of the complexity of modeling teams is embedded in the difficulty in acquiring and interpreting learner data and interaction data. Sensors, self-report mechanisms, external observations, and historical databases may be sources to inform team models, but they should be unobtrusive to avoid any negative impact on the learning process.

Collecting data is a challenge, but filtering that data to create information to inform tutor decisions is another challenge and a major contributor to the complexity of modeling teams. The development of accurate machine learning methods to select optimal tutor interventions to enhance team learning and performance is desirable and difficult. The sheer number of conditions present for individual learners on the team, the instructional environment, and the options available to the tutor are mind boggling, but perfect for machines.

As noted for individuals, team models, including models of instruction, may benefit from simple analysis of effectiveness where comparisons between models of teams, instruction, and domains are used to identify significant differences in learning, performance, retention, or transfer of learning from instruction to operations. Of course we need to build team tutors first [27], but eventually we need a playground to experiment and test. A testbed model used in GIFT [28] and based on Hanks et al. [29] has been used for evaluating adaptive instruction of individuals and might easily be extended to support team tutoring (Fig. 4).

Fig. 4.
figure 4

Testbed for evaluating the effect of adaptive instructional of teams

Full size image

In a modular system like GIFT, the team, instructional and domain models have common interfaces and message types that inform data shared between the models. This allows flexibility in swapping out or just changing the internal processes within these models to allow experimentation with different policies, strategies, and tactics in varying conditions represented in learner/team models and the environment or application (e.g., problem set, simulation or webpage). For example, an experimenter could examine instructional strategies for low performing teams in game-based simulation scenarios.

Since GIFT is also very data-centric, experimenters are permitted change out one set of parameters for another. For example, an experimenter could decide to examine instructional interventions for the common three-tier performance model in GIFT (at, above or below expectations) for a more granular performance model with five or more levels.

Given the complexity of examining the large number of conditions represented in the learner/team, instructional, and domain models with associated content, we believe it will be significantly faster and easier to share the analysis and development workload through community modeling schema.