Keywords

1 Introduction

In this chapter we present an extended version of a state-of-the-art review on recommender systems (RS) in the field of education and more specifically of Technology Enhanced Learning (TEL). The chapter is based on a previous study by Manouselis et al. in 2011 [66] in the first Recommender System Handbook, and a Springerbriefs book from 2012 by Manouslis et al. [67].

The initial version from 2011 was limited to 20 recommender systems and got extended by the 2012 publication to 42 systems. The report from 2012 did not only extend the previous review, it also introduced a classification framework that provides a detailed overview over research activities on TEL recommender systems (RecSys). The 2012 publication acts like a map that shows what recommender system approaches have been studied in the TEL field and summarises the main findings. It is also a kind of manual that can inform researchers about most prominent approaches chosen so far and highlights neglected areas of research that could be taken up by the research community. It tries to standardise the research on TEL RecSys by introducing reference datasets, evaluation methods and procedures, and finally outlines current challenges in the field.

The previous studies are highly cited and had a significant impact on the TEL RecSys field. Since their publication, the community has much more developed into a sustainable and coherent research field. Research results became more transparent and comparable through the use of educational datasets from Educational Resource portals such as OpenScout (http://learn.openscout.net/) or MACE (http://portal.mace-project.eu) that act as reference datasets like Movielens or Netflix [110]. The research community around TEL RecSys is continuously growing as an increasing amount of research projects, conferences, workshops, special issues in journals and books shows. Examples include the Workshop series of Social Information Retrieval for Technology Enhanced Learning (SIRTEL 2007–2009), the RecSysTEL Workshop series on Recommender Systems for Technology Enhanced Learning [65, 68], the dataTEL workshop series on datasets for Technology Enhanced Learning [25, 26], a specific track on Recommender Systems for Learning (ReSyL) at the 14th IEEE International Conference on Advanced Learning Technologies (ICALT 2014) [27], the data competitions from 2013 until 2014 of the LinkedUp project [19, 21], as well as several special volumes of journals and books [86, 87, 104, 109, 113]. The diversity of the events over the years shows how relevant the research topics and challenges are for the TEL community. Figure 12.1 shows a world map where we indicated the countries that contribute research results to this meta-study. It can be seen that research on TEL RecSys is of global interest.

Fig. 12.1
figure 1

The world map of TEL RecSys research. It highlights countries that contributed research considered for this meta-review study

Full size image

With the current chapter, we aim to go beyond the previous results by updating the classification framework as well as significantly increasing the amount of recommender systems that have been analysed in the state-of-the-art review. The current review almost doubles the number of systems analysed in the previous study (2012) and includes 82 recommender systems from 35 countries (see Fig. 12.1). Due to the growths of publications in the field, we needed to be more restrictive with the selection of suitable research papers that are added to the review. We therefore mainly considered new publications that are based on empirical data rather than conceptual drafts. We hope to provide a comprehensive overview about the TEL RecSys field, further standardise the research and development, outline new challenges, and increase the common knowledge about the most effective ways to apply recommender system technology in the educational domain.

Finally, we want to emphasis that all the bibliography covered by this chapter is available in an open group created at the Mendeley research platform and will continue to be enriched with additional references (http://bit.ly/recsystel). We would like to invite the reader to sign up for this group and to connect to the community of RecSysTEL researchers. Among gaining access to the collected bibliography, we are looking forward to colleagues that contribute new research articles and findings within this very fast developing research field.

The chapter is structured as follows. First, an overview of the TEL research field is presented. Next, the framework model used to classify the reviewed recommender systems is outlined. After that, the results of the meta review are described, presenting seven clusters in which the TEL RecSys have been grouped. Finally, some conclusions and future challenges are discussed.

2 Technology Enhanced Learning (TEL)

Technology Enhanced Learning (TEL) aims to design, develop and evaluate socio-technical innovations for various kinds of learning and education. This involves individual learners but also groups and organisational knowledge management processes. It is therefore an application domain that generally covers technologies that support all forms of teaching and learning activities. The research in this field is very heterogeneous as proven by Kalz and Specht [51] in their study on 3476 research articles collected from the web of science between 2002–2011. TEL research is widespread from web-based information systems over mobile and wearable computing [120] to large scale physical simulators that are used in medicine, military or public transport education [22, 119].

Within this diverse research area, research on personalisation technologies is a strong topic with a large amount of national and international funded research grants. Personalisation of learning gets even more important with the increasing use of digital learning environments like learning object repositories, learning management systems, personal learning environments, and devices for mobile learning scenarios that take into account the learners’ needs [8].

The uptake of personalised learning approaches and especially recommender systems nowadays is reasonable due to the high demand on interpreting data that is stored in educational institutions. In fact, we have never been so close to investigate the phenomena of learning as in the days of “big data”. Almost all digital behavior of learners is stored and saved on servers of educational institutes. Not so long ago, collecting data was limited in terms of cost, time requirements, scope, and authenticity of the data, as this was typically done using single groups or classes for an experiment. The digital way of learning has made data collection an inherent process of delivering educational content to the students. That means that the analysis of learning behavior is no longer only related to representative pilot studies rather than to the usage of the entire student population. This trend has even become faster with the appearance of Massive Open Online Courses (MOOCs) [72] and the emerging of the Learning Analytics field [41]. MOOCs provide massive amounts of student data and therefore provide new opportunities for recommender systems to offer personalised learning support. Learning Analytics is currently the research field within TEL that focuses on understanding and supporting learners based on their data.

As a consequence, recommender systems have become extremely interesting for TEL research. These efforts resulted in a number of interesting observations as described in [67]: (1) There is a significant increase of recommender systems applied in TEL due to the digitalisation of learning and the growths of educational data; (2) The information retrieval goals that TEL recommenders try to achieve are sometimes different to the ones identified in other systems (e.g. product recommenders). For instance, many TEL RecSys try to suggest most suitable learning activities to learners by taking into account their knowledge level. This level is measured by prior- or self-assessment methods and taken into account to build personalised sequences through the learning content or activities; (3) There is a need to standardise the evaluation of TEL recommenders as the effects of the systems on the learners are in the focus of the research—rather than the most accurate algorithm; and (4) TEL RecSys research tries to evaluate its impact on educational stakeholders ultimately in user studies, rather than in data-driven studies. The evaluation criteria therefore go beyond traditional recommender system criteria such as precision, recall, or F1 measures and include specific learning related evaluation criteria such as effectiveness and efficiency of the learning process.

3 Classification Framework for TEL RecSys Review

Several classifications and categories have been used in the past to provide an overview of recommender systems. Hanani et al. [43] provide a general framework for information filtering systems, whereas Schafer et al. [94] and Wei et al. [118] clustered recommender systems in the e-commerce domain by distinguishing information used for recommendations, the types of recommendations, and various techniques. Burke [12] focused especially on the recommendation techniques and listed especially new approaches to the dominating content and collaborative filtering approaches at that time. Adomavicius and Tuzhilin [2] followed up on this technology study and reviewed various systems that they clustered into content-based, collaborative, and hybrid ones. They provided a detailed summary of the different technologies applied by the investigated recommender systems.

There are also publications that provide suitable criteria to categorise and order recommender systems (e.g. [42, 44, 75]). Manouselis and Costopoulou [62] combined all these evaluation criteria in a comprehensive classification framework with three main categories: (1) Supported Tasks, (2) Approach, and (3) Operation. The authors used this framework to analyse and classify 37 multi-criteria recommender systems. This framework was adjusted in 2012 to TEL by adding specific Supported Tasks like Find peer learners and Predict learning performance [67]. In this chapter, we have used the adjusted version for the following review of the 82 TEL RecSys. A detailed description of the framework and its categories is not available in the chapter due to page limitations. The interested reader can find a summary of the current version of the classification framework under the following URL: https://sites.google.com/site/recsystel/. The additional items (support tasks, methods) that have been added to the original version of the framework [67] have been emphasised in Fig. 12.2.

Fig. 12.2
figure 2

Classification framework for TEL RecSys based on [67]

Full size image

4 Survey Results

4.1 Method and Overview of TEL RecSys

The review of recommender systems presented in Table 12.1 compiles a total of 82 systems. These systems have been identified in previous compilations of educational recommender systems [66, 67, 69, 86, 87, 89, 91, 111], and have been extended with works shared in the Mendeley group and complemented with a keyword search in Google Scholar. This review covers 15 years of research on educational recommender systems from 2000 until 2014. The extensive compilation of TEL recommenders offers new insights and trends for the evolution of the research field.

Table 12.1 Overview clusters
Full size table

Based on the current state-of-the-art review we have identified seven clusters that group TEL recommenders systems in terms of relevant contributions to the field. Within each cluster, papers are reported in chronological order aimed to represent the research evolution. The clusters identified are the following:

  1. 1.

    TEL RecSys following collaborative filtering approaches as in other domains

  2. 2.

    TEL RecSys that propose improvements to collaborative filtering approaches to take into account the particularities of the TEL domain

  3. 3.

    TEL RecSys that consider explicitly educational constraints as a source of information for the recommendation process

  4. 4.

    TEL RecSys that explore other alternatives to collaborative filtering approaches

  5. 5.

    TEL RecSys that consider contextual information within TEL scenarios to improve the recommendation process

  6. 6.

    TEL RecSys that assess the educational impact of the recommendations delivered

  7. 7.

    TEL RecSys that focus on recommending courses (instead of resources within them)

The systems grouped into the mentioned clusters produce recommendations for learners that either contribute additional learning resources, guide their learning process or suggest courses to take. However, recommender systems can also support teachers to improve their courses or monitor their learning resources [9, 32, 37, 38, 59, 96].

Papers included in Table 12.1 have been given an ID in the form of RS+ID+YEAR [RSID-YEAR] to facilitate its follow-up in the remainder of the chapter, since many of the systems analysed have not been named by the authors with a specific acronym.

4.1.1 Cluster 1: Recommending Resources for Learning Based on Collaborative Filtering

This first cluster contains seven papers that report the application of collaborative filtering techniques as used in other domains, such as e-commerce, to produce recommendations in TEL scenarios. CoFind [RS1-2000] guides learners to relevant resources that have been previously found as valuable by other learners. The system uses collaborative filtering in combination with folksonomies data [28]. Altered Vista [RS3-2003] considers user evaluations of learning resources and propagates them to users with similar tastes in the form of word-of-mouth recommendations about the qualities of the resources [82]. RecoSearch [RS5-2004] proposes a collaborative filtering infrastructure for authoring, searching, recommending and presenting learning objects to learners [34]. RACOFI [RS7-2005] uses a collaborative filtering engine that works with ratings that users provide for learning resources complemented with an inference rule engine that mines association rules between learning resources [57]. In QSIA [RS8-2005] traditional collaborative filtering is extended with a control mechanism to mark users who should be considered for recommendations [81]. In CYCLADES [RS9-2005] users search, access and rate learning resources available in repositories found through the Open Archives Initiative [4]. The last paper included in this cluster [RS10-2005] proposes a hybrid recommendation service on research papers rated by learners consisting in a clustering module (using data clustering techniques to group learners with similar interests) and a collaborative filtering module (using classic collaborative filtering techniques to identify learners with similar interests in each cluster) [102]. This last work served to span the research to improve collaborative filtering approaches, as compiled in cluster 2.

4.1.2 Cluster 2: Improving Collaborative Filtering Algorithms with TEL Domain Particularities

This cluster compiles 13 papers. A considerable amount of researchers have focused on multi-attribute criteria of educational resources in order to cover the complexity of the learning (prior-knowledge, expertise, available study time, etc.) when using collaborative filtering techniques. For instance, in [RS11-2006] resources have been described using SCORM learning resource specification [106]. In [RS78-2007] multi-dimensional ratings provided by the users on learning resources have been considered [63]. [RS29-2010] investigated multi-criteria ratings with data from MERLOT learning object repository [99]. [RS47-2011] considered the relationship (advanced learner, beginner learner) as the third dimension over the typical user x item in collaborative filtering [114]. [RS63-2013] used the learner tree to take into account explicit multi-attribute of resources, time-variant multi-preference of learner and learners’ rating matrix for implicit and explicit attribute based collaborative filtering [84]. In [RS71-2014] multi-dimensional ratings on learning objects are considered to correlate one user with another [103].

Other approaches to improve collaborative filtering algorithms have also been proposed. In particular, [RS14-2008] proposes a collaborative recommendation system with query extraction mechanisms [61]. [RS18-2009] stores the ratings made by similar students in the profile together with the learning goal at that time in order to take into account the learner’s evolution in time [40]. [RS30-2010] extends a collaborative filtering mechanism with the learners competencies [15]. The RSF system [RS49-2012] presents a collaborative filtering algorithm combined with an embedded web crawler to update learning material [35]. The DELPHOS system [RS64-2013] includes a weighted hybrid recommender (collaborative, content and demographic) that uses different filtering criteria to encode the relative importance of each particular filter. The weights of the filters can be assigned by the user him/herself or automatically calculated by the system [124]. [RS72-2014] shows that a graph-based collaborative filtering algorithm can improve accuracy of generated recommendations even when the user actions data is sparse and provide a balanced distribution of users degree centrality [31]. In [RS73-2014] sentiment analysis techniques on user-generated comments of a repository of educational resources are used to obtain valuable qualitative information for adjusting the perceived rating of a given resource by a specific user [52].

4.1.3 Cluster 3: Educational Constraints as Source of Information for the Recommendation Process

The 16 papers in this cluster consider the educational knowledge as information source for the recommendation process in order to produce recommendations that better address the educational goals in TEL scenarios. They require an explicit description of this knowledge in terms of rules, ontologies, concept maps, semantic relations, etc. They can overcome the lack of large datasets needed by collaborative filtering approaches, but in turn may require maintenance efforts to keep the user and domain preferences updated, unless semantic techniques and related approaches are used.

In this line, [RS6-2004] recommends learning objects based on sequencing rules that help users to be guided through the concepts of an ontology of topics [98]. In [RS19-2009] educational standards such as PAPI and IEEE LOM were used within an ontology framework to manage learners properties based on learning styles and reputation metadata [53]. Ontology-based multi-actor learning flows and competence driven user models as described in [RS31-2010] can provide advice on tasks and resources [70]. Ontologies have also been used in [RS55-2012] to recommend resources that match the identified knowledge gaps form the learners [7] and to support creativity such as in [RS54-2012], where a recommender system suggests creativity techniques to the users [100]. Networks of ontologies such as [RS53-2012] that conceptualise different domains and their characteristics to provide semantic recommendations have also been proposed [20].

Another approach to recommend learning resources based on knowledge gaps is CLICK [RS56-2012] that suggests resources to learners by comparing automatically generated domain and learner models from distributed learning repositories [79]. Conceptual relationships have been used in [RS33-2010] to semantically rank lecture slides and the search needs for the users [115]. Conceptual maps have also been built in the METIS system [RS51-2012] to recommend learning activities in the maths domain based on prior knowledge, skills, and abilities of the learners [107]. MetaMender [RS52-2012] supports the description of meta-rules written by domain experts to personalise the information to the learner [123]. In this sense, [RS50-2012] takes the needs and preferences of learners into account to suggest suitable learning resources from distributed learning repositories based on a rule approach [14].

Some issues that deal with the learner situation have also been addressed by several papers. [RS32-2010] considers the limited time available for learning when proposing a utility-based recommender based on concept knowledge modelling [73]. As discussed in [RS57-2012] TEL RecSys can also be used to enhance meta-cognition and make learners aware of the processes of their learning [125]. In this sense, [RS58-2012] recommends widgets for learning activities in the context of personal learning environments for self-regulated learning [78]. In ALEF [RS75-2014] information stored and maintained in the corresponding user and domain models can provide learners recommendations on how to achieve more successful collaboration [6]. Finally, Semantic Affective Educational Recommender Systems (SAERS) [RS74-2014] can provide appropriate emotional support with affective educational-oriented recommendations elicited with TORMES (i.e., Tutor-Oriented Recommendations Modelling for Educational Systems) user centered design methodology in order to recommend the learning activity to carry out [93].

4.1.4 Cluster 4: Exploring Non Collaborative Filtering Techniques to Find Successful Educational Recommendations

Specific solutions to produce recommendations for the TEL context have also been explored in the following 14 papers. An initial idea, suggested in [RS2-2002], was to consider data mining techniques (such as association rules mining) in order to build a model that represents learner behaviours, and use this model to suggest activities or shortcuts that can help learners better navigate the digital materials [122]. In this line, in RPL [RS21-2009] web mining techniques were considered together with a scalable search engine to compute recommendations against a repository of educational resources [54]. AHA! adaptive educational system was also extended with recommendations in [RS22-2009] using web usage mining together with hyperlink adaptation to learn learners browsing pathways for personalised link recommendation [83]. Additionally, in [RS66-2013] data mining techniques complemented with user centered design methods were used to identify recommendation opportunities in educational scenarios that promote active participation of learners and strengthen the sharing of learning experiences [88].

Other approaches such as [RS15-2008] have applied fuzzy logic and item response theory to recommending courseware with suitable difficulty levels for learners according to learners’ uncertain/fuzzy feedback responses [17]. In [RS60-2012] fuzzy knowledge extraction model is used to extract personalised recommendation knowledge by discovering effective learning paths from past learning experiences through an ant colony optimization model [116]. In [RS65-2013] MPRLS also uses fuzzy logic theory to construct an appropriate learning path based on the learners misconceptions to recommend most suitable materials [46]. Meta-rules derived from a Markov chain model have also been used in [RS20-2009] to calculate transition probabilities of possible learning resources in a sequenced course of study for discovering one or more recommended learning paths [48]. In [RS34-2010] social navigation techniques built upon traces of past user behavior and using the assembled collective wisdom have been used to guide users to the most useful information [11]. Peer-to-peer networks have also been used in [RS36-2010] for searching personalised and useful learning paths suggested by reliable (trusted) peers [13]. Semantic relatedness of open education resources metadata have been considered in [RS35-2010] [97]. [RS59-2012] apply factorisation techniques to generate accurate ratings and perform predictions to recommend most suitable items, as they take temporal effects into account and therefore accurately model and adjust to the increasing knowledge of learners [105]. A graph-based algorithm as defined in [RS76-2014] can be used to create recommendations from cross-platforms in order to make learners aware of relevant activities, resources and peers in self-directed learning scenarios [33]. Finally, geometrical description of the recommender space as in [RS77-2014] can lead to better recommendation and dynamics understanding [77].

4.1.5 Cluster 5: Consider Contextual Information in the Recommendation Process

As reported in a recent state-of-the-art review [111] contextual information can be of value to enrich the TEL recommendations process and there are many research opportunities in this direction, as the 13 papers clustered here show.

Some relevant approaches identified in the literature are the following. A2M [RS16-2008] proposed a hybrid approach to select the appropriate recommendation technique depending on the input received from the learning environment and filters the output by the course context and the user features to produce an ordered list of recommendations to be presented to the learner [85]. CoMoLe [RS23-2009] recommends activities (multimedia contents as well as collaborative tools) to learners depending on different criteria (user features, context, etc.), and workspaces through a context-based adaptive mobile educational environment [71]. [RS42-2010] recommends documents to students according to their current activity that is tracked in terms of semantic annotations (with Contextualized Attention Metadata) associated to the accessed resources [10]. [RS38-2010] recommends resources at the workplace using a context driven recommender system to effectively support knowledge workers to meet their individual information needs [95]. In a similar scenario, [RS39-2010] produces contextual recommendations in a knowledge-sharing environment to the employees of large organisations [5]. [RS37-2010] adapts a version of Googles PageRank algorithm to context-aware recommendation in personal learning environments which incorporates different types of relations, including social relations and relations between resources, to standard collaborative filtering techniques [30]. [RS41-2014] considers quality information about learning resources [16].

In some other works, physical sensors are used to collect information from the environment with educational purposes [91]. For instance, [RS43-2010] uses semantic web to adaptively recommend learning content according to various types of context obtained from physical sensors [121]. In the same sense, [RS40-2010] uses a sensor module to collect data from learners and recommends educational resources according to predefined context structure [60]. RFID is used in [RS79-2011] to sense de location of learning resources in the actual environment [117]. SCROLL [RS80-2013] collects context information with the sensors available in smartphones, as well as from the device features and actions done on it [58]. In the BISPA system [RS81-2013], physiological measures aimed to detect learners’ affective state are gathered [50]. Finally, AICARP [RS82-2014] proposes an interactive recommendation that is delivered through two complementary sensorial actuators taking as input physiological and environmental information [92].

4.1.6 Cluster 6: Assessing the Educational Impact of Recommendations in Educational Scenarios

Throughout more recent development cycles, it has been demanded that TEL recommender systems should be evaluated not only according to technical criteria, but rather by a combination of technical and educational criteria (see a review of 59 papers in [89]). Here, 12 papers compile works in this direction. [RS12-2007] analysed implicit feedback for navigational support in lifelong learning based on self-organisation principles to see the effect on effectiveness (completion rates and amount of progress) and efficiency (time taken to complete) in lifelong learning [49]. [RS68-2013] showed that recommendations can support learners to enhance their effort towards an ascending learning curve and better grades [112]. Additionally, in [RS69-2013] learning effectiveness, learning efficiency, course engagement and knowledge acquisition were measured to evaluate recommendations impact in a MOOC [89]. The study on learners perception as reported in [RS61-2012] suggests that recommenders can significantly enhance virtual learning communities and put the power of determining what constitutes a quality contribution in the hands of the community members [56].

[RS26-2009] evaluated the applicability of recommendations in mash-up environments that combine sources of users from different Web2.0 services [23]. In that context, [RS44-2010] discuss the applicability of recommendations for empowering learners to set up their personal learning environments so that they can connect to networks of learners and collaborate on shared artifacts by using the tools available [74]. Related to this, [RS45-2010] identified the advantages of using a discussion forum within an e-learning system to foster communication between learners [1] and MASSAYO [RS62-2012] suggested that recommendations on blogs contents can support dynamic interactions in the learning environment by improving the discussion as they provide contributions from students with different points of view [45]. In [RS67-2013] students who learned with articles recommended by a mobile learning system based on their preferences and reading proficiency levels achieved significantly better reading comprehension in comparison with the students who read non-adaptive reading materials [47].

Evaluations with users are also useful to compare the best approaches for the recommendations process. [RS24-2009] compared various cost intensive ontology based recommendation strategies with light-weight collaborative filtering strategies regarding their impact on the learning outcomes of the learners in informal learning networks [76]. [RS25-2009] report an experiment with real learners using an hybrid approach for recommending learning resources that combines social-based (using data from other learners) with information-based (using metadata from learner profiles and learning activities) that shows a positive significant effect on efficiency (time taken to complete the learning objects) of the learners after a runtime of 4 months [24]. In LMRF [RS48-2011] learner performance increased when the students use a recommender system based on content-based filtering and good learners ratings, compared to both collaborative and content-based filtering techniques [39].

4.1.7 Cluster 7: Recommending Courses

The previous clusters have focused on recommendations that can be provided within a course. However, some research works on TEL recommenders have addressed the problem of recommending appropriate courses to students by taking into account curricula information. The amount of papers that focus on course recommendations is less compared to papers that focus on recommendation tasks within a course or an online environment. Thus, course recommender systems are rather specific and mainly driven by universities that want to support the starting students. Nevertheless, the research on this area has progressed over the years. [RS4-2003] proposed course suggestions for students when they have trouble in choosing courses [18]. A few years later, a course recommender [RS13-2007] was developed for University College Dublin students for their online enrollment application [80]. This was followed up by the famous CourseRank system [RS27-2009] for Stanford University students with more than 70 % of students using the system [55] and another one [RS46-2010] at the University of Pittsburgh, which was evaluated based on a long-term evaluation experiment with students [36]. [RS17-2008] takes into account behavioral patterns to recommends potential courses for learners [101] and [RS28-2009] computes success probabilities of the student if enrolled in a certain course [108]. [RS70-2013] shows the integration of a course recommender in a Moodle instance [3].

4.2 Analysis According to the Framework

In the following section we cluster the 82 reviewed TEL RecSys according to the classification framework depicted in Fig. 12.2. We therefore start with the analysis of the Supported Tasks illustrated in Table 12.2, afterwards clustered all systems according to their Approach, in particular, the User Model (Table 12.3), Domain Model (Table 12.4), and Personalisation characteristics (Table 12.5), and finally Operation (Table 12.6). It needs to be mentioned that we could not cluster all systems into all categories exclusively and always end up with a total sum of 82 systems. This has mainly to do with the information that is provided in the papers and is sometimes incomplete. In other cases, the systems fit into several categories (e.g., provide a couple of supported tasks).

Table 12.2 Classification of TEL recommenders, according to the Supported Tasks
Full size table
Table 12.3 Classification according to the User Model of the Approach category
Full size table
Table 12.4 Classification of TEL recommenders, according to the Domain Model
Full size table
Table 12.5 Classification according to Personalisation characteristics
Full size table

From Table 12.2, the following issues can be identified regarding the Supported Tasks that TEL RecSys deal with:

  • There is a vast majority of TEL RecSys that aim to support the task of Finding good Items (content) to support learning activities. In total 61 systems (n=61) aim to support learners by providing new learning content to their current learning process.

  • The second most used recommendation tasks is recommend a sequence of items to learners (n=13). Recommend a sequence of items is a very important task within TEL RecSys because it is similar to instructional design methods. The aim of an instructional design is to guide a learner through a series of learning activities to achieve a certain competence. This didactical objective can be supported in recommender systems by suggesting the most efficient or effective paths through a plethora of learning resources to achieve a certain competence. Recommender systems with this task often considering the prior knowledge of a learner for their recommendations.

  • The Recommendation of peer learners is also a very central recommendation task for distance education settings and relatively often applied in TEL RecSys research (n=9). Online learners often feel isolated after a period of time without any physical meeting. Thus, courses with pure online presence tend to have higher dropout rates compared to normal courses or blended learning scenarios. To overcome this situation, recommender systems can be supportive by recommending peer-learners that the target learner can team up within an online course.

  • Interesting is that the above mentioned recommendation tasks are applied over all years in research. So there is not one specific recommendations tasks researchers have been focus on in a specific timeframe. In the more recent years some new recommendation tasks have appeared, such as Predict learning performance (n=1) and Suggest a learning activity (n=4) in contrast to just learning content. These developments show that recommender systems are increasingly applied to filter and personalise information in digital learning environments and are also applied for new educational goals.

From the analysis of the User Models that are illustrated in Table 12.3, the following aspects can be identified:

  • Regarding the Representation method, most TEL RecSys identified use classic Vector-space models with multiple attributes (n = 29) to represent the desired features or the user preferences. In addition, many systems rely on Ontologies (n = 18) that capture various attributes of users and relationships between those attributes. The ontology-based systems are closely followed by User-item ratings models (n = 13) that capture explicit ratings of users on items. History-based and Demographic features approaches have been applied less often (n = 5 and n = 2, respectively). Although there are few Associative networks approaches listed in the review (n = 3), we believe this approach will become more prominent through the increasing research on the Educational Data Mining field.

  • Regarding the Representation type, most are based on clear Measurable items (n = 17). A distinction needs to be made in this category between implicit and explicit ratings. Some systems apply explicit ratings like star ratings and tags given by the users to the content whereas other systems use implicit ratings extracted from the behaviour of the users such as user accessed a file, time spend on a resource, etc. Both types of ratings are together the most common types in TEL RecSys. Ordinal/Feature and Probabilistic approaches are not applied that often (n=4 and n=3, respectively).

  • With regards to the Generation, the initial user preferences engaged by the examined systems are usually acquired in a Manual way from the users (n=24). In many cases, the user model is initially Empty (n = 14), and then slowly created throughout the users interactions with the system. Stereotyping was also used in some cases (n=3). For learning, there is a trend in the recent years to apply more and more Clustering (n=10) or Classification (n=15) approaches for learning the initial user model from existing data.

Analysing the collected systems with respect to the Domain Model characteristics (Table 12.4), the following aspects can be identified:

  • Regarding Representation, there is not one major approach for the domain model for TEL RecSys to recommend items, but three almost equally applied approaches. The most often used approach is: Ontology (n = 23) followed by Vector-space (n = 18) approaches and finally Index/List (n = 16). Only a few systems engage a Taxonomy (n = 3), Graph (n = 1) or a Rule-based (n = 1) approaches. Interestingly, many of the first recommender systems for learning rely on Index/List or Ontologies representations of domain models and this approach seem to be kind of stable over all development cycles until today. The Vector-space approach is a more recent development starting in 2008.

  • Regarding Generation, most of the domain models are created in a manual way (n = 26). However, an increasing amount of systems in the recent years use automated metadata generation with classification (n = 17), clustering (n = 8) and sequential analysis (n = 1) methods.

Table 12.5 presents the analysis of the TEL RecSys based on the Personalisation aspect. As the extended review shows a broad variety of Personalisation approaches and different kinds of algorithms have been explored in the 15 years of research in the field.

  • In terms of Methods used for the personalisation of recommendations, Rule-based (n=22) and Collaborative filtering (n = 21) are the most applied techniques in the TEL field. It is followed by Hybrid (n = 13), Content-based techniques (n = 10), Graph-based (n=4) and Knowledge-based (n = 3). Other approaches explored (with n = 1) are Association mining, Raw retrieval and Manually selected. Interestingly, some techniques are time independent and are applied over all development cycles in TEL field. Examples for this are Collaborative Filtering (2000–2014), rule-based (2004–2014), whereas other methods are belonging to more recent development cycles such as Hybrid (2009–2014) and Content-based (2008–2014) techniques. There is an increasing interest in Graph-based (2010–2014) and Knowledge-based approaches (2013–2014).

  • The Algorithm type used in TEL recommenders are as diverse as the personalisation techniques. Although, Model-based are dominating (n = 24), there have been plenty of research on Memory-based systems (n=16), and Hybrid (n = 13).

  • As far as the engaged Algorithm techniques, Attribute-based is the most common (n = 17), followed by Hybrid (n = 13), and User-to-user (n = 10). Few item-to-item correlation approaches have been proposed in TEL RecSys (n = 4) as well as Vector space model (n = 2). User-to-user filtering seems the most often techniques over the whole period (2003–2014). Hybrid techniques started to become more relevant from 2009 until theses days, and Attribute-based systems significantly increased in the years 2013 and 2014.

  • Regarding the Output, a very clear picture is obtained. The produced output is most of the times a Suggestion (n = 54). However, there are also quite a few systems that predict the evaluation that a user would give to the suggested items in the form of Prediction (n = 12).

Concerning the Operation category of the dimensions, Table 12.6 indicates the following:

  • The Architecture of the majority of TEL RecSys is Centralised (n = 60), providing access to a single recommendation repository. Nevertheless, there are a few systems that rely on distributed architectures that provide access to a wide range of repositories (n = 11).

  • Regarding the Location, recommendations are usually produced at the recommendation server (n = 65). Only a few systems produce them at the information source (n = 5). Recent research on recommender systems is increasingly oriented to produce recommendations on the user side—i.e. for use on mobile devices in situated learning activities. Ongoing work in this area has been described in [111].

  • Until now, TEL RecSys Mode either provide their recommendations at an active Pull mode (n = 20) where users request relevant recommendations or in the more often used Passive mode where users receive recommendations as part of their natural interaction with the system (n=46).

Table 12.6 Classification of TEL recommenders, according to the Domain Model of the Approach category
Full size table

5 Conclusions

This chapter has extended the state-of-the-art reviews of TEL recommenders 2012 by doubling the amount of systems considered. In particular, the current chapter has reviewed 82 TEL RecSys along the 15 years of this specific research field (2000–2014). Research works have come from 35 different countries. The systems compiled and analysed have been classified into 7 exclusive clusters, namely (1) TEL RecSys following collaborative filtering approaches as in other domains; (2) TEL RecSys that propose improvements to collaborative filtering approaches to take into account the particularities of the TEL domain; (3) TEL RecSys that consider explicitly educational constraints as a source of information for the recommendation process; (4) TEL RecSys that explore other alternatives to collaborative filtering approaches; (5) TEL RecSys that consider contextual information within TEL scenarios to improve the recommendation process; (6) TEL RecSys that assess the educational impact of the recommendations delivered; and (7) TEL RecSys that focus on recommending courses (instead of resources within them). The framework proposed in [67] for the analysis of recommender systems has been applied with some extensions. The applied framework has been very valuable to analyse available TEL RecSys from a holistic perspective. However, in some cases it was not easy to extract relevant information from the content reported in the papers and to map those back to the framework categories.

After the state-of-the-art analysis of the field carried out in this chapter, we have perceived that the field is moving and new research approaches are emerging. For instance, initial TEL RecSys used very small and mostly internal datasets, whereas more recent studies apply larger reference datasets before they implement the systems in a real world scenario. Furthermore, the research community tries to make datasets available to other researchers and use additional reference datasets that are publicly available to make the results of their studies more comparable.

In the following sections a trend analysis in TEL RecSys for the last 15 years of research are summarised according to the framework categories.

  • Supported Tasks. Finding good Items (content) is the most applied task for recommender systems in the TEL field. But Recommendation of sequence of items that aims to create an effective and efficient learning path through digital contents is also an important task for the TEL community. Along this mainly content driven recommendations, the recommendation of other learners, so-called peers, that follow similar learning goals or have the same interest as a target learner are very central tasks. There are some new tasks appearing in the recent years, which go beyond recommending learning content, such as Predict learning performance and Recommend learning activity.

  • User Model. There is no clear trend identifiable regarding the user models in TEL RecSys. But there seem to be more research efforts going towards clustering and classification approaches. That is another indicator that the field increasingly adapts ideas and techniques from the educational data mining and learning analytics research communities. In this respect, the interested reader can consult the chapter on Data Mining Methods for Recommender Systems (Chap. 7).

  • Domain Model. Similar to the user model category, there is not one major approach for modeling the domain within TEL RecSys. The initial systems in the field almost always applied Index/Lists and Ontologies what is reasonable as TEL RecSys research was mainly driven by two communities: (a) Information Retrieval, and (b) Adaptive Hypermedia. Index/Lists have been used by the information retrieval community within TEL, whereas Ontologies have been extensively used by the Semantic Web and Adaptive Hypermedia community from 1998 until 2010. Both approaches are still used today but we see some converging approaches as described in [21]. In turn, like in the User Model category, more and more classification and clustering approaches are applied for the Domain Model as well. This emphasises once again the growing usage of data mining techniques in the field.

  • Personalisation. Within the personalisation category we were able to identify some trends over time regarding the used methods. Examples for this are Hybrid and Content-based approaches that started to be reported in 2008 and are increasingly applied in recent years until today. There is an increasing interest in Graph-based (2010–2014) and Knowledge-based approaches (2013–2014). These technologies are mainly applied to address two more common issues within educational datasets: (a) Sparsity, and (b) Unstructured data. When rating data are sparse, users are likely to receive irrelevant recommendations. Therefore, graph-based approaches, which extend the baseline of nearest neighbours in collaborative filtering by invoking graph search algorithms, have been applied successfully in TEL RecSys [31]. Collaborative Filtering and Rule-based approaches are still the most frequently used techniques over all development cycles (2004–2014).

  • Operation. Regarding the output, most of the TEL RecSys aim to suggest their recommendations directly to the users in a passive mode. The architectures, therefore, are in most of the cases centralised systems and the recommendations are usually created on the side of the recommendation server. There are some federated search approaches mentioned in the recent papers and also recommendations of learning objects from Linked Data sources have become a relevant topic in 2013.

To conclude the chapter, we have reviewed the challenges reported in [67] in the light of the meta-review carried out in this chapter and extended those from the previous publication. These are:

  1. 1.

    Pedagogical needs and expectations to recommenders. Recommendation opportunities in educational scenarios that go beyond recommending learning resources need to be further explored. For this, user centered design approaches [88] can be of value, such as to consider recommending learning activities that, for instance, foster communication [1] and metacognition [78, 89, 125]. At the same time, the potential of semantic technologies is being considered to describe the educational domain and therefore enrich the recommendation process [45, 53, 90, 97].

  2. 2.

    Context-based recommender systems. As reported in a state-of-the-art review of contextual TEL recommenders [111] contextual information can be of value to enrich the recommendations process and there are many research opportunities in this direction. Context-based recommenders can extend the input and output information to be considered in the recommendations process with the usage of appropriate physical sensors [91], such as reported in [50, 58, 60, 117, 121]. In this sense, the application of affective computing in TEL RecSys can provide added value to the recommendations when emotional and sentiment information is taken into account in the recommendation process [52, 93] and can provide interactive recommendations through sensorial actuators [92]. Details about Context-Aware Recommender Systems can be read in the corresponding Chap. 6

  3. 3.

    Visualisation and explanation of recommendations. An important line of research in this area is the use of visualisation techniques to provide users with insights in the recommendation process. Visualisations can help to explain recommendation results by explicitly exposing relationships among content and people. El-Bishouty et al. [29], for instance, researched the use of visualisation techniques to present the relationship between recommended peer-learners. Visualisation techniques can increase understanding of in- and output for a recommender system. It therefore also contributes to a higher level of trust of the user into the system that mainly acts like a black box to them. In this sense, guidelines for the design of this complex relationships should be taken into account as compiled in the chapter Guidelines for Designing and Evaluating Explanations in Recommender Systems Chap. 10

  4. 4.

    Demands for more diverse educational datasets. In 2011 most TEL recommender studies have still used rather small datasets which were not made public available [64, 65]. Since then, the dataTEL Theme Team of the European network of excellence STELLAR [25] collected an initial set of datasets that can be used by the research community [110]. These days we see many more studies that take advantage of this initial collection of datasets to start their research [31]. But the dataTEL collection can only be a first start to a comprehensive collection of datasets for RecSysTEL research. As TEL is a very diverse research field that starts at school level, over Higher Education until workplace learning and also is differentiated into informal, non-formal and formal learning, a larger collection with more diverse datasets is needed.

  5. 5.

    Distributed datasets. Big data architectures (such as Lambda, http://lambda-architecture.net) and technologies (such as Apache Drill, http://incubator.apache.org/drill/) that allow large scale and real time analytics over distributed data, are expected to change the way that research is taking place over federations or aggregations of learning information. Applications developed on top of Linked Open Data such as the ones piloted by the LinkedUp project (http://linkedup-project.eu), are also bringing new requirements to the infrastructures needed to support such research scenarios. We see the need for educational research of e-infrastructure components and services that can host, distribute and virtualise such big data powered recommendation applications for learning also to overcome the sparsity of single data silos.

  6. 6.

    New evaluation methods that cover technical and educational criteria. Recommender systems can be analysed to measure the effect on effectiveness (completion rates and amount of progress) and efficiency (time taken to complete) in learning [49, 89], towards an ascending learning curve and better grades [112], including mash-up environments that combine sources of users from different Web2.0 services [23] and mobile learning approaches [47]. For the RecSysTEL field it is important that upcoming developments on TEL RecSys should follow a standardised evaluation method as suggested in [67]. The method consists of four steps:

    1. a.

      A selection of datasets that suit the recommendation problem and tasks of the development.

    2. b.

      An offline comparison study of different algorithms on the selected datasets including well known datasets (if possible, educational oriented datasets in the same way that Movielens is to movie recommendations) to provide insights into the performance of the recommendation algorithms.

    3. c.

      A comprehensive user study in a controlled experimental environment to test psycho-educational effects on the side of the learners as well as on the technical aspects of the designed recommender system.

    4. d.

      A deployment of the recommender system in a real life application, where it can be tested under realistic and normal operational conditions with its actual users.

    The above four steps should come along with a complete description of the recommender system according to the classification framework presented in Sect. 12.3. A good example for this research approach is [32]. The used dataset should be reported and made publicly accessible. This would allow other researchers to repeat and adjust any part of the research to gain comparable results and new insights. A detailed description about how to run user studies with recommender systems is also available in Chap. 9

We hope the panorama of recommender systems to support learning that has been compiled in this chapter helps researchers, developers and users to get a clear view of the field.