Keywords

1 Introduction

Systems that retrieve and filter the data through content and similar profiles are known as recommendation systems (RS). These systems are usually used within the e-commerce domain. For example, some websites, such as Amazon, through the application of RS allow offering the user recommendations for products that users do not know and could be of their interest. Suggested recommendations help to overcome the distressing search problem for the user. But this technology is not only used to sell products, but it is also used to suggest videos (YouTube), movies (Netflix), friends (Facebook), among others.

This demand spans across several domains, among which is the educational domain. RS, which are applied in education, have the role of supporting teaching and learning activities through enhanced information retrieval. Nevertheless, there is limited information of the application of recommender systems in educational environments. Consequently, this study aims to summarize the current knowledge that is available concerning RS that have been employed to support educational practices.

This paper is structured as follows. Section 2 explains the research method and stages applied for the systematic mapping. Section 3 provides the results of the mapping study. And finally, Sect. 4 present our main conclusions.

2 Research Method

We have performed a systematic mapping study by considering the guidelines that are provided in the works of Kitchenham [1] and Petersen et al. [2] to obtain an overview of recommendation systems in education. Our systematic mapping study involved several stages and activities. In the planning stage, the need for the mapping is identified, the research questions are specified, the search strategy is established and the mapping protocol is defined. In the conducting stage, the primary studies are selected, the data extraction is performed, and the obtained data is analyzed and synthesized. Finally, in the reporting stage, the mapping study findings are presented. The details concerning the planning and the conducting of our systematic mapping are presented in the following subsections. The reporting stage is described in Sect. 3.

2.1 Research Questions

Our research aims to examine an overview of the areas in education that can be addressed by current use of recommendation systems. Thus, the following research questions are presented: “What are the educational areas covered by RS?” and “What are the approaches used to generate recommendations within educational scenarios?”. For further understanding, three research questions were added to support our primary goal. The idea behind the following research questions is that each one addresses different aspects of RS in education to identify gaps or issues with the current research. Table 1 shows all research questions of this mapping study along with their rationale.

Table 1. Research questions of the mapping study.
Full size table

2.2 Search Strategy

The conducted mapping study was based on an online search from the following digital repositories: Scopus, IEEE Xplore, ACM Digital Library and Web of Science. The mentioned repositories, which cover most of the specific conference proceedings and journals for our topic, are relevant to software engineering and computer science.

The first part of the search strategy was to define the search string used for query. In this part, the papers’ keywords, abstracts, and titles were searched, using the following terms with the combination of boolean operator “AND”, “OR”: (“recommender systems” AND “education”) OR (“recommendation systems” AND “education”). When using the search terms, synonyms and plurals were taken into account. e.g., “recommender” or “recommendation”; “system” or “systems”.

Table 2 shows the terms used in the different scientific databases with the results of the search, the asterisk “*” symbol was used in some databases as a wildcard to indicate any word variation on each term e.g. “recommend*” implies terms like “recommendation”, “recommender”.

Table 2. Research questions of the mapping study.
Full size table

2.3 Selection Criteria

In this process, we ensure that only relevant studies about RS in the context of education are included for further analysis. This was achieved by applying an inclusion-exclusion criterion on the mapping study to determine whether a paper should be included in the following steps.

Papers that met the following criteria were included: Papers supporting educational practices by using recommendation systems, full papers, academic journal and conference proceedings studies.

Papers that met at least one of the following criteria were excluded: Papers that are not focused on education, papers presenting only development and implementation of a recommender system, papers not written in english, duplicate reports of the same study in different sources, introductory papers for special issues, books, workshops and technical reports (grey literature).

2.4 Screening Process and Selection of Primary Studies

Once the search results were available, a pre-selection criterion was applied to these papers. Researchers read paper title, abstract and keywords to apply the inclusion criteria and consider that at least papers should mention the keywords presented in the search strategy. Then, the exclusion criterion was used during the full paper reading, generating what we call the primary studies list.

2.5 Data Extraction and Data Analysis

During data extraction, selected primary studies were fully read and analyzed by researchers for further classification. Data extraction was carried out by breaking down each research question into more specific criteria in which a set of possible options was established. The idea behind this approach is that each paper will be classified with the same extraction criteria, thus making it easier the document categorization and data extraction. The possible answers to each research question and criteria are detailed in Table 3, each paper can be classified based on the different options of each criterion.

Table 3. Data extraction criteria.
Full size table

Further, our data analysis involved a quantitative synthesis based on the number of primary studies that are classified in each answer from our research questions. As stated in Petersen et al. [2] bubble plots are useful to provide a map and give quick insights of a research field. Therefore, we make use of graphical synthesis methods to report the frequencies of combining the results from different research questions.

3 Results

By performing the search strategy, we identified 1181 papers extracted from digital database searches. Researchers read title, abstract and keywords of the papers resulting from the initial search; the application of selection criteria produced a list of 206 primary study candidates. Finally, as result of the whole screening process and data extraction, this work identified 44 primary studies. Each reviewed primary study was classified based on the possible answers of the main research questions. The results from the mapping study are structured according to the research questions, as shown in Table 4. Note that RQ1 and RQ5 are not exclusive; a study can be classified in one or more of the answers. The summation of the percentages is therefore over 100%. In addition, Fig. 1 shows a bubble plot that summarizes mapping results obtained from the combination of research question options.

Table 4. Results of the mapping study based on the possible answers to the research question.
Full size table
Fig. 1.
figure 1

Mapping results obtained from the combination of the first three research questions

Full size image

3.1 Areas in Education

Results of RQ1 show that around 47% of the papers reviewed are based on academic choices. Some studies like [3,4,5,6,7] reveal that the primary use of recommendation systems is to provide advice to students on their educational choices. This involves recommending students a place to study which could be a faculty, university, college or moreover support choices of specific academic courses or disciplines. Other studies focus on RS for e-learning courses. The authors from [7] use recommender systems to suggest online courses from different vendors as an embedded software.

Academic choices are not merely limited to the recommendation of universities and go beyond advising personalized courses; an example is [8] in which authors propose personalized curriculums. Other studies also focus on other aspects of education like recommending candidate students to available scholarships [9].

Another area supported by recommendations systems is the e-learning. From the reviewed primary studies, 15 papers (34%) were exclusively based on enhancing e- learning through RS. Studies like [10] proposed e-learning recommender system with the goal of helping students finding learning materials they need to study.

Few studies have been focusing on using RS for vocational and educational training. As shown in Table 4, only four papers, about 9%, considered some ways were students after graduation or during their studies could be introduced to the labor environment. Another study [11] proposed a hybrid recommender approach to support knowledge sharing and transfer among trainees, teachers and trainers for vocational education and training.

From the analysis and extracted data, 11 papers representing 25% of the reviewed studies used RS to enhance the academic performance. The study [12] applied a RS to identify students with learning deficiencies in assessments. The idea is through a recommender system find areas and indicators of achievement where students need to reinforce their knowledge and therefore, identify students with poor academic performance.

The remaining reviewed studies refer to resources and activities of the learning process suggested by RS. Around 18% of the reviewed papers use recommender systems on learning activities, [13] recommend online learning activities on a course web site. Besides, a reasonable amount of papers, about 29%, refers to recommending learning resources. Authors in [14] show a different RS with collective intelligence which suggests academic resources on the web including, educational videos, scientific literature, books, and all of the useful stuff for research.

3.2 Recommender System Approach

The results of RQ2 revealed that the most frequent type of recommender system used in education is the Hybrid approach, around 46% of the papers reviewed are based on a combination of different recommender types. The main choice of a hybrid approach is to improve the performance of the recommender model and overcome the problems of other types of recommender like collaborative and content based filtering.

Collaborative filtering (CF) accounts for around 30% of the reviewed papers. In CF the recommendation is assigned based on existing relations between users and items. Studies that apply the CF approach do not require content or any information about items, rather CF produce personalized recommendations, because they consider other people’s similar experience, therefore they can suggest appropriate academic or educational matters by observing same people’s behavior.

A small amount, 2% refer to using a Content based (CB) approach which aims to find similarities by the items’ properties. Moreover, less than 2% of the reviewed papers are based on a knowledge approach (see Fig. 1). Studies in this field suggest educational matters based on inferences about students or professors needs and preferences.

Other approaches of recommendations represent the rest 20%. Studies reviewed in this section consider other algorithms and classification schemes for the recommendation. The study [15] proposes a RS that exploits the knowledge, learns, discovers new information and infers preferences by using the knowledge from Smart Classroom.

3.3 RS Development Platform

The Web is the most adopted platform for using RS in education; it is reported by the 52% of the reviewed papers. As displayed in, Fig. 1 web-based platforms are present in many areas of education supporting different educational scenarios. In [16] authors describe an educational RS developed on a web platform, which recommends learning programming activities in a personalized way, i.e., according to the user profile. The web-based platform is named Wise Coach and enables the user to solve problems according to their programming level.

Other primary studies proposed different platforms for RS: based on desktops applications and mobile applications. The desktop-based platform was included in around 7% of the reviewed papers. Another study [17] describes the design of a stand-alone desktop platform that uses local information stored in databases and logs. Mobile based accounts for the 2% of the papers selected. [18] is an example where authors have designed and developed an android app which will recommend the graduate admission seekers to apply for suitable graduate schools.

Furthermore, there are 34% of the reviewed papers which propose some model or architecture. These studies do not develop or deploy a recommendation system and rather they explain possible conceptual models where recommendation systems can be based on. Some examples of this kind of study are presented in [19, 20].

3.4 Recommendation System Evaluation

The results for RQ4 revealed that 31% of the studies did not conduct any type of validation of the method (see Table 4). Around 18% of the studies presented RS which had been validated through a survey. For instance, [16] proposed a questionnaire for evaluating e-learning applications. Students answered a survey that aimed to validate the recommendations received, results obtained from the empirical evaluation allowed enhancement on the educational RS to be developed for it to be more reliable.

Around 47% of the papers report some kind of experimentation. For instance, [21] performed an experimentation to evaluate the effectiveness of a recommender system. Moreover, just one research representing 2% reported a case study. [22] conducted a case study to validate the proposed hybrid recommender system that demonstrates the effectiveness of using hybrid approaches in virtual learning environments.

3.5 Issues Addressed by the RS

One of the main issues addressed by RS is how to provide personalized recommendations. It accounts 43% of the papers reviewed. Some studies aim to give tailored suggestions to students and academics.

RS that solve the problems of Availability of information are presented in 31% of the selected papers. Studies like [4, 7, 9, 18] make available the scattered information of different online courses, universities, faculties, study plans, etc. in a single repository which is accessed through the RS.

Prediction accuracy and efficiency is addressed in about 29% of the reviewed papers. When implementing and building a RS, some studies test the system to provide the most accurately prediction using performance metrics like recall and precision with the idea to enhance the RS model.

Improve educational practices accounts for the 25% of reviewed papers. In those papers, the recommendation is targeted to a specific learning context, to engage students through the recommendation of educational resources.

4 Conclusions

Overall, there has been significant interest in the use of recommendation systems in educational scenarios. However, due to limited information about how recommendation systems have been used in education environments, what approaches are used and how they addressed different issues in the academic world, many researchers and practitioners are experiencing difficulties in retrieving relevant and useful information that summarize the utilities of using RS for educational purposes.

Therefore, in this work, we performed a systematic mapping study to investigate the use of recommendation systems in education. As the result of an automatic search on scientific databases, 44 primary studies were identified and thoroughly reviewed. From selected studies, relevant data was extracted and classified to get valuable insights about uses, approaches, and challenges addressed by RS. There are several uses of RS in education, the most reported is helping on academic choices, assisting in suggesting courses, or simply e-courses, research documents’ management, course complementary materials, resources and academic activities. Another area of education enhanced by RS is the e-learning education through web platforms.

Moreover, this work has also been useful to detect some research gaps and key areas where further investigation should be performed, like the introduction of data mining and artificial intelligence in recommender system algorithms to improve personalization of academic choices.

Furthermore, most of the existing recommendation approaches do not consider differences in the learner profile and characteristics. This issue can be addressed by introducing additional information about the student into the recommendation process and applying hybrid approaches that combine knowledge from learners. Although in this systematic mapping study we have not detailed possible drawbacks and gaps from the implementation of RS for education, as future work we intend to extend our study to improve the available knowledge of implementing and deploying models of RS within the educational context.