1 Introduction

Computer-supported education has been a field that has attracted the interest of numerous researchers worldwide. In parallel, the radical advancements in the area of computer science create a fertile ground for developing new methods and perspectives in this field [1]. To this direction, such novel improvements serve for the creation of a learning environment in which the student will play the leading role. In these environments, where the audience is characterized by a high level of heterogeneity, there are new technological instruments that provide an individualized learning experience by preserving the specific learning preferences and interests of students [2]. These instruments incorporate intelligence into the learning and tutoring procedures of the learning technology systems in order to adapt them to the individualized pace of instruction of each student.

The embodiment of intelligent methods to learning technology systems serves for determining the cognitive needs and abilities of learners [3]. As such, it helps the systems to create adaptive learning material for the students, offer dynamic assessment units, provide tailored teaching strategies, etc. For this reason, artificial intelligence (AI) mechanisms are employed to effectively manage learning paths tailored to each learner, monitor learners’ activities, elucidate them using appropriate models, reason about learners’ preferences and needs, and utilize learner and domain knowledge to dynamically expedite the tutoring procedure. Examples of AI methods are the technology of artificial neural networks and fuzzy logic.

There are many research studies that have presented adaptive educational systems [4,5,6,7,8,9]. These researches have focused on analyzing learners’ needs and perceptions, and creating adaptive educational material using the hierarchical relationships between domain terms. Moreover, they have dealt with the provision of appropriate advice to students based on their performance and the assessment of their knowledge towards offering adequate learning paths and enhancing learners' motivation through novel techniques. These researches have also explored the customization of the learning environments based on students’ learning styles, providing adaptive self-assessment and increasing students’ knowledge reflection by incorporating dynamic student models.

ANNs are the pieces of a computing system designed to simulate the way the human brain analyzes and processes information [10,11,12,13,14,15]. The technology of artificial neural networks (ANNs) has been utilized by researchers in the field of learning technology systems for providing personalization to students’ needs. According to [16, 17], the authors have focused on finding similarities pertaining to the domain knowledge patterns between the learners’ profiles and the learning material. As demonstrated in [18, 19], the adaptation of the learning technology system to the students’ specific needs and preferences using ANNs has been explored. In [20], the researchers have used ANN to construct a recommender system to assist students throughout the learning process. Other efforts have focused on the provision of adaptive instruction using ANN through the establishment of learning paths that align with the students’ needs and abilities [21, 22]. Finally, ANNs have been also employed for sentiment analysis in learning technology systems [17, 23].

Fuzzy logic models are mathematical tools for representing vagueness and imprecision. These models are used for recognizing, representing, manipulating, interpreting and utilizing data and information that lack certainty. Fuzzy logic has been applied to the field of learning technology mainly for analyzing the learners’ knowledge level, needs and behavior and for making the right decision about the instructional model that has to be applied for each individual learner. For example, in [24, 25], the authors employed fuzzy logic to explore the characteristics and behavior of students in learning technology systems. Other researchers have focused on determining the learning level [26], adapting the learning content and instruction according to students’ learning styles [27,28,29], and generating adaptive domain model [30, 31]. Furthermore, other works concerned the provision of emotion awareness [23], the delivery of feedback to students [32] and the assessment of the level of learner collaboration [33].

Moreover, recent developments in AI, particularly with regard to intelligent and adaptive learning technology systems, offer possibilities for modeling decision analysis. Decision analysis is utilized to create systems, aiming to give explanations to complicated problems by reasoning through sets of knowledge, mostly represented as if–then rules [34]. In learning technology systems, decision analysis can be employed to determine the appropriate learning material to be delivered to learners or the suitable test to be given to them. For instance, according to [35, 36], the authors have used decision analysis mechanisms in order to provide adaptive assessment units to learners. More specifically, in [36], the researchers explored the creation of dynamic adaptive tests using multiple-criteria decision analysis by taking into account multiple learners' characteristics as well as the kinds of exercises and the desirable learning outcomes. Moreover, in [3], the authors proposed decision-making models and methods to assess adequacy, acceptance and utilization of personalized learning objects. In addition, other researchers incorporated multi-criteria decision analysis approaches for selecting and evaluating digital learning units, delivered to students [37].

Based on the analysis of the afore-presented related scientific literature, we came up with the conclusion that the individualized tutoring provided by the systems mainly focuses on adapting learning content based on student knowledge level without considering characteristics as students’ misconceptions or using adaptation approaches like content granularity. Furthermore, to achieve this, they mainly adopt only one artificial intelligent technique, not a combination of them. More specifically, there are not any research efforts that involve ANN, WSM and fuzzy logic, as being proposed in this article, to further improve the hot research topic of tutoring modeling in intelligent and adaptive tutoring systems. As such, the need for this research emerged by the under-researched area of personalized tutoring systems. The aforementioned mechanisms are joined together in a novel way, and by considering different aspects of the students (e.g., cognitive characteristics, strengths and weaknesses), they offer a fertile ground for offering an individualized learning experience to them with the aim of further ameliorating their knowledge level.

In view of the above, this paper presents a multilayer inference engine to provide an individualized tutoring model to students. To achieve this, the students’ individual cognitive preferences and needs are taken into account. The multilayer inference engine involves a merge of algorithmic methods in different layers, namely artificial neural networks and multi-criteria decision analysis in one layer and fuzzy logic in the other layer. This engine provides individualized learning material to student regarding learning units’ adaptation and granularity. In particular, the artificial neural network is used for determining the way the domain knowledge is presented. It takes as input the learning style of the students according to the Honey and Mumford learning style model, which distinguishes learners to activists, theorists, pragmatists and reflectors [38]. The output of the artificial neural network is the way of presentation of the domain model, e.g., participation in problem solving activities, case studies, role playing, etc. The activation function of the presented ANN is calculated using the Weighted Sum Model (WSM), which is a method for multi-criteria decision analysis. Thus, the system provides adaptive learning units to students’ needs. The fuzzy logic is used for determining the granularity level of the learning material being delivered to students. To achieve this, it takes as input several students’ characteristics, namely the learning style, the knowledge level, the severity of mistakes occurred in assessment process and their frequency. In this way, the system further supports the individualized instruction and promotes students’ personalized learning enhancing their learning experience.

As a testbed for our research, we have designed and fully developed an intelligent and adaptive web-based system for the tutoring of the programming language Java. The domain to be taught includes basic to more advanced topics of Java, being appropriate to be delivered to undergraduate students of computer science. Our system was evaluated with the use of an established instrument and the statistical hypothesis test. The results of the evaluation are very promising, showing that our approach, to provide individualized domain knowledge using ANN, WSM and fuzzy logic, can be a powerful tool for personalized learning technology systems.

The remainder of this paper is organized as follows. Section 2 presents the architecture of the system developed. In Sect. 3, the approach of adaptive learning units using ANN and WSM is analyzed, whereas, in Sect. 4, the way of defining the learning units’ granularity using fuzzy logic is described. Section 5 includes the evaluation process of the proposed system and an extensive discussion on its results. Finally, Sect. 6 presents the conclusions deduced from the study and the future work for extending this research.

2 System architecture

This section presents the logical architecture of our system. As being an intelligent and adaptive tutoring system, it consists of three main modules, namely the domain, the student and the tutoring model (Fig. 1).

Fig. 1
figure 1

Logical architecture of the proposed system

Full size image

2.1 Domain model

The domain model deals with the knowledge of the subject to be learned, i.e., information on topics/concepts, exercises, problems and relationships between them. It includes different knowledge representations of the domain knowledge to support alternative instructional strategies, and testing items for student evaluation.

The domain knowledge of the system presented in this paper is comprised of the programming language Java. It has been divided into 11 chapters, covering knowledge in Java from basic to more advanced concepts. The domain knowledge is designed to be used by undergraduate computer sciences students (Table 1).

Table 1 Domain knowledge to be taught
Full size table

The learning units of the domain model range from course concepts presented through multimedia, namely text/images/videos, to learning activities, i.e., problem solving, pair discussion, role-playing and others. This instructional material is characterized by metadata indicating its difficulty, its granularity, the area of domain knowledge, the learning goals, etc. Moreover, a range of question items is provided through the domain model which is used for constructing the assessments dynamically. These elements include metadata regarding the area of domain knowledge and the nature of mistake each incorrect answer referred to. As the domain knowledge is the programming language Java, our system can diagnose two possible students’ mistakes, namely syntax and logical mistakes. Syntax mistakes can happen due to the fact that the syntax of the language is not respected. Examples of syntax mistakes can include opening brackets without closing them or entering several decimal points in one number, etc. On the other hand, logical mistakes concern programs that operate incorrectly producing unexpected results or undesired behavior. Examples of logical mistakes can include assigning a wrong value to a variable. Each incorrect answer is also characterized by a degree of severity ranged from 0 to 1, indicating whether a mistake is serious (near 1) or not (near 0). Therefore, the syntax mistakes have a low degree of severity, whereas the logic ones have a high degree of severity.

The proposed system tailors the presentation of the learning units to the individual needs of the student regarding the types of activities delivered. Moreover, it adjusts the granularity of the instructional material. For this, the performance of the learner in the assessment process plays a crucial role, namely the grades achieved and the mistakes occurred.

2.2 Student model

The student model includes data about student’s characteristics, cognitive and skills in the domain knowledge. It gathers measurement of student’s behavior and attitudes that are diagnosed through the interaction with the system during the learning process. Below, the student data used in the individualized instruction modules are described; since the fully analysis of student model is out of the scope of this research. Thus, the information about students that our system used is the learning style, the knowledge level, the types of mistakes usually made and their frequency.

The learning style refers to the way an individual learns. Students’ learning style is determined through a questionnaire delivered to them during their registration to the system. In our approach, we utilize the Honey and Mumford learning style model [38] in order to provide adaptive learning units with the intention to improve learning outcomes. The reason why we used the Honey and Mumford model is that it involves learning approaches that individuals inherently prefer. In Honey and Mumford model [38], the learning is maximized when students understand how they learn effectively and adopt approaches to learn in that way. Moreover, this model provides a highly sophisticated self-perception inventory being essential for individuals to find out their dominant learning style. The characteristics of the four learning styles proposed by this model are summarized as follows:

  • Activists (A): Activists are those people who learn by doing. Hence, they learn better through learning activities, such as brainstorming, problem solving, group discussion, puzzles, competitions and role-play.

  • Theorists (T): Theorists need to comprehend the theory behind the activities in order to learn better. The learning activities they prefer include models, statistics, stories, quotes, background information, applying concepts theoretically, etc.

  • Pragmatists (P): These learners are practical preferring applying the new concepts to problems of the real world. The learning activities, facilitating their knowledge acquisition, include experimenters, trying out new ideas, case studies, discussion, etc.

  • Reflectors (R): These people learn by observing and reflecting on results. They prefer to examine different perspectives, collect data and afterwards work towards a conclusion. Appropriate learning activities for reflectors are observing activities, feedback from others, paired discussions, etc.

During the assessment process, the system records students’ knowledge level based on scores achieved in the tests of the chapters. Moreover, it diagnoses students’ misconceptions through the mistakes they made, namely syntax and logical ones. This diagnosis can reason about the severity of the mistakes that the student is prone to, i.e., the average of severity’s degree with which each incorrect tests’ answer is characterized, and the frequency that the student makes mistakes, i.e., the ratio of tests where the student makes mistakes to the total tests given. This information is used in fuzzy logic inference for adjusting the granularity of the instructional material to students’ characteristics.

2.3 Tutoring model

The tutoring model exploits the information from domain and student model and generates adaptive scenarios according to teaching strategies. The system presented in this work provides two kinds of content adaptation (Fig. 2). The first one concerns the presentation of adaptive learning units according to the student learning style. The other one deals with the granularity of content delivered to students regarding their needs. These modules are described extensively in the following sections.

Fig. 2
figure 2

Individualized tutoring model

Full size image

3 Adaptive learning units using ANN/WSM

Our system supports fully the individualization of the domain knowledge to students’ cognitive needs and preferences by providing adaptive recommendation to students for enhancing their knowledge acquisition and an individualized pace of instruction according to a proper granularity.

For the types of activities presented for better knowledge acquisition, the system uses ANN combined with the weighted sum model (WSM), which is one of the most effective multiple criteria decisions analysis tools [39]. The reason why this approach is adopted is because it renders the system more dynamic and robust to select the most appropriate solution out of several alternatives. The proposed activities may include problem solving, pair discussion, role-playing and other activities through which students can learn the material regarding their learning style.

The learning style of each learner according to Honey and Mumford model [38] is the input of the ANN. In particular, each preference modality of this model (A, T, P, R) utilizes different ways, as seen in Fig. 3. For example, an activist prefers to participate in activities involving problem solving, role-playing and pair discussions; however, there is a different percentage for the preference of each activity. Accordingly, the remaining preference modalities of the learning style model are assigned weights by the same reasoning. Following, the activation function of the ANN, using WSM, produces the output of the ANN by calculating the resulting values. The weights and the activities of the ANN have been determined by 20 experts of the field of education. More specifically, 12 of them are university faculty members in the area of e-learning and pedagogy, 4 of them are primary school teachers and 4 are secondary school teachers. All these experts have a doctorate degree in educational sciences and also an extensive experience in the field of educational design. The experience of experts in the educational process is more than 12 years. Their knowledge and experience in the fields of education and pedagogy science were the cornerstone for their selection to determine the weights. The experts were asked to define descriptively the different degree of each learning style being involved in each activity, as well as the kinds of activities that are incorporated in our approach. In more detail, the collection of data was based on interviews with the experts. The interviews took place after a detailed presentation of our approach to the experts. The experts had three different rounds of elaboration. The results of each round were discussed and analyzed in the next round until the final one. At the final round, the experts delivered their proposals to the authors. It needs to be underlined that these weights are changeable and can be altered by the instructors using our system according to the educational design that they have made.

Fig. 3
figure 3

Adaptive learning units using ANN/WSM

Full size image

The artificial neural network, presented in this paper, is a kind of a network involving a group of sensory units. These units can be seen as cascading layers. These layers include an input layer (student learning styles using the Honey and Mumford model), one intermediate-hidden layer (learning activities offered to students) and an output layer of neurons (types of learning activities presented to students). The type of the ANN is FFNN (feedforward neural network) and the type of the activation function is the sigmoid. The ANN is fully interconnected in a way that all the neurons of each layer are connected to all neurons in the preceding layer. In our presented structure, the input nodes deliver the information into the units in the hidden layer and then the outputs from the hidden layer are delivered into the output layer. It is worth noting that the network is a supervised learning technique, i.e., mapping an input to an output based on input–output pairs. The output of the ANN concerns the type of activities as well as the percentage of them being presented to students and is summarized as follows:

$$y_{1} = \Phi \left( {X_{{\text{A}}} W_{{1{\text{A}}}} + X_{{\text{P}}} W_{{1{\text{P}}}} } \right)$$
(1)
$$y_{2} = \Phi \left( {X_{{\text{T}}} W_{{2{\text{T}}}} + X_{{\text{P}}} W_{{2{\text{P}}}} + X_{{\text{R}}} W_{{2{\text{R}}}} } \right)$$
(2)
$$y_{3} = \Phi \left( {X_{{\text{A}}} W_{{3{\text{A}}}} + X_{{\text{R}}} W_{{3{\text{R}}}} } \right)$$
(3)
$$y_{4} = \Phi \left( {X_{{\text{T}}} W_{{4{\text{T}}}} + X_{{\text{P}}} W_{{4{\text{P}}}} } \right)$$
(4)
$$y_{5} = \Phi \left( {X_{{\text{A}}} W_{{5{\text{A}}}} + X_{{\text{P}}} W_{{5{\text{P}}}} + X_{{\text{R}}} W_{{5{\text{R}}}} } \right)$$
(5)

To better clarify the functionality of the ANN, the case of a student is described. Maria is 65% activist and 35% pragmatist, based on the system’s log files. As such, XA = 0.65, XT = 0.0, XP = 0.35 and XR = 0.0. The corresponding weights receive the following values: W1A = 0.7, W3A = 0.2, W5A = 0.1, W1P = 0.4, W2P = 0.1, W4P = 0.1 and W5P = 0.4. These weights are the input to the activation function, which has the following outputs:

$$y_{1} = 0.65*0.7 + 0.35*0.4 = 0.595$$
(6)
$$y_{2} = 0.35*0.1 = 0.035$$
(7)
$$y_{3} = 0.65*0.2 = 0.13$$
(8)
$$y_{4} = 0.35*0.1 = 0.035$$
(9)
$$y_{5} = 0.65*0.1 + 0.35*0.4 = 0.205$$
(10)

The output of the ANN proposes that the domain model will mainly include problem solving activities and group discussion towards the enhancement of the knowledge acquisition.

4 Defining learning units granularity using fuzzy logic

The granularity refers to the level of detail regarding the domain knowledge that a tutoring system provides to students. In essence, the level of detail of instructional content should correspond to the knowledge levels and needs of the students. For example, in universities, students may have different learning capacities or background knowledge levels. As such, the granularity of the learning units should be tailored to their needs. In view of the above, our system diagnoses the granularity of the content that should be delivered to students regarding their characteristics in order to improve their learning outcomes. To this direction, a fuzzy inference module is applied for identifying the learning units’ granularity based on students’ learning style and their performance.

Fuzzy logic deals with the imprecision and uncertainty reasoning in comparison with traditional logic, which recognizes two values: true or false. The granularity level cannot be considered as a Boolean variable, and thus, fuzzy logic is the proper approach for our system to define it. In particular, it cannot be decided using simple if-clauses, since the value of each input and output (crisp values) has a degree of truth in which category it belongs, determined by the membership functions. For instance, if the knowledge level is 68%, it cannot be characterized as intermediate or high for sure, but there is a degree of truth for each category.

The fuzzy inference process consists of the following parts (Fig. 4): fuzzification of the input variables, choosing membership functions, constructing rules, making decision and defuzzification.

Fig. 4
figure 4

Fuzzy logic model for diagnosing content granularity

Full size image

The input set of the fuzzy model includes four variables, namely student learning style, knowledge level, types of mistakes and frequency of mistakes. All the input values are numerical and are mapped into fuzzy ones using mainly triangular membership functions, except from one variable where singleton membership functions are applied. The description of the input variables is as follows:

  • Learning style (LS): The learning style is based on Honey and Mumford model, as mentioned in Sect. 2.2. The fuzzy model takes as input the dominant learning style for each student as it is emerged from the system. The four types of this learning style model constitute the linguistic variables and are assigned using singleton membership functions to an integer, i.e., activists (A_LS) to 1, theorists (T_LS) to 2, pragmatists (P_LS) to 3 and reflectors (R_LS) to 4.

  • Knowledge level (KL): Student’s knowledge level is defined by his/her performance on chapters’ tests, namely the average of the scores achieved, and can be characterized from low to high. The rating of the provided assessments is based on 100-point scale. The linguistic variables of this input are low (L_KL), intermediate (I_KL) and high (H_KL).

  • Severity of mistakes (SM): This variable refers to the severity of the mistakes the student makes during the assessment process. The input value is calculated by the average of severity’s degree with which each incorrect tests’ answer is characterized. The linguistic expressions of this variable are low (L_SM), medium (M_SM) and high (H_SM).

  • Frequency of mistakes (FM): The frequency of mistakes is calculated during the assessment process based on the ratio of tests where the student makes mistakes to the total number of tests given by him/her. The linguistic variables of this input are rarely (R_FM), often (O_FM) and constantly (C_FM).

The output of the fuzzy model returns the level of granularity of the learning content delivered to each individual student based on the input set, i.e., student’s attributes, and the fuzzy rules, i.e., the reasoning of the model. This means that the system based on fuzzy logic provides different levels of detail in the proposed learning activities, and not different numbers of activities. In particular, the output can be described as follows:

  • Content granularity (G): It refers to the level of detail of the domain knowledge delivered to the student. The linguistic expressions of this output are abstract (A_CG), normal (N_CG) and detailed (D_CG).

Table 2 illustrates the fuzzy input and output set. Regarding the fuzzy variables’ representation, Fig. 5 shows an example of the equations of the singleton membership functions of learning style variable, whereas Fig. 6 shows an example of the equations of the triangular membership function for each linguistic expression of knowledge level variable. Moreover, Fig. 7 depicts the schemes of these two aformentioned variables. It should be noted that the type of membership functions was selected so that they approximately match the distribution of the data and their interval defined by experts after an interview process.

Table 2 Fuzzy input and output set
Full size table
Fig. 5
figure 5

Equations of learning style membership functions

Full size image
Fig. 6
figure 6

Equations of knowledge level membership functions

Full size image
Fig. 7
figure 7

Schemes of learning style and knowledge membership functions

Full size image

The inference mechanism produces the fuzzy outcome applying the fuzzy rules according to the fuzzy input and employing the Mamdani method for combining the active rules. The developed fuzzy model consists of 108 IF–THEN type fuzzy rules. The reasoning of the fuzzy rules’ design is based on the combination of the following facts emerged by the learning strategies of the Honey and Mumford model and the experience of experts involved in the process:

  1. 1.

    A more detailed content is more proper for Theorists and Reflectors rather than activists and pragmatists.

  2. 2.

    A student with low knowledge level needs simplified information regarding the domain knowledge in order to understand better the concepts, whereas a high granularity of content with advanced knowledge can be provided to an excellent student.

  3. 3.

    When a student makes serious mistakes in assessment process, such as logical ones, a more detailed content may be beneficial in contrast to students who make nonsevere mistakes.

  4. 4.

    A student who makes constantly mistakes may need shorter analysis of content in order to maintain a progress rate and avoid confusion of knowledge, whereas when the frequency of mistakes is low, the domain knowledge can be enriched with information.

A sample of these fuzzy rules is as follows:

  1. 1.

    IF LS is A_LS AND KL is L_KL AND SM is H_SM AND FM is C_FM THEN CG is A_G

  2. 2.

    IF LS is T_LS AND KL is L_KL AND SM is H_SM AND FM is C_FM THEN CG is N_G

  3. 3.

    IF LS is R_LS AND KL is I_KL AND SM is M_SM AND FM is R_FM THEN CG is N_G

  4. 4.

    IF LS is R_LS AND KL is H_KL AND SM is M_SM AND FM is R_FM THEN CG is D_G

  5. 5.

    IF LS is T_LS AND KL is I_KL AND SM is H_SM AND FM is O_FM THEN CG is N_G

  6. 6.

    IF LS is T_LS AND KL is I_KL AND SM is L_SM AND FM is O_FM THEN CG is D_G

  7. 7.

    IF LS is P_LS AND KL is I_KL AND SM is M_SM AND FM is R_FM THEN CG is N_G

  8. 8.

    IF LS is P_LS AND KL is I_KL AND SM is M_SM AND FM is C_FM THEN CG is A_G

This sample was chosen as representative rules showing the effect of each input to the output. Therefore, based on the first two rules, if a student has low knowledge level and makes serious mistakes constantly, he/she will receive simplest domain knowledge when being activist, whereas more information will be delivered to a theorist. However, this theorist student will not receive fully detailed content as this kind of learning style requires, since the other characteristics show that the student may face problems to the acquisition of knowledge.

Regarding fuzzy rules 3 and 4, a reflector student who makes rarely mistakes of medium severity may receive content of normal granularity if he/she is an intermediate student, whereas more advanced knowledge can be delivered if he/she is an excellent student. Therefore, the knowledge level affects the granularity of content, since, while reflectors prefer a variety of perspectives to investigate, in the first case, less detailed content is provided as we want the student to be improved progressively.

Being a theorist with intermediate knowledge level who makes often mistakes in assessment process, a normal level of content’s granularity will be delivered if the severity of misconceptions is high, whereas more information will be provided if it is low (rules 5–6). The reason why this has been decided is that in the first case, the serious mistakes occurred in combination with the other student’s characteristics indicates that the student may be confused if full information is provided.

The fuzzy rules 7 and 8 illustrate the impact of the frequency that a student makes mistakes in tests. If a pragmatist with intermediate knowledge level makes mistakes of medium severity rarely, a normal level of granularity in instructional material will be provided as it seems from the other characteristics that he/she can manage it, and thus, this individual instruction can help him/her to advance his/her knowledge, whereas, if he/she makes mistakes of the same severity constantly, he/she will receive simplest material in order to better digest the information.

The last stage of the fuzzy model is the defuzzification where the center of gravity (COG) technique is used to convert the fuzzy output produced by inference mechanism into a crisp output. Thus, the content granularity is estimated, and individualized instruction is provided to student.

To better clarify the functionality of this module in combination with the previous one, namely how the learning units granularity indicated by the fuzzy logic model is merged with the adaptive learning units produced by ANN/WSM, we extend the example of operation described in previous section illustrating the results emerged from the fuzzy model for this student. According to the ANN output, Maria’s domain model will mainly include problem-solving activities and pair discussion. Her dominant learning style is activist, and when she requests the Chapter 3 for studying, her instance of student model characteristics is KL = 35, SM = 0.81 and FM = 0.72. Thus, the fuzzy model diagnoses that the proper granularity of the learning units is abstract (CG = 0.289). As a result, the system delivers a simple problem solving activity for better content understanding and recommends a discussion in small group not to be confused with information overload. After a while, having given several chapters’ tests, Mary requests the Chapter 9. At that moment, her profile has formed as follows: KL = 70, SM = 0.38 and FM = 0.29. The output of the fuzzy model is that the proper content granularity is detailed (CG = 0.686). Therefore, the system now provides two advanced problem-solving activities promoting a challenging learning and proposes four group discussion rooms with more participants than the one recommended in Chapter 3 to interact with a range of ideas.

5 Evaluation results and discussion

The evaluation of software is considered to be a significant phase in the systems development life cycle (SDLC). For this reason, using established evaluation frameworks can provide more reliable results and enhance software’s usability. Specifically for the evaluation of the presented multilayer inference engine, incorporating ANN and WSM and fuzzy logic, the experimental measurement as presented in this section was chosen as one of the most widely used and effective techniques of performance evaluation [40]. Towards this direction, we employed the Lynch and Ghergulescu framework [2], which is oriented to the evaluation of adaptive and intelligent tutoring systems. This framework involves four discrete dimensions to be evaluated: (1) learning and training, (2) system, (3) user experience and (4) affective dimension.

The dimension of learning and training evaluates the efficiency and effectiveness of the educational process. In more detail, it assesses the pedagogical affordance of the system as well as the amelioration of the knowledge acquisition that has been achieved by the learners and the time needed for this amelioration to happen. The dimension of system evaluates the efficacy of the algorithmic approaches that have been utilized towards providing a better modeling of learners’ needs and preferences to ensure individualization and adaptivity. The dimension of user experience assesses the students’ attitude using the system. It involves the practical, experiential and valuable aspects of human–computer interaction, including user interface friendliness and ease of use and user satisfaction. Finally, the affective dimension evaluates the engagement of learners throughout their interaction with the system.

The population which took part in the experiment includes 60 students from the Department of Informatics of a public university in the country. All the students are undergraduate students of the same class year who attend the second-year course “Java Programming.” Furthermore, three faculty members gave their assistance in the evaluation process by taking part in the presentation and explanation of the system to students as well as in the supervision of students throughout the experiment. The whole experiment lasted for an academic semester. At the end of the semester, the students were given questionnaires to complete under the supervision of the evaluators and their instructors (faculty members). The questionnaires were based on the Lynch–Ghergulescu framework and included twelve questions, as follows:

  • two (2) questions for the evaluation of the “Learning and Training” dimension.

  • two (2) questions for the evaluation of the “System” dimension.

  • six (6) questions for the evaluation of the “User experience” dimension.

  • two (2) questions for the evaluation of the “Affective” dimension.

Table 3 summarizes the questionnaire that was delivered to students. These questions follow a 1-to-10 ranking model (lower is negative, higher is positive). In order to evaluate questionnaire’s reliability, a Cronbach’s alpha was run on the sample (Table 4). The alpha coefficient obtained was 0.826, higher than 0.7, indicating a high level of internal consistency for our scale with this specific sample.

Table 3 Evaluation questionnaire
Full size table
Table 4 Reliability analysis of questionnaire
Full size table

The results of the questionnaires were aggregated based on the framework’s dimensions are presented in Fig. 8. It needs to be noted that the students were very interested in using the software, and they became very familiar with it, since they are computer science students.

Fig. 8
figure 8

Results of questionnaire survey

Full size image

Analyzing the results of the evaluation study, there is considerable evidence that the creation of multilayer engine involving ANN and WSM combined with fuzzy logic mechanisms can be seen as a valuable tool for a more adaptive domain model for the students and an individualized learning experience. Concerning Question 2 which is about the efficiency in the use of time, students rated it with 7.56 points which is a high score indicating that the time they took using the system was very productive. This can be explained due to the high level of adaptivity to their cognitive preferences and needs provided by the system helping them learn efficiently. Concerning Question 3, evaluating the way of presentation of the domain knowledge, its score was 8.03 which further strengthens the reliability of the approaches to individualized instruction been incorporated into the system, namely ANN and WSM techniques. Concerning Question 4, evaluating the level of abstraction of the domain knowledge delivered to students, its score was 8 depicting a high acceptance of the results of learning unit’s granularity module. It should be noted that this module takes into consideration the learning needs of the students by employing fuzzy logic and delivers the domain knowledge to them according to these needs.

In total, 34 students (56.66%) attested the high pedagogical potential of our system and declared that their knowledge level was improved after their interaction with it (Learning and Training dimension). Thirty-nine students (65%) declared that the domain knowledge was delivered through an individualized way of presentation and at a proper level of abstraction, approving the ANN and WSM technique and the fuzzy logic mechanism. Also, 31 students (51.66%) attested their satisfaction while using the system and underlined the friendliness and easiness of the user interface. Finally, 30 students (50%) noted that they were very engaged and motivated throughout their interaction with the system.

The aforementioned results show that the proposed tutoring model achieves its goals for promoting an efficient learning experience. In particular, the pedagogical approach of providing individualization of not only the learning objects but also their granularity, boosts students’ engagement and productivity through the learning process. Moreover, the beneficial impact on learning outcomes and the positive attitude towards the developed system validate the choice of the intelligent techniques used for adapting tutoring to students’ needs and preferences.

In the evaluation process, the faculty members (who are also instructors of the students) used the system for a while as students, exploring the individualized learning content provided to them based on their characteristics in all chapters. Moreover, different case studies were presented to them in order to get an overview of adaptation approaches used in the system. Afterwards, they provided their feedback through oral interviews. They noted that the utilization of ANN and WSM as well as the fuzzy logic offered great results in the way of presentation and level of abstraction of the domain knowledge. In particular, they found the kind of learning activities to be appropriate to engage students into the learning process and the delivered learning activities to be pertinent and relevant to each individual’s characteristics. In addition, they mentioned that the approach of adapting content granularity is innovative and useful to students for learning better through well-aimed learning content. They underlined that these techniques offered an individualized learning experience to students and improved their knowledge level.

In order to provide more quantitative results, t test was also applied. The system presented in this article was compared to its conventional version. This conventional version had the same interface with the presented system without adopting the novel approaches for the domain knowledge model, namely the ANN and WSM techniques for individualized way of presentation of the domain model and the fuzzy logic mechanism for adaptive level of abstraction of the domain model. Instead, the conventional version delivered the domain knowledge to be taught progressively in chapters delivering learning activities of all categories in a static way solely based on students’ learning style. Our system was used by 60 students (Group A), as mentioned above. The conventional version was used by another 60 students (Group B), who are students in the same Department and with the same characteristics (Table 5). It needs to be noted that the classification into two groups (Group A and Group B) was made by the evaluators and the faculty members of the university, taking into account the demographic and psychometric characteristics of the students and as such they ensured that the two groups are comparable.

Table 5 Population characteristics
Full size table

For the t test, the alpha value was set equal to 0.05 and we analyzed the p value results. Based on these results (Tables 6, 7), there is a statistically significant difference between the means of the two trials concerning Questions 1, 3, 4 and 6 (of Table 3); since the P values were 1.45238E−08, 2.57714E−07, 1.4624E−10 and 0.022880914, respectively, less than the alpha value. This means that our system (holding the multilayer inference engine) surpasses its conventional version in terms of students’ learning outcome improvement, degree of acceptance of the way of presentation and level of abstraction of the domain knowledge model and their overall experience while interacting with the systems. These results were anticipated since our presented system incorporates the ANN and WSM technique for an optimized way of presentation of the domain knowledge as well as the fuzzy logic mechanism for delivering an individualized level of abstraction. These novel approaches also ameliorate the learning outcome improvement and the students’ experience while using the tutoring systems. An important clarification concerns specifically Question 2. As mentioned above, the efficiency in the use of time is related to a high degree of adaptivity to students’ needs and preferences. However, this question does not present statistically significant difference possibly because the learners are familiar with computers and the use of software, and thus they do not face difficulties interacting with them. All the other questions do not present statistically significant difference as both systems hold the same user interface which provides friendliness and easiness in its use.

Table 6 T test results for question 1 and 3
Full size table
Table 7 T test results for question 4 and 6
Full size table

To further investigate the pedagogical potential of the presented multilayer inference engine, a t test was developed between the performance data of Groups A and B, and specifically the grades of the students (Table 8). The mean of Group A grades was 7.42, while this of Group B was 6.52. The t test results showed a significant difference between these means, since the P value was 0.009286, less than the alpha value which had been set as 0.05, indicating that the system used by Group A outperformed this of Group B in terms of learning outcomes. Therefore, using the presented system, students achieved better grades and improved their knowledge level due to the successful individualized instruction.

Table 8 T test results for students’ grades
Full size table

Regarding the scalability and reusability of the proposed multilayer inference engine, it should be noted that its design is domain independent since it is developed based on pedagogical aspects emerged by the Honey and Mumford model and the experience of experts involved in the process. Thus, it is fully reusable in other domains requiring only the proper design of learning content. Moreover, the weights of the ANN model and the fuzzy rules can be parameterized by instructors according to their educational design. However, in this case, the pedagogical affordance and learning outcomes may not be the optimal, since the presented model has been evaluated thoroughly ensuring system’s reliability.

6 Conclusions and future work

With the proliferation of computers, a new form of education is emerged, namely digital one. The technological advances in this field lead to the development of online learning environments where students are in the center of learning process. Hence, they are designed to mainly provide individualized tutoring regarding students’ needs and preferences. To achieve this, intelligent techniques are applied in learning technology systems to enhance adaptivity and improve learning experience.

In view of the above, this paper presents a multilayer inference engine to provide personalization in tutoring model to students. The multilayer inference engine merges algorithmic methods in different layers, namely artificial neural networks and multi-criteria decision analysis in one layer for adapting learning units to student learning style, and fuzzy logic in the other layer for defining the granularity of learning units based on student profile. The student characteristics used in both methods emerge from student model that involves learning style, knowledge level, severity and frequency of mistakes made in assessment process. The scope of this approach is to enhance students’ learning experience incorporating effective tutoring strategies for improving learning outcomes. The novelty of this system is that it not only provides adjusted content to students’ needs, but also tailors its level of detail for better knowledge acquisition.

The system was evaluated using the Lynch and Ghergulescu framework, oriented to assess adaptive and intelligent tutoring systems based on four discrete dimensions: (1) learning and training, (2) system, (3) user experience and (4) affective dimension. Thus, a survey examining these dimensions was conducted and the statistical hypothesis test was applied. The results show a high level of acceptance of the presented model. The development of the multilayer engine incorporating ANN and WSM with fuzzy logic mechanism can be considered as a valuable tool for improving the adaptability of the domain model according to students’ needs and providing an effective individualized learning experience.

The limitations of this study concern the average of severity’s degree with which each incorrect tests’ answer is characterized. Future work includes the depiction of the severity’s degree using sophisticated techniques, such as cognitive maps. Another limitation is the number of the learning activities’ types which can be extended in a future research. Furthermore, other techniques for the selection of learning activities could be explored. It is worth noting that the approach of learning styles (Honey and Mumford model) was utilized in our research; however, arising from the study of Kirschner [41], there are also several other interesting and promising techniques for future investigation towards personalizing teaching and learning.

Moreover, future steps include the incorporation of further layers to inference engine enhancing system’s personalization. For example, machine learning techniques can be applied to predict students’ learning style and a second fuzzy logic model can be used to define the misconceptions in assessments. Moreover, it is in our future plans to incorporate to our artificial neural network the ability to be self-trained, in order to adjust the values of the weights automatically. Finally, it is desirable to investigate further the need for more triangulation in terms of data sources, and data collection, processing and analysis techniques.