Keywords

1 Introduction

The rapid progress of wireless communication and sensing technologies has enabled the development of innovative environment in learning activities. For this reason making learning systems innovative and smart has been the objective of many researchers in both the fields of computer science and education [1].

In the field of computer science an innovative environment is a digitally augmented physical world where sensor-enabled and networked devices work continuously and collaboratively to make lives of inhabitants more comfortable. Indeed, tremendous advances in smart devices, wireless mobile communications, sensor networks, pervasive computing, machine learning, robotics, middleware and agent technologies, and human computer interfaces have made the dream of smart and innovative environments a reality.

In the field of education learning theories provide insights into the very complex processes and factors that influence learning and give precious information to be used in designing instruction that will produce optimum results. The learning models are designed in order to supply to the students with practice, evaluation and improvement procedures which will adjust the model [2]. The teaching process first requires that the instructor creates a pedagogical design of the objectives and determines the content to be taught. Second, a pre-assessment is used to determine learning abilities. Third, pedagogical procedures are used when teaching is initiated. Finally, assessment is applied to determine what learners have achieved, and, according to the assessment results, instructors should use feedback to determine the cause of ineffective instruction [2, 3].

It’s crucial to improve the learning process and to mitigate problems that might occur in an environment with learning technologies. Besides, each student has its own particular way of assimilating knowledge, that is, his learning style. Learning styles specify a student’s own way of learning. Someone that has a specific learning style can have difficulties when submitted to another learning style. When the given instruction style matches the student’s learning style, the process is maximized which guarantees that the student learns more and more easily. Attention is a very complex process through which one individual is able to continuously analyze a spectrum of stimuli and, in a sufficiently short amount of time, chose one to focus on. In most of us, who can only focus on a very reduced group of stimuli at a time, this implies ignoring other perceivable stimuli and information.

However, for various reasons, students may not be predisposed to learning. In this sense, and in bigger classes, it is important that the teacher has instruments to point out potential distractions (namely in what concerns the applications being used by the students) that may indicate a lack of predisposition to learning.

This paper deals with the issue of AmI system, with the aim of proving a non-intrusive and non-invasive smart environment, reliable and easy tool that can be used freely in schools, without changing or interfering with the established working routines of students.

This paper is organized as follows. In the next Section the state of art of attention concept, learning styles, and AmI system where scientific literature is reviewed. Section 3 contains the proposed architecture, Sect. 4 presented the methodology applied with data acquisition and in Sect. 5 results are presented. Finally, in Sect. 6 some discussions and conclusions of this work are presented.

2 State of Art

In educational environments, attention is considered a fundamental factor in the evolution and success of the student. If the student is not concentrating and paying attention to what is being taught, he will not capture information that is being provided and consequently the academic course will be compromised.

Attention is a resource that allows the human being to be focused on a situation and to be able to ignore non-priority information. As happens with performance, several factors can influence attention, like, stress, mental fatigue, anxiety, emotion, new environments, and human health. Besides these factors, the advancement of technology has been a real problem which has increased the lack of attention. With the emergence of the smartphone that provides new and varied information in real time and new ways of communication, people’s attention is easily captured and the task that was meant to be done is left out [4, 5].

2.1 Attention Concept

In educational environments, attention is considered a fundamental factor in the evolution and success of the student. If the student is not concentrating and paying attention to what is being taught, he will not capture information that is being provided and consequently the academic course will be compromised.

Generally there is no universally accepted definition of attention because there is a diversity of disciplines that are focused on it. In the past, only psychologists studied attention, however in present days attention is highly important for other fields like philosophy, chemistry, anatomy, and even computational science [6].

The concept of attention may be defined as the transforming of a huge acquired unstructured data set into a smaller structured one where the main information is preserved. In Computer Science, attention means that there is a filtering input space that selects the most important data in processing and this is a key mechanism of behavioral control for tasks, which is related to planning, decision making, and preventing new situations, however there are limited computation capabilities [6, 7].

Attention means focusing on clear thinking, among one of several subjects or objects that may capture mind simultaneously. Attention implies the concentration of mental powers upon an object by close or careful observing or listening, which is the ability or power to concentrate mentally.

2.2 Learning Styles

In order to maximize the learning is also important to consider the concept of learning styles. A learning style is the method that allows an individual to learn best. Different people learn in different ways, each one preferring a different learning style.

Learning style not only specifies how a student learns and likes to learn, but it can also help a teacher to adapt to individual students, so that they might learn successfully. When the teacher’s methodologies do not support a specific learning style, the student will find it more difficult to learn and acquire knowledge. Everyone has a mix of learning styles, but some people may find that they have a dominant style of learning. Others may find that they have different learning styles in different circumstances.

Learning styles can be defined as cognitive, affective, and physiological features that serve as relatively stable indicators of how learners perceive interaction and respond to their learning environments [8].

There are several models developed by several authors that try to represent the way people learn [9]. Previous research suggests that, in the context of learning activities, different learning styles can influence learning performance [10, 11]. Learning styles are considered one of the more important factors influencing learning [12].

Some researchers have argued that learning style is also a suitable indicator of potential learning success because it provides information about individual differences in learning preferences and information-processing [10, 13].

However the field of learning styles is a very controversial field, because there are some authors that consider that scientific support for learning styles theories is lacking [14].

2.3 AmI System

Important features of AmI system or smart environments are that they possess a degree of autonomy, adapt themselves to changing environments, and communicate with humans in a natural way. An AmI system is a tool in which technology is embedded, hidden in the background, sensitive, adaptive and responsive to the presence of people and objects. This system also preserves security and privacy while using information when needed and with an appropriate context [15].

In our case we proposed an AmI system that aim to support the student learning process. This system is adaptability, which means that consider the student’s knowledge, background, interest, goals, targets and/or choices.

3 The Proposed Architecture

Once information about the individual’s attention exists in these terms, it is possible to start monitoring attentiveness in real-time and without the need for any explicit or conscious interaction. This makes this approach especially suited to be used in learning activities in which students use computers, as it requires no change in their working routines. This is the main advantage of this work, especially when compared to more traditional approaches that still rely on questionnaires (with issues concerning wording or question construction), special hardware (that has additional costs and is frequently intrusive) or the availability of human experts.

The present work adds a new feature to this previously existing framework, by providing the learning styles theory, where the applications of different type of exercises obtained different results of level of attentiveness. It constitutes a much more precise and reliable mechanism for attention monitoring, while maintaining all the advantages of the existing system: nonintrusive, lightweight, and transparent.

This work was detail in [16], but briefly as show in Fig. 1, it is possible to collected data that describes the interaction with both the mouse and the keyboard in the devices in which students work have software that generates raw data, which store the raw data locally until it is synchronized with the web server in the cloud. The Mouse and Keyboard collected data captured information describing the behavioral patterns of the students’, and receiving data from events mouse and keyword students’. This layer encodes each event with the corresponding necessary information (e.g. timestamp, coordinates, type of click, key pressed). These data are further processed, stored and then used to calculate the values of the behavioral biometrics. Mouse movements can also help predict the state of mind of the user, as well as keyboard usage patterns.

Fig. 1.
figure 1

Architecture of the system.

Full size image

After the raw data was stored in a data store engine, the analytic layer is responsible to process the data received from the storage layer in order to be evaluated those data according to the metrics presented. It’s important that in this process some values are filtered to eliminate possible negative effects on the analysis (e.g. a key pressed for more than a certain amount of time). The system receives this information in real-time and calculates, at regular intervals, an estimation of the general level of performance and attention of each student.

In the classification profile layer the indicators are interpreted. Based on data from the attentiveness indicators and building the meta-data that will support decision-making. When the system has a sufficiently large dataset that allows making classifications with precision, it will classify the inputs received into different attention levels in real-time, creating each student learning profile. With these results it is possible to obtain a profile of the learning style.

Finally, the actual students’ attention information is displayed in the visualization layer, and can be used to personalize instruction according to the specific student, enabling the teacher to act differently with different students, and also to act differently with the same student, according to his/her past and present level of attention. In visualization layer it’s possible to obtain some graphical modules that allow showing the information in an intuitive way to the user. These graphical modules stand out the user interface that allows the teacher to control the application. This graphical module user interface is composed by a module that allow the creation of charts (CHART) and the layer that allows the creation of virtual classrooms (ROOM) so that the teacher may view intuitively the students behavior.

4 Methodology

This work was applied on a vocational course while performing an activity based on Adobe Photoshop at the high school of Caldas das Taipas, Guimarães, Portugal. We want to determine how the class reacts during the lessons and the effect on mouse and keyboard dynamics, and attention level.

For this purpose one group of 22 (9 girls and 13 boys) students were selected to participate in this experience. Their average age is 17.6 years old (SD = 1.4 years). The experiment was applied in four different lessons, where they have access to an individual computer and 100 min to complete the task. Students received, at the beginning of the lessons, all necessary data with the goals of the task.

To quantify attentiveness the following methodology was followed. Apart from capturing the interaction of the students with the computer, the monitoring system also registers the applications with which students are interacting. Attention is calculated at regular intervals, as configured by the teacher (e.g. five minutes). The teacher may also want to assess, in real-time or a posteriori, the evolution of attention of the whole class.

In order to determine the learning style of each student, four different exercises were applied in four different days where the room had similar conditions in terms of lightning, temperature and humidity. The exercises applied were the following: on the first day a video exercise without audio; on the second day, an exercise only with images; on third day, an exercise only with text; and on the fourth day an exercise only with audio. In the end of each class, the exercise was saved in order to be assessed by the teacher.

4.1 Features Extraction

The process of feature extraction starts with the acquisition of interaction events, which is carried out by a specifically developed application that is installed in each of the computers, laptops or tablets. The first stage in the life cycle of the proposed system takes place in the data generating devices, which was designed and implemented using a logger application.

The data collected by the logger application characterizing the students’ interaction patterns is aggregated in a server to which the logger application connects after the student logs in. The privacy of the students is ensured, since the necessary data that is collected in the registration process are an ID that does not identify the student, password, and gender. Furthermore, the privacy issues of the system are assured, since the teacher will only have access to the final results on the level of attention.

The Mouse and Keyboard Sensing layers are responsible for capturing information describing the behavioral patterns of the students while interacting with the peripherals [17].

5 Results

During the lessons the monitoring system was used to assess the interaction of the students with the computer and to quantify their level of attentiveness as well.

On each lesson the level of attention of each student was quantified. However, at the beginning it is necessary that the teacher define the task-related applications that the students will use during the class. For that he/she uses a graphical interface to set rules such as “starts with Photoshop” or “Contains the word Photoshop” which are then translated to regular expressions that are used by the algorithm to determine which applications are and are not work-related [16]. In this sense it is necessary to measure the amount of time in each interval, which the student spent interacting with task-related applications. By default, applications that are not considered task-related are marked as “others” and count negatively towards the quantification of attention. The teacher may also determine the regular intervals at which attention is calculated.

Figure 2 shows the output of the evaluation of attention worked task-related of a number of students in the four different lessons.

Fig. 2.
figure 2

Detail of evaluation of attention of the students in the four different lessons.

Full size image

The first lesson was a video exercise without audio; the second lesson, was an exercise only with images; the third lesson was an exercise only with text; and the fourth lesson was an exercise only with audio.

In each of these lessons it’s analyzed the level of attention, which is measuring with the work task-related. This is an example of a set of students, but the teacher had access to all students and the global of the class, which allows the teacher to assess the temporal evolution of attention. These results consider the entire length of a class and give the percentage of time spent in task-related or other applications, for each student.

6 Discussions and Conclusions

The main goal of this paper was to present an AmI system approach that analyzed the interaction of student’s in learning activities using technologies connected to the Internet. In this case, a specific subject was focused (Adobe Photoshop) and it was analyzed in four different lessons, using four different learning style approach. For this case it was observed the performance of the class and each student. An example of the results of one student was showed and we can observe that these students for the same subject react differently depend on the learning style applied.

In general, the lesson that obtained better results is the lesson where it was applied Text and Images. However the student 12P505, have a better result for the lesson where the exercise was video. In the opposite, the lesson that the level of attention was worst was the exercise where it was applied only text. Although, the student 12P503 have the worse result in the lesson where the exercise applied was video.

These indicators are useful to predict the behavior of a student and identify potential problems in the course of their learning.

AmI system make possible the enhanced of learning/teaching processes. The architecture of an ambient intelligent learning environment is proposed to address these issues, especially to monitor the students’ attentiveness students in learning activities, through the use of a developed log tool. With this architecture it is possible to detect those factors dynamically and non-intrusively, making it possible to foresee negative situations, and taking actions to mitigate them. In this case the door is then open to allow to analyze students’ profile, taking into account their individual characteristics, learning styles, and to propose new strategies and actions, minimizing issues such as stress, anxiety, and new environments, which can influence students’ results and are closely related to the occurrence of conflicts. Moreover it’s possible to maximize the performance and attentiveness since the teacher is informed about the behavioral of each student.

Enlarging this study to the use of smartphones and tablets, taking advantage of their new features such as several incorporated sensors, and high resolution cameras, is the next step that may allow a wider characterization of the student, making it possible to enhance the learning experience, though better recommendation and personalization.