Keywords

1 Introduction

Student attendance is the process of recording information about the presence of a student at a particular place, for a particular period. It has been one of the most significant criteria to keep a count on the participation of a student in a program, or course. Therefore, maintaining attendance for a course becomes very much essential. Different institutions have different methods of taking attendance of the students varying from pen-paper based approach to the use of biometric-based methods. Biometric-based techniques have several advantages over the pen-paper based approach. They still contain a massive scope for future enhancements because the total time taken for the attendance process to finish for every student depends on the number of students’.

In the last decade, face detection and recognition strategies have undergone a large-scale change. As a result, several face recognition based methods have been proposed for attendance management intending to overcome the limitations of the traditional methods. Different methods using a fixed camera setup [1,2,3,4,5,6,7], multi-camera setup [8, 9], rotating camera setup [10, 11] or High Definition (HD) video camera setup [12,13,14] have been proposed. Also, methods using recognition technologies with Radio Frequency Received Signal Strength (RF-RSS) [15], Radio Frequency Identification (RFID) [16, 17], Bluetooth Low Energy (BLE) beacons [18] or Internet of Things (IoT) [19,20,21] have been proposed. The proposed methods generally vary in the choice of hardware and recognition algorithms used. Figure 1 depicts the block diagram of a face recognition based attendance management system. Initially, the faces are detected from the input image, and region of interest is marked. Then features are extracted of each of the detected faces, which later on are used to identify the faces using the existing features set.

Fig.1
figure 1

Block diagram of a face recognition based attendance management system

Full size image

The literature survey part gives a brief review of all the previous works on face recognition based attendance systems. The methods, features, and limitations in practical implementation have been discussed. After that, a comparison table has been presented which performs the comparison of previous works on various parameters. Lastly, the conclusion and the scope of improvements and enhancements in the future have been discussed.

2 Literature Survey

Several authors have proposed systems that use face recognition methods to automate attendance management in an institution. They were successful in providing solutions to the problems of the traditional approach of attendance. There still exists a large scope for improvement in terms of efficiency, accuracy, and cost optimization.

In [2], the authors proposed a model of automatic student attendance management which used the eigenfaces-based method for detecting faces. It was used because of the good ability for detecting faces. They used Principal Component Analysis with facial features. The two main components that were used in the implementation part were OpenCV and Fast Light Toolkit (FLTK). They used a fixed camera for the acquisition of the image of the students. The system worked by first analyzing the image of the faces and computing eigenfaces comprising eigenvectors, which later were used for recognition. They observed that change in face angles to the camera resulted in a change of accuracy. The accuracy decreased as the face angle increased.

In [9], the authors proposed a model of student attendance management using face recognition with an additional feature of percentage analyzer. The proposed system used Artificial Neural Network (ANN) with Principal Component Analysis (PCA) for face recognition. In this model, maximum varied face points were identified and feature vectors at those points were created. Also, the covariance column matrix was created using PCA for creating facial features. They used the extracted features for computation of eigenfaces, which served as an input to ANN for training. The ANN once trained could identify faces with a threshold.

If the value of the result was less than the threshold, it discarded the result.

In [16], the authors proposed a model of student attendance management using Raspberry Pi 2 and MATLAB. The system contained a camera for image acquisition with a Raspberry Pi 2 module. In the proposed system, they detected the faces in the image using the Viola-Jones algorithm. They extracted local features of the image using Histogram of Oriented Gradients (HOG) and Local Binary Pattern (LBP). They used the local features for recognition. The classification and comparison of images were performed using Support Vector Machine (SVM). Figure 2 depicts the working of face recognition using HOG with SVM. The model was tested on a group of 12 students. It gave an accuracy of 92% and could recognize all the detected faces.

Fig.2
figure 2

Block diagram of face detection using HOG with SVM

Full size image

In [19], the authors proposed a prototype of a model similar to the previous one as defined in [15, 31] but used Viola-Jones algorithm to identify faces from the picture, and fisher faces algorithm for face recognition. It had an additional feature of the Global System for Mobile (GSM) notification. It used the internet, modem, personal computer, webcam, and PostgreSQL database for storing records. After marking the attendance in the database, it sent a notification to parents and the student. This model had a limitation that it required a stable internet connection for the attendance to be marked successfully. This had a medium accuracy close to 72.00%.

In [1], the authors proposed a prototype of a model for classroom attendance using a deep learning model. In the proposed model, they used a fixed camera in the classroom for image acquisition. Open-source machine learning library, Torch was used for building the detection system. They performed the face detection using Convolutional Neural Network (CNN) cascade of twelve CNNs. They trained deep CNN using deep learning-based OpenFace library with a pretrained FaceNet face recognition system’s network. They performed classification using a Linear Support Vector Machine (SVM). They tested it on images having less than 20 faces and had an accuracy of 95.02%.

In [3], the proposed model used a video sequence instead of an image as an input to the face recognition based attendance system. Viola-Jones algorithm as described in [19] was used for face detection. They tracked the faces frame to frame by analyzing the video sequence by using a correlation tracker. They obtained distinct features from the face images present in the pre-processed image using the Convolution Neural Network (CNN). CNN was used to get low-dimensional distinct features from the images. CNN comprised many layers with a loss function L. The training was performed on the neural network to find θ that minimizes L. They computed it using back-propagation with differentiable layers. The optimal value was obtained iteratively by using the first-order method. Figure 3 depicts the block diagram of the deep neural network-based attendance system, and the flow of working of the neural network. This model wasn’t able to identify every student in the classroom every time.

Fig.3
figure 3

Block diagram of deep neural network-based attendance system

Full size image

In [20], the proposed model used Region-Based Convolutional Neural Network (R-CNN) and SeetaFace, a C++ based open source face recognition engine. R-CNN was used here because of its robust capability for face detection. 4 K High Definition (HD) video camera was used for image acquisition to overcome the limitation of low resolution. The model achieved a satisfactory result for the class attendance system with Faster R-CNN and SeetaFace. The proposed model wasn’t cost-effective because it used costly hardware and required a high-end Graphics Processing Unit (GPU) for computations. The computation was not in real-time.

In [6], the authors proposed a prototype of a model that used the Inception V3 model for recognition, which was created and trained at the time of student registration. This model used the Maximum Margin Face Detection (MMFD) technique. MMFD was used by using Histogram of Oriented Gradients (HOG) with Maximum Margin Object Detection (MMOD). The image acquisition in this model took place with the help of a rotating HD camera. In this work, the author also compared the accuracy of Haar Cascade (94.71%), Viola-Jones (94.77%), HOG (96.40%), and MMFD (99.74%) for face detection. This model had an advantage over the models in [15, 19, 23, 31] as it did not require an internet connection. The limitation of this work was that it required a high Graphics Processing Unit (GPU) and high-end computers.

In [9], the authors proposed a prototype of a model that used the Viola-Jones algorithm for detection and creating a feature vector. For image classification, it used Linear Discriminant Analysis (LDA), Linear Regression (LR), and KNearest Neighbors (KNN). It was one of the first secure models to use an android application for GUI. The android application was used to generate QR Code by the lecturer, and students had to scan the QR code and verify their faces for marking of attendance. In this work, the author showed that KNN and LR outperformed LDA.

3 Review of Face Detection and Recognition Techniques

After the capturing process, the faces of the students are detected from the captured images or video frames. The detected faces are further processed and are identified using the existing database. Different approaches and their combinations have been used in face recognition based attendance systems to boost performance. Table 1 covers eight papers in this area. An overview of all the papers has been given with the choice of algorithms, results, and limitations.

Table 1 Comparison of face recognition strategies utilized for face recognition based attendance management system
Full size table

In the deep learning-based methods, the use of Artificial Neural Network (ANN) has proven to be more superior in terms of recognition accuracy compared to other methods. It has been utilized in [11, 18, 19, 24] giving a better system performance. On top of that, the combination of the traditional methods and ANN-based methods has also been utilized in [8, 12, 15,16,17, 22]. This combination has given some tremendous results in terms of accuracy of face recognition. Various papers have been proposed with Principal Component Analysis (PCA) with Convolutional Neural Network (CNN) [17, 22]. Support Vector Machine (SVM) with CNN [12], Histogram of Oriented Gradients (HOG) with CNN [10], K-Nearest Neighbours (KNN) with CNN [16]. Each of the works shows that the accuracy of the system is better than the previous works in which ANN was not used with the respective traditional methods. A more detailed analysis of the features and limitations of several papers has been performed in Table 2.

Table 2 Comparison of previous works on face recognition based attendance management system in terms of capturing method, features, and limitations
Full size table

4 Review of Capturing System

The first part of any face recognition based technique is the capturing system. It is very essential to have a proper capturing system that captures each of the faces. The attendance management system using face recognition uses a camera for image acquisition. In some systems [6, 19, 25] smartphone camera was used for image acquisition for the attendance system. In [8, 12, 26] they used video camera for capturing video of the student’s sitting in the classroom. They used the captured video for the detection of faces in the frames and recognition. Cloud controlled camera [19], 4 K High Definition (HD) camera [11] and multi-camera setup [23] have been used in some proposed systems. The concept of periodic attendance [3, 27] and random capture were also introduced to increase the security and reliability of the systems. Table 2 contains the list of capturing systems used for image acquisition in different proposed works on face recognition based attendance management systems.

5 Review of Storage Methods

It is very important to store the attendance in a manner that suits the system and increases the overall performance and usability of the system. Authors have used different methods in the proposed systems for storing attendance. The main aim is to mark and store the attendance for recognized persons in a way that helps in getting insights and information from the data easily. In most of the proposed systems [2,3,4,5, 22, 23] a Structured Query Language (SQL) based database is used for storing the attendance. Also, in some systems [25] NoSQL database is used. Storage system used either work locally [15] or through a server [2,3,4,5, 7, 23]. Each of the methods used for the storage system has some advantages and limitations when compared to the other. The choice of the method depends upon the requirements and expectations of the system. A proper storage system, when used with a face recognition based attendance system can add more functionality and features to the system.

6 Some Other Approaches

Besides the face recognition based method for student attendance management, [28] suggests an attendance method that uniquely identifies wireless devices using Radio Frequency Received Signal Strength (RF-RSS) frequencies. It uniquely identifies a student’s registered smartphone device in the specified area. The received RF-RSS frequencies are assembled and are formed as time series. Similarly, [29] analyses the practice of Bluetooth Low Energy (BLE) beacons for attendance management. The paper [29] also studies the vulnerabilities of BLE based attendance management system and suggests the measures to overcome them. In [30], the author proposed a model of anti-cheating smart attendance management using Radio Frequency Identification (RFID). Similarly, [31] proposed a two-factor authentication architecture RFaceID based on an RFID tag and a cascade of two face recognition stages. They discussed its performance when applied to VidTIMIT, Extended Yale B, and MOBIO face datasets. Besides these works, [32] proposed a more advanced model that not only maintains attendance but also optimizes classroom usage by using the Internet of Things (IoT) and Artificial Intelligence (AI).

7 Results/Findings

The above-carried work aims to compare the previous works on student attendance management that are based on face recognition. Each of the proposed methods has some advantages and some limitations. The methods vary in the choice of hardware, and algorithms used. Most of the authors have used ViolaJones, eigenfaces, fisherfaces, and HOG-based algorithms. Also, PCA and LDA have been used for dimension reduction. SVM and KNN have been widely used for the classification of the feature vectors. The problem with these algorithms is poor accuracy. Also, there is a light sensitivity issue in the case of LBP and LBPH. The neural network-based methods have also contributed to the attendance management systems. The above study shows that CNN-based methods have given tremendous results when combined with the traditional approaches because of the strong classifier. The main challenges with the practical implementation of the CNN-based methods are the high associated cost and high GPU requirements for real-time computation. There is a trade-off between speed, cost, and efficiency. Also, most of the proposed models depend on network connection for working. Therefore, the need for a cost-effective student attendance management system with good efficiency and ability to work in the absence of network connection still exists.

8 Conclusion and Future Work

To improve the existing face recognition based student attendance management systems, there are some points on which one can work. In [17, 22] ANN with PCA was used to solve blocking issues of attendance management system using face recognition. However, it still had issues with the accuracy and performance of the system. In the future, Fast PCA with back-propagation can be used to resolve this issue. Also, different image processing algorithms can be tried and used to increase the accuracy of the model and make attendance monitoring more efficient without compromising the speed and memory requirements. Cost-effective and network independent models can be proposed without compromising with efficiency.