Keywords

1 Introduction

Traditionally, the attendance of students is taken by calling out each student names and marking it out on the student’s attendance registers which is provided for the faculty members. This method may sound easy and less complicated, but the problem arises when the number of students are more it may result with errors in the registers; it is common for humans to make mistakes, while being in a rush to avoid this, we can use face recognition application where the data of each students are done when they come in front of the camera and click recognize, and it will be stored in the data automatically by recognizing them from the faces which is been already stored in the data with faces.

This method has many advantages like its time efficient and storing of data doesn’t cause any mishaps. One more advantage is there won’t be any proxies. When the process is completed, the teacher must click on mail where the database of attendance will be sent to the faculty, and the students who are absent the respective parent will receive the message.

2 Problem Definition

Though many systems are developed for face recognition, but still, it has many challenges like in computer vision and recognition of pattern researches like features are computed slowly, it needs auxiliary information and for broad practical applications. The problem of face recognition can be stated as follows: face recognition human facial features like the mouth, nose, and eyes in a full-frontal face image.

3 Proposed System

The process is done by two algorithms which is haar cascade, used for face detection and lPBH, used for face recognition. The application is created using Python module in graphical user interface. Tkinter is the package used which makes it easier in labeling and other things.

The process follows totally 3 steps first by taking image of the students along with their details and storing them in the database and training those images in the database finally testing them, so when the student comes in front of the camera, it tries to recognize them by comparing them from the database which as the faces already trained and tested and enter them into the attendance sheet. If the person doesn’t belong to the database, it will be stored in the unknown image file. The stages are clearly explained in the further sections.

4 Dataset and Methodology

Face recognition-based attendance system: The graphical user interphase of the system has been shown in Fig. 1. The system has been divided into 3 sections: creation of database, dataset training and dataset testing, sending alert messages in an extension.

Fig. 1
A screenshot of the face recognition software's data input page. The requested information includes student ID, name, mobile number, parent's name, and email address.

System GUI

Full size image
  1. (1)

    Creation of Database

  2. •

    Initialize the camera and to get the attention of student’s alert message is set

  3. •

    Input of user id

  4. •

    Conversion of the image from RGB to gray scale, face detection and

  5. •

    It will be stored in the database by labeling them up to 100

  6. (2)

    Dataset Training

  7. •

    Recognizer of LBPH face will be initialized

  8. •

    Next, faces and ID from the folder are taken to train the LBPH recognizer

  9. •

    The trained data will be saved as xml or yml file

  10. (3)

    Dataset Testing

Loading from xml and yml of haar classifier, LBPH face recognizer and data that has been trained is done.

  • the image will be captured from the camera,

  • next convert them into gray scale,

  • the face will be detected from it,

  • Finally, prediction of the face is done by using above recognizer.

The system in this uses haar cascade algorithm in detection of face which in turn uses modified haar cascades in detection. We can use any camera like USB Webcam to capture photos. The system will access console of SSH in laptop. Most important of all, it will require a lot of positive and negative images in training the haar cascades. In which, the positive images are pseudocode for the attendance system.

Input: Real-time video of the student’s face will be taken output: Excel sheet of the student’s attendance.

  1. (1)

    Each frame of the image will be transferred from RGB to gray scale

  2. (2)

    Next, apply haar cascade classifier for face detection and get the region of interest

  3. (3)

    Now, we need to apply the LBPH algorithm on the region of interest to get the features

  4. (4)

    If it is for enrollment, then features will be stored in the database else if it is for verification then do the post-processing

Haar Cascade Algorithm

Haar cascade is a machine learning algorithm used for detection of objects in an image or video. It is a process where a cascade function is used to train a lot of positive and negative images.

It is used for detection of objects in images. It will also be able to identify any kind of object.

As per Fig. 2, the first step is to collect all the haar features. It selects those features which are adjacent to the rectangular regions at a specific location in the detection window. Then, it sums up all the pixel intensities in each of the region and calculates the difference between these sums.

Fig. 2
An illustration of how the Haar cascade algorithm works. As input, a function receives positive and negative images. The output is passed to a cascade classifier that returns a system object.

Haar cascade algorithm

Full size image

Loading the dataset

We will plot the first image in our dataset and check its size. By default, the shape of every image in the dataset is 141 × 141, so we will not need to check the shape of all the images. Sometimes, it depends on the size of the face and distance from the camera.

Data preprocessing

We need to have a target variable. That means the column will get created for each output, and variable will be assigned to each of them; the first layer will be also be taken in an input shape. This is the shape of each input image, 141, 141, 1 as seen earlier on, with the 1 signifying that the images are gray scale.

Training the model

Now, it will train the model. For training, we will use the ‘fit()’ function in the model with the following parameters: Data training will be taken as x; target of data will be taken as y and validation data, and all the number of epochs. For the data validation, we going to use the test set provided in the dataset, which we have split into x and y. Now, this will be using the model to make predictions.

Local binary patterns histogram

The goal of face detection is to detect and locate faces from the image, to extract and to use the in other areas. Also, nowadays, there are many different algorithms to accomplish this face detection and recognition, such as Eigen faces, fisher faces, scale-invariant feature transform and speed up robust features. In this section, we going to use LBPH-based face detection algorithm as depicted in Fig. 3. LBP is an easy but powerful way to extract all features and label those pixels from the image.

Fig. 3
A flowchart starts from an input face image, divides the face image into blocks, calculates the histogram for each block, combines L B P H into a single histogram, face image processing, and recognition results.

LBPH algorithm flowchart

Full size image

The algorithm used to follow this step

  1. (1)

    Initializing them temp = 0 is the first step

  2. (2)

    where I those are the training images

  3. (3)

    where H = 0, then initialize them to the pattern histogram

  4. (4)

    The model of the label of LPBH is calculated

  5. (5)

    Keep adding the corresponding bin by one

  6. (6)

    Next step is to get the greatest LBP feature during each of the face image and later merging them into the unique vector

  7. (7)

    Then, it is time to compare all those features.

  8. (8)

    Last step, finally, when it resembles with the image that is stored in the database, the image will be recognized

5 Results and Outcome

In the experiment, first, we need to create a dataset of each student which will have a distinct ID number with an image of a face in the database, and then, those features of the face from all the image will be extracted At the end, it gets segregated, and recognition of those face of each student along with information will be done (Figs. 4 and 5).

Fig. 4
The data flow diagram of the proposed system presents the steps involved in recognizing a student's face and processing their attendance.

Data flow diagram of face recognition

Full size image
Fig. 5
A sequence flow diagram of the student's face and attendance involves faculty, students, capturing, processing, comparison modules, and database.

Sequence diagram of face recognition

Full size image

Now, it is required to compare the input image of the face which has been detected and extract all the facial features and compare them with the face images in the database; if failed to recognize, they will be stored in the unknown images as shown in Fig. 6.

Fig. 6
A screenshot of the proposed system's U I page has a photograph of a man labeled unknown, and the input values I D, name, date, and time are displayed as null.

Unknown person

Full size image

Using the algorithms, the details of all the faces of each student known and unknown will be compared that have been trained in the system. In this research, three major tasks have been performed those are face detection, train, and finally recognize the face of each student from the real-time video camera.

  1. A.

    Face detecting and preprocessing

In this step, the system will detect the face as image as the input with real-time camera with less resolution or high resolution depending taken individually or in group.

Firstly, we need to convert all those frames from RGB to gray scale to do the detection of the faces. We will apply haar cascade function that has been trained and detect those features in other images. In the system, we will use haar features like edges, lines, and four rectangles that are nose, mouth, and eyes. For those large or huge image of a variable size of an image, it will take or need a lot of computations, and all those features most of the time will be irrelevant. But, by using AdaBoost, we will select the best among those. Then, region of interest like which will contain faces is extracted and will be sent to the next stage as shown in Fig. 7.

Fig. 7
A screenshot of the proposed system's U I depicts the photo of a person. It highlights a person's face with a solid line box.

Detection of face

Full size image
  1. B.

    Images of face is trained

First, the image is captured, and we will be preprocessing them (Fig. 8); later, we need to train those images in the dataset. In the training phase, those images of recognition will be applied that will get stored of the values of histogram of images of the face (Fig. 9).

Fig. 8
A screenshot depicts the student's face extraction and preprocesses storage location. It exhibits 61 facial reactions of a student.

Extraction and preprocessed faces of those students stored in the dataset

Full size image
Fig. 9
A screenshot of the proposed system's U I has a file on the left. It highlights a person's face with a solid line box.

Dataset training

Full size image
  1. C.

    Post-processing and recognizing image of the face

The last and final task is to recognize the image of the face. The cascaded haar classifier and training them to recognize and training recognition will be used for the recognition of the face. Then, at the end, classifier will be comparing those stored images of face with the input images of face, and if all the features of input images of face get matched with in the images stored in the database, the recognition of face result gets displayed with real-time video camera of the screen with name and university seat number (Figs. 10 and 11).

Fig. 10
A screenshot of the proposed system's U I has a file on the left. It highlights a person's face with a solid line box with the student I D and name.

Recognizing face images

Full size image
Fig. 11
A screenshot depicts two spreadsheets. The first is a student database with the student I D, name, phone number, parent name, and email I D. The next is the attendance sheet after completion of the program.

Attendance sheet absent list of students

Full size image

We can also get an absent list on teacher’s mobile number as well as on parent’s mail ID by choosing following options (Figs. 12 and 13).

Fig. 12
Two screenshots of the proposed system's U I and an opened email depict the email notification functionality when a student is absent for a class. It is sent to the student's parents.

Absent list for parents

Full size image
Fig. 13
A screenshot depicts a message of teachers absent with today's attend and dated attendance. It exhibits a message as you have selected.

Absent for teachers

Full size image

Teacher can also get an absent students list on their mobile number for that same day as well as previous day.

6 Conclusion

Compared to all other algorithms, this one is much faster and has less false rate. This application is very much useful in schools and colleges since it saves time of teacher which will be wasted in talking attendance manually; thus, even preventing from proxies and even parents can have a track of their child attending each classes or not. Since messages will be automatically generated and passed to parents if the child doesn’t come under the database of that class and teachers or faculty will have the complete details of students who are absent with a Excel sheet to their mails which has been stored in the database already. The great advantage of this application is, it will be able to recognize the face of the students even when there is a slight changes like sometimes students wear glasses or arrive without them and if they have beard or not also it can recognize them easily.

When the students come in front of the camera, they will be recognized with the images that have been trained; even when it fails to recognize them, it will be stored in unknown images which later can be used to check for errors and managed attendance according to it.