Keywords

1 Introduction

Modern web applications and web information systems are complex and sophisticated solutions that typically process information with different levels of confidentiality. Such solutions often become targets of various hacker attacks when the attackers try to obtain the data circulated in web information systems by exploiting the weaknesses of web solution algorithms or bugs in software. According to sources [1,2,3], faults related to the authentication stage (or “Broken Authentication” in [1]) are rated as the second among the top 10 web application security risks, with up to 45% of similar vulnerabilities found during the conducted analysis [2]. Besides, there are several essential moments mentioned in [1] as crucial, like usage of well-known passwords, weak password database protection, susceptibility to brute-force attacks, and (what is important) missing or non-effective multi-factor authentication.

Using multi-factor authentication and adopting additional authentication factors can indeed increase the efficiency of the identification and authentication stage of web applications and web information systems [4, 5]. The research report “The State of Strong Authentication 2019” produced by Javelin Strategy and Research [3] states that there are many widely used supplementary authentication factors (that follow the first traditional password factor), such as pre-generated one-time passwords, SMS passwords, hardware cryptographic keys, and security questions. At the same time, biometric information inextricably linked with the web system user should also be considered the reliable authentication factor. Nowadays, fingerprint scanning (44%) and face recognition (10%) are actively used due to the widely spread mobile platforms already equipped with the necessary scanners [3, 6].

Biometric authentication requires certain cooperation on the user side and hardware and software support to acquire the user’s biometric data. However, modern smartphones, laptops, tablet computers, and workstations are capable of obtaining various biometric data easily with minimal cooperation of the user. For example, behaviometric data can be collected silently, and the voice and face image of a user can be acquired with a built-in microphone or camera and simple instructions. The latter (face image, in particular) appears to have the potential for being used in biometric solutions and systems.

Not all current biometric face identification and recognition solutions can be considered open source and free to use [7]. It is also noted that their functionalities differ significantly. Some solutions are inclined to deprecated methods of utilizing multimedia devices and require additional supporting software components to be downloaded or installed on the user side.

Therefore, it is of importance to propose a solution for web information systems with single-factor authentication. The solution should introduce an additional authentication factor based on the biometric face recognition and be open source. Several requirements should also be considered, such as the capability to be incorporated into the existing web system authentication procedure, support of the standard web browser multimedia functionality, and usage of available (or built-in) image capturing devices (photo and web cameras).

2 Solution Design and Software Implementation

2.1 Concept of the Solution

The proposed solution includes all the necessary steps and modifications for the user side and server side of the web information system. The overall design is shown in Fig. 1.

The solution can be incorporated into the web information system algorithms right after the first authentication step. The starting point of the proposed solution, in general, does not require any specific parameters for its operation. However, the solution should comply with the already implemented web information system application program interface (API). If several parameters inherited from the previous step have to pass through to the web system operation step after authentication, then they should be collected and passed at the exit point of the solution. Thus, compatibility with the web system API is achieved.

Fig. 1.
figure 1

The proposed solution overall design and its structure elements.

Full size image

The proposed solution starts the pre-arranged HTML code on the user side to initiate the biometric authentication step. The HTML code contains verbal instructions for users and program instructions to activate available multimedia devices on the user side for face image capturing (typically, a web camera or frontal photo camera). The web browser on the user side fully handles all operations with the multimedia devices. The introduction of the HTML5 standard enhanced web browser multimedia capabilities significantly [8, 9]. Support of video streaming, capturing images from video streams, and operating the built-in multimedia devices on mobile platforms are of particular interest. WebRTC (Web Real-Time Communication) is a technology that enables Web applications and sites to capture and optionally stream audio and/or video media, as well as to exchange arbitrary data between browsers without requiring an intermediary [9]. Access to the data from photo and web cameras is provided by the getUserMedia() function that is supported by almost all latest web browsers on a variety of platforms running various operating systems [10] (Fig. 2).

Parameters of the getUserMedia() function can specify the data source and the desired format and quality of the video stream data. The JavaScript language is used to handle the operations. Program instructions necessary for engaging the camera and transmitting the acquired face image to the server side are incorporated into the web page designed for the biometric authentication step on the user side.

Fig. 2.
figure 2

Support of getUserMedia/Stream API by web browsers [10].

Full size image

The captured face image is transmitted to the server side for processing and analysis. The server side algorithms can be developed separately and assembled into a complete project later [11]. Source codes of the algorithms can be produced independently or taken from the appropriate software library.

2.2 Processing Algorithms and Methods

Comprehensive studies [7, 12] suggest that the open source computer vision library OpenCV [13] and its PHP frontend PHP-OpenCV seem to be among the most effective open source software libraries for video and image processing and analysis. OpenCV includes several hundreds of computer vision algorithms, and there are many adaptive (trainable) algorithms based on artificial neural networks (ANN). These algorithms are highly effective for processing images, calculating specific features (feature vectors), localizing and detecting objects in images. Therefore, it seems prospective to utilize the algorithms included in the OpenCV software library.

In this study, the performance of several face localization methods is tested and evaluated to select the best performing method for its further implementation as the main one. The following face localization algorithms available in the OpenCV library are utilized: linear binary patterns (LBP) [14], FacemarkLBF [15], Haar cascades [16]. This part is necessary due to the importance of successful face localization. Once the face is localized in an image, it is possible to continue with face recognition. Otherwise, appropriate actions should be taken to acquire a new face image.

Face recognition is conducted using the trainable classifier based on the local binary pattern histograms (LBPH) [14, 17,18,19]. The LBP operator [14] is applied to the localized face image to calculate feature histograms representing local texture and shapes over the processed areas. The localized face image is divided into small regions from which LBP histograms are extracted to produce the resulting feature histogram vector (feature vector) (Fig. 3) [19]. Calculated feature vectors for successful classifications are stored in the Biometric feature database (when the user face image is processed for the first time) for further accumulation and enrichment of the training dataset.

Fig. 3.
figure 3

Calculation of the feature vector using the LBPH method [19].

Full size image

The transmitted image undergoes normalization of brightness and contrast before being passed to the localization step. Unsuccessful localization of a face in the image or localization of the facial profile leads to disqualification of the image under processing. In this case, all operations start from the beginning, and the user side should provide another face image. The overall flowchart of face image processing and analysis is shown in Fig. 4.

3 Testing and Evaluation

Several available algorithms are taken for testing and evaluation of their performance to select the most efficient ones. These algorithms provide face localization and detection in processed images which can be acquired in various environments and be of different quality.

Testing and performance evaluation of face localization algorithms is conducted using the assembled set of 514 images of various faces. The assembled set contains images from several groups, and each group has almost the same number of images. These groups are named according to the specific ‘trait’ that all images within the group possess. The assembled set includes images from the following groups:

  • ideal images with a proper light balance and faces positioned at the center (“Ideal”);

  • images with faces out of focus or images contain parts of faces only (“Part-Face”);

  • images with partially covered faces (“Hidden”);

  • images with excessive darkness or images with shadows on faces (“Dark”);

  • images with excessive light (“Light”).

Experimental results are shown in Table 1 with percentages that represent successful face localization for each group.

Fig. 4.
figure 4

Flowchart of face image processing and analysis.

Full size image
Table 1. Experimental results of face localization for several groups of face images.
Full size table

“Haarcascade alt” and “Haarcascade alt2” algorithms demonstrate good results with clear tendencies to localize small faces or embossed objects. These algorithms provide the best results when processing images with excessive light or images from the “Part-Face” group. The algorithms are suitable for processing images acquired using mobile platforms. The “Haar cascade alt tree” algorithm shows good performance when dealing with images with partially covered faces or with faces out of focus.

The “LBF” algorithm is able to detect facial key points and fit for detailed localization. However, “LBP” and “LBP improved” algorithms demonstrate accurate and stable results for all groups of images in the assembled set. Thus, the “LBP improved” algorithm is selected as the best performing algorithm for its implementation as the main localization algorithm of the proposed solution.

The LBPH-based trainable classifier is used further to perform face recognition. In this study, the LBPH based classifier produces two possible outputs – “affirmative” (recognition is done, even if the face is matched incorrectly) and “negative” (face is not recognized / no match found).

There are two experimental samples prepared to train and evaluate the LBPH based classifier. Test sample “A” contains 58 various face images of the same person taken under various conditions. These conditions are equivalent to the ones for the image groups of the previously discussed assembled image set for localization testing. The training sample includes 20 images from the sample “A” and 100 images of other (different) persons.

Test sample “B” contains 50 various high-quality face images of the same person. There are no images with partially covered faces or images with faces out of focus in this test sample. However, images with excessive light or darkness are included in the test sample “B”. The training sample for this case includes 20 images from the sample “B” and 100 images of other (different) persons. All images comply with the conditions mentioned earlier.

Table 2 demonstrates the performance results (in percentage) of the trainable LBPH-based classifier with the two experimental samples.

Table 2. Performance results of the LBPH-based classifier.
Full size table

Here, the “1st type error” stands for erroneous recognition of face images belonging to one user as belonging to another. The “2nd type error” stands for unsuccessful recognition when the face is not matched with the face of any user.

Results of the experiment demonstrate a high rate of correct face recognition. It can be seen that the most optimal case would be direct utilization of two output classes and adoption of the “1st type error” results as successful face recognition. However, the biometric authentication factor, in this case, should be interpreted as a very soft authentication step with only two outcomes:

  1. 1.

    the face is matched, and so the user is known to the web information system;

  2. 2.

    the face is not matched, and so the user is unknown to the web information system.

The rate of correct face recognition can be increased by using more robust adaptive algorithms and models [17]. Also, the proposed biometric authentication step can be improved by utilizing additional biometric data and processing algorithms. For example, face recognition can be supplemented by iris scanning [20] or any other biometric technique.

After the biometric authentication step is acknowledged as successful, the control returns to the web information system algorithms at the point located right after the initial authentication procedure of the web information system.

4 Conclusion

This paper presents the solution that extends the capabilities of web information systems with single-factor authentication by introducing an additional authentication factor based on biometric face recognition. The proposed solution is based on algorithms from the open source software library. It uses the standard web browser multimedia functionality and available (or built-in) image capturing devices (photo and web cameras).

The proposed solution can be easily integrated into existing authentication procedures of web information systems. Face localization and recognition algorithms are based on adaptive ANN techniques and demonstrate a high success rate. They can be replaced during further operation by more robust and prospective ones to fully satisfy the specific requirements of a particular web information system and incorporate all achievements in the area of biometric data analysis and face recognition.