Population Index and Analysis Based on Different Geographies; Using Distance Measurement, Social Distancing, and Deep Learning

Abstract

Nowadays, every individual is familiar with the COVID-19 pandemic which has caused great turmoil in everyone’s life. Also, they are aware that there is no medicine or drug to cure COVID immediately, and people are at the risk of losing their lives. Lack of vaccines or delay in vaccine production for mass results social distancing being the only measure to tackle this pandemic. As a result, social distancing has proven to be a very reliable and efficient way to diminish the growth of this disease; the reason why lockdowns are imposed, and people are asked to keep some distance from each other, for their safety as there will be minimal physical contact. Machine learning and artificial intelligence come into the picture in every solution to a generic problem the community faces nowadays like in medical, supply chain management, face detection, etc. Using the power of AI algorithms, the paper aims to develop a robust system to monitor and analyze social distance measurement protocols at public places during the COVID-19 pandemic with the help of CCTV feed and check whether they abide by the safety protocols or not by measuring the distance between them. The proposed approach is implemented to enumerate the number of violations at a popular public place to prevent massive crowds at particular periods. The proposed method is suitable to construct a scrutiny system at a public place to alert people and eschew mass gatherings that can be concluded using achieved results. The paper also has an analysis of the performance of different models of R-CNN, Fast R-CNN, and YOLO. YOLO architectures are validated based on object detection and object tracking rate in real time.

1 Introduction

To control the spread of COVID, “social distancing” is without a doubt the best measure to do unless each person gets fully vaccinated. Social distancing is observed as an unprecedented measure by all the health agencies. That is the reason, countries across the world had to bring nationwide lockdowns for months which surely slowed and diminished the growth of the virus in public. Taking the example of the first lockdown in China’s Hubei province in January 2020, the confirmed cases were doubling up each day in February by forcing a compulsive lockdown, and then for 5 consecutive days in March, they did not get a single new confirmed case. This proves that social distancing when enacted in China reduced the cases; hence, this measure was adopted worldwide.

1.1 How is Social Distancing Efficient?

The COVID-19 virus is spreading easily with physical contact among people. According to the World Health Organization (WHO), “COVID-19 is transmitted via droplets and fomites during close unprotected contact between an infector and infectee”. Fomite is the concept of an object being probable to transmit the virus due to contact with it. Fomite may include utensils, furniture, food, clothes, etc. Thus, keeping a social distance from people and fomites by avoiding proximity is the main step to avoid COVID-19.

1.2 Social Distancing

A minimum of 2 m distance from another individual is recommended by experts. Social distancing is a way that does not use any medicines or antibiotics and is still the strongest way to counter COVID-19. The long-term goal is to bring back normality in our way of life and enjoy our freedom. Social distancing protocol can be ensured in public places and the workplace. To do that, the research paper implements human detection using advanced AI algorithms and calculates the distance between these people in real time from live camera footage to check whether people are abiding by safety protocols or not. The work carried can be used at various places like workstations, offices, schools, academies, hospitals, restaurants, etc., thus building a safe environment by managing efficiently. The different ways in which social distancing can be achieved has been illustrated in Fig. 1.

Fig. 1

Various ways to achieve social distancing

2 Research Background and Relevant Work

The research paper presents solutions to avoid social gatherings for saving the lives of people at public places by utilizing real-time pedestrian detection and distance measurement methods to calculate the number of violations at the time. The authors are willing to suggest that the reason behind increasing the infection and spread of the virus among people is avoiding social distancing norms. Figure 2 consolidates the need of pedestrian detection and distance measurement, object detection, and social distancing as the main factors of the model needed to avoid the spread of COVID-19.

Fig. 2

Need of the model

2.1 Pedestrian Detection and Distance Measurement

The deep learning-based methodology includes a video frame from the camera as an input and the pre-trained open-source object detection to assess the distance between pedestrians for determining the social distancing measures in a video to assuage the impact of the COVID pandemic [1]. Punn et al. [2] An obtained result from the suggested framework is compared with faster region-based convolution neural network (CNN) and single-shot detector (SSD) in terms of object classification and localization which are loss values, mean average precision (mAP), and frames per second (FPS). The result shows that YOLO v3 displayed superior outcomes with a balanced mAP and FPS score to monitor the social distancing in real time. Nguyen et al. [3] proposes technologies like machine learning, computer vision, ultrasound, thermal, etc. It presented and reviewed various technologies with an overview, inspected the state of the art, and considered how it can be utilized to uplift and promote social distancing measures and facing open issues in it with latent solutions to address it as per Fig. 3.

Fig. 3

Need of pedestrian detection and distance measurement

2.2 Object Detection

Detecting instances of perceptible objects of a certain class in digital images (such as plants, humans, animals, birds, or cars) in object detection is an important computer vision task. It can be grouped into two subtopics: cognition and “general object detection” include methods to simulate the human vision under a unified framework for detecting different types of objects, and “detection applications” include specific application scenarios for detection, such as face detection, text detection, and pedestrian detection [4, 5] as per Fig. 4.

Fig. 4

Need of object detection

2.3 Social Distancing for COVID-19

The paper presents a scientific study of busy hours at public places to avoid the glut of people. The economic impact of social distancing measures targets decreasing infections among the people. Studies have proved that social distancing measures fall off the aggregate economic loss during the lockdown [6]. Many scientists and researchers put their best foot forward to find a solution to stop the spread of COVID, and many articles have been published related to severe acute respiratory syndrome (SARS), Middle East respiratory syndrome (MERS), and COVID-19 [7]. Automating diagnosis of COVID-19 consisting of computed tomography (CT) and chest X-ray (CXR) imaging and using a deep learning approach to get better results is commendable and can be found in [8] and Fig. 5.

Fig. 5

Need of social distancing

3 Object Detection and Distance Measurement

3.1 Object Detection

Object detection is used to detect objects in the given frame and to place a bounding box around each detected object in that frame. Deep learning is a powerful method for object detection.

Algorithm Selection:

3.1.1 R-CNN

Region-based CNN is a method based on selective search. The main principle used behind this method is to extract limited regions from the image. Thus, the research works on these limited regions. The regions which are predicted can be overlapping with each other and may be vary in size as well. So, based on intersection over union (IOU) score: Where there is minimum suppression that region is used to ignore the bounding boxes.

3.1.2 Fast R-CNN

The fast R-CNN method follows the R-CNN algorithm. Feature extraction in the original image takes place only once. Then based on the location of the region proposals, the relevant ROI features are chosen.

3.1.3 Haar Cascade

The “Haar wavelet” is the origin behind the Haar cascade classifier. To analyze pixels into squares by function in the image, this technique is used, which uses “integral image” ideas to figure out the “features” detected from an image. The algorithm used by Haar cascades includes the selection of a small number of essential features from a wide dataset that is already provided and the result of classifiers, i.e., well-organized additionally detection of face using various cascading techniques in an image.

3.1.4 YOLO

The you only look once (YOLO) algorithm, shown in Fig. 6, is among the finest algorithms for object detection in the current era and works in a completely different manner than what traditional algorithms used to have. As the name suggests (you only look once), in a single view, the algorithm is able to predict and give results. The bounding boxes are created using the method of probabilities and thresholds. This algorithm is used for real-time processing for object detection.

Fig. 6

YOLO for object detection

R-CNN and fast R-CNN are selective search algorithms, and they slow down the performance and are time consuming algorithms, while YOLO predicts with single network evaluation, in contrast to others which require substantial amounts for a single image.

As per Fig. 7, in object tracking first, a new ID is given to every detected person, and then, a bounding box is drawn around each detected person and has to measure centroid for each bounding box. In the next frame, the purple one is the old centroid, and the yellow one is the new centroid. To find out which person has moved, where or which two persons are the same person and then calculate Euclidean distance between every old centroid to every new centroid, and the close pairs will be detected as the same person. When the centroid disappears from the frame, if any is not found for consecutively 50 frames, erase the ID from our array.

Fig. 7

Object tracking

3.2 Distance Measurement Between Detected Objects

• 1. F = (P × D)/W

• 2. D’ = (W × F)/P

After a deep understanding and discussion of all these methods, the conclusion is that the paper is best suited by using the YOLO algorithm. The Figs. 7 and 8 clearly describe how YOLO works and how it suits the vision of this project to bring it to fruition.

Fig. 8

Distance measurement formula

4 Proposed Methodology

The paper proposes a solution-efficient proposal to use the YOLO algorithm to detect the crowd density at an area chosen by the user, which follows COVID-19 safety protocols by maintaining social distancing norms. The ML model also finds violations over some time and presents it to the user for a gasket of pre-verified data of population index for a particular area for them to plan their future visits to that particular area with utmost protection as per Fig. 9.

Fig. 9

Overall working methodology

4.1 Stage I: Detecting Violators

In Fig. 10, the stage mainly focuses on detecting pedestrians in the frame and then calculating the distance between every pedestrian who is detected in the frame, and then, it will show how many people are at high, low, and not at risk.

Fig. 10

Detection of violations

It contains the following features:

• 1. To get a camera frame or video feed.

• 2. Deep learning is firstly used to detect and then to localize the pedestrians in the frame.

• 3. Find the distance between pedestrians: To plot pedestrians equidistantly using the Euclidean distance formula and thus calculate the approximate distance between them.

• 4. Find and detect violators who do not follow the minimum distance rule.

The respective flowchart of object detection to detect if it is at safe or unsafe distance has been illustrated in Fig. 11.

Fig. 11

Stage 1—flowchart of object detection

4.2 Stage II: Finding Busy Hours

Figure 12 shows a stage mainly focused on findings of busy hours based upon the number of violations at a particular time to avoid overcrowding at public places. It contains the following features:

Fig. 12

Busy hours in a day

• 1. On a particular day and time, using a camera frame, detection of people can be done by using the YOLO algorithm.

• 2. After detection, distance is measured between two humans, and if that distance is less than the safest distance, those humans have violated social distancing norms.

• 3. If the number of violators is more in a particular region, then that region is occupied by a large number of people.

• 4. Compared to the capacity of the region, people are more so they are not following social distancing. One could avoid going to these particular regions where there are more violators, or they can choose different timing to visit that place.

The respective flowchart of detection of busy hours to further detect if it is at safe or unsafe to visit the place has been illustrated in Fig. 13.

Fig. 13

Stage 2—flowchart of busy hours

5 Experiment and Result Analysis

5.1 YOLO Outputs

5.1.1 Ideal Case

In this Fig. 14, humans are detected in the video as being sufficiently far away from each other. That is why a green bounding box is created to showcase that they are socially distant.

Fig. 14

Ideal case of social distancing

5.1.2 Intermediate Case

In Fig. 15, the paper shows an observation that there are a few violations showcased by the red bounding boxes. Our project keeps a record of these violations with their timestamp.

Fig. 15

Intermediate case of social distancing

5.1.3 Worst Case

In this Fig. 16, all people detected in the frame are violating the social distancing norms.

Fig. 16

Worst case of social distancing

5.2 Database of Violations

Using a MySQL and Python connector, database is created which is used to store the number of violations at a particular time (Fig. 17). It also helps to give an idea of how many people are maintaining social distancing norms and able to monitor COVID-19 social distancing violations more efficiently.

Fig. 17

Database of violations

5.3 Busy Hours

The proposed method has been implemented at a famous cafe in Pune, India. The following graph shows the busy hours of that cafe in terms of people visiting from Monday to Friday. It also shows an average of all the days for a better observation.

It is observed that in a pandemic, people are gathering more at public places. To avoid this, one could use these visualizations and analysis and avoid going to that particular place at that specific time and choose a less crowded time as per their convenience.

At this place, human detection is carried out via video frames for a while from Monday to Friday during working hours; and after detection, distance is measured between the two humans. If the distance is less than the safest distance, a violation occurs. If more violations are observed, that means the number of people present at that place is more. So to follow the social distancing norms, one could avoid such situations via these visualizations and find a better and suitable time according to their comfort and convenience. The research was conducted at a place having an area equal to 5293 m² where FSI is calculated to around 2.5. Considering the rules and regulations given by the government and the capacity of the restaurant, 150 people can be accommodated at a particular time under normal conditions. But in case of a pandemic according to the government’s guidelines, restaurants should operate with 50% of their full capacity [9]. So, let us consider two pedestrians that are 6 feet apart, 50 people approx. can visit the place at the same time (with 0 violations). As per the Figs. 18, 19, 20, 21, 22 and 23, as the number of people visiting goes on increasing, number of violations also increases because the distance between two pedestrians will not be safe.

Fig. 18

Busy hours of Monday

Fig. 19

Busy hours of Tuesday

Fig. 20

Busy hours of Wednesday

Fig. 21

Busy hours of Thursday

Fig. 22

Busy hours of Friday

Fig. 23

Weekly average busy hours

5.4 Discussion

Strength: In the research, use of YOLO is a great strength because the YOLO algorithm is one of the finest object detection algorithms, and compared to other object detection algorithms, YOLO is the fastest. Using YOLO, real-time detection is achieved. If a person wants to go out in a public region, he could use such a module to get information about that region before stepping out and could avoid going to crowded regions.

Challenges: For human or pedestrian detection, which algorithm should be used was a difficult task. Before going for YOLO, the Haar cascade algorithm was used but it did not have the required result and contained some inaccuracies. Getting a high-resolution video frame was quite challenging.

6 Conclusion

The paper aims to be effective for making social distancing between people at all times, making things more systematic and safer not only during COVID-19 but after that also. With the help of artificial intelligence (AI), the same technology that is the foundation of self-driving Tesla and Netflix recommendations, combined with edge computing, the technology that is reshaping the Internet of things (IoT), the paper aims to practice social distancing with least disturbance to our daily lives.

Using algorithms like R-CNN and YOLO, the research performed real-time human detection for fast and accurate responses enabling us to monitor social distancing at public places like malls, companies, institutions, hospitals, etc. With efficient monitoring using database management skills, our tool helps in bringing about discipline among people with its robust and specific approach to achieve maximum safety for all people. The paper monitors a particular area and predicts the number of people visiting that place in a week, giving us weekly insights of the traffic seen per day, which in turn gives insights on the following week’s traffic, and then alerts the general public accordingly for their safety.

7 Future Scope

In addition to the research, real-time analyzes of the violators could be implemented along with an application of face detection. Immediate action can be taken, for example, alerting the violator by sending them a message. To make this project more effective, the inclusion of mask detection along with social distancing could be added. Another future scope could also be analyzing the previous week’s data and using an ML model to predict the following week’s population traffic in a particular area. The paper aims to create a surveillance app soon which would have all of this, and as soon as the user puts in the desired location of where they want to visit, it entails predictive analysis of last week data, gives insights on this week as per the day of week and time of day, and suggests which time is best suited for the user instead of randomly visiting at any place and finding a busy traffic present.