Keywords

1 Introduction

The term Smart City encircles the people dwelling inside it and, the technologies and devices they use. Everyone wants to live in an urban setting and smart cities are a sort of technological trend moving the people towards more urban settings and lifestyle. With increasing standard of living, the demand for materialistic desires is also increasing with rapid pace. Everyone wants to own at least the basic luxury which includes own house, car, fridge, television, air conditioner, and many more that a man requires to live a so-called satisfying life. Smart cities are connected mostly with mobile devices whether it be a cell phone or a vehicle. This paper mainly focuses on motorcycle related effects in a smart city setting.

With increasing traffic in todays world, a lot of problems have emerged. Of them the problem of traffic policemen to keep a check on riders for helmet has also become overwhelmingly difficult. We could see more vehicles on road than there are people on road. Everyone is in haste to reach his or her destination. So, policemen cannot stop every vehicle and check them for following traffic rules. This solution will be solving this problem of policemen and making it automatic.

There have been solutions like this in the developed nations but they are mainly focused on cars. Scenario in less developed countries is a little bit different. For instance, in India there are more two-wheelers than cars. And the major traffic rule for bike riders is with the helmet. Traffic rule instructs riders to wear helmet while riding the motorcycle but people think that wearing helmet is a tedious and uneasy task. People find it suffocating wearing the helmet during riding the motorcycle. Police check the vehicles mainly on intersections so riders take advantage of this. They wear the helmet on their hands, maybe to protect their elbows, and by the time they reach near the policemen they wear the helmet. It might be more tedious to carry the helmet in hands while driving, but riders find it easier to have it in hand rather than on head. People do not ponder over the point of wearing the helmet, they just overlook this rule. They wear the helmet just for the sake of showing it to the policemen. The riders think policemen the judges of a fashion show in which wearing helmet is mandatory.

This paper talks about the problems caused by not wearing helmet, the statistics behind it, and the lastly the implementation that this paper is mainly written for. This solution could prove to be a technology helping in leveraging the Intelligent Traffic Systems of Smart Cities.

2 The Concept of a Smart City

A city can be defined as an entity in which a large collection of individuals live and thrive in a social, material and cultural environment. A city encircles a group of amenities and resources which are housing, transportation, sanitation, utilities and communication [1]. The proper coordination and interaction between these amenities provided by a city, makes up what is called a city. With the advent of industrial revolution, the migration of rural dwellers to an urban setting has increased manifold. And this caused the population in a fixed area of a city to exponentially increase. Due to this a lot of forest areas surrounding the city area got cut down to accommodate more city dwellers. This urbanization increased the rate of growth of industries, so there demands for labour, and thus more labourers came to city to get job, thereby contributing to overpopulation.

All these causes deteriorated the serene environment that once existed when the rural and urban population was in balance. But now, a lot of problems have emerged, that are continuously degrading the lifestyle and health of the city dwellers. To overcome these problems, a smart solution is required to fight it. And here comes the concept of a Smart City. A Smart City is an umbrella term describing a bouquet of digital technologies working and interacting in order to make a better city and eliminating the causes of its deterioration. These technologies mainly comprise Internet of Things (IOT), Artificial Intelligence, Machine Learning, Blockchain, Edge Computing, Networking and Communication. The surveillance cameras, smart dustbins, sensors on sanitary outlets and industrial chimneys, sensors in vehicles, cell phones and many others act as devices to collect data from around the city. This huge amount of real-time collected data carry a lot of useful insights that can be studied and analysed to create a better solution to fight a city-deteriorating factor [2].

2.1 Challenges

Smart Cities contribute to development and enhancement of various aspects of life. These include but not limited to Waste Management, Intelligent Traffic Management, Environmental Pollution Control, Smart Lighting, Smart Roads, Better Healthcare, Forest Fire Detection, Early Earthquake Detection, Potable Water Monitoring, Smart Electricity Management, Smart Security and Emergency Facilities, Intelligent Shopping and Payment at POS, Smart Agriculture, Smart Energy Management and Intrusion Detection Systems. To incorporate these many technology applications and running them in sync with each other is another challenge that is experienced in deploying a Smart City. With great technology, comes a great deal of challenges with it. Smart City is no such exception. The major challenges [3] with Smart City are:

  1. 1.

    Infrastructural resources: The whole architecture of a Smart City employs so many technologies that have to run in sync with each other and be able to process information captured in real-time with least possible latency. For the accomplishment of this a lot of compute resources are needed which is expensive and not in the budget of every nation. Therefore, only some nations are accepting and implementing this technology. Before implementing these technologies, one should first ponder over power supply, cost incurred in maintenance and complications involved in running a city-wide technology.

  2. 2.

    Security: Wherever a remote connection, communication or internet is involved, there always exists a danger of security issues. Because the systems are monitored remotely via a channel that is connected to internet, it is vulnerable and open to attacks by hackers breaching the security of the intelligent systems infrastructure. So, tackling the security issues stands a major issue in implementation and maintenance of Smart City deployment. It will be very risky if the whole thread to puppeteer the entire city comes under the control of a malicious mind.

  3. 3.

    Privacy of city dwellers: With surveillance cameras installed, the privacy of individuals is snatched without consent. The cameras monitors behaviour and activities of every individual and this may just kill his or her privacy. But wherever it is required to collect data, privacy concerns arise automatically. To fight this, many organizations are working on to develop technologies that will enable systems to collect data but without harming or killing peoples right to privacy. Such technologies are Federated Learning and the concept of Differential Privacy [4, 5]. By using these two concepts the privacy concerns getting in the way of Smart City implementations can be done away with.

  4. 4.

    Smart citizens: For a Smart City to exist and function properly, it must be able to coexist and be in sync with its dwellers. The citizens must be educated and informed enough to adapt to these technologies and play a key role in better development of the city of future. This can be done through mass crowd-sourcing, citizen empowerment and email campaigns.

  5. 5.

    Updated and socially inclusive population: If half of the population, like elderly citizens, cannot afford to use mobile phones or applications, then the concept of smart city wont be able to spread much and benefit the city as a whole. A Smart City planning must involve these subtleties and consider all the citizens, and make them informed about or think another way out to make the technology accessible to those who cant access it using traditional means like mobile phone.

2.2 Smart Cities in India: A Case Study

The concept of Smart City has been employed by India under the Smart Cities Mission by the Ministry of Housing and Urban Affairs, Government of India. Under this plan India aims to conduct this mission under three components, which are, retrofitting (city improvement), redevelopment (city renewal) and greenfield development (city extension) [6]. India is an under-construction and fastly evolving country. It ranks second in terms of population. The waste management, traffic management, pollution management and other like issues are unable to be handled by the governmental organizations responsible for it. The population is ever increasing and the land area is that much only, so a lot of problems are arising due to large scale urbanization.

Through this programme, government wants to develop the country as well as fight the problems that are making Indias growth stagnant. Government conducted proposal submissions for smart city for various cities by various individuals and organizations. The government is implementing the proposals in some major cities. The problem of citizen empowerment is highly lacking in India. So the challenge of benefiting each and every citizen adversely affects the spread of this concept around the nation.

The traffic management in India is not very good. People dont want to follow traffic rules. When traffic policemen stop them for fines, they shut their mouth with bribe. The traffic policemen is also corrupt so he also accepts the bribe and lets the rider go and make his life or someone others life a hell. Not wearing a helmet while driving is thought to be cool. Moreover, people give the excuse of excessive hot weather for not wearing helmet. But is it better to not wear helmet and tolerate some heat than die on road without helmet by head injury? Absolutely not at least in theory, but it needs to be implemented in practice. Every citizen should not be just responsible for him or her but also think in advance what his or her wrong deeds on road will do to others on road. But people dont understand till they get someone to monitor their deeds. So the implementation that this paper talks about helps to keep a check on the riders for helmet and penalize them for not wearing one. The system also contributes to a better Intelligent Traffic Management System in India (Figs. 1 and 2).

Fig. 1
figure 1

Dholera SIR is the first smart city in India located in the state of Gujarat. There are more smart cities to come up in the future as per the government’s plans [7]

Full size image
Fig. 2
figure 2

(Data Source [9])

Statistics showing the effect of wearing a helmet during an accident

Full size image

3 Motivation

One of the major concerns of Smart Cities is the traffic management. Smart Traffic Management includes solutions providing commuters with facilities like easily finding a parking space using a mobile phone. But Smart Traffic Management does not only concern with providing facilities to its citizens, but also to keep a check on its dwellers that they follow the traffic rules properly, so that no issue may occur in proper functioning of the Smart City. With smarter city and more urbanisation, the vehicles on roads will increase and checking them for following traffic rules manually would be too cumbersome for the traffic policemen. So, there is a need for a solution that could check riders and drivers for following traffic rules properly.

Driving a motorcycle on road is more vulnerable than driving a car or other heavy vehicle [8]. This is due to the fact that in the case of any collision or accident, the motorcycle riders body is more open to impact than other vehicles like cars in which the driver is inside a case, which can protect the driver to some extent. According to the World Health Organization (WHO), wearing helmet reduces head injury by around 20–45%, and those not wearing a helmet while riding are three times more prone to head injury than those wearing a helmet. Young passengers rarely wear a helmet and on the top of it, they want to flaunt and over-speed. It is also evident that in countries where helmet wearing laws are strict, the injuries caused by accident to two-wheeler riders has decreased by 20–30%. The implementation of strict helmet wearing laws in Italy increased the helmet wearing from 20% in 1999 to more than 96% in 2001, and decreased the deaths and head injuries by a huge margin. Laws implemented in Malaysia reduced the head injuries by 30% [10]. As per the National Highway Traffic Safety Administration studies, riders who were not wearing helmet had higher health care costs, longer hospitalization, longer recovery periods, and more severe disorders or disabilities [11].

Mostly people do not wear helmet because they feel hot and suffocating to wear the helmet. But, feeling hot for a small duration is better than death. Moreover, riders should buy good quality helmets with proper air vents and tie them properly for optimum security. There are many excuses people give for not wearing a helmet, but according to Nicole Levy, MD, a primary care sports medicine specialist at Rush University Medical Center, there exist no valid excuses for not wearing a helmet. A serious injury can occur to the brain as a result of the accident. The helmet absorbs most of the impact during the accident and saves the rider from serious consequences of brain trauma. A serious brain injury can lead to skull fracture, lost consciousness, memory loss, concentration loss, sleep disorders and even death [12].

4 Concept

The solution that this paper talks about will be able to detect riders for wearing helmet and penalize them if they are found not wearing helmet. The solution is built using YOLO and darknet framework and was trained with TensorFlow as backend. The model is trained on custom manually annotated image dataset and trained on two-GPU server for 2500 epochs.

The detection of faces and number plates were performed by traditional machine learning methods for computer vision available in OpenCV like Haar Cascade for face detection and, Canny Edge Detection and Contour Detection for number plate detection.

The implementation works in two phases. In the first phase, the riders are detected for helmets. If the rider is found to be wearing a helmet and his face is detected or not, then he is let go. If the rider is found to be not wearing the helmet, and his face is detected, that means he is without helmet then another phase of the solution will start. In the second phase, the number plate of the rider who is found to be not wearing a helmet, is captured and read. Then the riders motorcycle number is stored in a server. Afterwards, a fine notice is generated automatically and is sent to the address of the rider [13] (Fig. 3).

Fig. 3
figure 3

Implementation flowchart of the solution

Full size image

The implementation employs three major modules. One for helmet detection, other for face detection and the last for Automatic Number Plate Recognition (ANPR). The face detection is performed using OpenCV and Helmet Detection is performed using trained YOLO model and OpenCV. The Automatic Number Plate Detection (ANPR) is performed using the API from Plate Recognizer [14]. And these detections go on in real time. It is like binary classification at first step and then conditional steps afterwards. The flowchart representing the implementation flow plan is as depicted in figure above.

4.1 Object Detection

It is a task in computer vision and image processing that aims to detect objects in a digital image, video or real-time camera feed. It detects objects for which it is trained for. It has many applications in areas of computer vision like image retrieval, pedestrian detection, video surveillance, and face detection. Nowadays it is being used in many other tasks like fault detection through image, behaviour detection and object segmentation. This solution is also an application of object detection that will be using real-time traffic camera feeds.

There exist two approaches for performing object detection, which are machine learning approach and deep learning approach. In machine learning approach, features have to be explicitly specified and defined. These use one algorithm for extracting features from the image and another for classification of the detected object based on the features. For feature extraction, the algorithms commonly used are Viola-Jones algorithm based on Haar features, Scale invariant feature transform (SIFT) and Histogram of oriented gradients (HOG). For classification mostly Support Vector machines (SVM) is used.

4.2 YOLO

YOLO or You Only Look Once is an object detection framework based on darknet. Darknet is a framework written in C that performs object detection on images, videos and camera feeds. It is now in its third iteration (YOLOv3). Though many object detection frameworks exist in the market, YOLO is one of the most widely used and the reliable one. It is always updated with its codebase and pre-trained weights that one can use if one wants to train a custom object detector or use pre-trained weights to make a general object detector. The default weights and configuration files provided by the YOLO are trained to detect almost all the commonly used objects and these objects are included by Microsofts Common Objects in Context (COCO) objects dataset. The YOLO models come in various shapes and sizes; they have different number of layers that corresponds to better mean average precision(mAP). This model is most widely used nowadays because of its performance along with least inference time per frame or image. YOLO gives better mAP as well as has least inference time in comparison to its closest competitor Facebook AI Researchs RetinaNet (Fig. 4).

Fig. 4
figure 4

YOLO architecture [15]

Full size image

In deep learning approach, it is not required to explicitly define and specify the features. These employ convolutional neural networks (CNN) that are competent enough to create and learn features. The commonly used techniques to perform the task of object detection are R-CNN, Fast R-CNN, Faster R-CNN, Single Shot Multi-box detector (SSD), You Only Look Once (YOLO), Facebook AI Researchs RetinaNet and TensorFlow Object Detection API.

YOLO works by passing the frame of the video or image through a single neural network that divides the image into different regions and calculates bounding boxes with some probabilities. The bounding boxes with most probability for the desired class is chosen [15].

Other detectors like RCNN and Fast RCNN calculate bounding boxes for different portions of the image on different scales. So, these algorithms calculate many, around thousands of bounding boxes for detecting a single object. It is due to this fact that YOLO performs around 1000 times faster than RCNN and 100 times faster than Fast RCNN.

4.3 Curse of Dimensionality

This is an inherent problem with machine learning and neural networks. There exists a trade-off between number of features in the data and number of epochs for which the model is trained on the data. Although this term has deep roots emanating from Pure Mathematics, yet it has different interpretations, one of which is used in reference to machine learning algorithms. Bellman coined this term to describe the problem of exponential increase in volume when a lower dimensional Euclidean space is taken to higher dimensions [16].

In reference to machine learning, dimensionality refers to the number of features that are there in the data. It comes in to play during the training process of the algorithm. When the training starts, at this point of time, the model has learnt very little and is not yet capable enough to perform its desired task, that is it is a very simple approximation of the original to-be model. This is known as underfitting. After some number of epochs or iterations over the entire dataset, the models best state is achieved after which the models accuracy stalls, precision and recall start approaching 100%. This state of the model is known as sweet spot and this is the state which every model aims for. After this spot, any increase in epochs dwindles the accuracy and generalization power of the model in real-world scenarios and unseen cases. This unwanted state is known as overfitting of the model [17] (Fig. 5).

Fig. 5
figure 5

Bias-variance tradeoff

Full size image

In the case of underfitting, the model is too simple and too biased, while in the case of overfitting, the model produced is too complex and its variance is too high. In other words, it can be said that an underfitting model learns little to nothing about the data it is trained upon, and an overfitting model tends to learn too much about the data, that is it just memorizes the data, so it becomes useless to be used on unseen data. So, in both cases validation error is too high, though training error may be small in the case of overfitted model [18].

But this problem of dimensionality can be dealt with by using regularisation. Regularisation is a technique that helps to downgrade the complexity of a model, so that the element of overfitting from the model gets removed. By this the model approaches the sweet spot and a better model is achieved. Regularisation is an umbrella term for various techniques. These techniques are ln-regularisation, data augmentation, early stopping, batch normalization, and dropout. These techniques all work to reduce to find the sweet spot of the model. Of them, dropout has been considered the best for most of the cases till now. While data augmentation is a technique that is mostly used in the case of computer vision to augment the data by modifying the images, that is by rotation, colour changing, reverting, etc. Early stopping as the name suggests stops the algorithm from further training when that sweet spot is found (Fig. 6).

Fig. 6
figure 6

Annotating image using the open-source LabelImg tool

Full size image

5 Preparing the Data and Training

The data for the implementation was created manually. First the images from Google Image search for keyword ‘helmet’ and ‘traffic’, and 100 images related to these keywords were downloaded. The model had to be trained on a one-class data, that is “Helmet”. Thereafter, the images were annotated and marked to specify the desired portions of the images for detecting helmet. The data was labelled using LabelIMG tool which is a free open source tool to label and annotate images. This tool provides feature of making bounding boxes on desired portions of image and can generate annotations of two types YOLO and Pascal VOC format [19].

After the annotation step, the whole directory of darknet was configured to meet the needs of the required model. The weights chosen for training the model was darknet53.conv.74. The configuration file was identical to yolov3 configuration file. [YOLO] The model was trained for over 2500 epochs on dual Nvidia Quadro P6000 GPU server. It took around 25 min for training to complete. The training procedure was halted after 2500th epoch because on reaching around 2500 epochs, the model seemed to start overfitting and since a real time general model is required so overfitting is a curse for this application.

6 Results and Discussions

The system is able to perform detection of helmets in static images, videos and real time camera feeds. The detector is made using OpenCV to implement all the required utilities for detecting of helmets [20]. I chose OpenCV over the usual, original darknet detect utility, because of the speed and latency factor. OpenCV is faster than darknets detect because it performs much less calculations and moreover since it uses CPU it is less hardware-resource demanding. And these all benefits make OpenCV a good choice for real-time detection.

The output image or image or camera feed from the helmet detector module shows detected bounding boxes (in orange) with the label Helmet along with confidence percentage with which the trained model detected the object, that is helmet.

The automatic number plate recognizer(ANPR) module uses the API from Plate Recognizer. The detector module sends a request of the image or video frame being fed into it, and receives a json response. The field containing vehicle plate number in the response is then extracted using normal Python methods.

The face detector module utilizes the Haar Cascade features available in OpenCV [20]. The image is first converted to grayscale and then face is detected using Multi-scale face-cascade algorithm.

The detection procedure on an image or a pre-recorded video is a little slow, and of course is not performed in real time, but this can be extended to real time using OpenCVs VideoCapture() function. Using this the system can be deployed along with a real time recording camera, to make the camera intelligent enough to detect helmets in the feeds it captures.

The helmet detection module and the face detection procedure, both work simultaneously, and return boolean results for detections for a frame of a camera feed. After receiving results from these two procedures, the further processing is decided. If helmet is detected, then irrespective of the result of face detection, the rider is let to pass without any fine. If the helmet is not detected and a face is detected, then that means the rider is present in the video frame, but rider is not wearing a helmet. In this case, number plate in the current video frame is detected and stored in the traffic fine database. This stored fine database information is then used to generate fine automatically based on the vehicle registration information (Fig. 7, 8 and 9).

Fig. 7
figure 7

Result of helmet detector module on an image

Full size image
Fig. 8
figure 8

Result of face detector module on an image

Full size image
Fig. 9
figure 9

Result of ANPR module on an image

Full size image

The results are demonstrated using a Flask web application [21]. The application is a two-page application, one to upload and save the image or video or camera feed, and the other to show the results. The experiment was performed on three types of images

  1. 1.

    With rider having helmet but his or her face could not be seen.

  2. 2.

    With rider having helmet with his or her face being seen.

  3. 3.

    With rider not wearing a helmet, and his or face being seen.

Fig. 10
figure 10

Result of the system on an image with person wearing helmet with his face not being seen

Full size image
Fig. 11
figure 11

Result of the system on an image with person wearing helmet with his face being seen

Full size image
Fig. 12
figure 12

Result of the system on an image with person without helmet

Full size image

As can be seen in Figs. 10, 11 and 12, the system performs well on the three use cases that have been considered in making this application. This system has a lot of potential to better contribute for developing an Intelligent Traffic Management System in a Smart City.

This application or its like can be deployed on a secure environment and connected to Traffic Police database wherein different analysis can be carried out on the data collected [22]. The camera feeds from the traffic surveillance cameras will be fed into the application in real-time, and then the further procedures will be carried out - like letting the rider go or automatically penalizing the rider for not wearing a helmet by generating a fine for the rider’s motorcycle registration number. The cameras can use IOT-enabled smart sensors which could just manipulate the recorded camera feeds in real time thus decreasing latency and increasing the overall response time of the system.

7 Future Work

This new century has embarked on the path towards the Fourth Industrial Revolution, wherein technologies like Artificial Intelligence (AI), Internet of Things (IOT) and Blockchain will be helping in building a smart city - a city of the future. A city where each and every entity whether living, non-living or abstract will be interacting with each other to enable a smart planet in totality. Smart and intelligent systems will be able to sense and gain useful insights from the data collected from IOT sensors. These systems will be utilising the power of AI to instil the smartness of a human being to an abstract of non-living entity. These systems could be made more secure using the Blockchain paradigm.

The system we talk about here could serve to leverage the services of an Intelligent traffic management system deployed in a smart city [23, 24]. Using this system, we will be able to enforce the traffic rules regarding helmets in a much better automatic way. Here we used the AI paradigm and IOT paradigm. The sensors including the cameras which will convey the traffic surveillance data to a server leverages IOT [25], and the analysis that will take on the server is the task of AI. But we did not talk about the third paradigm that we mentioned above - the possible uses of Blockchain in this application.

Blockchain is one of the future technologies. The notion of it came into existence when Satoshi Nakamoto presented a paper in 2008 that aimed to solve the problem of double spending in transactions without the need of a mediator or a financial institution like bank [26]. In simple terms they wanted to enable exchanging money between two entities in a secure manner without each needing a bank between them.

That was the story regarding financial transaction. In computer science, transaction is defined in layman-terms as a collection of tasks that must be completed in order for a process to accomplish. Since a blockchain is concerned with providing a decentralised base to a transaction and securing it on top of it, it can be used in our application as well. Our system can use blockchain to secure the communications between the smart surveillance cameras and the processing server. The transactions will be the data packets that would be sent by the camera systems to the database for processing [27].

Moreover, these servers can be done away with by employing a full blockchain network of smart surveillance cameras. The cameras could then be embedded with smart sensors and edge computes. These cameras will be smart enough to process the camera feeds themselves, and this information can be then verified by the blockchain of cameras, where each camera will be a block, and transactions will be the decision of penalising a rider. Through this, the concept of an Intelligent Traffic Management System would be fully successful in terms of technology stack utilisation and implementation.

Since we are talking about applying blockchain to our system, we can not leave the concept of incentives here. The riders who are wearing helmets can be given rewards by the blockchain network [28, 29]. Through this, the message of wearing helmets will spread nation-wide. Every rider will then have no chance to bribe a traffic policemen or hack into the blockchain system, thereby providing a safe, secure and intelligent surveillance system.