Keywords

1 Introduction

An autonomous car is capable of learning its environment through sensors and camera which will then process the data received through these external devices, which will help in taking decisions. With great development in hardware technology and embedded devices, small components can do computations effectively with all forms of data. There are major car manufacturers including BMW, Google and Tesla that are building and actively testing these cars. Latest results show that autonomous cars have become very efficient and already are driven without any human intervention [1]. Advanced and complicated control systems, algorithms and software take all the sensory data and information to recognize and identify suitable and right routing paths [2, 3]. Autonomous vehicles have complicated control systems that can able to take in sensor data, analyse the data to differentiate different objects in the surrounding environment and identify vehicles and other obstacles in the environment, which will be very helpful for planning to the desired destination [4].

The National Highway Traffic Safety Administration has classified autonomous vehicles as belonging to zero of five levels: i.e. a. no automation, b. assisted automation, c. partial automation, d. high automation and e. full automation. It is important that the driver’s attention is needed within level 0 to level 2 modes. Any car that has been manufactured will be considered to be in any one of those levels and also as the level increases, the automation also increases but there are no cars which are in level 5 automation, i.e. high automation, and companies are working towards it. An autonomous car construction implies a mechanical and electrical design. There are two solutions: transform a real car into an autonomous car or to design a new vehicle [5]. Audi expressed that their new A8 would be autonomous up to 60 km/h [6]. The driver does not require safety checks such as frequently gripping the steering wheel. The Audi A8 is claimed to be the first production car to reach level 3 automation, and Audi would be the first car manufacturer to use laser scanners along with cameras and various sensors for their system [7].

Autonomous vehicles have excellent scope as they tend to do fewer errors compared to human drivers. Road accidents are major cause of death in the world and most of these accidents happen due to mechanical problems and driver’s distraction. Several perspectives towards vision-based self-driving rely on certain features of the road such as lane markers and systems usually have a specific lane detector [8]. A new study shows that most of the traffic jamming is caused by three major issues: improper car parking, street dwellers and people walking on the roads [9]. These accidents can be brought down if properly trained autonomous vehicles are implemented. Apart from saving lives, consumption of fuel rate can also be reduced with the help of autonomous vehicles. One of the major advantages of autonomous vehicles is that these vehicles can be used in military and this will help in keeping troops out of harm. In 2002, DARPA announced grand challenge which focused on building autonomous vehicles for a prize of $1 million dollars offered to researchers from top institutes if their vehicles can travel a distance of 142 miles through the Mojave Desert. This was the first grand challenge that took place and later few more challenges took place which motivated researchers to build a self-driving car that can navigate as far as possible.

In this survey, we try to look at and compare various methodologies used in order to build autonomous cars which are popular and also efficient with respect to their performance. There are a good number of concepts and technologies through which autonomous vehicles can be implemented but choosing the right one considering the external factors is prominent. Also, this survey focuses on providing information to the researchers to learn more about trends and technologies in this area since the scope of this area is vast and is open for contributions.

2 Background and Related Work

The history of autonomous cars begins in 1920s, where experiments were conducted on self-driving cars. In 1925, radio-controlled car named ‘American Wonder’ was demonstrated on New York City streets. The trend of radio-controlled cars continued for years, and later in 1980s, vision-guided Mercedes-Benz robotic van was designed in Munich, Germany. Since then, the major focus has been on development of vision-based guiding systems which uses computer vision, LIDAR and GPS technologies. To enhance the research of autonomous system, there are comparisons between various methodologies and their results shown in Table 1.

Table 1 List of various methodologies and their results
Full size table

The concept of neural network training for self-driving cars was introduced in early 1990s by Dean Pomerleau, Carnegie Mellon researcher, how raw images of roads can be captured and trained in order to control the steering of the vehicle based on the condition of roads. Also, this method was considered to be more efficient compared to the other methods used for self-driving cars. This was the major turning point as even to this day self-driving cars use neural network method for training and implementation purposes. There are various neural network training methods like convolutional neural network, R-CNN. The working and performance of neural network vary based on the concepts and more importantly, the input provided. The neural network representation is as shown in Fig. 1.

Fig. 1
figure 1

Neural network representation

Full size image

Neural network training can be done by live streaming of video or using images but the drawback is that it is only limited to 2D images, but LIDAR technology changed everything as it used to gather real-time data of the car surrounding. Spinning light detection and ranging system (LIDAR) was used in order to gather real-time 360-degree maps of the surrounding. LIDAR technology made car even more automated and later cars were using both LIDAR and neural network in order to gather live data and hence made car more autonomous. With the help of built-in GPS, the cars were able to navigate around without any trouble as they had everything to navigate. Also, cars use ultrasonic sensors and odometry sensors for distance measurement and motion detection, respectively. All these components and car controlling system were linked and controlled together by a central computer, which enabled a certain level of automation. Overview of control system is as shown in Fig. 2.

Fig. 2
figure 2

Overview of control system

Full size image

2.1 Convolutional Neural Network

When we consider a neural network, the network is made up of neurons with weights and biases. CNN is also a neural network, where the neuron takes in several inputs and calculates a weighted sum of the inputs. This is passed through an activation function which generates the output. The difference between neural network and CNN lies in the input given-in neural networks the input given is vector, whereas in CNN the input given is a multi-channelled image. Image recognition, image classifications, object detection, face recognition and so on are some of the major application fields of CNN.

2.2 Regional-Based Convolutional Neural Network

In R-CNN of CNN, it is mainly focused on the single region so that the interference is minimized as it expects only the single object of which the interest lies will dominate in the given region. By the method of selective search algorithm, we can detect the regions in the R-CNN, and it is done by resizing the regions with equal size so that it can be used in the CNN classifier and also in bounding box regression. Bounding box regression is needed because the starting proposal might not coincide with the region that is given by the features of CNN.

2.3 LIDAR Technology

Light detection and ranging, or LIDAR, is a remote sensing method which uses light in the form of a pulsed laser. The major components for such a device include a laser, scanner and a GPS receiver. A sensor in LIDAR is used to continuously fire beams of laser light and then determine the time taken for the light to return to the sensor.

2.4 AlexNet

Alex Krizhevsky designed a convolutional neural network called AlexNet, which addresses the problem of image classification. He proposed to take as input an image from one of 1000 various classes, which produces an output that is a vector of 1000 numbers. The element at the ith position in the output vector is considered as the probability of input image present in the ith class. As a result, the sum of all elements of output vector is 1. The input image should be an RGB image of size 256 × 256, which includes all images in training set and testing set as well. If it is not of that size, then it has to be converted to the size before being able to use it for training the network.

2.5 AdaBoost

AdaBoost is an ensemble classifier; ensemble classifiers are made up of multiple classifier algorithms whose output is the combined result of output of those classifier algorithms. AdaBoost classifier forms a strong classifier algorithm by combining various weak classifier algorithms. Using a single algorithm may result in objects being classified poorly. Combining several classifiers by considering the right amount of weight for the selected training set for every iteration can result in greater accuracy for the overall classifier.

2.6 TensorFlow

TensorFlow is an open-source framework that has an artificial intelligence library. Models are built using the data flow graphs generated by this library. Creating a large-scale neural network with many layers is easy for the developer to build. The main application area of TensorFlow includes classification, perception, understanding, discovering, prediction and creation.

2.7 Viola–Jones Object Detection

Viola–Jones algorithm is mainly used for the purpose of object detection. The main property of this algorithm is that the detection fast despite the fact that training is slow. Haar basis feature filters are used in this algorithm, so that multiplication is not used.

2.8 Hardware-in-the-Loop Simulation

Hardware-in-the-loop simulation or HIL simulation is a type of real-time simulation used for testing control systems. In simple words, the physical part of a machine or system is replaced by a simulation. This provides a rigorous testing method with great variety of benefits. Actuators and sensors are used to connect the machine with the control system.

2.9 DEDLUS Platform

It consists of three module: image capture module, data management node and data processing node. The data management node stores the information passed by image capture module capture by webcam or camera; it meagre the image with the result archive by the data processing module and stores it. An object detection algorithm is run on the GPU by the processing module to process the data.

2.10 Yolo

It is an object detection technique in which it divides the image into different sections using neural network and predicts the boundaries of each section and then assumes the weight to different sections and identifies the different objects in the image. Steps for object detection are depicted in Fig. 3.

Fig. 3
figure 3

Steps in object detection

Full size image

2.11 High Dynamic Range Imaging

A greater dynamic range of exposures in images can be obtained in HDR Imaging when compared to the standard methods [19]. The objective here is to present a similar range of luminance as perceived by humans through their eyes. When compared to the traditional methods, HDR images represent a greater luminance in scenarios of real world containing very bright, direct sunlight to extreme shade. A number of different narrower range, exposures of same subject matter is combined after the image is captured. The counterpart of HDR, which is the non-HDR cameras capture images with a limited exposure range, which results in loss of detail in highlights or shadows.

2.12 Deep Learning

Deep learning is a part of artificial intelligence concerned with surpassing the learning approach used by human beings to gain certain types of knowledge. At a much simpler level, deep learning can be considered as a way of automating the process of predictive analytics. Deep-learning algorithms are arranged in a hierarchy of abstraction and increasing complexity contrary to the traditional machine learning algorithms. The algorithm in the hierarchy makes use of a nonlinear transformation on its input and creates as output a statistical model from what it learned. Once the output has reached an acceptable level of accuracy the iteration is stopped.

2.13 Hebbian Learning Algorithm

Hebbian learning algorithm tries to explain synaptic flexibility, i.e. about how the synaptic strengths are adapted in brain. If the neurons on the either side of the synapse have linked output, then the synapse linking two neurons is strengthened. When an input neuron triggers, the output neuron triggers, and the synapse is strengthened. By following the resemblance to an artificial system, the weight of tap is increased with high linking between the two sequential neurons.

3 Approaches and Working

Artificial neural network is inspired by the biological neural network, and ANN itself is not the algorithm but it is the combination of many different machine learning algorithms working together to process the given input. This type of system learns to perform task by considering previous example without being programmed to perform a specific task. It is the combination of connected nodes called artificial neurons. The node receives inputs from an external source or from other nodes and computes the output. Every node has a weight associated with them and with the combination of input value and weight, the node produces an output. This output can be an input to other nodes or can be an output to a system. A typical ANN is the combination of many layers. Different layers perform different kinds of functions on input. Data traverse through the first layer or input layer to the final layer or output layer by traversing through multiple processing layers.

An autonomous car requires five essential functions, i.e., localization, insight, scheduling, vehicle control and system management in order to autonomously drive without human involvement [20]. The system consists of three input nodes camera module, LIDAR, sensors and the combination of these input nodes is called input layer. The camera module takes an image or a video as input, the LIDAR creates 360-degree map of the surrounding of the car while there are many other sensors which sense the environment around the car. This process uses the ANN and machine learning algorithms for processing the data in the hidden layer. Combination of the three inputs will be given as the input to the hidden layer through input node and in each layer many nodes will receive the input from the above layer and will process the input and pass the output to the next layer as its input. The output data from the hidden layer are not part of the global output but are only present inside the network. Based on the weighted combination of its inputs, every single one of these hidden units computes a single real-valued output. The data processed by the hidden layer is given as input to the output layer nodes. The output layer nodes are steering, breaks and acceleration. The steering will control the direction in which the car must move, while accelerator will provide the speed in which the car must travel and the break will be applied when it is needed to control the speed of the car or to stop. The diagram to the above explanation is as shown in Fig. 4.

Fig. 4
figure 4

Working of autonomous car

Full size image

The system accepts live data from the inputs present in Fig. 4 and will process them to produce output but there are situations where one of these input devices may fail to gather data which will only result in failure of the system. This is how the above system might fail while working. Another possibility is that the system may fail to process and return the output in time and the delay may sometimes lead to an accident. There are many other ways which will result in failure but training system well with all the possible scenarios will result in a strong working system.

4 Conclusion

In this paper, we discuss all the existing and current technologies that are used in autonomous vehicle development. Autonomous vehicles have a history starting from early 1920s, and still, there is a huge room for development. Initially, it started with radio-controlled cars, and now, we have cars at level 4, i.e. high automation, and we do not need much time to witness level 5 automated cars. The development of Mercedes-Benz vision-based autonomous van by Ernst Dickmanns and introduction of neural network in computers Dean Pomerleau in 90s changed everything. Later on, every car that claimed to be an autonomous had vision-based system in them but it had its own limitation and was later equipped with LIDAR technology which made cars more autonomous. Along with those technologies, various types of sensors were used to gather more data, which would make system perform more accurate. Even after usage of all these technologies and equipment, we are still finding hard time to reach level 5 automation because there are certain external factors that cannot be controlled, and those factors do count at times. This shows that still there is a vast scope for new methodologies and development.