Bayesian Artificial Intelligence-Based Driver for Fully Automated Vehicle with Cognitive Capabilities

Abstract

At automation level 5 as defined by the Society of Automotive Engineers (SAE), a driver will not be in the loop even in a complex driving environment featuring among other challenges, the presence of vehicles with automation levels ranging from 1 (no automation) to 5 (fully automated). This paper defines the safety and ride quality requirements that a fully automated vehicle should meet when operating in a mixed traffic environment featuring vehicles of various automation levels and proposes a Bayesian AI-based driver algorithm as a solution. Design advances that can potentially overcome the safety and ride quality issues are described. Microscopic level data sourced from driving simulator studies are used in applications. Finally, conclusions are presented on the abilities of the Bayesian AI-based driver to meet safety and ride quality criteria while operating in driving environment characterized by uncertainties. The Bayesian AI-based driver is likely to enhance consumer and safety regulator acceptance.

1 Introduction

The Society of Automotive Engineers (SAE) has defined levels of automation that range from 1 to 5. At automation level 5, a driver will not be in the loop even in a complex driving environment. There are many factors that can contribute to complexity in the driving environment. A notable future factor will be the presence of vehicles with automation levels ranging from 1 (no automation) to 5 (fully automated). According to the SAE [1] and the U.S. Department of Transportation [2], the level 5 automation has “The full-time performance by an automated driving system of all aspects of the dynamic driving task under all roadway and environmental conditions that can be managed by a human driver”. Here, it is understood that the human driver is not distracted and also drives in a non-aggressive manner in terms of maintaining safe distances in the longitudinal and lateral direction as well as accepting a safe gap in traffic for merging or lane change manoeuver. This vision can become a reality with a Bayesian artificial intelligence (AI)-based driver algorithm of the “cognitive vehicle”.

This paper defines the safety and ride quality requirements that a fully automated vehicle should meet when operating in a mixed traffic environment featuring vehicles of various automation levels and advances a Bayesian AI-based driver algorithm as a solution.

2 Cognitive Vehicle

Researchers and automotive industry aim to develop advanced technology vehicle designs that exhibit ‘cognitive’ features. The cognition capability for a fully automated vehicle is very challenging but is essential. In addition to non-distracted and non-aggressive driving, the situation awareness capabilities can be designed that should extend human driver capabilities. These include faster perception and reaction time to avoid hazards, based on acquiring and analysing vast amounts of data almost instantaneously.

Heide and Henning [3], Stiller, Farber, et al. [4], and Hoch, Schweigert, et al. [5], made the calls for cognitive features of an automated vehicle. This preliminary, but ground-breaking research work defined capabilities of the “cognitive car” for perceiving itself and its environment, and collecting and analysing information autonomously for making driving decisions. That is, the cognitive vehicle is expected to acquire and process vast amount of data systematically, to make driving decisions autonomously while enhancing its ability to reason and to learn [3].

On the technology side, The National Research Council of Canada (NRC) developed a Cognitive Vehicle Technology Platform that featured advanced automotive information and communication technologies (ICT). The NRC’s R&D was among early attempts to rationalize application of on-board sensors and electronic controls for enhancing safety and performance. In addition to safety and performance objectives, it was also stimulated by rising demand for integrated and wirelessly connected communications and infotainment devices [6].

The cognitive vehicle features are presented in Table 1. Based on occupant and societal expectations, these should address safety, comfort, efficiency, eco-driving, and convenience requirements. There is a strong role for artificial intelligence-based algorithms supported by technology in meeting these objectives. The following sections of the paper describe these challenges. Here, as an example, progress in technology in support of situation awareness is briefly noted.

Table 1. Cognitive features and their functions

Awareness of position and surroundings is obtained by integration of technology and methodology. A scan of technology shows that competitive forces are causing the industry to advance these while bringing the cost down. The following are the commonly used technologies for enhancing accuracy and reliability:

• (a) Latest available communication system (e.g. G5 when it will become commercially available, dedicated short range communication system (DSRC), long term evolution (LTE))

• (b) Multiple camera surround view

• (c) Sensor hub with multi-camera input

• (d) Low latency sensor data availability for supporting cameras

• (e) Radar, Lidar, GPS, etc.

• (f) Road geometric feature processor

• (g) Traffic data in subject and surrounding lanes obtained from multipurpose sensors and communication system

• (h) GPS-map/curve speed processor

• (i) Lane tracking sensor data processor

• (j) Road weather scanning feature, and static object location scanner.

This list of technologies and methods should be viewed as incomplete since it is intended to only illustrate advances underway in support of the situation awareness model that integrates the data inputs and dynamically updates the driving environment information for making autonomous driving decisions.

3 Challenges of Accuracy and Reliability

Accuracy and reliability attributes are essential for fully automated driving. Although progress has been made in these areas, setbacks have also been encountered recently by some developers. The reliability capability ensures that system performance does not degrade under difficult driving environments characterized by geometric design features of facilities, traffic control, traffic flow encompassing vehicles with various automation levels, vehicular traffic density, cyclist and pedestrian traffic, and environmental conditions.

Much uncertainty can be expected in quantifying variables that define in real time the states of the driving environment. Although the technology in support of the automated driving is improving, realistically these states will remain uncertain. Two examples are relevant here. In urban or suburban medium speed driving environment, a distracted pedestrian may pose a hazard. In the vehicle-following context in highway driving, stopping distance required by a vehicle in an emergency may not be known with certainty.

Even when the sensors and models that provide tire-pavement friction data are operational, the safety distance to the vehicle ahead requires information about the road section ahead of the decelerating vehicle in order to avoid a collision. Although the connected vehicle technology can source the pavement condition information from the vehicle ahead, validity of data cannot be assured due to potentially different tire condition. Under automation, the estimation of distance and processing of friction data enables a better estimate of safety margin as compared to driving under human control. However, the certainty assumption is not realistic.

The issues of accuracy and reliability have to be overcome for occupant and safety regulator acceptance of fully automated vehicles. Although safety is of utmost importance, the requirements of efficiency, comfort, convenience, and eco-driving should also be met in automated driving. For this reason, the vision of the intelligent vehicle became increasingly ambitious over the years. From a safety perspective, as noted earlier, an intelligent vehicle in its advanced form should have cognitive features that can perform driving tasks in all driving environments. In its fully automated form, a cognitive vehicle should have the capability to make a corrective active safety decision under all driving conditions, as an experienced human driver is expected to do. For this reason, according to a news article, development of ‘human-like’ self-driving technologies is attracting investor capital [7].

4 Automated Driving Based on Bayesian AI

The high-level architecture of the Bayesian AI-based automated driving system is shown in Fig. 1. For a general introduction to the subject of Bayesian AI, the reader is referred to References [8] and [9].

Fig. 1.

High level architecture

Major functions of the automated driving system are noted in Fig. 2. A detailed explanation of these functions is presented in other papers of the author [10, 11]. Here, due to space limitation, only brief notes can be provided. Likewise, Fig. 3 presents only highlights of the Bayesian AI-based driver algorithm for longitudinal control of the vehicle. Details can be found in References [10] and [11]. The capability for lateral control is almost identical, but it is not included in this paper.

Fig. 2.

Major functions of the longitudinal and lateral control system

Fig. 3.

Algorithm for the longitudinal control system (Khan, Reference [11])

The high level architecture items shown in Fig. 1 are presented in Fig. 2 as major functions of the automated control system. These are noted below.

• Model of the driving environment

• Location of vehicles and separation distances

• Model for tracking safety margins

• Self-calibration model

• Model for characterizing driving states

• Bayesian AI model for driving decisions:

• optimal timing and nature of driving actions

• (i.e. acceleration/deceleration, speed level)

In this paper, the primary focus is on the vehicle-following task, but the model is equally valid for maintaining target side-separation distance between vehicles. The Bayesian AI-based system treats uncertainties of the traffic environment in making control decisions.

Figure 3 presents the high level logic diagram of the automated driving algorithm. According to the formulation of the algorithm, the longitudinal control system in the vehicle-following context can be deployed any time to perform the following functions:

• a. Decelerate to avoid a collision

• b. Accelerate to reach the target distance or attain a desired speed

• c. Maintain the target distance to the leading vehicle.

According to the design, the transition from function to function is seamless and automated under machine control. If the subject automated vehicle is following another vehicle which is not operating under longitudinal control, or if a vehicle from a neighbouring lane cuts in front of the subject vehicle, the system has to adapt and to accommodate these demands.

There is a need to improve the capability to adapt to prevailing traffic flow condition so as to ensure safety and at the same time to avoid abrupt speed changes while maintaining a target distance to vehicle ahead or to attain the desired speed in the vehicle-following driving environment. Further, other design challenges of the cognitive vehicle noted in Table 1 have also to be met by the automation system.

For on-line application, the location and distance estimation capability is built in the cognitive connected vehicle. The system is able to identify driving states with potential for a rear-end crash. Vehicle speed, road condition and other driving environmental factors are used as a basis for this task. The longitudinal control model has the capability to automatically update key driving parameters, namely the probabilities of critical and longer distances, as well as the probability of correctly estimating distance to the leading vehicle.

The algorithm shown in Fig. 3 can identify optimal control actions. These are the timing of deceleration/acceleration action (i.e., immediate or wait) and the magnitude of speed change. In the case of deceleration, the options are no change, normal deceleration and emergency deceleration. In the case of acceleration, the options are no chnage, normal acceleration, and somewhat higher acceleration.

The variabels used in the algorithm are defined as follows:

• d_c is critical distance (requires emergency deceleration in order to avoid a crash); d_1.25c and d_1.5c are used for monitoring and analysis of deceleration cases

• d_1.5c, d_1.75c and d_2.0c are used for monitoring and analysis of acceleration cases

• P’(d) is the prior probability of distance d

• e_o represents no additional information acquisition decision, and therefore calls for immediate action without waiting for additional information

• e_w implies waiting for additional information

• r₀ corresponds to e₀ (i.e. no new information on distance)

• r_c, r_1.25c, r_1.5c, r_1.75c, r_2.0c are readings on corresponding distances (i.e. d_c, d_1.25c, etc.)

• Reliability in recognizing the driving condition by the longitudinal control system is expressed as P(r|d,e) – the conditional probability

• P(r|d) is the marginal probability

• P”(d|r,e) is the posterior probability

• G matrix represenre utilities (also called rewards) that apply to a and d combinations

• For deceleration case, decision options are: a₀ no action – maintain target distance, a_N calls for normal deceleration, a_E calls for emergency deceleration

• For acceleration case, decision options are a₀ (maintain target distance), a_N (normal acceleration) and a_H (higher than normal acceleration).

During a driving mission, the system has to estimate safe target separation distance between vehicles and also the critical distance (d_c) for emergency stopping. This applies in the longitudinal direction in the vehicle-following driving environment. Likewise, in order to avoid a lateral collision, the safe distance between the envelopes of vehicles must be known. These are compared with the available distances on a real-time basis.

5 Example Application

A distracted driver log was extracted from a driving simulator experiment carried out with participation of over 80 young drivers. The driver was searching for music (a song) and did not perceive a hazard posed by the vehicle ahead in the form of abrupt lane change and brake application. After a perception time of 4 s, which is much higher than the usual 1.5 s for alert drivers, the distracted driver was able to avoid a collision as a result of high emergency level deceleration (Fig. 4). Figure 5 shows the speed profile of the distracted driver and Fig. 6 presents the distance to the vehicle ahead. This high level of emergency brake action is uncomfortable for the occupants of a vehicle and also results in a shock wave that travels upstream in the traffic flow and may result in accidents.

Fig. 4.

High level of deceleration by the distracted human driver

Fig. 5.

Comparison of speed profiles of distracted driver and the automation system

Fig. 6.

Comparison of distance to vehicle ahead for the distracted driver and the automation system

A fully automated vehicle was following the vehicle driven by the distracted driver. In contrast to the distracted driver, the automated driving system was able to detect the hazard including the follow-up delayed action of the distracted driver and avoided abrupt speed change. Figures 5 and 6 present a comparision of speed profiles and distance to vehicle ahead for the automated vehicle vis-à-vis the distracted human driver. The Bayesian AI-based algorithm presented in Fig. 3 is credited with the automated driving system performance.

The results clearly show that the Bayesian AI-based algorithm performed according to the design. The distance to vehicle ahead ensures safety even under an emergency condition and the change in speed is comfortable for the occupants.

6 Conclusions

The model development and its application described in the paper lead to the following conclusions.

• 1. Bayesian AI-based driver of the fully automated vehicle is likely to overcome the known deficiencies of current versions of automated drivers as reported in research literature.

• 2. The Bayesian AI-based driver will enable the automated cognitive vehicle to meet safety and ride quality criteria while operating in driving environment characterized by uncertainties.

• 3. The cognitive vehicle design is likely to enhance consumer and safety regulator acceptance and consequently safety and ride quality benefits will be achieved.