Keywords

1 Background

Today both traditional and non-traditional auto companies are investing heavily in research projects to develop automated applications for vehicles (traffic jam assist, automated braking etc.) that minimize the role of the driver. Increasingly, a car’s capabilities are determined more by its electronics and software than by its mechanics. The industry is envisioning a system of vehicle-to-vehicle communications, cameras, variety of sensors (Radar, LIDAR, RFID, etc.) and other devices integrated with advanced algorithms that can monitor the road in a variety of roadway, weather and traffic conditions to enable driverless systems.

Vehicle automation can be divided into the following major subsystems that implement a control loop, as described in automated vehicle reference model architectures such as the National Institute of Standards and Technology/4D Real-time Control System (NIST 4D/RCS) architecture:

  1. 1.

    Sensing—acquire raw data from vehicle sensors

  2. 2.

    Perception—process the raw sensor data and create usable information on the vehicle and its environment

  3. 3.

    Planning—use the perception of the vehicle’s current environment and situation as well as a priori data to plan short and long term actions and behaviors

  4. 4.

    Control—implement the short and long term actions and behavior.

This high level control loop is only intended to illustrate common approaches for many automated vehicles to date. This control loop also shows an example for conceptual processes and components of each subsystem. It is not intended to be an all-inclusive list of sensors, algorithms and behaviors, and in fact specific hardware and software approaches can vary wildly across the industry. The control loop in Fig. 1 mirrors the approach that we as humans take in sensing, perceiving, and interacting with our world.

Fig. 1
figure 1

High-level control loop

Full size image

To implement these subsystems, a wide variety of well-entrenched and emerging technologies are integrated to provide the appropriate level of vehicle automation. These enabling technologies are grouped into the following high-level categories and were researched as part of this research effort (Fig. 2).

Fig. 2
figure 2

Research categories

Full size image

2 Research Approach

The research team scanned a combination of approximately 250 popular and technical press sources, investigating over 100 key observations to extract needs, trends and gaps in each technology area.

Based on these insights next steps for USDOT considerations were developed. A graphic representing this high-level process is provided in Fig. 3 for clarity. In addition to exploring the current state of technology, the research team also developed predictions about near- (5 years), mid- (10 years), and long-term (15 years) technology milestones. Through an analysis of the team’s observations, trends were extrapolated to illustrate possible future technology advances that will enable higher levels of vehicle automation. These predictions are designed to help USDOT better understand the role they can play in shaping and accelerating the deployment of automated vehicle features.

Fig. 3
figure 3

High level process

Full size image

Different levels of automation are enabled by advances in the above mentioned technologies. The NHTSA [1] defines different levels of automation as given in Table 1.

Table 1 Levels of automation (NHTSA)
Full size table

3 Summary of Findings

3.1 Position, Navigation and Timing

Positioning, Navigation, and Timing (PNT) technologies require high accuracy and reliability to enable vehicle automation. To meet these needs, researchers and product manufacturers are continuing to develop positioning systems that are smaller, faster and much more capable than their predecessors and are at much lower price points. These positioning systems are evolving to not just provide an output based on a combination of Global Navigation Satellite System/Inertial Navigation System (GNSS/INS) but also to provide frameworks that can incorporate other data such as wheel speed encoder odometry, visual odometry, and data from other sensors. In parallel, redundant techniques are emerging to provide precise timing in situations where GNSS is unavailable. This capability could greatly help automated vehicle applications that are dependent upon precise time synchronization for cooperative aspects.

To address the gaps that still remain for PNT, advancing research to provide corrections to vehicles from roadside infrastructure and how that applies to vehicle automation has high potential. Enhanced navigation information could be provided from the infrastructure as part of the digital infrastructure initiative. This information could include detours and safe harbors for automated vehicles in scenarios where the technology decides to intervene with the human driver. Investigating redundant timing techniques and approaches for safety critical key infrastructure components is also critical to advance PNT to enable automated vehicle applications. Furthermore cyber security is a major concern but many do not consider the vulnerabilities of GNSS signals. More research is needed for this area as well.

3.2 Mapping

Mapping technologies are focused mainly on macro-level maps that would allow a vehicle to navigate along corridors, cities, etc. as well as mapping the immediate area around a vehicle to determine its relative position and orientation with respect to everything around it. These algorithms must operate in real-time and be able to handle challenging conditions and dynamic environments. At the macro level, private sector companies are continuing to collect and develop extremely high resolution maps that could be used by vehicle automation systems ranging from Advanced Driver Assistance Systems (ADAS) to fully automated, driverless vehicles. In parallel, researchers are continuing to advance Simultaneous Localization and Mapping (SLAM) techniques to improve the capabilities of visual SLAM in challenging lighting conditions (dawn, dusk, night, rain, fog) as well as changing environments (seasonal changes, cities). Researchers have also been developing techniques for cooperative vehicle mapping to allow one vehicle to utilize the perception capabilities of another vehicle to more safely navigate through its environment. Finally the increasing availability and performance of Graphical Processing Units (GPU) is enabling these complex algorithms to run in real-time.

While advancements are continually being made in mapping technologies, there are significant gaps that still remain. Information that is being proposed to be provided from the infrastructure is limited in nature (e.g. SAE J2735 SPAT/MAP/TIM). This could be significantly enhanced by sharing a more rich set of situational awareness information between static infrastructure points and vehicles to enable vehicle automation. Research in this area is conducted to investigate reliable and efficient methods of exchanging situational awareness between infrastructure points and vehicles which could greatly improve safety of vehicles (at all levels of automation) around intersections, work zones, school zones, etc. A richer set of map data to be shared from the infrastructure can also be very useful. Finally, building off of research that was conducted on sensor-friendly infrastructure markings that take into consideration many of the new vehicle perception and localization techniques will help advance mapping to the next stage.

3.3 Communications

The successful implementation of connected/automated vehicles depends on a reliable and secure communication infrastructure. Recent advances in communications, controls, and embedded systems are transforming vehicles to sensor platforms, collecting data from other cars and the cloud, and feeding it back to drivers and infrastructure to assist in safe navigation, pollution control, and traffic management. The next step in this evolution is utilizing Machine to Machine (M2M) communication and Internet of Things (IoT) and creating the Internet of Autonomous Vehicles. The promise of 5th generation wireless systems (5G) with its unique capability of transmitting gigabits of data will pave the way for this evolution. Autonomous vehicles rely on spectrum for high-performance radar, laser, LIDAR (light detection and ranging) systems, and satellite communications for location mapping. Given the tight spectrum resource, it is important to consider alternative technologies for autonomous vehicles including cellular networks, 4G/5G, Wi-Fi, and VLC/Li-Fi among others.

Expansion of M2M and telematics in the vehicular networks exposes the network to external cyber-security threats. As such ensuring vehicular network security is an important part of these developments. Security areas that need attention include: security of the infrastructure, encryption of all the communications on the cloud, and building robust authentication schemes. Also, as smart devices are integrating with vehicles there is an added threat of personal information theft that should be addressed. Moreover successful implementation of connected/automated vehicles requires infrastructure investment, coordination among international standard bodies, and enabling policies, laws, and regulations development.

3.4 Sensing

The need for advanced in-vehicle technologies and intelligent infrastructure constantly drive innovations in sensor technology. The new sensor technologies are either enhancing the current sensor capabilities to have a better range, accuracy, and robustness or to develop completely new smarter and smaller sensors. Overall, the focus of sensor technology has been on generating quality data that supports development of new safety applications and capabilities. These improved sensors not only provide accurate data but also provide data at a higher frequency. As a result the market has also seen advances in microprocessors that manage sensors and analyze sensor data to provide meaningful outputs for control systems. These microprocessors enable sensor fusion to provide redundancy for safety applications, making sensors more powerful. Sensor fusion thus improves efficiency and enables the development of several automated vehicle safety applications. The use of cloud based systems to analyze real-time data from a variety of sensors and other dynamic events is also gaining popularity and further enables sensor fusion. Technologies which process data in real-time provide opportunities to develop several safety applications.

Accurate and reliable 360° sensing of vehicle surroundings, sensor fusion and data accessibility, enhanced performance in adverse weather conditions, and more robust standards and safety mandates for V2V and V2I applications are some of the needs and gaps identified. Though most of the sensor related technologies will be vetted by the market place, tracking these advances could be of potential importance for infrastructure applications that support automated vehicle applications. For example using stereo cameras at a complex urban intersection to get a 360° view of the intersection could help develop better signal timing strategies.

3.5 Human Factors

A high level of trust must exist between the user and the technology to enable vehicle automation dependency. Consequently, the human factor research of automated vehicles is vital for the acceptance/usage of such a technology. Based on the recent technology observations, there is a need for cooperative and non-distracting devices to support the different level of automation and the smooth transition from human control to the automated control. In addition, the mixed environment of automated vehicles and legacy vehicles still need to be investigated. Some of the observed trends based on the conducted research showed that the driver state sensing technology to mitigate driver distraction is underway by several OEMs. It was also found that many driver information/assistance display technologies are claimed to be ready for market usage. Although the automated vehicles technologies are rapidly evolving, there are many research gaps that still remain. For example, the research related to interaction of automated vehicles with other vulnerable road users (also pedestrians, cyclists, and legacy vehicles) is still lacking. Another example, is that, the impact of many of the current technologies (including after-market devices) related to HMI on safety is still unknown; especially for the different levels of automation.

Though many of the current technologies (including after-market devices) related to Human Machine Interface (HMI) may potentially enhance safety of legacy vehicles and improve driver awareness, their impact on safety is still doubtful. At the end, it can be stated that there is a need to investigate the influence of this type of technologies on enabling vehicle automation on the streets; especially when it comes to elderly drivers or drivers with disabilities and/or special needs. This kind of analysis is very critical in the development of a framework to assess safety of self-driving algorithms. Safety for pedestrians and bicyclists also can be assessed using surrogate safety studies that can model autonomous vehicles explicitly.

4 Discussion

Based on research findings, a high-level summary of how different technologies map to different levels of automation is depicted in Fig. 4. These technologies are seen as cumulative, i.e., for communications, the advent of 5G vehicle automation applications does not replace DSRC. This figure shows the progression of technologies relative to other technologies in the same technology area, and does not imply that technologies that are vertically aligned will occur at the same time. It is critical to note that there are many technologies and combination that can enable different levels of automation, e.g., to solve the problem of high accuracy self-localization, one application may rely more on sensor fusion while another may rely on more expensive inertial measurement units. Therefore this figure represents only those technologies that were encountered in this research process.

Fig. 4
figure 4

Advances in key technologies will enable higher levels of automation

Full size image

The research observations were used as the basis for predicting which technologies will most profoundly impact vehicle automation in 5, 10, and 15 years. These key technologies, described in Fig. 5, will be instrumental in achieving the high level of reliability and safety that different automated vehicle applications require.

Fig. 5
figure 5

Predictions for near, mid, and long term advances in technology that will profoundly impact and advance automated vehicles

Full size image

Significant improvements in self-localization are expected due to contributions from GPS/INS, SLAM, maps, infrastructure support, and cooperative localization. Self-localization can currently be estimated within about 1.5 m in ideal conditions. Within 5 years, advances in GPS, INS, and SLAM [2] with higher definition maps [3] will enable “limited self-localization” to within about 10 cm. To achieve the next level of accuracy within 10 years (approximately 2 cm accuracy), better SLAM and real-time maps [4] will be needed that capture dynamic changes in the environment. Within 15 years, further advances in GPS, infrastructure support (e.g., RF localization like UWB), and cooperative localization [5] will allow for “advanced self-localization” to within 1 cm, which is critical for Level 3 and 4 automation.

Algorithms are an area where considerable advances are expected. Stereo vision cameras have been recently used more as a less expensive alternative to improve the reliability of sensors and provide a wider range of detection for different ADAS applications. Currently fully autonomous vehicles are using a combination of LIDARs, radars, and vision based sensing systems to complement the weaknesses of each kind and provide redundancy [6]. Based on the observations the team made during the course of this research, there is a push towards vision based sensing as the potential first class player in the area of perception technology [2]. The need for sensor fusions to combine the data elements received from multiple sensors in order to improve performance and reliability of the overall system has also been focused on. Advances in sensor fusions are needed to address challenges in the higher levels of automation, such as robust object detection and classification, adverse weather conditions, etc. In particular AI techniques, such as machine learning, are proving to be effective at improving processing speed and performance [7]. AI will be crucial to achieving Level 4 automation, and is predicted to reach a high level of functionality within 15 years.

Many companies are exploring the use of cloud data sharing and computing and potentials to wirelessly connect cars, cyclists, and pedestrians to the cloud. Using cloud data along with advanced sensing technologies may potentially improve the detection, warning, and auto braking stages of collision avoidance applications. Also by analyzing the data in real-time and updating the map on the cloud, information such as adverse environmental conditions [8], traffic jams, accidents, and other dynamic events can be systematically transmitted to the vehicle so it can adjust its route or other course of actions [9]. Low latency, high bandwidth communication is needed to enable this information sharing. Current advances in communication are occurring rapidly [10], which gives the research team confidence that 5G will be delivering value to automation within 10 years.

The complex interaction between human and machine will continue to be a challenge for years to come, especially as infotainment continues to compete for driver’s attention [11]. Achieving safe Level 3 operation will require a better understanding of transition of control between driver and machine, which require simulators and test beds that are just now coming online [12]. As with automation in the aviation industry, challenges related to transition of control will likely continue to persist, perhaps beyond the horizon of predictions made in this report. Driver monitoring technologies are still being researched [13], and could begin to play a significant role in automation in the next 10 years. The research team found research on the impact of automated vehicle technology and other road users sparse [14], and expects this challenge will persist until much higher market penetration of automated vehicle technologies and aftermarket solutions [15] are achieved.