Driver assistance systems increase both safety and driving comfort. AVL and Daimler describe here a method for the automatic assessment of quality and safety in these systems. This is illustrated by the example of three important features: adaptive cruise control (for longitudinal guidance), lane keeping assistance (lateral guidance) and automated respectively autonomous driving. Thus, AVL implements new rating procedures for assisted driving into the tool AVL-Drive.

Initial Situation

Advanced Driver Assistance Systems (ADAS) increase on the one hand the safety on the road by alerting drivers to critical situations which can lead even to taking action autonomously to avoid collisions or reduce the severity of accidents (for example emergency braking function). On the other hand, they enhance driving comfort with features like automatic parking, lane keeping assistance and adaptive cruise control. Drivers and passengers will only perceive a gain in comfort and safety if the support given by the assistance system is provided in a safe, reliable and pleasant manner.

Objective assessments of driveability perception, safety and perceived safety of ADAS are a great help to developers, because they give precise feedback on the shortcomings of the application in question, allowing developers to eliminate such weaknesses effectively.

From the broad variety of new or advanced driver assistance systems, this article deals with these ADAS functionalities: distance control as longitudinal guidance using Adaptive Cruise Control (ACC), lane assistance as lateral guidance using Lane Keeping Assistance (LKA) as well as functions for automated/autonomous driving.

Concept

The methodology pursued by the AVL tool AVL-Drive to achieve an objective assessment of subjective impressions covers the objective assessment of longitudinal dynamics (operating and driving behaviour of engine and transmission), lateral dynamics (handling, steering, chassis suspension) and vertical dynamics (chassis suspension comfort). Based on measured variables supplied by sensors and control devices, a variety of operating modes are detected automatically, using fuzzy logic and behaviour patterns. These are physically evaluated in real time and assessed online by applying ratings in much the same way as driveability experts would do on the basis of their own subjective perception. For developing the rating modules for ADAS, this methodology was extended by adding an environment detection system. The methodology shown in ➊ was applied in the project.

➊
figure 1

Methodology for the objective assessment of subjective perception

Full size image

Measuring Techniques and Analysis

The choice of sensors and measuring technique was made on the assumption that the assistance system was to be assessed either by the environment detection system built into the vehicle as standard or by special additional sensors. This would allow the use both of series-production sensors and additional sensors. Radar sensors and lane-measuring video cameras serve as additional sensors.

The subjective ratings were provided by male and female subjects of various ages and with different levels of driving experience. The study was conducted on vehicle end-customers and driveability experts, who gave their subjective ratings on the driver’s seat (as driver) and on the front passenger seat (as passenger). Based on the evaluation of the surveyed criteria in conjunction with the comments in the questionnaires, it was possible to determine the necessary criteria for an assessment of both ACC and LKA.

Correlation of Subjective Perception and Physical Parameters

To objectify subjective perception, it is necessary to define physically measurable variables that correlate with subjective perception. In the case of the driver assistance systems we are dealing with, the physical parameters are fundamentally very different, as the ACC has an impact on the vehicle’s longitudinal dynamics, whereas the LKA and automated driving also affect the lateral dynamics. Even so, each key criterion results in an obvious main input variable. The accuracy of the ACC distance control is, of course, dependent on the distance from the vehicle ahead. In addition to what is probably the most important key criterion, a large number of additional physical variables were measured too to ensure a good correlation.

LKA Systems

Before developing the objective assessment criteria and functions for tracking stability, lane control and choice of driving line for an autonomous vehicle with lateral guidance system or LKA system, extensive subjective ratings were carried out, as mentioned before. An analysis of the measured data from the test drives revealed that achieving a mathematical description of the subjective perception of tracking stability and choice of driving line is an extremely complicated matter. The perception of choice of driving line and lane control is not only characterised by a multitude of physical vehicle variables but also by the way the environment and the course of the route are perceived. The perceived safety provided by the LKA system’s choice of course depends on the difference between the current and the expected choice of course.

Lane control rating is perceived as the control accuracy of steering wheel angle and yaw angle, as well as the choice of a driving line that seeks to follow a trajectory within the lane, which the driver would perceive as pleasant and safe in all driving situations. Vehicle passengers will perceive the chosen driving line as calm, controlled and safe if, due to its combination of curves, the driving path largely corresponds to an ideal line with radiuses as large as possible and smooth changes of direction.

Studies on chosen test routes showed that when driving curves with very low lateral acceleration humans drive approximately in the middle of the lane and, already at low lateral acceleration, they have a tendency to move toward the ideal driving line with radiuses chosen as large as possible.

Here an example for choice of driving line assessment: ➋ (left) shows the driving line chosen exactly in the middle of the lane (blue line) for a slight left-right-left curve combination. In the analysis of subjective perception, this choice of driving line is rated as acceptable due to the relatively clear directional changes at road points 2 and 3 with regard to lane control.

➋
figure 2

Subjective assessment of lane control on curvy lanes

Full size image

➁ (centre) shows a slightly delayed lane control system reaction with regard to the centre of the driving line (blue line). Here, the relatively sharp changes in direction are perceived and rated as too late and, accordingly, also as unsafe at point 3, as the car steers too far towards the oncoming lane and traffic.

The green choice of driving path in ➁ (right) corresponds to an ideal line with the largest possible curve radiuses and the smallest possible directional changes within the lane. This behaviour was rated best in view of safety and superiority perception. On the one hand, the example shows the demands placed on the control quality and the tuning of ADAS for automated driving and LKA. On the other hand, it reveals the high complexity involved in finding a way to represent human subjective perception of lane control based on physical measurement parameters in an objectively assessable way. In addition, the traffic situation (in ➁, the oncoming traffic) may also affect subjective perception, and thus the choice of the driving path.

ACC Systems

The physical parameters which needed for assessing an ACC system objectively are divided into vehicle mode data and environmental interaction data (situation relative to vehicle ahead or vehicle cutting in, traffic situation) as mentioned before for the LKA system.

Research on subjective perception has shown that car occupants exhibit potentially the highest sensitivity to the ACC system’s start of braking and braking behaviour when the vehicle approaches a vehicle ahead moving at a slower speed or braking. In such cases, sensitivity is even higher than when encountering a so-called cut-in manoeuver.

Application Examples

With regard to the development and tuning of ADAS, the use of AVL-Drive offers developers great advantages, especially for the analysis and evaluation of test drives. ➌ shows the objective ratings of the LKA system with the lane precision criterion of the LKA system on a longer test drive in semi-automated/autonomous driving mode.

➌
figure 3

Test drive in autonomous driving mode with objective rating, lane precision criterion (DR = driveability)

Full size image

In ➍ the ACC systems of four different vehicles are compared. Clear differences can be seen in the way they were tuned by the individual car manufacturer. Vehicle 3 exhibits a weakness in its stopping behaviour because during the tests it occasionally stopped far behind the vehicle ahead, leading to a poor rating. On the other hand, Vehicle 3 was very agile and pleasant when pulling away from standstill.

➍
figure 4

Comparison of four ACC systems of different manufacturers following a preceding vehicle with activated function

Full size image

Benchmark Results

The AVL-Drive tool has been used in the vehicle and powertrain development process for many years to improve the quality of the vehicles and to enhance development efficiency. ➎ shows the achieved quality improvements in series-production vehicles of different vehicle classes from 2006 till today (area in blue). The slight decline in the increase is caused by fuel consumption measures such as start-stop systems and downsizing.

➎
figure 5

Benchmark results for ACC and LKA systems of series-production vehicles

Full size image

➄ also shows the overall assessments of ACC and LKA systems of different series-production vehicles. In current series-production vehicles, ACC systems all reach consistently a very high level of quality. Substantial improvements are also seen in the relatively new LKA systems. Nevertheless, there is furthermore room for improvement, achievable by the use of better sensors and improved evaluation and control algorithms.

Summary and Outlook

The presented method for objective assessment of driveability quality in driver assistance systems is implemented in the AVL-Drive tool. New driving mode detection functions and new objective assessment functions for subjectively relevant driveability criteria for ADAS, such as semi-automatic driving with ACC, LKA and highly automated/autonomous driving, have been developed. In future, the new AVL-Drive module for assessing driver assistance systems can be employed in the vehicle development process to achieve improved tuning for the ADAS much more efficiently. The module additionally enables automatic evaluation of test drives in view of quality and safety.

The vast range of subjective ratings delivered essential insights into what customers would perceive as the perfect ADAS design. Particularly important, in this respect, is the factor “perceived safety” alongside the factors comfort, dynamics and real safety. These insights will be incorporated into the further development of ADAS on a continuous basis.