euroFOT: Field Operational Test and Impact Assessment of Advanced Driver Assistance Systems: Final Results

Abstract

Within the first European large scale field operational test euroFOT the effects of eight different Advanced Driver Assistance Systems (ADAS) were investigated. About 1,000 vehicles equipped with different ADAS were used. The Institute of Automotive Engineering (ika) of the RWTH Aachen University analysed the effects of ACC usage in combination with FCW under normal driving conditions of 100 passenger cars. The results of the data analysis show positive effects on traffic safety and fuel consumption. In addition, the complete process chain from the data collection to the analysis was defined and conducted at the ika.

F2012-F07-012

1 Introduction

One of the main challenges of road transport is the reduction of fatalities. The worldwide number of fatalities exceeds more 1.2 million per year [1]. To identify possible approaches to improve traffic safety various initiatives were launched in Europe within the last years. One of these approaches is the wide deployment of Advanced Driver Assistance Systems (ADAS).

These systems are designed to support drivers in their daily driving routine by increasing driving comfort, safety and efficiency with regard to traffic flow as well as fuel consumption. In addition to different studies [2] Field Operational Tests (FOT) were conducted within the last few years that aim to investigate short- and long-term effects of ADAS under normal driving conditions. The first large-scale FOT for investigating effects of ADAS was started within the seventh framework program of the European Commission. Objective of the euroFOT project was to assess the impacts of eight different ADAS with regard to driver acceptance, traffic safety and efficiency as well as fuel consumption. All tested functions are in production and are integrated as such in the vehicles. Overall, about 1,000 vehicles were used for data collection in the project.

This paper presents the results of the conducted analysis of the collected data of 100 passenger cars equipped with an Adaptive Cruise Control (ACC) and a Forward Collision Warning (FCW) function.

2 State of the Art

Field operational tests belong to experiments under realistic settings and are an important method to assess the impact of ADAS because of low controlling factors. This allows investigating the naturalistic driving behaviour under real driving conditions which is highly important for the assessment of ADAS. Based on the collected data the analysis answers predefined research questions to assess the impact. To gather a sufficient amount of data, an experiment duration of several months is necessary.

In the U.S. large-scale FOTs are used as an evaluation method since 1996 [3]. In addition to FOTs of Intelligent Speed Adaptation Systems (ISA) [4], Forwards Collision Warning (FCW) [5], Lane Departure Warning (LDW) [6], Adaptive Cruise Control (ACC) [7], and naturalistic driving studies were conducted [8].

Also in Europe a growing number of large-scale FOTs can be observed. First activities were started on a national level and afterwards extended to cooperative projects between different partners in the European Union (e.g. euroFOT). The following table highlights some of the conducted FOTs in Europe and the US (Fig. 1).

Fig. 1

Overview of FOTs in Europe and the US

The first large-scale FOTs were carried out in Sweden (1999) [4] and Great Britain (2000) [9] and focused on the safety potential of Intelligent Speed Adaptation (ISA). On European level the TeleFOT [10] started at the same time as the euroFOT project.

3 Methodology

In the following, the analysis approach for the ACC and FCW used in euroFOT is presented. To that end, first the defined experimental design of both systems is described and afterwards the data analysis approach of the collected data is given.

3.1 Experimental Design

The participants of the FOT are non-professional drivers that were recruited by different car dealerships in Germany. They were contacted by the dealerships after buying a vehicle of the specific manufacturer equipped with ACC and FCW. To collect all relevant data (CAN data and GPS data) data loggers were installed in all 100 vehicles. The data loggers are able to record, temporarily store and transmit the data afterwards to a centralised server system. Overall, about 100 CAN signals were recorded. In addition to dynamic measures (velocity, acceleration, yaw rate, steering angle, wheel speed etc.) status information of different systems were considered (e.g. state of turn indicator, wiper, ADAS). To avoid modifications of the customers’ vehicles and influences to the driving behaviour no additional data sources like video systems were used.

For the impact assessment it is necessary to assess also trips without the influence of the system as a reference (baseline) besides trips with activated system (treatment). The comparison of the collected data in both phases (baseline/treatment) forms the basis for the assessment of the systems’ impact. Based on these research questions hypotheses to be tested (e.g. ACC decreases the number of incidents) have been defined. By means of the hypotheses the required signals and data sources for data collection have been identified.

The experimental duration of the FOT for the vehicle fleet of 100 passenger cars was twelve months of which the first three months served as a baseline. During this baseline period the ADAS were not available and the naturalistic driving behaviour of the participants was analysed. In the following treatment period the systems were activated and the drivers were free to use the systems as desired. Neither further instructions were given to the drivers nor were they accompanied by a supervisor. Drivers used the vehicles in their daily routine (e.g. way to work). The experimental design for the 100 passenger cars is shown in Fig. 2.

Fig. 2

Experimental design for the vehicle fleet of 100 passenger cars

In addition to the selection of relevant driving scenarios (e.g. incidents, lane change manoeuvres) different indicators are necessary for the impact analysis. Especially for the hypothesis testing the so called performance indicators (e.g. number of harsh braking events, average speed) are required for the statistical analysis. Moreover, so called situational variables such as weather conditions or road type are relevant for the hypothesis testing in order to ensure that the comparison of baseline and treatment is done under similar circumstances to avoid influences from external effects.

Furthermore subjective data is collected by means of four time-based (Time 1 to Time 4) questionnaires to analyse the effect on user acceptance. The first two questionnaires (Time 1 and 2, see Fig. 2) were spread during the baseline phase. The remaining questionnaires (Time 3 and 4) were distributed at half-time and at the end of the treatment phase.

3.2 Data Analysis

The required signals were recorded with defined sampling rate. By means of the installed data loggers the collected data were transmitted wirelessly to the centralised server system at the ika (see Fig. 3). After the successful transmission of the data to the server (within the data management) the data processing is conducted. Firstly, the data quality analysis includes the assessment of missing data parts and plausibility checks of the recorded signals. In a second step, the gathered data was enriched with map attributes (e.g. road type, speed limits) from a digital map based on the recorded GPS information. Afterwards, additional information (e.g. time-headway, Time-to-Collision (TTC), average speed) was calculated from the collected data. Finally, situational variables and relevant driving events were identified by means of an automated recognition process [11].

Fig. 3

Processing steps between data collection and impact assessment

The data analysis starts with the calculation of the performance indicators that are necessary for the hypothesis testing. Afterwards, the processed data was stored on a database, which served as a basis for the data analysis. The results of the hypothesis testing were used as input for the impact assessment.

The processing of the raw data led to an increase in the data amount due to the enrichment and derivation process. Overall, about 1 TB of data is available for the impact assessment, see Fig. 4.

Fig. 4

Overview of data amount

4 Results of the Impact Assessment

The magnitude and the experimental set-up of the conducted FOT allow detailed insights into various aspects of the daily use of ACC and FCW by normal customers. In the following some of the main outcomes of the conducted statistical analysis are presented. Thereby, the analysis focuses on changes in traffic safety, driver behaviour and environment while driving on motorways.

4.1 Safety

The combination of ACC and FCW show a positive impact on driving safety. While there was no decrease in average speed (an indicator previously linked to increase in safety [12]), the average time-headway (THW) shows an increase of about 16 % (see Fig. 1) and leads therefore to larger/greater safety margins. Due to the predefined settings of the ACC time-headway the number of (intended or unintended) close approaching manoeuvres is highly reduced and prevents therefore critical driving situations. The analysis of critical time-headways (<0.5 s) reveals a reduction of 73 % on motorways. As a consequence of the safer distance behaviour the frequency of harsh braking manoeuvres is lower when driving with active ACC. Two out of three harsh braking events (67 %) can be avoided by the use of ACC. Like for the harsh braking events the number of incidents is lower when using ACC and FCW. The incidents based on vehicle kinematics show more than 80 % reduction. Details on the incident definition can be found in [13] (Fig. 5).

Fig. 5

Overview of safety indicators for driving with active ACC and FCW on motorways

Explanations for the increase in average time-headway and the reduction of critical time-headways, harsh braking events and incidents can be found in the selectable ACC settings that can never be lower than the legally prescribed value which is not always considered by drivers in their normal driving behaviour. Resulting from the increase in average time-headway the reaction time to avoid (unintended) close approaching events is higher. If the driving situation exceeds the braking capacities of the ACC because of a highly decelerating vehicle in front, the presented warnings (by the ACC and FCW) give the driver appropriate time to react on the driving situation. It could be shown in the analysis that this effect can be mainly attributed to the ACC by comparing situations where only one of the functions was active (Fig. 6).

Fig. 6

Number of harsh braking events within different experimental phases

In phases of active ACC (independent on the FCW status) a significant decrease of harsh braking events was observed. In case of deactivated ACC and activated FCW no such effect could be shown. To specify the contribution of each individual function the change in issued FCW warnings was investigated. To that end, it can be seen in Fig. 7 that the highest reduction during the treatment period was found in phases when the ACC was active. The number of warnings was significantly decreased by about 80 %.

Fig. 7

Number of FCW warnings in different experimental phases

4.2 User Behaviour

ACC related changes in the driving behaviour can be deduced from the same performance indicators that are associated with safety. Those are mainly based on objective data and indicate especially a safe distance behaviour which leads to safety benefits. Additionally, the questionnaire data indicates that the expectations of drivers to the ACC are fulfilled, i.e. the scores on satisfaction and usefulness that drivers gave before gaining access to the systems match those given during and after the trial. In Fig. 8 it can be seen that the acceptance rating on the Van der Laan scale (scaling from −2 to +2) shows low variation.

Fig. 8

Acceptance rating for ACC and FCW

The acceptance is based on the average of the questionnaire items related to satisfaction and usefulness. In terms of usage, drivers mainly use ACC on motorways. Here, the proportion of kilometres driven with active ACC reaches almost 50 %. The increase in the usage frequency can be confirmed by evaluating the travel time and distance with and without active ACC. Comparing the first months to the last months of the FOT there is a significant increase of ACC usage in terms of travel time with active ACC (31 %) and frequency of ACC activations (53 %). The drivers seem to get used to the positive perception of the ACC and use the system longer and more often over time even though they do not indicate a change in their usage behaviour within the questionnaires. This increased use of ACC is in line with the perceived increase of safety and comfort which is self-reported by the drivers. In contrast, self-reported ratings on trust do not change over time and thus do not reflect the positive perception related to safety and comfort. Confidence which is a sub-criterion of trust even decreased thus expressing that the drivers had higher expectations than the system could fulfil.

The majority of the drivers (close to 70 %) perceive the FCW as safety increasing and most helpful on motorways in normal traffic conditions. The ratings on satisfaction and usefulness remain high throughout the study, but are in general slightly lower than those of ACC. The slight decrease in the ratings can be interpreted by high expectations of the drivers at the project start. In addition, drivers are not uniformly positive to the investigated FCW’s audio-visual interface. Some reported that they perceive the timing of the warnings as to early and therefore annoying. This can be attributed to varying individual comfort zones in terms of following distances and reinforces the need for investigating new and creative ways of offering individual adaptations possibilities. Acceptance is a key parameter for the effectiveness of ADAS since unsatisfied drivers tend to switch of the system and therefore no benefit can be achieved.

4.3 Fuel Consumption

Based on a lower variation in speed when driving with active ACC, it is hypothesised that there are also positive effects on environment in terms of fuel consumption and CO₂ emission. The analysis of the fuel consumption shows a significant reduction of 2.77 % while driving on motorways. This system related change in fuel consumption and the measured average fuel consumption of 7.3 l/100 km is combined with the usage rate of 49.4 % that is reached during the FOT on motorways and projected to the EU-27 level (see Fig. 9).

Fig. 9

Fuel saving potential of ACC for driving on motorways

For the European passenger car fleet which consists of approximately 62 % petrol and 35 % diesel powered vehicles the overall fuel saving potential of 1.37 % accounts for almost 700 million litres of fuel every year and 1.7 million tons of CO₂ based on the average fuel consumption that is evaluated with the objective data. The statistical data on the total mileage of passenger cars and the fleet composition can be found in [14] and [15].

5 Conclusions and Recommendations

During the euroFOT project data of about 1,000 vehicles was collected. With the help of the gathered data the impact of eight different ADAS was evaluated. The analysis of the effects of ACC and FCW usage showed positive effects on traffic safety, driver behaviour and driver acceptance as well as fuel consumption.

The relevant factor for the reduction in harsh braking events, incidents etc. can be attributed to changed distance behaviour. The analysis shows that the average time-headway was increased at about 16 %. In addition to the usage rate of 50 % the analysis of acceptance rating revealed a positive perception of the ACC and FCW. Furthermore, a reduction in fuel consumption of 2.8 % was observed which results in less CO₂ emissions.

Based on the gathered insights with regards to how drivers use the systems valuable input for the various stakeholders (suppliers, vehicle manufacturers, and research institutes) for improving system design and promoting product development is provided. Moreover the positive results are a powerful tool to raise public awareness about the potential of ADAS. These will provide further impulses for consideration of ADAS in the customer’s purchase decision. By means of increased penetration rates a major contribution for improving traffic safety can be achieved.