1 Introduction

Car-following (CF) and lane-changing (LC) behavioral model has been the focus of traffic flow theory research [47, 56]. They are often regarded as two basic driving tasks and have been analyzed separately by extensive researches [22, 52].

While in actual driving practice, CF and LC are inseparable [42, 45, 46, 57, 58], the driver could consider LC in the process of CF and may also consider the influence of the leading vehicle in the target lane in the LC process [42]. Some researchers have also recognized this problem and proposed LC model considering CF [49], and the CF model considering LC [39]. Toledo et al. [42] proposed an integrated driving (ID) behavior model framework and verified the ID model outperformed the independent models in different cases by field data [43]. However, the ID driving model is rather complicated with different modules and many parameters to be calibrated, limiting its further development and application.

The lately rapid development of traffic detection technology, connected vehicle (CV) and machine learning methods [30, 38] has laid the data resource and methodology foundation for the application of data-driven modeling method in the field of traffic simulation modeling. Researchers have attempted to apply data-driven methods in the CF and LC behavior modeling and gained satisfied results, [4, 26, 27, 47, 57, 58].

Meanwhile, the social force (SF) model has been first applied in pedestrians’ walking behaviors [19, 21], then in other road users’ behaviors such as cyclists’ conflict avoidance behaviors [25] and drivers’ CF behaviors [8] and crossing behaviors at intersections [32].

As Helbing and Tilch [20] mentioned, the SF model could be applied to model the car following behavior. Later, Anvari et al. [1] pointed out that the SF-based models offer the possibility of a unified theory that can explain different road user’s movements (including vehicle and pedestrian).

Hence, it seems reasonable to build the drivers’ ID behavior model framework by SF theory and calibrate the parameters by the data-driven methods. Very lately, researchers used SF theory and data-driven methods to model the driver’s operational level LC behavior and verified the proposed deep neural network (DNN)-based LC model by the trajectory data of NGSIM dataset [26, 27].

Here, we proposed a new work to built the NN-based operational ID model on highway, under the hierarchical SF behavioral model framework. The operational ID behaviors on highway refer to all driving behaviors of the driver when he/she is driving a car toward a particular off-ramp or a lane (such as HOV lane), including CF, LC and gap acceptance, etc.

We first present the hierarchical SF behavioral model framework and then analyze the impact forces for the social force-integrated driving (SF-ID) model. On this basis, the input and output of SF-ID model are determined. Then feed-forward neural networks (FNN)-based SF-ID models are proposed. The model structure, training sample size and input historic periods of the SF-ID models with different structures are optimized by field data extracted from the NGSIM dataset (FHWA [13], respectively. Then we refine our FNN SF-ID (L1) model by cutting off some unsignificant factors. Finally, the FNN SF-ID (L1, L2, L3, and R) models are compared with the field trajectory data. Model test results showed that the FNN SF-ID (R) is a little bit outperformed than the other 3 FNN SF-ID models by comprehensive evaluations on CF and LC behaviors. Our study provides inspirations to capturing drivers’ ID behaviors on the highway and to new traffic simulation modeling.

Following is the structure of paper: Sect. 2 reviews the studies on SF behavioral models, ID models and the data-driven CF/LC models; Sect. 3 presents the ID model structure in the hierarchical SF behavioral model framework; Sect. 4 configuration of the SF-ID model by field data; Sect. 5 the model calibration and verification; and Sect. 6 discussions and main conclusions.

2 Literature review

2.1 The SF behavioral models

In the original crowds’ SF Model [19], the movement of pedestrians is affected by a driving force from the destination and by repulsive forces from other pedestrians and obstacles around.

SF models successfully reproduce many crowd phenomena such as lane formation [19] and faster-is-slower effects [21]. By modifying the SF model of Helbing and Molnar [19], different force-based models can reproduce more realistic pedestrian behavior.

Recently, researchers have extended the SF model to other road user behavior modeling and achieved good results. Coverage includes improved car-following models [8, 55], driver’s lane-changing decision behavior [53], right-turn behaviors at intersections [32], motorcar–pedestrian mixed traffic in share space [1], to the cyclist’s conflict avoidance behaviors in mixed traffic at un-signalized intersections [25].

In addition, researchers also attempt to apply data-driven methods to SF models. Liu et al. [35] propose a video data-driven social force model for simulating crowd evacuation: the initialization of pedestrian position, path navigation, and goal selection in the improved SF model were guided by real video data.

2.2 Integrated driving models

Psychologists have noticed and proposed the concept of driving style to explain individual’s stable tendency in the car following, speeding and road safety [6, 10, 37]. On the other hand, researchers also found that different driving behaviors (such as CF and LC) interact with each other in some way (Zhu and Lei 2007), [39].

Later, Toledo et al. [42] proposed a hierarchical integrated driving (ID) behavior model framework, integrating LC, CF, gap acceptance and the inter-dependencies between these behaviors. The framework is based on the concepts of short-term goals and short-term plans, and drivers are assumed to conceive and perform short-term plans in order to accomplish short-term goals. The ID behavior model framework supports a more realistic representation of driving tasks. And the ID model outperformed the independent models in different cases by field data [43]. However, the ID model is rather complicated with different modules and many parameters to be calibrated, which limits its further development and application.

However, with the development of physiological and psychological theories on driving behavior and data-driven methods, new LC models [9, 48] and traffic flow simulation models [11, 33] have emerged, and they all inspired by the idea of ID behavior.

2.3 Data-driven CF/LC models

Recently, with the advancement of data-driven methods, field data-based CF/LC models are put forward. On the CF model, He et al. [17, 18] present the k-nearest neighbor (kNN)-based CF model and verified the validity, transferability. Zhou et al. [57, 58] study the recurrent neural network (RNN)-based CF model, which exceeds the intelligent driver model (IDM) [44]. Lately, Wang et al. [47] validate the Gate Recurrent Unit (GRU) neural network-based CF model considering 10 s historical data perform better than the IDM and the feed-forward neural network (FNN)-based CF model.

On the LC model, Bakhit et al. [2] verified an artificial neural network (ANN)-based LC detection models by Next Generation Simulation (NGSIM) data collected from a weaving freeway segment in Arlington, Virginia. And NN-based models were also built to predict drivers’ lane-changing behaviors [7, 31, 36, 40]. Lately, under the theory of SF, Huang et al. [26, 27] built ANN and GRU NN-based LC behaviors models on the operational level.

From these earlier studies, we have some preliminary ideas: (1) SF-based model framework seems to offer the possibility of a unified theory that can explain the vehicle, bicycle, and pedestrian movements, both separately and in interaction with each other. (2) From the perspective of psychology and behavior analysis, ID behavior generally exists in reality, which could have a significant influence on traffic flow. (3) The incorporation of the ID behavior could enhance the accuracy of simulation models. (4) Data-driven CF/LC models have been tested to outrun the classic CF/LC models. (5) The application of the SF model framework and data-driven methods on the ID model is not fully studied yet.

Therefore, in this paper, we first propose the ID model in the hierarchical SF behavioral model framework and then built a data-driven SF-based ID model using the artificial neural network (ANN) to depict the operational ID behaviors on the highways.

3 Integrated driving model in hierarchical SF behavioral model framework

3.1 Hierarchical SF behavioral model framework

In the operational integrated driving behaviors in hierarchical SF behavioral model framework, a number of salient characteristics are considered for modeling:

  • Under the premise of obeying traffic rules and safety, drivers tend to follow the shortest/direct path to the temporal destination (TD) or short-term goal, according to impacts from infrastructures and other surrounding influencing vehicles. This reflects the motivation effect of the TD in the SF behavioral model.

  • Environmental characteristics of infrastructure EI include the location of lanes and other related infrastructure (curbs, ramps, HOV, etc.). The location of lanes and curbs are assumed to define the space limits of the movements of the subject vehicle and surrounding vehicle.

  • Personal characteristics P of the subject driver including age, driving experience, gender, occupation, etc., are assumed to be incorporated in the driving behaviors (speeds, gap acceptance, etc.). That means if there are ample samples on vehicle dynamic movement behaviors and the corresponding environmental characteristics E, the driver’s Personal characteristics P can be extracted in some way.

It is noted that not all considerations of the problem are listed above. The environmental characteristics of weather and climate EW are also assumed to be incorporated in dynamic movement behaviors of the subject vehicle and corresponding influencing vehicles [28]. However, due to the limited time period coverage of our research data, our research does not consider EW here.

In the hierarchical SF driving behavioral model framework (Fig. 1), the driver’s behavior is also categorized into three levels: operational, tactical and strategic [23]. The strategic level behavior model mainly deals with trip planning-related choices, such as choice on trip mode, trip departure time, and trip route. The output of the strategic level model would be the planned trip, which is also the inputs of the following tactical level model. The tactical level behavior model mainly deals with path planning-related decisions, such as whether enter an off-ramp/HOV lane. And the output of this level model would be the planned path formed by temporal destinations, which is also the inputs of the following operational level model. The operational level behavior models mainly deal with specific dynamic changes of the vehicle’s trajectory, i.e., the longitudinal acceleration (including the so-called car-following behaviors) and the lateral acceleration (including the so-called lane change behaviors).

Fig. 1
figure 1

Hierarchical SF driving behavioral model framework

Full size image

The relationship between the three hierarchical models is a causal relationship, specifically: the output of the strategic level model is the input of the tactical level model, the output of the tactical level model is the input of the operation level model, and the output of the operation level model is the vehicle dynamic behavior/trajectory change. Here, we proposed a social force-based integrated driving behavioral model (SF-ID) for the operational level driving behavior model. The inputs of the SF-ID model include the planned path from tactical level model, the information on surrounding infrastructure and related vehicles, and the information on weather and climate, etc. Because the SF-based models usually have complex formulas and numerous parameters, we adopt the powerful neural network models and data-driven methods to build the SF-ID model.

3.2 Impact forces analysis for SF-ID model

The complexity of the SF-ID behavioral model lies in the numerous impacting forces/factors. According to the characteristics listed above, the first impact force for the SF-ID model considered is the driven force f0 caused by the driver’s temporal destination (TD) or short-term goal.

In the SF-ID model, the function of driven force f0 is to motivate the driver i to drive in the desired direction \(\vec{\varvec{e}}_{i}^{0} \left( t \right)\) at his/her desired speed \(v_{i}^{0} \left( t \right)\) adapted to his actual driving speed \(v_{i} \left( t \right)\) within a certain time \(\Delta T_{Di}\)

$$f_{0} = \frac{{v_{i}^{0} \left( t \right)\vec{e}_{i}^{0} \left( t \right) - v_{i} \left( t \right)) }}{{\Delta T_{Di} }},\quad {\text{where}}\quad \vec{e}_{i}^{0} \left( t \right) = \frac{{\vec{D}_{i} \left( t \right) - \vec{s}_{i} \left( t \right)}}{{\left| {\vec{D}_{i} \left( t \right) - \vec{s}_{i} \left( t \right)} \right|}}$$
(1)

where \(\vec{D}_{i} \left( t \right)\) and \(\vec{s}_{i} \left( t \right)\) are the position of TD and the actual current trajectory of vehicle i, respectively.

Accordingly, the impact factors accounting for the driven force often include the driver’s TD’s location \(\overrightarrow {TD}_{i} \left( t\right)\) and the expected arriving time \(T_{Di}\). Then the driver’s TD would be input of the SF-ID model as:

$$\overrightarrow {TD}_{i} \left( t\right) = \left\{ {\Delta \overrightarrow {TD}_{i} \left( t\right),\Delta T_{Di} } \right\}$$
(2)

where \(\Delta \overrightarrow {TD}_{i} \left( t\right)\) is the vector difference between position of TD and vehicle i, \(\Delta T_{Di}\) is the vehicle i’s expected running time to TD:

$$\Delta \overrightarrow {TD}_{i} \left( t \right) = \overrightarrow {TD}_{i} \left( t \right) - \overrightarrow {s}_{i} \left( t \right)$$
(3)
$$\Delta T_{Di} = T_{Di} - t$$
(4)

where \(T_{Di}\) is the vehicle i’s expected arriving time at TD. It is worth noting that the information on the location of related off-ramps and HOV lanes of the environmental characteristics on infrastructure EI is incorporated in the TD—the outputs of the tactical level model. For example, if a driver wants to leave the highway via an off-ramp nearby, he/she will naturally take it as a TD/short-term goal. In addition, we assume that only the location of current lanes and adjacent left and right lanes would have a direct impact on vehicle movement, and the farther lanes will not have an impact. Therefore, the inputs of infrastructure EI in the SF-ID model only include the central locations of the current lane, left lane and right lane of the subject vehicle:

$$\vec{E}^{I} = \left\{ {L_{\text{C}} \left( t \right), L_{\text{L}} \left( t \right),L_{\text{R}} \left( t \right)} \right\}$$
(5)

where LC(t), LL(t) and LR(t) stand for the latitude position of the centers of the current lane, left lane, and right lane of vehicle i at time t.

As assumed above, both the drivers’ personal characteristics P and environment characteristics E can be extracted from ample samples of the dynamic movement behaviors of the subject vehicle and corresponding influencing vehicles by powerful NN models. The influencing vehicles include adjacent vehicles in the current lane, the left lane and the right lane of the subject vehicle as illustrated in Fig. 2.

Fig. 2
figure 2

A typical scenario of ID behavior on the highway

Full size image

In the two-dimensional SF-ID models, the acceleration/deceleration along the travel direction and LC behavior are integrated; the trajectory and velocity of all vehicles are vectors, such as the subject i’s position:

$$\vec{s}_{i} \left( t\right) = \left\{ {x_{i} \left( t \right), y_{i} \left( t \right)} \right\}$$
(6)

where xi(t) and yi(t) stand for the subject i’s longitudinal and lateral position at time t, respectively.

The relative distance \(\Delta \vec{s}_{j,i} (t)\) of vehicle i to the surrounding vehicles j is also vectors, and if there is no surrounding vehicle j, the relative distance is set as infinity:

$$\Delta \vec{s}_{j,i} \left( t \right) = \left\{ {\begin{array}{*{20}l} {\vec{s}_{j} \left( t \right) - \vec{s}_{i} \left( t \right)} \hfill & {{\text{if }}\;j\;{\text{exists}}} \hfill \\ \infty \hfill & {\text{else}} \hfill \\ \end{array} } \right.$$
(7)

and

$$\vec{s}_{j} \left( t \right) - \vec{s}_{i} \left( t \right) = \left\{ {x_{j} \left( t \right) - x_{i} \left( t \right),y_{j} \left( t \right) - y_{i} \left( t \right)} \right\}$$
(8)

Vehicle j includes the current-lane leading (CL) and current-lane following (CF) vehicles, and the nearby vehicle in the adjacent lanes, denoted as left-lane leading (LL), left-lane following (LF), right-lane leading (RL) and right-lane following (RF) in the subscripts, respectively.

And the velocity \(\vec{v}_{i} \left( t \right)\) of i and relative velocity \(\Delta \vec{v}_{j,i} \left( t\right)\) of vehicle i to j can be calculated by:

$$\vec{v}_{i} \left( t\right) = \left( {\vec{s}_{i} \left( t\right) - \vec{s}_{i} \left( {t - \tau } \right)} \right)/\tau$$
(9)

and

$$\Delta \vec{v}_{j,i} \left( t \right) = \vec{v}_{j} \left( t \right) - \vec{v}_{i} \left( t \right) = \frac{{\vec{s}_{j} \left( t \right) - \vec{s}_{j} \left( {t - T} \right)}}{\tau } - \frac{{\vec{s}_{i} \left( t \right) - \vec{s}_{i} \left( {t - T} \right)}}{\tau }$$
(10)

where τ is the time step, when τ is very small, (9) can represent the momentary speed of i.

The velocity-related variables are derived from previous and current position data. However, in view of the SF model, the moving direction and speed of the surrounding vehicle will produce “forces”, so the relative velocity will also be one of the impact factors of the SF-ID model.

What is more, considering the drivers’ driving experience or prediction capability [47], the vehicles’ previous position data a few time steps before are also taken into account in our ID model.

According to SF model, the output of the SF-ID model should be the “force” representing behavioral change. Here, we use the “results” of behavioral changes—trajectory change of i\(\Delta \vec{s}_{i} \left( t \right)\)–as the output of the model:

$$\Delta \vec{s}_{i} \left( t \right) = \vec{s}_{i} \left( t \right) - \vec{s}_{i} \left( {t - T} \right)$$
(11)

where \(\Delta \vec{s}_{i} \left( t \right)\) is vehicle i’s the position difference from time t–T to t.

Then the inputs and outputs of the SF-ID model would be:

$$\begin{aligned} \varvec{Input} & = \left\{ {\Delta \overrightarrow {TD}_{i} \left( t \right),\vec{E}^{I} ,L_{i} ,W_{i} ,L_{j} ,W_{j} ,\Delta \vec{s}_{j,i} \left( {t - T} \right)}\right.,\Delta \vec{v}_{j,i} \left( {t - T} \right), \\ & \Delta \vec{s}_{j,i} \left( {t - T - \tau } \right),\Delta \vec{v}_{j.i} \left( {t - T - \tau } \right), \\ & \Delta \vec{s}_{j,i} \left( {t - T - 2\tau } \right),\Delta \vec{v}_{j.i} \left( {t - T - 2\tau } \right), \ldots \ldots , \\ & \left. \Delta \vec{s}_{j,i} \left( {t - T - N\tau } \right),\Delta \vec{v}_{j.i} \left( {t - T - N\tau } \right) \right\} \\ & j = CL,CF,LL,LF,RL\;{\text{and}}\; \, RF \\ & \varvec{Output} = \left\{ {\Delta \vec{s}_{i} \left( t \right)} \right\} \\ \end{aligned}$$
(12)

where L and W stand for the vehicle lengths and widths, respectively; and the subscripts i and j stand for vehicle’s identification. As we know, the vehicle’s length and width can represent the type and size of the vehicle and thus are taken as part of ID model inputs.

Here, T is the predict time interval of the model, which is taken as 1 s, while τ stands for the time step for vehicle trajectory sampling and it is closely related to the accuracy of derived velocity. Considering the data acquisition technology and data accuracy requirements of intelligent transport system (ITS) and intelligent vehicle (IV), we take τ = 0.2 s. N is the parameter that defines the range of the influencing historical time Nτ, which will be discussed in the model calibration section.

3.3 Feed-forward NN-based ID model

From the hierarchical SF behavioral model framework and the analysis on the impact factors of the operational ID behaviors on the highway, we propose a feed-forward neural network (FNN)-based SF-ID model, formulated as:

$$\widehat{{\Delta \overrightarrow {s}_{i} }}\left( {t|\theta } \right) = \left\{ {\begin{array}{*{20}c} {f\left( \begin{aligned} \Delta \overrightarrow {{TD_{i} }} \left( t \right),\overrightarrow {E}^{I} ,L_{i} ,W_{i} ,L_{j} ,W_{j} ,\Delta \overrightarrow {s}_{j,i} \left( {t - T} \right),\Delta \overrightarrow {v}_{j,i} \left( {t - T} \right), \hfill \\ \ldots \ldots \hfill \\ \Delta \overrightarrow {s}_{j,i} \left( {t - T - N\tau } \right),\Delta \overrightarrow {v}_{j.i} \left( {t - T - N\tau } \right) \hfill \\ \end{aligned} \right),\quad j = CL,CF,LL,LF,RL\;{\text{ and}}\; \, RF} & {{\text{ if }}\left| {\widehat{{\Delta \overrightarrow {s}_{i} }}\left( {t|\theta } \right)} \right| < \left| {\Delta \overrightarrow {s}_{i}^{Max} } \right| \, } \\ {\Delta \overrightarrow {s}_{i}^{Max} } & {\text{else}} \\ \end{array} } \right.$$
(13)
$$\hat{\vec{s}}_{i} \left( {t|\theta } \right) = \vec{s}_{i} \left( {t - T} \right) + \Delta \vec{s}_{i} \left( {t|\theta } \right)$$
(14)

where \(\Delta \vec{s}_{i} \left( {t|\theta } \right)\) is the estimated vehicle i’s position difference during T, \(\Delta \vec{s}_{i}^{Max}\) is the maximum position difference vector during T, defined by the speed limits of the vehicle and infrastructure. Note that (13) is a model built on the basis of ideal data, and infrastructure \(\overrightarrow {E}^{I}\) can be omitted, considering that most driving behavior dataset lacks infrastructure data, including the NGSIM dataset we use in Sect. 5.

The ID function f(∙) will be learned by NNs. And θ stands for the parameters of the NN, mainly including the transfer functions and weight vectors.

The FNN-based SF-ID model contains three kinds of layers (see Fig. 3). The input layer that takes the inputs, the hidden layer (one or more) that calculates, and the output layer that generates the outputs [31].

Fig. 3
figure 3

The Structure of SF-ID model based on FNN

Full size image

The transfer function for neurons is the sigmoid function, which has been extensively used and proven to be suitable for the recent DNN-based CF model [47]. For data-driven method, it is important to define the range of model inputs. Table 1 lists the range of input parameters for the FNN SF-ID model.

Table 1 Input parameters range of the SF-ID model based on FNN
Full size table

Theoretically, if we have ample high-quality samples, which could reflect all the mechanisms of drivers’ ID behaviors, then with proper transfer functions and neuron quantity, a simple NN with only one hidden layer has the power of capturing the behavior rules of ID behaviors. Thus, we believe the NN-based SF-ID model has the capability to reproduce the drivers’ operational level ID behaviors properly. However, NN models with over one hidden layer (also known as deep NN) have obtained up-to-day accomplishments in many challenging works, including traffic flow forecasting, CF modeling, natural language processing, computer vision, and speech recognition, etc. [5, 28, 34, 47, 54]. Therefore, we intend to try and compare the simple and complex NN structures in the SF-ID model.

4 Configurations for ID models

The configuration of NN model contains the performance index, decisive parameters and training algorithms for NN. Conforming to the calibrating principles [24] and other related researches [25, 47], we adopted the widely used correlation coefficient (R) and mean squared error (MSE) of predicted and empirical positions as the performance indices:

$$R_{x} = \frac{{\mathop \sum \nolimits_{k = 1}^{M} \left( {\vec{x}_{k} - \bar{X}} \right)\left( {\vec{x}_{k} - \bar{X}} \right)}}{{\sqrt {\mathop \sum \nolimits_{k = 1}^{M} \left( {\vec{x}_{k} - \bar{X}} \right)^{2} \Delta \mathop \sum \nolimits_{k = 1}^{M} \left( {\vec{x}_{k} - \bar{X}} \right)^{2} } }}$$
(15)
$$R_{y} = \frac{{\mathop \sum \nolimits_{k = 1}^{M} \left( {\vec{y}_{k} - \bar{Y}} \right)\left( {\vec{y}_{k} - \bar{Y}} \right)}}{{\sqrt {\mathop \sum \nolimits_{k = 1}^{M} \left( {\vec{y}_{k} - \bar{Y}} \right)^{2} \Delta \mathop \sum \nolimits_{k = 1}^{M} \left( {\vec{y}_{k} - \bar{Y}} \right)^{2} } }}$$
(16)
$${\text{MSE}} = \frac{1}{M}\sum\nolimits_{k = 1}^{M} {\left( {\hat{\vec{s}}_{k} - \overrightarrow {s}_{k} } \right)^{2} } = \frac{1}{M}\sum\nolimits_{k = 1}^{M} {\left[ {\left( {x_{k} - x_{k} } \right)^{2} - \left( {y_{k} - y_{k} } \right)^{2} } \right]}$$
(17)

where Rx and Ry stand for the correlation coefficient for x and y-axis coordinate values of trajectory. \(\vec{x}_{k}\), \(\vec{y}_{k}\), \(\vec{x}_{k}\) and \(\vec{y}_{k}\) stand for the kth estimated and empirical xand y-axis coordinate values of trajectory, respectively. M stands for the total number of the validation sample.\(\bar{X}\), \(\bar{Y}\), \(\bar{X}\) and \(\bar{Y}\) stand for the means of \(\vec{x}_{k}\), \(\bar{y}_{k}\), \(\vec{x}_{k}\) and \(\vec{y}_{k}\), respectively.

The followings are decisive parameters in our FNN-based SF-ID models.

  1. (1)

    Historical time steps N of inputs: Parameter N defines the range of subject vehicle i’s historical position data being input to the model. The larger N implies more historical information being considered by the NNs. The process of finding out the best value of N is presented in Sect. 5.2.

  2. (2)

    Activation function: According to the DNN-based CF and LC models [26, 27, 47], we take the sigmoid function as the activation function for neurons in the FNN’s input and output layers.

For neurons in the hidden layers, we take the Parametric Rectified Linear Unit (PReLU) as the transfer function, because it could reduce the risk of overfitting [17, 18]. The Relu function is:

$${\text{PReLU}}\left( {x_{i} } \right) = \left\{ {\begin{array}{*{20}c} {x_{i} } & {{\text{if }}\;x_{i} > 0} \\ {\alpha x_{i} } & {{\text{if }}\;x_{i} \le 0} \\ \end{array} } \right.$$
(18)

where α stands for the threshold parameter. In the experiment of finding out the appropriate α, we set α from 0.2 to 0.9 with a step size of 0.05. Finally, the value of α is 0.80.

  1. (3)

    The NN structure: The NN structure contains the number of the hidden layers and their corresponding neurons. Since there is no direct way to find out the most appropriate NN structure, researchers usually get appropriate NN structure by field data testing. A number of typical NN structure schemes are tested and compared in Sect. 5.

  2. (4)

    The loss function: The loss function (or objective function) in the SF-ID model considers the relative error in both lateral and longitudinal positions of the vehicle trajectory. The MSE is adapted to measure the error between the estimated and empirical positions.

  3. (5)

    Epoch: The epoch stands for the iteration times in training. ‘Early Stopping’ is used in training to avoid overfitting or underfitting. When the results are no better than 50 times, the model stops training. The epoch is set from 50 to 3000.

In line with the recent work of Wang et al. [47] and Huang et al. [26, 27], the widely used backpropagation (BP) algorithm is adopted to train the FNN-based SF-ID models.

5 Model calibration and verification

Here, the FNN-based SF-ID models will be calibrated and verified by field data. First, the empirical field data are presented. Then structures of the FNN-based SF-ID model are compared with diverse training sample sizes. Next, the historical time steps N are determined with the suitable structure (from simple one hidden layer to complex 3 hidden layers NNs) and training sample size. Last, the performance of the FNN-based SF-ID models is verified by comparing model outputs and field data from different aspects of driving behaviors.

5.1 Data preparation

We take the Next Generation Simulation (NGSIM) (FHWA [13] as the testing dataset. NGSIM contains accurate and reliable highway vehicles’ trajectory data, related vehicle information (including the vehicle’s length and width, the leading and the following vehicle, lane, etc.), and infrastructure information. The dataset is of U.S. Highway I-101. The study area of I-101 is about 2100 ft long with one on-ramp and one off-ramp (Fig. 4).

Fig. 4
figure 4

The study area of U.S. Highway I-101 in the NGSIM dataset

Full size image

The I-101 dataset also has a total of 45 min of data: 7:50 a.m. to 8:05 a.m.; 8:05 a.m. to 8:20 a.m.; and 8:20 a.m. to 8:35 a.m. These periods represent the buildup of congestion, or the transition between uncongested and congested conditions, and full congestion during the peak period. The NGSIM I-101 dataset has been widely used for CF and LC modeling [16,17,18, 26, 27, 29, 41, 47, 51, 53, 57, 58].

Three data filtering procedures had been conducted for the NGSIM dataset before extracting the training samples:

  1. (1)

    Primarily, vehicles with discontinuous time records are filtered out. That means part of the vehicles’ trajectory records were missing. The trajectory data of such vehicles are unable to provide valid samples for the model.

  2. (2)

    Then, the vehicles running shorter than 200 m are filtered out. This is to provide valid samples for CF and LC behavior modeling in the SDF-ID model [17, 18].

  3. (3)

    Last, vehicles type as motorcycles were filtered out. Some vehicles in the NGSIM dataset are motorcycles, which are not included in our study.

Extracting SF-ID model training sample from the original NGSIM trajectory dataset mainly includes three parts: position and time of vehicle i’s TD, determination of the surrounding influencing vehicles’ ID and extraction of relative dynamic information, and verification of the continuity of vehicles’ trajectory data.

To determination of vehicle i’s TD is to extract the trajectory position and timestamp just before the vehicle i leaving the vehicle trajectory study area. And the data related to TD in the samples can be calculated by (2)–(4) according to vehicle i’s trajectory and timestamp.

Figure 2 shows that the number of influencing vehicles of vehicle i changes from 0 to 6. The judgment of the surrounding influencing vehicles is based on the minimum relative distance \(\left| {\Delta \vec{s}_{j,i} } \right|\) between vehicle j and vehicle i. According to the car-following theory (FHWA [12], there is a correlation between vehicles in a range of inter-vehicle spacing, from zero to about 100–125 m. Here, if the minimum relative distance between vehicle j and vehicle i is less than 125 m, the ID of the vehicle with the smallest relative distance is recorded as influencing vehicle; otherwise, the ID would be Null. i.e.,

$${\text{ID}}_{j} = \left\{ {\begin{array}{*{20}c} {j|\mathop {\hbox{min} }\limits_{j} \left\{ {\left| {\Delta \overrightarrow {s}_{j,i} } \right|} \right\}} & {{\text{if}}\left| {\Delta \overrightarrow {s}_{j,i} } \right| \, < \,125\,{\text{m}}} \\ {\text{Null}} & {\text{else}} \\ \end{array} } \right.$$
(19)

In case of the missing records of some vehicles’ trajectory in the original NGSIM data, the continuity of the vehicle trajectory has been judged in sample data preprocess as the training samples need historical information.

After preprocessing, we get samples of 2193 vehicles and about 1,225,000 records of vehicle trajectories for training/tests. Samples with excessive deviations of trajectory displacement within 0.1 s (for example over 200 ft) are filtered out. Then we get over 1,150,000 valid sample records, and they are denoted as empirical ID samples.

The empirical ID samples are first randomly divided into three parts: training datasets (40%), cross-validation datasets (30%), and test datasets (30%).

5.2 Calibration results of SF-ID models

In this section, we first propose 12 schemes of the model structure with 1 to 3 hidden layers for the FNN for SF-ID models. We take preliminary tests with a small sample size to obtain the initial suitable value of historical time steps N for the schemes. Then we search for the appropriate network structures and sample size by training and cross-validation on field data with the initial N. With the appropriate network structures and sample size, we go back to determine the suitable value of N again. In this way, we finally find out the suitable FNN SF-ID models.

5.2.1 Choice of Model Structure and Sample Size

When implementing NNs to model driving behaviors, we should be cautious about the problem of overfitting, as the driving behavior dataset is relatively small. And complex structured NN tends to overfit with a small sample size [3]. What is more, as it is hard to define the correlation of training sample size and NN structures [47], we test different schemes from one to three hidden layers (see Table 2) to find the appropriate sample size and NN structure.

Table 2 NN structure schemes
Full size table

The NN schemes in Table 2 are trained by sample size from 50,000 to 450,000 (randomly pick from the training dataset) and tested by 200,000 samples randomly pick from the cross-validation dataset. Here, the historical time steps N of input are taken as 5. In the test, the weight coefficients of the NN are mostly not close to 0, showing the effectiveness of the NN model.

Figure 5 shows the MSE values in ft2 of FNN-based SF-ID models with diverse structure schemes and training sample sizes. The correlation coefficients Rx and Ry of all FNN-based SF-ID models are very high, up to about 0.999.

Fig. 5
figure 5

Comparisons of MSE results of FNN with structural schemes in Table 2 and training sample sizes (N = 5)

Full size image

The results show that the FNN SF-ID model with one hidden layer is scheme S2 (see Table 2) with the corresponding training sample size of 100,000, and the MSE is 16.53 ft2 (1.53 m2). And the best FNN SF-ID model with 2 and 3 hidden layers are schemes S7 and S11, with a corresponding training sample size of 200,000 and 250,000, and the MSEs are 15.78 ft2 (1.47 m2) and 15.80 ft2 (1.47 m2), respectively.

As shown in Fig. 5, the FNN models with more complex structures (S7 and S11 in Table 2, with 2 and 3 hidden layers) performed a bit better than the simple FNN model (S2 in Table 2, with 1 hidden layer). However, the overall MSEs under different training sample sizes of the simple FNN model (S2) seem smaller than the other two models, indicating that the simpler FNN model may outperform the complicated FNN models in some cases.

To further explore the model performance of FNN models with different hidden layers in operational ID behavioral modeling, we compare the best three FNN models with different number of hidden layers—S2 in Table 2, with one hidden layer, S7 with two hidden layer and S11 with three hidden layer—in the following work, and named them as FNN SF-ID (L1), FNN SF-ID (L2) and FNN SF-ID (L3), respectively.

5.2.2 Suitable value of historical time steps N

Before testing the suitable structure schemes and training sample size for the FNN model, we randomly pick 100,000 samples from the training dataset to test and compare different values of historical time steps N for all the structures schemes of Table 2. The preliminary results showed that the MSE of most schemes were lowest when N is 5 or 6.

Then, we take N = 5 to find out the suitable structure schemes and the corresponding training sample sizes for FNN SF-ID (L1, L2, and L3) models.

Next, with the suitable NN structures and corresponding training sample sizes, we compare the FNN SF-ID (L1, L2, and L3) model with different historical time steps N. The results (Fig. 6) show that N = 5 for all the FNN SF-ID (L1, L2 and L3) models can achieve the best performance. Table 3 shows the parameters of the best fitted FNN SF-ID models with different hidden layer numbers.

Fig. 6
figure 6

MSE of FNN SF-ID (L1), FNN SF-ID (L2) and FNN SF-ID (L3) with different historical time steps N

Full size image
Table 3 Parameter range and calibration results of best fitted FNN SF-ID (L1\L2\L3\R*)
Full size table

5.3 Model structure features

We use statistical methods to analyze the weight matrix in the FNN SF-ID (L1) to explore the model structure features. The weight between the input layer and the hidden layer is W1, and the weight matrix between the hidden layer and the output layer is W2 (see Fig. 3). In W1, inputs with weights ranking over 75% of all are considered as main influencing factors of hidden features, and inputs with weights ranking below 15% are considered as insignificant factors. Table 4 lists the main and insignificant influencing factors for the FNN SF-ID (L1) model.

Table 4 The main and insignificant influencing factors for the FNN SF-ID (L1) model
Full size table

The main influencing factors listed in Table 4 are basically consistent with the influencing factors of the traditional CF behavior and the LC behavior model [14, 15, 50]. In the traditional CF model, the vehicle is mainly influenced by the state of the leading one (relative distance and velocity) [14, 50]; in SF-ID FNN, the presence and the velocity of the leading vehicle have greater weights. In Gipps’ LC model, the driver needs to perform a comprehensive evaluation on the current lane and the target lane (the presence of heavy vehicle, whether there is a safe distance for lane change, etc.); in the FNN SF-ID (L1) model, the presence and state of the vehicle on the adjacent lane (including the vehicle size and relative distance) also have greater weights.

In W2, the hidden features that have a greater influence on the outputs \(\Delta \overrightarrow {s}_{i}\) are mainly affected by the adjacent vehicle state (including presence and relative position \(\Delta \overrightarrow {s}_{j,i}\)), and the lateral position of temporal destination \(TDx_{i}\).

Interestingly, the main influencing hidden features on output \(\Delta y_{i} (t)\) are basically only affected by the historical longitudinal positions; while the main influencing hidden features on output \(\Delta x_{i} \left( t \right)\) are not only affected by the historical lateral positions \(\Delta x_{i} \left( {t - T - n\tau } \right)\), n = 0,1…5, but also the historical longitudinal positions \(\Delta y_{i} (t - T - n\tau )\), n = 0,1…5. This seems to indicate that the driver’s LC behavior and CF behavior cannot be completely separated.

We also analyzed the insignificant influencing factors in W1. The insignificant influencing factors for the FNN SF-ID (L1) model include the relative velocities of surrounding vehicles EXCEPT for the leading one, length and relative longitudinal distance of the following vehicle.

Considering that the relative velocities of the surrounding vehicles can be derived from the historical relative position by Newton’s kinematics, they can be considered as redundant inputs. And to our common sense, the driver generally does not care much about the relative velocities of the surrounding vehicles except the leading car, not to mention the length and the relative longitudinal position of the following car.

5.4 Refined FNN SF-ID model

Based on the analysis of the model structure features, we refine the original FNN SF-ID (L1) by removing the insignificant inputs (the relative velocities of surrounding vehicles EXCEPT for the leading one, all the information of the following vehicle in current lane CL). The refined FNN SF-ID (L1) is noted as FNN SF-ID (R).

We calibrate FNN SF-ID (R) by the method in Sect. 4. To facilitate comparisons with the original FNN SF-ID models, the historical time steps N are taken as 5. The parameter calibration results are shown in Table 3.

Overall, FNN SF-ID (R) performed a bit better than the original FNN SF-ID (L1, L2 and L3) models, with smaller MSEs, smaller NN structure, smaller training sample size, and less epoch needed for model calibration.

Then the refined FNN-based SF-ID model—FNN SF-ID (R)—can be formulated as:

$$\widehat{{\Delta \overrightarrow {{s}}_{i} }}\left( {t|\theta } \right) = \left\{ {\begin{array}{*{20}c} {f\left( \begin{aligned} \Delta \overrightarrow {{TD_{i} }} \left( t \right),L_{i} ,W_{i} ,L_{j} ,W_{j} ,\Delta \overrightarrow {s}_{j,i} \left( {t - T} \right),\Delta \overrightarrow {v}_{CL,i} \left( {t - T} \right), \hfill \\ \ldots \ldots \hfill \\ \Delta \overrightarrow {s}_{j,i} \left( {t - T - N\tau } \right),\Delta \overrightarrow {v}_{CL.i} \left( {t - T - N\tau } \right) \hfill \\ \end{aligned} \right),\quad j = {\text{CL,LL,LF,RL and RF}}} & {{\text{ if }}\left| {\widehat{{\Delta \overrightarrow {s}_{i} }}\left( {t|\theta } \right)} \right|{ < }\left| {\Delta \overrightarrow {s}_{i}^{Max} } \right| \, } \\ {\Delta \overrightarrow {s}_{i}^{Max} } & {\text{else}} \\ \end{array} } \right.$$
(20)

5.5 Comparisons with field data

To examine the validity and accuracy of the FNN SF-ID models, we compared some specific driving behaviors of FNN SF-ID(L1, L2, L3, and R) model and field data randomly extracted from the empirical ID test dataset.

5.5.1 Comparison on avoiding collisions

In the traditional CF models, the collision-free can be mathematically proved (FHWA [12]. The FNN SF-ID assumed that the driving behaviors can be speculated from historical behaviors and surrounding impacts. Here, we will validate the collision-free situations in statistics. Therefore, we calculate the relative error of the estimated space headway as follows:

$${\text{RE}}^{hs} \left( t \right) = \frac{{h^{s} \left( t \right) - h^{s} \left( t \right)}}{{h^{s} \left( t \right)}} = \frac{{y_{i} \left( t \right) - y_{i} \left( t \right)}}{{y_{CL} \left( t \right) - y_{i} \left( t \right)}}$$
(21)

where \(h^{s} \left( t \right)\) and \(h^{s} \left( t \right)h^{s} \left( t \right)\) represent the estimated and real space headway at t; \(y_{CL} \left( t \right)\) represents the leading vehicle’s longitudinal position at t; other denotes are the same as above.

Totally 455 pairs of leader–follower vehicles moving over 800 ft on US I-101 of the test dataset are estimated by the FNN SF-ID (L1, L2, L3, and R) models. In the results, the percentage of relative error \({\text{RE}}^{hs} \left( t \right)\) smaller than 0.1 was 96.50%, 98.04%, 82.67% and 94.38% for FNN SF-ID (L1, L2, L3, and R), respectively. And the results of Lane 1 to 4 are presented in Fig. 7. It shows that FNN SF-ID (L1, L2, and R) have outperformed than FNN SF-ID (L3) in collision avoidance situation, as their relative errors are near zero.

Fig. 7
figure 7

Histogram plots for the REhs of different FNN SF-ID models on different lanes

Full size image

As the minimum space headway is about 18 ft, the relative errors imply collision-free exists in all our FNN SF-ID (L1, L2, L3, and R) models. From Fig. 7, we can see that for each type of FNN SF-ID model, the relative errors of space headway in different lanes are quite close. This shows that the SF-ID models are in line with the actual situation in car-following and collision avoidance.

5.5.2 Comparisons on the trajectories in different lane change scenarios

To further analyze whether the FNN SF-ID model can truly reproduce the operational lane change behavior on highway, we compare the estimated and empirical trajectories of four different lane change scenarios in US I-101 dataset: (1) merging vehicles via the on-ramp with multiple lane change operations (i.e., Vehicle 2006) and (2) diverging vehicles to the off-ramp with lane change operations (i.e., Vehicle 2107).

In merge scenario, the subject vehicle 2006 changes from lane 6 to 5. Before lane changing, vehicle 2006 was only with CF vehicle 2010, and without CL, LL, LF, RL or RF vehicle. After lane change, it was with CL vehicle 1996 and CF vehicle 2007, LL vehicle 1991, RL vehicle 2000, respectively. The maximum longitudinal trajectory deviations were 29.74 ft (9.06 m), 18.33 ft (5.59 m), 12.47 ft (3.80 m) and 16.92 ft (5.16 m) for FNN SF-ID (L1, L2, L3 and R), respectively. The maximum lateral trajectory deviations were 4.95 ft (1.51 m), 2.89 ft (0.88 m), 2.94 ft (0.90 m) and 2.55 ft (0.78 m) for FNN SF-ID (L1, L2, L3 and R) models, respectively.

In diverge scenario, the subject vehicle 2107 changes from lane 5 to 6. Before lane changing, vehicle 2107 was with CL vehicle 2104 and CF vehicle 2116, LL vehicle 2112, RL vehicle 2119, without LF or RF vehicle. After lane change, it was with CL vehicle 2099 and CF vehicle 2117, and without LL, LF, RL or RF vehicle. The maximum longitudinal trajectory deviations were 14.80 ft (4.51 m), 16.00 ft (4.88 m), 33.40 ft (10.18 m) and 7.02 ft (2.14 m) for FNN SF-ID (L1, L2, L3 and R), respectively. The maximum lateral trajectory deviations were 3.66 ft (0.12 m), 1.96 ft (0.60 m), 2.60 ft (0.79 m) and 2.40 ft (0.73 m) for FNN SF-ID (L1, L2, L3 and R) models, respectively.

Figure 8 shows the comparisons of the estimated by FNN SF-ID (L1, L2, L3, and R) models and empirical vehicles’ trajectories. We can see in the enlarged details of Fig. 8 that the refined model FNN SF-ID (R) performed better than the original FNN SF-ID (L1, L2, and L3) model in both merge and diverge situations.

Fig. 8
figure 8

The target vehicle’s trajectories reproduced by different FNN SF-ID models in merge and diverge situations in US I-101

Full size image

All vehicles in the test dataset with full LC behaviors (totally 384 vehicles, 84,482 samples) were tested. A vehicle with full LC behaviors means that it keeps moving for 300 fts on lanes before and after LC operation in the test dataset. Overall, the FNN-based SF-ID models could all reproduce the drivers behaviors in merge and diverge situations satisfactorily with MSEs of 19.69 ft2 (1.83 m2), 18.06 ft2 (1.68 m2), 18.59 ft2 (1.72 m2) and 16.05 ft2 (1.49 m2) for FNN SF-ID (L1, L2, L3 and R), respectively.

Table 5 presents the performance of FNN SF-ID (L1, L2, L3, and R). The refined model FNN SF-ID (R) outperformed a bit than the three original models, as most of the performance indexes are better.

Table 5 Performance results of best fitted FNN SF-ID (L1, L2, L3, and R) models
Full size table

6 Discussions and conclusions

Our effort in this paper is an attempt to model the driver’s operational ID behavior on the highway. The driver’s ID behavior phenomenon has been recognized by researchers, and Toledo et al. [42] have proposed the ID modeling incorporating CF, LC and gap acceptance behaviors and the model calibration results. However, the further research and application of the ID model have stagnated since then due to two main obstacles: the lack of proper psychological and behavioral theoretical framework to explain the underlying mechanism of ID behavioral, and a powerful method to handle the complex model parameter calibration.

The recent emerging of hierarchical social force (SF) behavioral model framework and data-driven method seems to clear the obstacles on the theoretical basis and calibration method of the modeling of driver’s ID behaviors.

In the process of modeling, we are surprised by the power of data-driven methods. A relatively simple NN model with a single hidden layer (compared with image recognition NNs) can capture the driver’s ID behavior features on the highway by using only a small sample size on positions and other information on the subject and surrounding vehicles.

During NN modeling, we have also tried other more complex NN models, such as GRU (Gated Recurrent Unit) [47] and LSTM (long short-term memory) [26, 27] models. However, the performance of GRU and LSTM models is no better than that of FNN models. Instead, the inputs of the model need to be adjusted according to the structural characteristics of the GRU and LSTM model, making the modeling process more complex. Therefore, the FNN model is adopted in the end. Test results showed that the FNN SF-ID models on the highway can capture underlying driving behavioral features, as it could reproduce both the different LC maneuvers and collision avoidance in CF.

Then, we analyze the FNN SF-ID (L1) model structure features and finding out the main influencing factors and insignificant ones for the hidden features for FNN SF-ID (L1). The main influencing factors basically include all the considered factors of the traditional CF and LC behavior models [14], 1986; [50]. In addition, TDx as the main influencing factor shows that the impact of goal driving force in SF theory on driving behavior is significant. This proves that the SF theory is suitable for ID driving behavior modeling.

Interestingly, in the FNN SF-ID model structure, the main influencing hidden features on output \(\Delta x_{i} \left( t \right)\) are not only affected by the historical lateral positions but also the historical longitudinal positions. This seems to verify that driver’s LC behavior and CF behavior are integrated [42].

Through the analysis of the insignificant factors, we exclude the CF vehicle from the surrounding vehicles, because both the static and dynamic states of CF belong to the insignificant factors. Thus, we refine the original FNN SF-ID (L1) by removing some of the insignificant factors.

We also compare the performance of the four FNN SF-ID (L1, L2, L3, and R). From the comprehensive evaluation of collision avoidance in CF and LC situations, the FNN SF-ID (R) seems to outperform a bit than the three original models in our study. Performance results presented in Table 5 show that our FNN SF-ID models have a good capability in reproducing/predicting the operational ID behaviors on the highway.

The satisfactory performance of the FNN SF-ID model may be partly attributed to the relatively specific and simple operational ID behavior on the highway, but also benefit from the appropriate behavioral model framework.

In addition to providing the theoretical model basis for traffic flow microscopic ID simulation models on the highway, the FNN SF-ID model can also be applied in advance driving assistant system (ADAS) of IV, connected vehicles (CVs) and autonomous vehicles (AVs). The FNN SF-ID model could also be used to predict the positions of the surrounding human-driven vehicles.

Due to the powerful parameter fitting capability of data-driven methods, we must be very cautious when applying these methods in driving behavior modeling. As many researchers don’t want to find that the microscopic traffic flow simulation model becomes a huge black box one day. However, by exploring the NN structural features, we still can have some understanding of the relationship between model input and model hidden features and can qualitatively understand whether the data-driven driving behavior model conforms to our common sense. This can alleviate the black box problem to some extent. Moreover, our refined FNN SF-ID model also fully demonstrates that the data-driven driving behavior model can be refined by analyzing its structural features.

This can improve the black box problem to some extent. And our refined FNN SF-ID model also fully shows that the model can be improved by analyzing the structural characteristics of NN. This can improve the black box problem to some extent. Moreover, our refined FNN SF-ID model also fully demonstrates that the model can be improved by analyzing the NN structural features.

The limitation of this method lies in the general limitations of data-driven modeling methods, that is, the model performance has a large dependence on the accuracy, reliability, and sample size of field sample dataset. Moreover, the trained model has strong location adaptability. For example, the model trained on the sample dataset of I-101 in this paper may not be able to apply to other highway sections directly. This limitation will be further improved in our subsequent studies.

Our work is only a preliminary small part of this course. Due to data limitations, our model does not incorporate information on the infrastructure. Our future work is an attempt to build ID models with infrastructure information and in more complex driving situations such as urban road sections by applying more data-driven methods.