1 Introduction

Collisions at railway level crossings (or grade crossings) are a persistent problem that, so far at least, have proven resistant to what would normally be considered ‘common sense’ engineering solutions. Collisions occur at busy urban crossings with the full gamut of engineering counter-measures, such as barriers, lights, red light cameras and advanced train protection and warning (e.g. Kirknewton crossing near Edinburgh, Scotland; Network Rail 2016), just as they occur at little-used rural crossings where drivers have clear sight of oncoming trains from several miles distant (e.g. Kerang crossing in Victoria, Australia; Salmon et al. 2013a, b). In the ten years between 2002 and 2012, there were 601 crashes between road vehicles and trains at Australian level crossings (Bureau ATS 2012). In the UK, which has over 6000 crossings and one of the best safety records in Europe, there were still 56 collisions between 2009 and 2015 leading to 10 fatalities (RSSB 2015). In the European Union, there are approximately 400 similar deaths over the same period (Cirovic and Pamucar 2012) and in excess of 300 in the United States (Mok and Savage 2005). The problem appears to be universal and not confined to any one country. Compounding matters further is these fatalities represent the tip of a much broader risk pyramid (e.g. Heinrich 1941; Prem et al. 2010; Walker et al. 2015; Mulvihill et al. 2016) so that, collectively, fatal, non-fatal and near miss accidents represent a significant cause of harm and disruption to transport users and operators. Indeed, many road and railway networks are running at close to capacity and are ill-equipped to withstand prolonged route closures while repairs and investigations take place. It is not surprising, therefore, that railway level crossings represent a significant strategic risk for the railway industry globally (e.g. RSSB 2014; Stroud 2010), one that has become increasingly exposed by improving safety and operational trends elsewhere in the system.

There is a strong desire to improve the current situation with many nations setting bold targets, such as zero fatalities at level crossings (e.g. Transport Committee 2014). The challenge is where to search for the transformative insights which will deliver on ambitious commitments like these? There can be no doubt existing methods and approaches have yielded significant improvements (Evans 2011; Mok and Savage 2005) but to achieve zero fatalities at crossings, there is a clear need to go further still. Specifically, to continue deepening our understanding of human behaviour in these contexts to better understand what constrains and affords behaviour in ways that existing methods do not readily detect (Read et al. 2013; Salmon et al. 2016). A more recent approach to this problem is to switch perspectives. Rather than viewing the human as the ‘weak link’ in the railway level crossing system and placing ‘error’ in the foreground, it is instead possible to view level crossing users as flexible and adaptive decision-makers (Read et al. 2013; Salmon et al. 2016). Exploring the issue of level crossing safety from this perspective is the purpose of the current paper.

The applicability of systems thinking to railway level crossings has been heavily advocated by Read et al. (2013; 2016) and Stefanova et al. (2015) and further supported by a robust case study (Salmon et al. 2016). A sociotechnical system is one in which social and technical elements combine to achieve the system goals (Vicente 1999; Walker et al. 2008) and railway level crossings fall within this definition. The level crossing system operates based on interactions between road users (e.g. motorists, cyclists, pedestrians) and railway users (e.g. train drivers, signallers). Users also interact with technology, such as vehicles, equipment and devices (e.g. gates, alarms) and infrastructure (e.g. road, railway tracks, signage etc.; Read et al. 2013; Salmon et al. 2016). A model which combines all these system elements and allows decision-making to be mapped on to it is the Abstraction Decomposition Space (ADS), as outlined by Rasmussen and Jensen (1974). previous research using the ADS approachThe ADS is based on the more common Abstraction Hierarchy (AH) method (see Naikar, 2013 and Jenkins et al. 2009 for detailed guidance), itself a component of the Cognitive Work Analysis (CWA) framework. Here, system elements are placed within one of five levels of system abstraction (Table 1).

Table 1 Five levels of abstraction that can be applied to a railway level crossing system (
Full size table

These five levels of abstraction are based on studies by Rasmussen and colleagues into decision-making and how humans think and reason about complex systems (Rasmussen and Jensen 1974; Rasmussen et al. 1994). It was discovered that people reason about ‘concrete’ physical information (i.e. “how does this work”) and ‘intangible’ higher-level functional abstractions (i.e. “why is this here?”) in ways that are linked to ‘how’ and ‘why’; or the ‘means’ that a system can use to achieve defined ‘ends’ (Jenkins et al. 2009; Rasmussen et al. 1994). From a system’s point of view, these linkages between nodes at different levels of abstraction are as important as the nodes themselves, and this is captured in Rasmussen’s AH through the use of so-called ‘means-ends links’ within a hierarchy. The ADS extends this framework by further subdividing the system according to the systems level at which the nodes reside: system, sub-system and component (or variations thereof). By crossing levels of abstraction with levels of system decomposition, and populating the resulting matrix with specific system nodes and means-ends links, an ADS for any system can be derA good example from Hoffman and Lintern (2006) is shown in Fig. 1.

Fig. 1
figure 1

Source: Hoffman and Lintern (2006) p. 212

An Abstraction-Decomposition matrix for a weather forecasting task (Hoffman and Lintern 2006) showing levels of abstraction on the y-axis, levels of system decomposition on the x-axis, system elements arranged across those levels and a trajectory based on the sequence of activities.

Full size image

The ADS is intended to depict the way subject matter experts think about their work domain. In Rasmussen and Jensen’s (1974) original study, which was based on professional technicians troubleshooting faults in electronic equipment, it was noted that experts typically navigate through the ADS starting with purposes or values at the higher system level, and then work down towards finer decompositions at physical object and function levels (see again Fig. 1). In other words, expert decision-making of this kind, when mapped onto an ADS of a system, tends to describe a left to right downward diagonal trajectory. The trajectory can exhibit irregular, opportunistic and iterative loops (Lintern 2006; Naikar 2013) but this downward diagonal path is common and has been detected in other similar studies, such as those involving high-fidelity nuclear power plant simulators and thermal hydraulic microworlds (e.g. Itoh et al. 1995; Vicente 1999). A key strength of the ADS approach is its focus on the actors’ environment rather than their behaviour. It is this feature which permits a shift in perspective away from simple notions of humans being the ‘weak link’ to instead consider the system as a whole and the way in which it shapes behaviour. In other words, to shift the perspective away from simple notions of driver error, stricter enforcement and/or more complex engineering counter-measures and reveal instead new opportunities for improving level crossing system performance.

From previous research using the ADS approach (Rasmussen and Jensen 1974; Itoh et al. 1995; Vicente 1999), it might be expected that expert driver’s decision-making at level crossings will follow a similar downward diagonal trajectory through the ADS. Not because these drivers are experts in Rail Level Crossings (RLX’s), but because the environment is making the system constraints and affordances obvious. On the other hand, it may be the case that our current level crossing systems are inadvertently constraining desirable behaviours, affording undesirable ones, and that unexpected system elements are driving behaviour in ways not previously considered. Indeed, the nature of the level-crossing task may be entirely different to those previously reported and the decision-making trajectories likewise different. This study aims to shed light on precisely these issues. The following sections describe the naturalistic on-road study that was performed: the manner in which verbal protocol data were extracted from drivers and mapped into an ADS of level crossing systems; the decision-making trajectories described by expert and novice drivers; and the implications of this for future thinking and reasoning about how level crossings could be designed in future.

2 Methodology

2.1 Design

The objective of the study was to examine formative driver’s decision-making at level crossings using the ADS approach. Data to populate the ADS were collected from in and around a regional town within the Australian state of Victoria. Participants were recruited in two groups (expert and novice) and drove an instrumented vehicle around a pre-defined route featuring ten railway level crossings with two main types of warnings (six active, four passive). The study utilised a within-subjects design, with all participants experiencing the same route, crossing types and sequence. Data collection took place in light traffic conditions as established through pilot tests, and two observers rode in the vehicle. The participants were required to provide a concurrent verbal protocol for the duration of the drive. The study was naturalistic which meant that apart from two instances, no trains were approaching the crossings when participants encountered them. Therefore, for the six crossings that included active warning devices (such as flashing lights, descending boom gates, etc.), these warnings were inactive at the time of approach. As per road rules, drivers were nonetheless required to be aware of the crossing, the possibility of trains and warnings, and to verbalise how they were thinking and reasoning about the system such that appropriate behaviour in that environment could be emitted. Participants were not informed of the focus on driver behaviour at railway level crossings. The pre-briefing instead described a more general study into driver behaviour in different environments, as the route encompassed diverse road types including urban and suburban streets, highways, and unsealed rural roads.

The verbal protocols for each driver were professionally transcribed verbatim. The sections of transcript which referred to the immediate approach and transit of the crossing were then extracted. These were subject to a non-mutually inclusive theme-based content analysis (e.g. Walker, 2005) enabling all level crossing-related themes (pre, during and post crossing) to be captured. The themes were anchored to the ADS headings and categorised according to 1: Level of Abstraction (e.g. Physical Object, Object-Related Purpose etc.) and 2: System Level (e.g. System, Sub-System, Component). This involved an initial categorisation undertaken by a lead-coder followed by scrutiny, checking and verification by the wider author team. Further coding iterations were undertaken as required to ensure the meaning and content of the themes was consistent and demonstrated good ecological validity. Following this, formal inter-rater reliability was established by employing a further five independent raters, rating eight of the transcripts. These were divided into equal expert/novice driver transcripts and equal active/passive crossing types, selected at random. The mean agreement between the independent raters and the previously analysed transcripts was 72.19%.

The dependent variable of verbal protocol themes was mapped into an ADS of the level crossing system. The development of the ADS involved considerable industry and other stakeholder input and was performed as part of a wider study (see Salmon et al. 2016). The mapping process involved stepping through the verbal transcripts in time sequence, plotting the system and abstraction levels of each coded theme using a variation of Rouse et al. (2017) ‘abstraction trajectory’ visualisation methodology (see Figs. 5, 6, 7, 8). The independent variable was experience. This manipulation was employed to help compare the decision-making trajectories to those of previous research (e.g. Rasmussen and Jensen, 1974; Itoh et al. 1995; Vicente 1999) and to explore differences and their meaning. Controlling variables included pilot testing to ensure unimpeded experimental runs in clear weather, standardised vehicle and test protocol, and matching of participants on age and gender.

2.2 Participants

Twenty-two drivers (10 males and 12 females) took part in the study. All participants regularly drove in the study area but were not experts in RLX system design. They were sorted into experienced or novice driver groups. Experienced drivers were required to be aged 30–55 years and possess a full (unrestricted) driver’s licence, whereas novice drivers were required to be aged 18–24 years and within 1–2 years of obtaining their provisional (restricted) drivers’ licence. The experienced driver group (n = 11, Mean Age = 45.1 years) had an average of 27.3 years solo driving experience (SD = 7.6). The novice driver group (n = 11, Mean Age = 19.3 years) had an average of 1.6 years solo driving experience (SD = 0.3). Genders within these subgroups were balanced in both cases: five males and six females. Participants were recruited through local newspapers, notice boards, community groups and word of mouth, and were compensated for their time. The research complied with the tenets of the Declaration of Helsinki and was approved by the Institutional Review Board at Monash University’s Human Research Ethics Committee (protocol number CF12/0077–2012000016).

2.3 Materials

Prior to the drive all participants completed a paper-based demographic questionnaire, were pre-briefed on the requirements of the study, and the desired form and content of the verbal protocol. Additional coaching (if required) was provided in the early part of the on-road course which was designated a practice section.

Participants drove the route in Monash University’s On-Road Test Vehicle (ORTeV), which is a Holden Calais instrumented to collect vehicle and video data. The focus of the current study were the verbal protocols captured by the audio recording equipment, although video and other data were used by the observers and analysts to provide context if required.

The study route was approximately 30 km long and was situated in and around the Greater Bendigo district of Victoria, Australia. The route encompassed a range of road types, including city streets, residential and suburban roads, highways, unmarked roads, gravel and dirt tracks. Speed limits varied along the route, ranging from 40 km/h to 100 km/h. The route featured a total of nine level crossings of two primary types: ‘active’ and ‘passive’. There were six active crossings featuring bells and flashing light controls, with five of these also featuring boom gates. There were three passive crossings with stop or give way signs only, one of which drivers passed through twice in reverse directions. All the crossings conformed to current Australian and State regulations and were in normal use at the time of the study. Figure 2 shows examples of active and passive crossing types as encountered by the drivers in the study. The route took approximately 40 min to complete. All drives were completed on weekdays at either 10 am or 1.30 pm to avoid peak traffic conditions for the area. These times had been assessed by the authors prior to the study to ensure participants experienced similar traffic conditions.

Fig. 2
figure 2

source: Google Earth)

On-road views of an active urban and passive crossing used in the study (Image

Full size image

2.4 Procedure

The study began with the completion of an informed consent form and demographics questionnaire, then a briefing on the research and its aims. These were expressed broadly and described a study of everyday driving (i.e. not one specific to railway level crossings). Standard procedure was followed in terms of advising participants to drive as they normally would and to adhere to all pertinent traffic regulations. The in-vehicle observers would halt the drive should hazardous conditions or driving practices persist.

Participants were then escorted to the test vehicle and instructed to find a comfortable driving position. Two observers were present in the vehicle throughout the drive, one seated in the front passenger seat, the other in the rear. Upon commencing the drive, participants first completed a short practice route whilst providing a concurrent verbal protocol and receiving feedback and instruction if necessary. A key element of the feedback was for driver’s to vocalise whatever came into their mind, not just what they could see, or what they are currently doing, but to describe what they did not currently know, what they were expecting, and how decisions were being made. At the end of the practice route, participants were informed the test had begun and that data collection had now commenced.

While on the experimental route, the front seat observer provided directions while the participants provided verbal protocols constantly for the duration of the drive. At the conclusion of the route, participants were instructed to park and were provided with an experimental debrief during which the specific study aims were revealed. Questions, feedback, and the formalities associated with participant compensation marked the completion of the test.

3 Results and discussion

Verbal transcripts provided by 22 drivers, 11 novice drivers and 11 expert drivers, each encountering ten railway level crossings on a 30 km test route were subject to content analysis. This was to discern what parts of the ADS template were activated and in what sequence. This represents a total of 220 level crossing encounters, two of which featured a train. These encounters were grouped into two main crossing types (active or passive) each with two sub-types. This gives rise to the following: active crossings with boom gates, bells and flashing light controls; active crossing with bells and flashing lights only; passive crossings with a stop sign only; and a passive crossing with a give-way sign only. There were six crossing encounters with no or missing content (i.e. blank transcripts from one novice driver in the passive crossing with give way sign; one experienced and four novice drivers in the lights only crossing condition).

The coding of the verbal transcripts is of an immediate form meaning that themes are mapped directly onto Levels of Abstraction and System Level. This enables the sequence of ADS nodes referred to by drivers in those transcripts to describe a trajectory when they are plotted into the ADS framework. This will be used to inform judgements about the patterns of decision-making drivers employ. The results and discussion are divided into two sections. The first section deals with ‘activation’ of the ADS elements (the system constraints and affordances), comparing the number of times the elements in the level crossing system were associated with the drivers’ decision-making process. The second section deals with the ADS ‘trajectories’, with a comparison between novice and experienced drivers in each type of crossing.

3.1 Activation of level crossing system constraints and affordances

The Abstraction Hierarchy (AH) which underlies the ADS was developed in a separate study and reported in detail in Salmon et al. (2016) and a summarised version appears in Fig. 3. In the full, unabridged version of the AH, there are 115 nodes representing individual system constraints and affordances across the five levels of abstraction. These nodes, and their interconnection via means-ends links, were developed using extensive user and industry input (Salmon et al. 2016).

Fig. 3
figure 3

Summarised abstraction hierarchy for railway level crossings

Full size image

The nodes referred to (i.e. ‘activated’) by drivers in their verbal commentaries provide a high-level descriptive account of what was guiding behaviour in that situation and context. It is important, however, to note at the outset that many drivers did not seem to realise they were even in a crossing area. When they did, the most frequent task-related nodes in the ADS, across all drivers, are those shown in Table 2. Here, it can be seen that the process of checking for trains was the most commonly activated node (n = 56). Drivers also referred to the need to reduce speed on railway level crossings (n = 25), especially after noting road markings and signs (n = 44 and 22, respectively). A small number of drivers also checked for warning lights at the crossing (these nodes were activated n = 12 times in the analysis) and behaved appropriately according to the environment. It was expected that more drivers would perceive the lights in the ‘active’ crossings, but only 27% of the participants mentioned them in the verbal transcripts, presumably because they were (apart from two encounters) not active. Even despite this, it might be anticipated that more drivers would seek out more of these cues to help them with the task of checking for trains, which is clearly something drivers place an emphasis on. To put this in context, every one of the 220 crossing encounters created the opportunity to mention at least one piece of overt crossing/warning infrastructure but the actual frequency with which such artefacts were mentioned was much lower. Tables 2 and 3 present the full list of results. The items in the Constraint column are extracted from the full in-depth AH (see Salmon 2016) which underlie the summarized version showing in Fig. 3.

Table 2 Most commonly activated ADS nodes mentioned by all drivers (n = 22) in their verbal transcripts
Full size table
Table 3 ADS items activated by fewer than four drivers in their verbal transcripts. Shaded objects represent engineering interventions present in the situation(s) that were not activated at all
Full size table

One clear overall finding emerges using the formative ADS approach, it is evident that drivers’ natural tendency is to look for trains (even if not required), slow down (again, even if not required), and to rely less than expected on the overt engineering artefacts put there by railway authorities (the visual, auditory and tactile aids). To clarify, engineering solutions, such as warnings, are clearly an important part of driver situation awareness, and the results are consistent with earlier studies in showing that the more of this infrastructure present in the situation, the more of it is identified by drivers (Salmon et al. 2013a, b). The issue seems to be that their role in driver situation awareness only occurs when such artefacts are perceived, which is less evident than expected, and only in the context of other constraints and affordances in the system which also shape behaviour. This represents a challenge to simplistic ‘information provision’ approaches to design which tacitly assume drivers lack needed information and when they are provided with it they will act accordingly. Clearly this assumption is only partially correct. There was, for example, a noticeable difference in attitude between novice and experienced drivers that came through in the verbal transcripts. This attitude could be summed up for expert drivers as “even if there are lights, I will make my own checks”. For novice drivers, it can be summed up as “I only have to check the crossing because there are no warnings”. In other words, drivers seemed to prefer it if there are warnings but mainly as a source of reassurance or security, with many experienced drivers still not fully trusting them and preferring to supplement these engineering aids with their own checks. This can be further examined with reference to the least activated system nodes as shown in Table 3. Interestingly, a large number of (sometimes costly) engineering elements of the railway level crossing system, such as signage, boom gates and audible warnings, are not often mentioned in the verbal transcripts, sometimes not at all. Again, this is likely to be an artefact of there being only two crossing encounters with a train (and active warnings) present, but even so, only three out of 220 level crossing encounters made explicit reference to being alerted to the presence of a level crossing by an active or passive device placed there by external authorities specifically for that purpose. Boom gates, for example, were noted by only one driver although all 22 participants encountered the same crossing. The boom gates were of course in the upward position in this study, but were nonetheless an obvious feature (see Fig. 2). Table 4 summarises key features of the results obtained across the two levels of driver experience and the four types of crossing. The extent of ‘train checking’ occurring at the (in)active crossings is particularly marked, with expert drivers checking more than novices at crossings with gates, and vice versa at crossings with lights only. Strictly speaking, the active infrastructure performs the ‘train checking’ task for the driver, yet there is a clear compulsion on the part of the driver to perform their own checks. This could be viewed as a redundant behaviour or it could be harnessed in new crossing designs if it is something this situation strongly affords. Relative to these ‘train checks’, Table 4 shows the actual infrastructure of lights, gates, signs and road markings etc. is activated less frequently. Of course, it could be the case that these artefacts are prompting the ‘train checks’ but this was not clearly evident from the verbal transcripts. Even if it were, it could be argued that the infrastructure is affording behaviours that are not necessarily needed or expected. The following section explores these results in detail.

Table 4 Activated ADS elements from the verbal transcripts of experienced and novice drivers for each railway level crossing (highest number of activations in bold)
Full size table

It was expected that the system element ‘detect train’ would appear more in the verbal transcripts for the ‘passive crossings’, where the onus is placed on the driver to look. In the data, however, a strong finding was that drivers check for trains at all types of crossing, even ‘active’ crossings with engineering measures designed to perform this task for them. This behaviour was found to be predominant in the experienced driver group, and in particular at the crossing with boom gates. This crossing type had the most ‘engineering’ present, yet experienced drivers activated the ‘check train’ node 53 times compared to novices’ 15 (Table 4). Experienced drivers seemed more cautious and perhaps less trusting. Phrases, such as “I will check anyways because the lights might not be working” were quite common, for example. The Verbal Protocol Analysis (VPA) technique introduces a little ambiguity on this point as it is not always certain what, precisely, drivers are meaning when they report they are ‘checking for trains’. That being said, these findings are again consistent with Stanton et al.’s previous work (2013).

Another system constraint well evident in the verbal transcripts was ‘speed reduction’. 17.8% of the experienced driver group mentioned speed reduction on railway level crossings, whilst only 11.5% of the novice group did (Table 4). Only one crossing had an explicit requirement to slow down (the one located on Williams Rd) where the required speed reduction was to 60Km/h from 80Km/h, and even then in only one direction. The crossing at Schumaker’s lane did not have any posted speed reduction signs but there were rumble strips and painted train/train track icons painted. The crossing located on the Loddon Valley Highway had advanced warnings including a sign that read ‘Prepare to Stop’, but again, no explicit instruction to slow down. This strong finding, that drivers register an intent/need to slow down on the approach to level crossings, even where technology is present but does not indicate a train approaching, is consistent with previous observational research (e.g. Shinar and Raz 1982).

For ‘passive’ crossings, the system element ‘road signs’ was activated by half of the drivers in the study but at ‘active’ crossings cited by only one. In this particular driving context (and doubtless others too), the stop sign has particular meaning and consequences (i.e. it is strictly enforced) which means drivers focus on this much more than a simple ‘give way’ sign. More widely, it can be noted that other studies suggest placing signage earlier to decrease approach speeds, but even then driver compliance remains unchanged (e.g. Ward and Wilde 1995). In this study, around one-fifth of all drivers became alert to the rail level crossing by seeing the railway line within the wider road landscape, rather than ignoring it pending overt signage or other indications. Changes in design to make the railway line itself more visible could offer an interesting urban design approach, as it is clearly a feature of the road-scape that drivers use in decision-making.

Although the system constraint ‘visual warning of RLX’ did not match any driver’s verbal transcript, the ‘flashing lights assembly’ was activated by 12 (out of 22) drivers. This is somewhat lower than might be expected. The former ADS element is a process, whilst the latter is a physical object, and this is how it was manifest in the verbal transcripts: ‘lights’ as an object rather than ‘warning’ as a process. This same issue was encountered widely in the transcripts, suggesting that reasoning at lower levels of abstraction tends to dominate.

3.2 Decision-Making trajectories

Approximately 95% of all system nodes mentioned in the verbal transcripts were contained in these lower three levels of system abstraction (i.e. Purpose Related Functions, Object-Related Processes and Physical Objects). The abstraction level ‘Object-Related Processes’ contained the largest amount of task nodes (37%) whilst the highest Functional Purpose level contained the smallest amount (< 1%) as Fig. 4 shows. It is legitimate to ask whether this arises because of a bias introduced by the verbal protocol technique. This seems unlikely given that Itoh et al.’s (1995), Vicente’s (1999), not to mention Rasmussen and Jensen’s original (1974) study also used verbal protocols, often with smaller samples than the research described here. Given this, a number of further questions need to be explored.

Fig. 4
figure 4

Percentage of nodes at each level of abstraction activated by drivers in their verbal commentary

Full size image

From previous research (e.g. Rasmussen and Jensen, 1974; Itoh et al. 1995; Vicente 1999), there might be an initial expectation that expert decision-making trajectories fall from left to right on a downward sloping trajectory. These previous studies were dealing with a particular type of trouble-shooting/problem-solving task somewhat different to that under consideration here. Driving, in particular, is an over-learned activity compared to the kind of ‘overt’ problem-solving tasks tackled in previous studies. This may affect the extent to which participants can call into consciousness higher-level functional purposes, and give voice to them via a VPA. More likely is that mention of higher-level functional purposes would only occur where those functional purposes align for different actors in the scenario (e.g. Burns and Hajdukiewicz 2004; Burns 2000) and clearly car drivers would not be expected or maybe even required to know or to ‘own’ all of those shown in Fig. 3. This is worthy of further exploration because the trend appears to be clear. Drivers encountering level crossings are, in general, focussed on lower level objects rather than higher-level purposes, and this could be important in terms of future design interventions.

The results are shown in more detail graphically in Figs. 5, 6, 7, 8 for experts and novices for each of the four-level crossing types. These representations are inspired by Rouse et al. (2017) novel ‘usage trajectories’. In this case, however, instead of showing types of recurring action they show the level of abstraction each driver is referring to in their verbal protocol as they approach and transit the RLX. These representations will be labelled ‘abstraction trajectories’. The y-axis shows the five levels of abstraction in the ADS. The x-axis represents time/sequence, expanded or contracted over fixed sequence steps as required to achieve a clear visualisation. The lighter trajectories are those for individual drivers while the bold trajectory represents the median.

Fig. 5
figure 5

Abstraction trajectories for experts and novices as they approach and transit a railway level crossing with gates

Full size image
Fig. 6
figure 6

Abstraction trajectories for experts and novices as they approach and transit a railway level crossing with a give way sign

Full size image
Fig. 7
figure 7

Abstraction trajectories for experts and novices as they approach and transit a railway level crossing with lights only

Full size image
Fig. 8
figure 8

Abstraction trajectories for experts and novices as they approach and transit a railway level crossing with stop sign

Full size image

Figures 5, 6, 7, 8 present a detailed picture of constraints-based decision-making at level crossings. Notable differences between the drivers’ decision-making processes are clearly evident. Rasmussen’s previous studies suggest that experienced actors are more likely to adopt ADS trajectories that fall from top to bottom and right to left. It is also believed that actors change the abstraction level when reasoning (Naikar 2013). In this study, however, the major part of experienced drivers’ ADS trajectories progressed from left to right (across a reduced number of abstract system layers) while tending to conclude at Purpose-Oriented Functions rather than the lowest level of Physical Objects. A lot of iteration and looping was still noted, especially within the three lower abstraction levels (Purpose-Oriented Functions, Object-Related Processes and Physical Objects).

A notable feature of the analysis was the common paths present in the ADS trajectories. These common paths indicate a standard decision-making process for some drivers. For instance, in the crossing with gates, five out of eleven experienced drivers reduced their speed and then checked for trains, in this order. The novice driver group had generally fewer common paths, suggesting that their decision-making is founded on a more disparate set of affordances and constraints.

Another notable feature is that more diversity in the abstraction trajectories, across all groups, is present in the initial stages of the level crossing task. For all crossing types, except the crossing with gates in the expert category, the abstraction trajectory finishes with alignment to the mode, with all participants converging on this point. This seems to suggest a highly formative element to decision making, with drivers beginning their decision-making process at different levels of abstraction but ending at the same level.

3.3 Statistical analysis

To test whether certain types of level crossings have a more positive influence on behaviour, further quantitative analysis is performed. First, the data acquired through content analysis were categorised into the level of expertise, level crossing type, and the five levels of abstraction from the ADS. Second, for each type of level crossing, the number of mentions for each one of the five levels of abstraction was counted and the proportions of these node counts were calculated. Tables 5 and 6 show the summary results for the experienced drivers and novice drivers’ group, respectively.

Table 5 Proportion of node counts for each level of abstraction for experienced drivers (colour coded according to relative size, with darker grey representing higher proportions)
Full size table
Table 6 Proportion of node counts for each level of abstraction for novice drivers (colour coded according to relative size, with darker grey representing higher proportions)
Full size table

In both experienced and novice drivers’ groups, and in all level crossings, the majority of the task nodes refer to the two lower levels of abstraction, these being “Physical Objects” and “Object Oriented Processes”. In the experienced driver group, all level crossings except gates only presented a total proportion higher than 20% of nodes related to “Purpose-Oriented Functions”, “Values and Priority Measures” and “Functional Purpose”. In the novice driver group, the level crossing with gates is also the one with the smallest proportion of nodes that relate to high levels of abstraction, with the level crossing with lights presenting the highest proportion based on these data, it could be argued that drivers tend to focus on more physical elements of the driving task when encountering RLX’s with gates, which is expected given that the gates act as a physical barrier. An unexpected outcome is that novice drivers generally presented a higher proportion of nodes in the three highest levels of abstraction than the experienced drivers.

To test whether these results are significant, a Multivariate Analysis of Variance (MANOVA) was performed to compare the effect of each type of level crossing on the abstraction levels. The results from this test show that the level crossing type only has a significant impact (p < 0.05) in the two lower levels of abstraction, “Physical Objects” and “Object Oriented Processes”. Therefore, one could not reject the hypothesis that different types of level crossing will impact on the decision-making process of each driver when driving through a rail-level crossing. In this study, the level crossing with gates might have induced drivers’ reasoning process to be at lower levels of abstraction.

4 Conclusion

This paper has mapped the verbal transcripts provided by 22 drivers at 220 level crossing encounters into Rasmussen’s ADS. The motivation is the pressing need to drive out new solutions to the long standing, and persistent, problem of driver behaviour at level crossings. This paper provides a demonstration that how we might assume or expect drivers to behave is not necessarily a good guide for meeting increasingly ambitious level crossing safety aspirations. Indeed, it is in the margins of human performance variability that transformative approaches could be developed, and the method applied in this paper is an attempt to advance this important agenda.

There are three key findings. The first is that driver’s naturalistic behaviour at level crossings, regardless of type, is to look for trains (even if not required), slow down (again, even if not required), and for their behaviour to be shaped by a wide variety of constraints and affordances (some—but not all—put there for that purpose by railway authorities). The situation and context are affording these behaviours in ways that are not always expected and could be expanded upon in future research. Two possibilities exist. The first is that current level-crossing technology at active crossings does not differentiate between a train coming or the technology itself having failed in some way and a train is not coming. In other words, most level crossings only have a ‘prepare to stop’ or ‘stop’ phase not a ‘proceed’ phase, such as a green light. Perhaps a solution like this would place the driver’s focus back on to the engineering aids and help avoid drivers slowing down unnecessarily. The second possibility is that current level-crossing constraints and affordances could be harnessed further. In other words, current behaviours, such as slowing to check for trains at active crossings could, under some circumstances, be seen as a desirable emergent feature that could be amplified through design.

The second finding is that, counter to what previous research might hint at, expert decision-making in these situations does not describe a left–right downward trajectory from high-level system purposes to low-level physical objects. Instead, drivers remain at intermediate and lower levels of system abstraction, with numerous loops and iterations. An interesting question around the type of system (causal vs intentional/public vs work system) arises. The strong possibility arises to design crossings in such a way as to reveal their functional purpose much more clearly than they do at present. This is within the purview of future work, but for example, the functional purpose “provide access across railway line” could be made manifest by the green light solution mentioned above. “Priority access for railway traffic” and “protect[ing] railway [and] road users” could be made ‘self-explaining’ through the socio-technical design of the urban environment (see Stevens et al., 2016). “Minimise delays to road network” could be made manifest by a countdown timer display (showing that the delay will be short) or suggestions for alternative routes made on key approach roads. In whichever case, the goal would be to develop solutions which stimulate the kinds of decision-making trajectories we know from previous research are indicative of good performance. In other words, the ADS analysis provides a benchmark against which human-centred solutions can be evaluated.

The third finding is that the highly formative nature of decision-making at level crossings represents a direct challenge to easy engineering or common-sense solutions. This, in turn, opens up intriguing new margins of human performance variability within which novel practical solutions await discovery. As noted at length in the paper, the ADS is a formative model that has been deployed here as a way to invert the normal perspectives on level crossing safety. Specifically, to help us regard humans in these systems as adaptive and flexible decision-makers (rather than focus exclusively on the notion of human error) and to view the level crossing as a sociotechnical system comprised of multiple actors, agents and entities which combine to give rise to desirable or undesirable outcomes. The study demonstrates that our current level crossing systems are inadvertently constraining some desirable behaviours, affording undesirable ones, and that unexpected system elements are driving behaviour in ways not previously considered. This exploratory study has used the ADS method to shine a light into these previously hidden margins of human performance, and the scope for rethinking existing and future practice appears considerable.

5 Key points

  • Car driver’s naturalistic behaviour at level crossings does not necessarily adhere to engineering norms.

  • Railway level crossings do not currently communicate their functional purpose(s) well.

  • Green lights, countdown timers, and sociotechnical urban design are practical design solutions which could be adopted to evoke more positive road user behaviours.