Keywords

1 Introduction

Technical advances in the automotive industry strive in the direction of full automation. However, besides advantages like improving traffic and fuel efficiency, people do not always trust autonomous vehiclesFootnote 1 (AVs) to make critical decisions (Giffi & Vitale, 2017). Nevertheless, trust in the technology is needed for user acceptance and, therefore, a successful market entry. In order to reduce anxiety in AVs, evaluation experiments are recommended to design AVs in the most suitable and usable way for future passengers. However, in the context of AVs, evaluation experiments remain difficult in real-life conditions and could lead to safety issues for participants. Therefore, an exploratory study using Virtual Reality (VR) has been conducted with a participatory design approach that included users in the design process of Human-Machine Interfaces (HMI) for passengers of an AV for public transport.

2 Related Work

Studies that investigate current user acceptance for AVs reveal concerns from users’ side regarding AVs especially in the public transport sector. Schoettle and Sivak (2014) conducted a survey that revealed that 45.9% of the 1533 respondents from U.K. and U.S. were very concerned about AVs in public transport. In the study of Dong et al. (2017) with 891 respondents only 13% of respondents stated that they would be willing to ride a bus without an employee of the respective public transport agency on board. Anania et al. (2018) found within a workshop with 50 participants that less people would allow a child to ride an autonomous school bus compared to a bus driven by a licensed human driver. A global study including 17 countries and approximately 20,000 respondents from Deloitte, (2017) revealed that 62–81% of potential consumers from China, India, Germany, USA, Japan, and South Korea had concerns that AVs will not be safe. The following reasons of concern have been identified (Deloitte, 2017; Dong, DiScenna, & Guerra, 2017; Pillai, 2017; Schoettle & Sivak, 2014):

  • System performance in poor weather conditions (safety)

  • The interaction of AVs with manually driven vehicles (safety)

  • AVs do not drive as well as human drivers (safety)

  • System and AV security

  • The interaction of AVs with vulnerable road users (safety)

  • Lack of assistance and information.

Currently, most HMI concepts for AVs that have been published by car manufacturers consider concepts for other traffic users, like for instance the Mercedes F 015 that projects a zebra crossing on the street to indicate that passengers may cross the street in front of the vehicle (Mercedes Benz, 2015). Additionally, HMIs give information about the AV’s intention and detection while being in autonomous mode to the drivers of partly-autonomous vehicles like the BMW Vision Next 100, the Rinspeed Oasis, or the Toyota Concept-I (BMW, 2016; Rinspeed, 2017; Toyota, 2017). However, published concepts for AVs that focus on passengers (and not solely the driver) are sparse, especially in the public transport sector.

In order to ensure an appropriately functioning system or product, evaluation of interaction in the actual field of application is required (Heufler, 2004). Since experiments in real-life conditions are not feasible yet though, the presented study uses the technology of Virtual Reality (VR). VR constitutes an advantageous alternative for evaluation if experiments are not feasible in real-life conditions or would turn out to be very expensive (Mihelj, Novak, & Beguš, 2013). The is based on studies in pedestrian and HMI research that have demonstrated the suitability of VR for investigations that involve AVs and human participants (Berg & Vance, 2016; Deb et al., 2017; Pillai, 2017; Stadler et al., 2019; Sween, Deb & Carruth, 2016).

3 Method

The study focuses on autonomous buses in public transport (called ‘AVs’ in the rest of the paper). The method consisted of three steps: (i) design of concepts and variants of HMIs for passengers of an AV, (ii) evaluation of the HMI concepts and variants thanks to an experience simulation in VR, and (iii) selection and refinement of one HMI concept for AV passengers.

For (i), users’ and experts’ interviews have been conducted and compared with published literature to derive a requirements catalogue for identifying fundamental needs for passengers inside AVs. With the help of the requirements catalogue, a morphological analysisFootnote 2 was conducted to derive a range of HMI concepts (including variants).

For (ii), after implementing the HMI concepts and variants in the virtual environment, experience simulation tests have been conducted in VR with participants. This method enables to identify how people might behave or interact in given situations and is especially useful for studying new services, environments, and interactions (Kumar, 2012). The scenario was defined as a riding experience inside an AV driving around a block. The simulation was created in a virtual environment in which six events were programmed for the tests (Fig. 1).

Fig. 1
figure 1

Scenario for experience simulation

Full size image
  1. 1.

    Start of scenario: The participants were standing in the entrance of an AV and had the task to choose a spot to sit or stand for the ride. After that, the AV started to drive.

  2. 2.

    The AV approached a zebra crossing with pedestrians (virtual agents) on the sidewalk. The AV stopped and the pedestrians crossed the street.

  3. 3.

    The AV approached a junction at which it had to join lanes of an intersecting street. Directly afterwards, due to a temporary construction side, the AV had to change lanes again.

  4. 4.

    The AV stopped at a bus stop.

  5. 5.

    The AV had to make an emergency brake for a jaywalker who was running across the street.

  6. 6.

    End of scenario: The AV returned to the starting point and the scenario ended.

The experience simulation was combined with observations, questionnaires, and participant interviews.

After a briefing and consent agreement, the participants were equipped with a head-mounted display (HMD) (i.e. HTC VIVE). The VR setup consisted of an empty tracked area of 4.0 m × 4.0 m with a chair to sit down (when the participants decided to sit during the test). Besides the HMD, the participants were equipped with noise-cancelling headphones.

At first, a tutorial familiarized the participants with the VR technology. Subsequently, the scenario was played without any HMI concept inside the AV. This represented the baseline scenario (control group) for later comparison. Afterwards, the scenario was conducted in a randomized order for each previously defined HMI concept. Video-recorded observations gathered the participants’ behaviours and reactions (e.g. being surprised or amused) to the events. At the end of the experience simulation in VR, the participants were requested to fill out a questionnaire regarding their perception of the HMI concepts using a five point Likert scale (Likert, 1932). The questions were based on the System Usability Scale (SUS) defined by Brooke, (1996). Finally, the researchers conducted interviews to get insights about subjective justifications from the participants why they liked or disliked the HMI concepts.

For (iii), as a last step, all collected data (i.e. observations, the answers of the questionnaire, and answers of the interviews) were analysed to derive concept features that improved or impaired the respective concepts. Thus, the participants’ preferred HMI concept was refined with the gathered feedback in order to develop the most suitable and usable concept out of the users’ perspective.

4 Results

With the help of ten user interviews, five expert interviews (including psychologists, engineers, and designers), and a literature research, a requirements catalogue was developed as a basis for a HMI concept generation. The requirements were divided into three groups (must have—should have—could have) (Table 1).

Table 1 Requirements catalogue
Full size table

Out of the requirements catalogue, two solution approaches were generated (Figs. 2 and 3).

Fig. 2
figure 2

Solution approach 1

Full size image
Fig. 3
figure 3

Solution approach 2

Full size image

Within solution approach 1 (Fig. 2), a LED stripe is installed at the lower end of the front windshield. This stripe shows the AV’s intention via light indications that point to a direction (e.g. LED lights up at the right side shows that the AV will steer to the right). Additionally, small light indications visualize other road users (blue lights symbolize pedestrians and yellow lights symbolize other vehicles).

Solution approach 2 (Fig. 3) consists of a semi-transparent screen-like surface that shows various information during the ride like the AV’s intention, detection of obstacles, other road users as well as the AV’s direct surroundings. Furthermore, four variants were built for solution approach 2. While variant A shows the AV’s detected direct surrounding through its cameras, variant B shows the vehicle and its direct surrounding on an abstracted two-dimensional map. Variant C shows the AV and an abstracted map from a three-dimensional bird perspective. Lastly, variant D combines variant one and three (Fig. 4).

Fig. 4
figure 4

Solution approach 2 concept variants

Full size image

The experience simulation was conducted with overall seven participants. The reason for keeping the sample size small was based on usability principles, defined by Rubin and Chisnell (2008) as well as Nielsen (2000) who stated that a sample size of five to ten participants is the most efficient for identifying the most relevant usability problems.

The participants’ observations revealed that during the scenario without any included on-board HMI concept, the participants showed behaviours that implied discomfort and uncertainty. The behaviours consisted of covering the body with hands and arms (protective position), lifting shoulders, and/or trying to hold on to something (e.g. a handrail that was visible in VR) (Fig. 5).

Fig. 5
figure 5

Noticeable participant behaviours

Full size image

Usability scores were derived from the questionnaires (Table 2). It shows that solution approach 2 had an overall higher usability score than solution approach one. Only one out of the seven participants rated solution approach 1 higher than solution approach 2.

Table 2 Average usability scores of questionnaires
Full size table

A further question was to pick a favourite variant of solution approach 2. Five out of the seven participants chose the third variant (i.e. abstracted map from a three-dimensional bird perspective, variant C of Fig. 3) as their favourite concept. During the interviews, the participants stated that they felt uncomfortable when being confronted with the baseline scenario since they did not know where the AV would drive and if it would be capable of behaving correctly in the traffic environment. Overall, the participants claimed that the HMI concepts helped to make the AV’s intention and detection system transparent. However, in total, solution approach 1 was not perceived as usable and supportive as it was less easy to understand and less visible. Solution approach 2 was subjectively preferred by the participants. Two major improvements for variant C of solution approach 2 were brought forward by the participants:

  • The screen size and position must be improved since it blocked the view outside the vehicle and was not well visible from every spot inside the AV;

  • Additional content like speed of the AV, overall route of the AV and real-time information should be added.

With the help of the user feedback, the favoured HMI concept was refined (Fig. 6).

Fig. 6
figure 6

Refined HMI concept

Full size image

Figure 6 shows the final HMI concept with adapted screen size and positioning above the AV’s windshield. Additional information like overall route, current speed, as well as real-time notifications was incorporated into the concept.

5 Discussion

This exploratory study showed that the HMI concepts were helpful for the participants for feeling more comfortable while driving with an AV for public transport.

The solution approaches and variants consisted of information about the AV’s driving behaviour to minimize uncertainties for passengers inside the vehicle. Even though Carsten and Martens (2018) considered lower level AVs (i.e., not fully autonomous) in their study, they also found out that comfort can only arise as soon as the passengers know and can predict how the AV will behave, and thus, no automation surprises occur.

The study was intended to generate, evaluate, and refine HMI concepts for AVs in a quick and exploratory way. Therefore, the focus was not to test the concepts with a statistically representative quota sample of participants, but to identify basic requirements, user preferences and usability indicators for future HMI systems. This provides a basis for future design studies and concept developments.

VR turned out to be a useful tool for experience simulations since it created an immersive experience even though the chosen scenario is currently barely feasible in real-life conditions. The participants stated that the experience was engaging and convincing and thus conveyed authentic situations to them. Furthermore, it constituted a highly flexible low-cost solution that enabled the creation of complex scenarios in a short period of time. In this context, Deb et al. (2017), who conducted VR experiments in the field of pedestrian research, also concluded that the technology of VR has many advantages over real-life studies like safety, validity, time and costs.

The presented study showed however some limitations. Since every scenario consisted of the same events, the learning curve for participants was anticipated to be high. The baseline scenario was always played as the first scenario in order to reveal the discomfort for participants during the ride. Even though the subsequent scenarios involving the HMI concepts were played in randomized order, it is suggested that the stress level for participants already decreased since the participants could expect the upcoming events in the subsequent scenarios. In future studies, having a bigger set of events and varying scenarios could prevent this limitation.

6 Conclusion

Technological progress in automation in the field of transportation is not only advantageous but also creates challenges that have to be addressed to ensure user acceptance and thus, a successful market entry. The presented study showed in an exploratory way that Human-Machine Interfaces have the capability to release discomfort for passengers through showing information like the AV’s intention or detection of its surroundings. In future studies, more complex scenarios and events will be created to test new concepts in a variety of situations and thus provide holistic.