1 Introduction

Wildfire threat to communities is a significant problem in many countries around the world (Guha-Sapir et al. 2015). Public early warning plays a critical role in protecting lives and reducing injuries by raising risk awareness and promoting protective actions (Mileti and Sorensen 1990; Quarantelli 1984). Currently, wildfire warnings issued by local officials are predominantly presented as text and published on local officials’ websites (Emergency Management Victoria (EMV) 2014). They usually offer a comprehensive range of information concerning fire setting and movement, influence of prevailing winds, location of communities under threat, and suggested protective actions. Due to the salient spatial nature of wildfire warning information, discussions surrounding an alternative, map-based approach have emerged in recent years with the intent to add spatial clarity to the text-based messages. However, the strategies employed by emergency management agencies for providing wildfire warnings using maps are diverse. The predominant approach is to furnish simple maps illustrating cursory fire locations using point-based markers as a supplement to comprehensive textual messages (Fig. 1a). More recently, a growing number of local emergency agencies have begun to supply the public with maps depicting enriched wildfire warning information including fire perimeters, wind conditions, and warning polygons via interactive web-based mapping applications (Fig. 1b, c; Emergency Management Victoria (EMV) 2015; Google Crisis Response 2015; Government of South Australia 2015). Also, some of these agencies have adopted interactive mapping applications as the default information portal for public warnings, providing conventional textual warnings as secondary information (e.g. Fig. 1c; Emergency Management Victoria (EMV) 2015; Government of South Australia 2015). Yet, a high degree of diversity still exists in the content provided on the maps, in the cartographic representation used to depict warning information, and in the interactive functionalities allowed by the mapping interfaces.

Fig. 1
figure 1

Examples of existing mapping approaches used to deliver wildfire warnings: a alerts and warning maps provided by the Department of Fire and Emergency Services (Western Australia), b California Fire Map provided by CAL FIRE; and c VicEmergency provided by Emergency Management Victoria and supported by fire authorities in the state of Victoria, Australia. The amount of spatial content provided by the maps escalates, respectively

Full size image

The discrepancy in approaches used for delivering map-based early warning information is attributable to two largely unanswered questions: (1) what is the effectiveness of using maps for delivering wildfire warnings compared to using text-based warnings, and (2) what elements and designs constitute an effective wildfire warning map? The first question has been addressed by Cao et al. (2016b) illustrating that communication of spatially related wildfire warning information using appropriate cartographic representations can provide increased appeal, improved understanding and heightened risk perception when compared to similar text-based messages. However, this leaves open the question of how interactive mapping applications can best be designed to improve the effectiveness of wildfire warnings.

To address this knowledge gap, a wildfire mapping application was prototyped, and an evaluation of the tool was conducted in the Australian context, to develop a blueprint for creating usable and effective map-based wildfire warnings. Before introducing the study questions and methods in specific, we feel it is necessary to clarify that the map-based warning instrument developed for this study was intended to serve as an alternative warning communication approach to text-based messages. Therefore, the mapping application had to provide access to sufficient warning information such that citizens who prefer visual depictions of wildfire information over text would not need to seek additional information. However, in reality a map-based approach does not necessarily discount the use of text-based warnings, and both options can be provided to suit varied predilection of the users. Furthermore, ‘effectiveness’ of a warning instrument in this paper is defined as achieving its communication goals, that is, to provide information that can promote timely and appropriate decisions regarding protective behaviours in response to an impending threat (Quarantelli 1984). To this end, this study sought to address two specific questions: (1) what information should be included in a wildfire warning mapping tool to facilitate decision-making; and (2) how can an interactive mapping interface be best designed to present this information to ensure usability and ease of information access?

In the remainder of this paper, we first present a review of the key information elements necessary for effective warnings, as well as the general principles for designing usable, interactive mapping interfaces for non-experts. The current public response to wildfire emergencies in the Australian context is then introduced, followed by a demonstration of the methods for designing, prototyping, and evaluating an Australian-based wildfire mapping application. Findings from the evaluation study are then presented and discussed, identifying the imperative content for map-based wildfire warnings whilst highlighting some of the principles for effective mapping interface design. The study results provide valuable empirical evidence with respect to how to design effective and usable tools to truly harness advanced mapping technologies to benefit the general public in the face of wildfires, potentially saving lives by providing adequate information and facilitating appropriate decision-making.

2 Important warning information for effective warning

Whilst there has been an absence of the literature discussing the appropriate content of effective warning maps, the content of text-based warnings has been a topic of interest for decades. Based on a review of public response to hazard warnings, Mileti and colleagues (Mileti and Peek 2000; Mileti and Sorensen 1990) developed a generic formula for effective warning messaging, specifying several crucial components, including the hazard and its consequences (hazard), the location and effective time of the warning (warning location and time), guidance for protective actions (guidance), and a credible source for the information (source). However, these information elements may be of differing importance, and their relative significance is subject to the warning delivery mechanism, hazard type, and behavioural characteristics of the warning receivers (Mileti and Sorensen 1990). For example, Baker (1991) reported that ‘aggressive’ communication, such as face-to-face delivery of mandatory evacuation orders (guidance), effectively stimulated evacuation behaviour in hurricane events prior to the 1990s. By contrast, scientific prediction of hurricane strike probability and severity (hazard) was found to be relatively less effective in terms of eliciting evacuation behaviours in these events (Baker 1991). Such a salient effect of evacuation orders implies people’s trust in local officials’ risk assessment and protective action recommendations. However, residents’ perception of officials’ trustworthiness is contingent upon their credentials and past job performance (Lindell and Perry 1992; Kramer 1999), which may change over time along with a residents’ experiences. Residents’ trust in official guidance during early hurricane events may not be generalisable to other types of hazard or temporal and social contexts. For example, studies by Riad et al. (1999), Gladwin et al. (2001) and Smith and Kain (2010) have documented non-compliance to evacuation orders by a majority of residents during several hurricane events after the 1990s. Dow and Cutter (2000) further identified that hazard ‘savvy’ populations who previously experienced similar hurricane events and evacuation guidance did not blindly adhere to an evacuation order, but rather based their evacuation decision on self-dependent risk assessment using accurate, timely, and specific hazard information. Conversely, in response to the 2009 American Samoa Tsunami, Lindell et al. (2015b) found that environmental cues and incomplete warning messages were sufficient to elicit effective risk recognition and timely evacuation by a majority of residents. The implication may be that for hazards characterised by rapid onset, people have instinctive heuristics that allow them to infer protective actions from simple information concerning the hazard. Consequently, the specific recipe for warning messages should be scrutinised by accounting for particular hazard characteristics, and social and risk perception contexts (Mileti and Sorensen 1990).

In contrast to the abundant research examining what constitutes effective text-based warnings, there is a paucity of systematic investigation concerning how to construct effective warning maps, leading to large discrepancies in the content of current online maps used for public warnings: some warning maps only provide simplistic visual information, whereas others attempt to offer more enriched visualisation of the hazard and warning. Specifically, simplistic warning maps often focus on the visualisation of one selected aspect of information, such as those that provide cursory incident locations (hazard; e.g. Fig. 1a) or designated warning areas (location; e.g. National Weather Service (NWS) 2008), serving as a supplement to textual warnings. However, the visual communication of only one warning element may not yield positive effects on warning comprehension and responses. For instance, the communication of warning location maps (i.e. depicting warning polygons) alone was found to result in erroneous risk interpretation and decisions (Ash et al. 2014; Lindell et al. 2015b), potentially ascribable to the inadequacy of the information provided for elucidating the spatial heterogeneity of the hazard and risk situation. Enriched visual warnings, on the contrary, often attempt to provide a relatively more comprehensive picture of the hazard and warning context by offering visual details regarding both the hazard and warning location. Yet, diversity continues to exist in the specific map content adopted for portraying hazards and warnings. For example, tornado warnings in the USA employ real-time radar imagery to provide visual evidence of the location and extent of advancing tornados (hazard), in association with warning polygons maps to delineate warning locations (National Weather Service (NWS) 2008). By contrast, the ‘cone of uncertainty’ (COU) maps widely used for hurricane warnings provide an example of how maps are used to represent a more complex range of hazard- and warning location-related information, involving the current location of the tropical cyclone centre, its predicted track, statistical errors associated with the prediction, projected storm intensity at different timepoints, as well as coastal areas for which watches or warnings have been issued (National Hurricane Center (NHC) 2015). However, studies have identified the deficiency of both mapping approaches in providing appropriate information to facilitate comprehension and decision-making. First, Casteel and Downing (2013) demonstrated the lack of effectiveness of radar imagery in enhancing people’s risk perception and motivating behaviours. Despite the small sample size adopted by the study, the results warrant further examination of how useful radar imagery is for communicating warnings. Furthermore, in relation to the COU map, several studies have suggested that residents can generally understand the relatively higher strike probability along the centreline of the predicted storm track and within the uncertainty cone (Cox et al. 2013; Wu et al. 2014, 2015a, b). However, people tend to be overconfident with the reliability of the forecast centreline (Broad et al. 2007), assuming significantly less risk at other locations within and outside the cone (Cox et al. 2013; Wu et al. 2014, 2015a, b). Moreover, several studies have revealed that some residents erroneously relate cone size to storm intensity and misinterpret cone area as a warning zone (Broad et al. 2007; Ruginski et al. 2016), potentially due to the visual prominence of the uncertainty cone in relation to the point and polygon symbols depicting hazard intensity and warning areas on the map. The confusion caused by the uncertainty cone thus warrants reconsideration of the inclusion of strike probability information on hurricane warning maps. In conclusion, it is important to identify both the most useful information elements and the most useful method of their delivery for providing holistic and effective visual warnings. When supplying insufficient or inappropriate visual information, warning maps may have no added, or even adverse effect on the public’s comprehension and response when compared to text-based warnings.

In the specific context of wildfire warnings, as with other hazards, protocols exist for conventional text-based warning communication. For example, the Common Alerting Protocol Australian Profile (CAP-AU standard; Australian Government 2013) stipulates the provision of location and time for each warning issued, along with information detailing the exact location of the fire and its likely impact, direction of fire movement, wind conditions, and time remaining before impact for a community (hazard), as well as guidance for protective behaviours (guidance). Text-based wildfire warnings in Australia often encompass all these elements in an order of warning location, time, hazard, and guidance. However, the scarcity of empirical studies scrutinising the relative importance of these information elements leaves open whether the current information presentation approach is effective.

In relation to map-based wildfire warnings, much of the information identified in the CAP-AU standard for text-based warnings is geographic in nature and can be visualised using maps. Specifically, Cao et al. (2016b) identified seven spatial elements from typical Australian textual wildfire warnings following the CAP-AU standard, including: (1) fire location/perimeter (hazard), (2) fire suppression status (i.e. location of active/contained fire edge; hazard), (3) current and forecast wind condition (hazard), (4) predicted fire movement (hazard), (5) fire warning areas and associated warning levels (warning location), (6) closed roads (guidance for evacuation), and (7) evacuation centre locations (guidance for evacuation). Analogous to the diverse visual warning approaches used for other hazards, maps currently used for wildfire warnings have adopted varied selections of these seven spatial information elements but not necessarily all. For instance, the California Fire Map provided by California Department of Forestry and Fire Protection (CAL FIRE) (2016) delivers both general wildfire locations and accurate fire perimeters (element 1); and the VicEmergency and Alert SA supplied by two Australian state agencies (Emergency Management Victoria (EMV) 2015; Government of South Australia 2015) provide maps of warning polygons (element 5), fire perimeters (element 1) and wind conditions (part of element 3). However, a search of scholarly literature to date has found a paucity of research examining the effectiveness of existing wildfire warning maps for eliciting appropriate public responses, or contemplating which of these spatial information elements are important and should be included in map-based warnings to facilitate decision-making.

3 Usable map interactivity designs for information access

Web-based mapping technologies allow for active control of spatial information through interactive displays (Crampton 2002). Fundamental functions of interactive maps include the ability to search locations, turn map layers on and off, zoom in and out and pan across a map view, retrieve additional information pertinent to particular features, and measure distances (Steinmann et al. 2005). Whilst the general public’s ability to use interactive maps is mediated by experience, expertise, motivation, and spatial cognition, interactive features can be designed in a way that caters to a wide audience with varying abilities (Roth 2013). Generally, interactive maps designed for novice and non-expert users should follow several heuristics, including simple functionalities (Roth and Harrower 2008), intuitive presentation of information (Hagemeier-Klose and Wagner 2009), and flexibility to support a diversity of users’ habits (Nivala et al. 2008; Roth and Harrower 2008). Moreover, an introduction to interactive controls, or short training, was found to have a significant influence in enhancing people’s ability to use sophisticated mapping tools (Andrienko et al. 2002). Whilst training of general users on the Internet before an emergency event is challenging, adequate ‘tool usage’ instructions delivered through effective interface design, such as tooltips and explanatory text (Roth and Harrower 2008), may heighten the ability of novice users to interact with advanced functionalities.

The effectiveness of interactive mapping tools is also impacted by the type devices for which they are designed, i.e. smartphone, tablet, and/or desktop computer. These types of devices exhibit intrinsic distinction in their screen sizes and user input techniques (Chittaro 2006). For example, a mapping tool designed for a desktop computer’s monitor may not display properly on a mobile device with a small screen. Given the current prevalence of the variety of devices, to construct effective mapping applications requires respective system design and user evaluation on a number of platforms.

In the context of hazard risk visualisation, a particular challenge is the presentation of data at a scale that captures the extent of the hazard ‘footprint’ whilst allowing for individual contextualisation (Lieske 2012). For instance, a regional flood map often conceals local information related to one’s particular location, preventing risk personalisation (Lieske 2012). Theoretically, this problem can be alleviated through interactive manipulation of map scales enabled by web-based applications (Lieske 2012); however, empirical evidence regarding user ability to personalise risk through dynamic map viewing is currently lacking.

4 Individuals’ decision-making and response to wildfires in the Australian context

In this section, we present an overview of Australian wildfire management and the Australian public’s likely response in the case of wildfire emergencies in order to provide the context for which the wildfire mapping application presented in this study was designed and evaluated. Australian wildfire management is renowned for its unique national policy that does not mandate evacuation. Rather, it prioritises ‘leaving early’ as the safest option at all times, whilst endorsing residents’ choice of ‘staying and actively defending one’s property’ under restricted conditions (Handmer et al. 2010; Teague et al. 2010). Such a policy was informed by the fact that both evacuation and staying and defending may ensure the safety of humans, but can be significantly dangerous if not conducted appropriately. Specifically, evacuation at the last minute has accounted for significant wildfire fatalities in Australia (Haynes et al. 2010), and staying and defending without adequate preparation is likely to result in failed defence, leaving the defenders in severe peril (Cao et al. 2016a; Handmer et al. 2010; Tibbits and Whittaker 2007). The policy therefore stresses the importance of establishing a household ‘fire-plan’ at the start of a wildfire season that entails an explicit decision concerning which protective action will be enacted (i.e. either ‘leaving early’ or ‘staying and actively defending a well-prepared property’), coupled with adequate planning and preparation for the execution of the identified action (Llewellyn 2012). Specifically, those who intent to evacuate early should not only assemble an emergency kit for their escape but also specify a trigger for determining when evacuation should be activated; those who plan to stay and defend should make sufficient physical and psychological preparation for actively defending their property throughout a wildfire, coupled with contingency plans for when defence fails (Llewellyn 2012). Ideally, with explicit and adequate planning, householders can be prepared for various emergency scenarios, and make easy, early and safe decisions regarding either leaving early or staying and defending when a prospective wildfire threat is identified.

In reality, however, whilst many Australian citizens have espoused a ‘fire-plan’ to either ‘stay and defend’ or ‘leave early’, the planned actions are often not adequately deliberated or prepared. Specifically, research has found that many of the individuals who plan to evacuate are unable to identify an explicit trigger for evacuation prior to an emergency (McLennan et al. 2015). Those individuals are therefore likely to not enact prompt evacuation upon receiving a warning. Rather, they often demonstrate a desire to assess the fire situation and contemplate whether, and at what point, evacuation is warranted, resulting in delayed evacuation (Tibbits and Whittaker 2007). In a similar vein, many of those who have decided to stay and defend are found to not be fully committed to active defence, and they intend to evacuate if they ‘feel threatened’, such as when a possible passage of fire front is identified (Tibbits and Whittaker 2007, p. 289). However, without sufficient information, it is difficult, if not impossible, for residents to make an accurate assessment of the prospective fire threat at an early stage. Those who plan to stay and defend until they feel threatened thus are essentially planning to wait before making a final decision and are likely to evacuate late, when it is no longer safe to do so (Tibbits and Whittaker 2007).

Moreover, an appreciable proportion of Australian residents is found to not be able to make an explicit choice between ‘staying and defending’ and ‘leaving early’ prior to an emergency (Heath et al. 2011; McLennan et al. 2015; Trigg et al. 2015). Many communicate a plan to ‘wait and see’, and intend to leave the decision to the day of an event (McLennan and Elliott 2012; McLennan et al. 2015; Rhodes 2007; Whittaker et al. 2009). However, when a potential wildfire threat is identified, those residents often still cannot make the decision (McLennan 2014). Such a reluctance, and/or difficulty, in deciding between ‘staying and defending’ and ‘leaving early’ is driven by residents’ desire to both save their property and protect the lives of family members (McLennan and Elliott 2012; McNeill et al. 2015). McLennan (2014, p. 5) hence has characterised the response of these types of residents in wildfire events as ‘wait, seek more information, and hope for the best that the fire will not impact their property and they will not have to make a choice’.

In summary, a deficient ‘fire-plan’ prior to an event, meaning either an underprepared plan to ‘stay and defend’ or ‘leave early’, or a deliberate plan to ‘wait and see’, entails a complex psychological process of contemplation and decision-making for implementing the ‘fire-plan’ when a wildfire actually strikes. This often results in delayed decisions and responses, putting residents in significant peril. Therefore, given the inadequacy of many Australian residents’ fire-plan, effective warning information is critical for aiding decision-making during a wildfire event. However, specific needs tied to warning information may vary by individual and intended action (i.e. ‘staying and defending’, ‘leaving early’, or ‘waiting and seeing’), as each approach requires a unique type of situational reasoning and decision-making. Guided by this assumption, the current study sought to identify the most important information elements for providing effective map-based wildfire warnings for people with various fire-plans. Details of the methodology are introduced in the next section.

5 Methods

To understand what constitutes an ‘effective’ map-based warning approach, a preliminary wildfire mapping tool was prototyped and evaluated. A formative evaluation approach (Kaplan and Maxwell 2005) was used to identify how the mapping tool could best be designed to achieve the goal of facilitating individual decision-making and easy information access. Specifically, the evaluation focused on exploring the important information elements to be communicated, and identifying optimal design characteristics for presenting warning information through an interactive mapping interface. In the following, we introduce the prototyped tool and evaluation procedure in detail.

5.1 The prototyped mapping tool

The wildfire mapping tool prototype was designed to serve as a comprehensive portal for citizens who prefer the visual communication of wildfire warning information. ‘Comprehensiveness’ was achieved by supplying an inclusive array of information (Table 1) identified from examples of text-based warnings currently disseminated across Australia. The information content first included spatial information elements identified by Cao et al. (2016b) concerning hazard, warning location, and response guidance (Table 1). Results from Cao et al. (2016b) suggest, however, that a combination of appropriately designed map elements and text-based descriptors may provide for an optimal communication strategy. Therefore, the prototyped mapping tool adhered to these design characteristics by coupling spatial information elements (e.g. burnt area represented using a map in Table 1) with text-based descriptors (e.g. number of hectares of the burnt area in Table 1). The map representations of spatial warning information were designed based on the most effective symbols and design principles identified by Cao et al. (2016b).

Table 1 All the spatial and non-spatial information elements provided by the prototyped wildfire mapping application
Full size table

In addition to the spatial-related hazard, warning location, and response guidance information communicated in current text-based warnings disseminated across Australia (Table 1), the prototyped mapping tool delivered two additional pieces of spatial information, namely a map-based indicator of the user’s home and text-based descriptions highlighting the distance from the user’s home to the nearest fire front (i.e. personalised information in Table 1). The user’s home was mapped based on an address search and depicted by marking the location using a blue house symbol. The marked location was used to centre the map view, with the map scale adapted to show details of the local extent whilst ensuring the inclusion of all necessary wildfire information. The distance between the nearest active fire edge and the marked location was then calculated and displayed to the user.

In addition to spatial information elements, the prototyped mapping tool also conveyed non-spatial information delivered in Australian text-based warnings, including fire danger rating (FDR) and specific action advice provided for areas under warning (Table 1). The FDR information was provided to indicate fire weather severity and can be related to one’s decision to ‘stay and defend’ or ‘leave early’. That is, under severe FDRs, agencies’ recommendation is that ‘leaving early’ is a safer option, and those with a plan to ‘stay and defend’ need to re-evaluate their decision (Department of Fire and Emergency Services (DFES) 2015b). The action advice provided by wildfire warnings often suggests the appropriate timing (e.g. immediately) for activating one’s fire-plan (i.e. ‘staying and defending’ or ‘leaving early’). Notably, the prototyped wildfire mapping tool aimed to provide personalised FDR and action advice information by identifying and presenting information associated with one’s search location (Table 1). Finally, several Google base maps were provided to users during the assessment process (Table 1) to help identify preferences. These included Google Street Map as a default background, with Google Satellite Imagery and Google Terrain Map as additional background options.

The prototyped tool also allowed for fundamental interactive functionalities such as address search and map manipulation as supported by Google Maps, control of map layer visibility, retrieval of text-based descriptors through callout boxes enacted by clicking on pertinent map features (see footnote for Fig. 2 for more details). The web-based mapping tool (Fig. 2) was prototyped using open source web-GIS development tools including OpenLayers (2015; version 2.13.1) and GeoServer 2.4.4 (2015; version 2.4.4).

Fig. 2
figure 2

A screenshot of the prototyped interactive mapping tool used for communicating wildfire warnings. The interface contains three sections: a map table of contents (section 1) and a map view section (section 2) conjointly showing information elements 1–18 and 21–23 (Table 1), and a Household Risk Advice section showing information elements 19 and 20. Interactive control of map visibility allowed for the presentation of all wildfire-related map layers and the two additional base maps (i.e. Google Satellite Image and Terrain Map). In the current image, three wildfire-related map layers are visible on the map: fire control status, current wind conditions, and closed roads. Textual descriptors associated with the spatial information elements are shown in the map legend or as map annotation. Personalised spatial information (Table 1), including the map of a user’s home and its distance to the nearest fire front, is automatically shown following the search of one’s address: the legend of the former is shown in the map table of contents, and the latter is presented in text under the name of the fire at the top of the same section

Full size image

5.2 Participants

Participants for the current study were elicited from respondents of a previous survey-based study conducted in three wildfire-prone suburbs within Western Australia (Cao et al. 2016b). Out of those previous respondents (N = 264), 168 survey respondents expressed an interest in engaging in a subsequent interview study through a question asked at the end of the survey. The final interview sample was selected from these survey respondents. With an intention to identify varied information needs by residents with distinctive fire-plans, a purposive sampling approach was adopted to obtain a mixed sample involving residents with different fire-plans. The survey responses revealed that 28.9% of the previous survey participants (N = 235Footnote 1) planned to ‘defend as much as they can but leave if it’s too severe’, and 41.7% planned to ‘stay and defend or leave early depending on the day’, both indicating a strategy to ‘wait and see’. Furthermore, 10.2% of the survey participants planned to ‘stay and defend throughout the fire’, 13.2% planned to ‘leave early’, and the remaining 6.0% did not identify a plan. To represent the distribution of fire-plans revealed by the survey responses, we developed a sampling strategy to elicit 12 participants with a plan to ‘defend as much as they can but leave if it’s too severe’ (representing those who plan to ‘wait and see’), six participants with a plan to ‘stay and defend throughout the fire’ and six participants with a plan to ‘leave early’. In addition, to represent the varied characteristics of the population, each ‘fire-plan’ group was designed to contain a mixture of rural and urban residence, and people of different genders, education levels and e-map habits. Using such a sampling approach, 24 potential participants were randomly identified, of which 21 eventually consented to participate. The demographic characteristics of the 21 interviewees are presented in Table 2.

Table 2 Demographics of the 21 interviewees
Full size table

5.3 Test scenarios

As an individual’s decision-making may differ under different emergency circumstances, three test scenarios were devised to simulate three escalating wildfire warning levels currently employed within Australia (Advice, Watch and Act, Emergency—in order of escalation). The mapping tool presented information for each scenario that corresponded with fire characteristics associated with each warning level (National Bushfire Warnings Taskforce 2009; Table 3). One of the three test scenarios was randomly assigned to each participant, whilst ensuring that an equal number of participants were provided with each of the three test scenarios, respectively, within each fire-plan group (i.e. ‘wait and see’, ‘stay and defend’, or ‘leave early’).

Table 3 Specification of the simulated scenario for each warning level
Full size table

5.4 Interview procedure and usability tasks

The study adopted a formative evaluation approach to identify the important information elements and optimal interactivity design by observing and analysing users’ interaction with the prototyped mapping tool (Hix and Hartson 1993). This was achieved through a semi-structured interview approach. All 21 interviews were conducted in a private room at local community centres, each taking approximately 1 h. Participants were informed of the location and duration of the consultation, and permission was obtained prior to each interview. The interview procedure was designed with a set of tasks to guide and prompt participants’ navigation through the warning tool, in order to explore whether the objectives of ‘effectiveness’ were met (Gabbard et al. 1999). Verbal protocol analysis (VPA) a means of capturing user experiences as participants ‘think aloud’ (Ericsson and Simon 1993) was employed throughout the interview process to assess user needs and expectations as they ‘talked through’ their interactions (Gabbard et al. 1999; Nielsen 1993). Each interview was video and audio recorded, with one web camera in front of the participant for recording facial expressions and audio, and one camera behind the interviewee recording laptop use patterns.

The semi-structured interview procedure comprised four sections. In the first section, participants were asked to complete three sub-tasks including: (1) finding your home on the map; (2) identifying the fire’s location in relation to your home (how far it is from one’s home, and how the fire might threaten one’s property); and (3) making an evacuation plan by identifying at least one safe destination and egress route. The completion of the three sub-tasks required an exploration of all information presented in sections 1 and 2 of the mapping tool (Fig. 2). Throughout participants’ interaction with the mapping tool, they were asked to speak aloud communicating what they were doing and what they were thinking. During the procedure, special attention was given to prompt participants’ articulation of the information elements they were interested in using, how they interpreted the information elements they explored, and how they used each information element to complete the three tasks. Additionally, in cases where participants overlooked certain information elements, their user behaviour was recorded, followed by a prompt to explore the missed information.

The second portion of the interview began with participants reviewing the FDR for their area, and the suggested protective actions for the corresponding warning level (i.e. section 3 of the mapping tool as shown in Fig. 2). This was followed by questions focused on the particular wildfire scenario that had been presented to them: ‘in this current mock scenario, how do you intend to respond to the wildfire now?’. To facilitate participants’ decision-making and help researchers interrogate their reasoning process, two additional questions were asked: ‘on a scale from 1 to 7, how likely is it that the fire will impact your property and why do you believe this?’, and ‘on a scale from 1 to 7, how severe do you think the impact will be if the fire does reach your property and why?’. Participants were given time to re-familiarise themselves with the scenario they were presented with prior to answering these questions. They were then asked to rate the threat likelihood and severity of the event, describe the protective action they would take, and explain the reason for their assessment and decision, highlighting the information elements their perception and decision was based on.

In the third section of the interview, participants were given a stack of index cards, each representing one of the 23 information elements shown on the mapping tool (Table 1), and were asked to rate the importance of each on a Likert scale from 1 to 5 (5 = critical, and 1 = not important at all). Following the rating of each information element, the participants were prompted to explain their rating, if the usage of the information element for one’s decision-making had not been discussed during the previous interview sections. In the final section (Sect. 4), participants were asked a series of demographic questions and asked to indicate their self-rated wildfire knowledge, their level of household preparedness, and their past fire experience.

5.5 Data analysis

The first research question, i.e. what information should be included in a wildfire warning map to facilitate individual decision-making, was addressed using both qualitative and quantitative results. First, the audio recordings of the 21 interviews were transcribed and analysed. Using a scissor-and-sort technique (Stewart and Shamdasani 2014), materials were identified and analysed to understand participants’ rationale for using the heterogeneous information presented in completing the three sub-tasks in the first section of the interview, as well as in making response decisions in the second section of the interview. By examining similarities and differences in participants’ responses, a coding scheme was established to group participants’ logical approaches to using the warning information into three types. The three groups are introduced in the next section. The results were then used to inform the importance of the various warning information elements with an aim to assist the decision-making of a diverse audience. Second, the third section of the interview process also yielded quantitative results regarding the importance of each information element as subjectively rated by the participants. These data were used to complement the qualitative results in understanding the significance of the content delivered by the prototyped wildfire warning mapping tool.

The second question, i.e. how an interactive mapping interface can best be designed to present this information to ensure usability and ease of information access, was informed by examining participants’ interaction with the mapping application throughout the interview. The video recordings were first examined to identify user interaction with the mapping tool, which were documented in words. A scissor-and-sort technique was then used again to identify sub-themes pertinent to participants’ map using behaviours. A coding schema was subsequently established, and participants’ responses were generalised in relation to each sub-theme. Details regarding each sub-theme are also discussed in the next section.

6 Findings and discussion

Generally, all participants complimented the mapping application for how informative, comprehensive, explicit, and visually efficient it was:

When you read (textual) information, you got to stop and visualise it, you have to imagine where the roads is, so this is really quick and easy, and you can choose what information you want.

(Subject 8, female)

To me, the most important is to have all the information on the map. Because you can have too much text, and it’s too hard to correlate information and link ideas from one paragraph to something said somewhere else down here, or across different pages. So to me, map is one way of having all the information clear, all in the same place at the same time.

(Subject 10, male)

Interestingly, the only participant who declared her general dislike of maps and abstract visual communication at the beginning of the interview reversed her attitude after exploring the mapping application:

That (the interactive mapping tool) is far far better (than textual warnings). There is so much information there to help you make decisions. And it’s far more informative than those text messages. So that is mindset changed when I looked at that.

(Subject 17, female)

However, as results from the study by Cao et al. (2016b) suggested participants’ subjective preference may not align with objective measurement in indicating warning effectiveness. Therefore, we focused on the objective findings regarding participants’ usage of the information supplied by the prototyped mapping application for decision-making, and their interaction with the mapping interface for information accessing.

6.1 Content for facilitating decision-making

6.1.1 Qualitative results

Participants’ account of how they wanted to respond to the simulated wildfire scenario in the interviews revealed varied logical approaches to using the warning information for making decisions. Such a variance was mainly attributable to the difference in the response strategies that the participants intended to employ. In the following, we specify the three types of response strategies identified from the interviews, the decisions each response strategy entails, and their associated information usage styles.

First, three participants planned a strategy of ‘committed defending’, for which they were committed to defending their property against any possible threat, including both ember attack and the impact of a major fire front. A comparison between these participants’ response strategies identified in the interviews and the general fire-plan they espoused in the previous survey study (shown in Table 4) demonstrated that one of them espoused a plan to ‘do as much as possible to defend their property but leave if the fire threat becomes too severe’, but was found to actually be committed to actively defending against any fire threat whilst retaining evacuation as a contingency plan. The other two participants espoused a fire-plan that is consistent with their actual response strategy (Table 4). All of the three committed defenders identified a need to keep monitoring the fire situation for a purpose of obtaining a sense of ‘timing’ for getting prepared for active defence. They therefore depended on the information regarding the specific location of the active fire front, its distance to their home, and wind impact. In addition, two of the committed defenders intended to subject their decision regarding staying and defending to the weather conditions on the day, and would abandon the plan of defence on a ‘hot windy day’ and evacuate early, suggesting the importance of communicating wind conditions, temperature, and/or FDR levels.

Table 4 Number of interviewees adopting the three different response strategies in the face of the simulated wildfire threats in comparison with their espoused general fire-plans in the previous survey study
Full size table

Second, six participants explicated response strategies to evacuate early before the fire threatens their home without attempting to defend their property. This response strategy was categorised as ‘active evacuation’. Notably, all of these participants espoused a consistent general fire-plan in the previous survey study (Table 4). These participants’ decision-making process was thus one that focused on deciding the timing of evacuation. A profound distinction, however, was revealed in the reasoning concerning this decision across the six active leavers, indicating their varied information needs. Specifically, one active leaver demonstrated a high level of vigilance, attributable to a high level of perceived fire impact on human health and lives. Her plan was to leave as soon as the fire was ‘close’. Simple awareness of an official suggestion of ‘immediate evacuation’ for her area stimulated a perception of the impact likelihood being ‘definitive’, and an intention to leave ‘immediately’. With this said, the communication of fire location and predicted fire course provided essential knowledge regarding the safe egress routes or which side of the house would provide the safest shelter if it is too late to leave and she needs to shelter in place.

I don’t care if my house burns down. To me it’s just a house at the end of the day.

(Subject 14, Female)

The other five active leavers, whilst intending to leave if there was ‘any potential threat’, were unlikely to be motivated to leave by simply receiving a warning suggesting ‘immediate evacuation’. Rather, explicit communication of the fire location and predicted fire movement was essential to facilitate evacuation-related decision-making. In their past experiences with fire, when such specific fire information was absent, most participants (4/5) drove around to ascertain the exact fire location and direction of spread. Generally, a fire 7–9 km away moving in the direction of one’s home was sufficient to trigger an immediate evacuation. One participant also intended to use the time of impact-related information indicated by the fire spread estimation to make his evacuation decision, as he intended to evacuate as soon as the fire was predicted to reach his home within 6 h.

Finally, the majority of the participants (12/21) were identified to be reluctant to make an explicit decision to either ‘stay and defend’ or ‘leave early’ at an early stage. Most of these participants espoused a fire-plan to ‘do as much as possible to defend their property but leave if the fire threat becomes severe’, whilst two participants in this group advocated for a plan to ‘stay and defend throughout the fire’ (Table 4). However, the interview results revealed that the definition of ‘defending’ by all these participants was restricted to the combatting of a minor ember attack only, as they all planned to evacuate when they recognised that the ember attack would become too fierce or a major fire front would confront their property. In addition, all of these participants believed that late evacuation would be safe as they could ‘outrun fire’. Such a response strategy is thus inconsistent with the doctrine stressed by the national policy to psychologically prepare for active defence throughout a fire whilst making rational contingency plans (Llewellyn 2012). As concluded by Tibbits and Whittaker (2007) and McLennan et al. (2015), a response strategy to defend against minor ember attack but leave when fire threat is expected to reach a more severe degree is essentially a plan of late evacuation. On this account, this response strategy was named ‘reluctant evacuation’.

A salient distinction between the strategies of ‘reluctant evacuation’ and ‘active evacuation’ was that the evacuation decision associated with the former hinged on participants’ judgment of the severity of the fire threat, whilst that associated with the latter was driven mainly by a confirmation of the presence of a fire threat regardless of its intensity. Evidently, the decision-making of the ‘reluctant leavers’ involves more complicated judgment in comparison to that of the ‘active leavers’. Moreover, the ‘reluctant leavers’ evacuation decision was further confounded by the fact that these participants could not feel certain about making a judgment about the fire threat severity until the fire reached close proximity due to their understanding of the capriciousness of wildfires. For example, many ‘reluctant leavers’ indicated a plan to make the final decision to leave whether the fire was to jump a major road (acting as a fire barrier) 1–3 km away from their home. Some mentioned the use of time-related information provided by the fire spread estimation map to make an evacuation decision, such as leaving when the fire was predicted to reach the property within 1–2 h. Meanwhile, most of these participants ignored the warning levels and action advice suggested by agencies.

So I would really want to see whether fire crews can contain the fire, especially because the wind is going to change to the east (based on the map of wind forecast). If it comes past the Brookton Highway (1.5 km away), and the wind is going to change, it’s going to blow it towards my house. But if it gets as far as Brookton Highway before the wind changes, it’s going to blow north of it. So it is not going to burn directly in my direction. So I think that information (on the maps) would be useful for me. (It) would give me a great sense of confidence in my own decision-making process when that information is visually available like this. I think again text is rubbish.

(Subject 4, Male)

I guess what I kept seeing here is estimated time 0-4 h (based on the map of fire spread estimation). So if I’ve got everything ready to go, if my wife and daughter have fled, it’s really just me deciding to leave. When I look at this map, I can see my escape routes. There are three main routes (I can take). I will probably wait until I was told the fire was an hour away, and make a decision then, because an hour gives me a lot of time. I don’t know. I have to admit, when it came down to only one hour away, how do I decide it’s worth staying or not?

(Subject 15, Male)

A further analysis of how various warning information elements were utilised by participants with different response strategies revealed two distinctive information usage patterns. That is, despite response strategy-dependent differences in the specific purpose of using each information element, a majority of the participants (18/21), including all of the three committed defenders, five out of the six active leavers and ten out of the 12 reluctant leavers, mainly relied on hazard information such as the wildfire location, wind conditions, and fire spread estimations. The decisions of these 18 participants appeared to be partially informed, but not necessarily steered by the response guidance provided by emergency management officials. Rather, they tended to weigh their decisions on their own assessment of the hazard threat, and hence were referred to as ‘self-reliers’. A closer examination identified that the majority of the ‘self-reliers’ (15/18) only valued their own risk assessment, whereas the other three participants intended to use the warning and guidance as a general reference (Table 5); however, when the ‘self-reliers’ own assessment of the hazard situation conflicted with what was suggested by the warning and guidance, the latter was overruled. Such self-reliance appeared to be caused by the mistrust of the warnings and action advice provided by officials, a sentiment ascribable to participants’ perception of the warnings as being too general or inaccurate in describing a fire threat for one’s specific location.

A lot of the alert information is sort of general. You think yeah, but what’s happening to me now? What’s happening to my house? You want to come and have a more detailed look.

(Subject 8, Female)

However, it needs to be highlighted that there was a difference in the ‘self-reliers’ reliance on the various hazard information. Specifically, all ‘self-reliers’ identified the existing location of wildfire perimeters and wind condition as critical. Moreover, most ‘self-reliers’ trusted forecast wind information to a great extent in planning their evacuation time. However, only four of the 18 ‘self-reliers’ used the map of fire spread estimation to identify evacuation triggers. Many did not completely trust the modelled fire spread prediction, due to the uncertainty (i.e. the 4-h time window) it comprised. Some participants also were concerned with its accuracy, and would trust their own assessment of ‘time’ using other hazard information more than the modelled results. Nevertheless, it was found that participants’ self-assessment were not always accurate. For instance, a number of participants (n = 5) mistook the wind speed demonstrated on the map as the speed of fire spread and used it to infer the time of fire arrival for their location, resulting in significant errors. Further research is therefore needed to examine whether the displaying of a fire spread estimation map can help reduce the risk of mal-assessment of impact time by ‘self-reliers’, and how to best present this type of spatial information to promote its usage.

Table 5 Number of participants showing different information needs (i.e. advice followers and self-reliers) in relation to the response strategies identified in the interviews
Full size table

Another point to highlight is that ‘self-reliers’ were information driven, and most desired to acquire accurate spatial information updated at least half hourly to hourly. However, current update frequency for text-based warnings ranges from 1 h to half a day in Australia (Department of Fire and Emergency Services (DFES) 2015a). It thus needs to be improved to satisfy the ‘self-reliers’ demand. In fact, half hourly to hourly warning update frequencies can be realised by implementing semi-automated warning generation systems to incorporate timely spatial information from the emergency management system (Cao et al. 2017). In a case that a warning map cannot be updated timely, however, effective time of the data presented and expected time for the next update should be explicitly communicated to the users. Otherwise, an individual reading an out-dated map without knowledge about its effective time will make erroneous interpretation of the current situation. Furthermore, an ignorance of the long warning update interval may cause ‘self-reliers’ unnecessary complacency. This is especially risky for those who are ‘reluctant leavers’ and intend to closely monitor the fire development and delay their response decision. Consequently, prominent presentation of time information is critical to ensure proper warning information usage by the ‘self-reliers’. Warning update capacity of the emergency managers should also be propagated before the onset of a wildfire to advocate a suitable decision-making strategy, such as less reliance on timely warning information in the context of unsatisfactory warning updating capacity.

In contrast to the self-dependence of the ‘self-reliers’, a minority of the participants (3/21) trusted the warning and guidance provided by the agencies, and depended on such information for making their decisions (Table 5). These three participants were thus categorised as ‘advice followers’. Upon viewing an agency’s advice to ‘leave immediately’, all three participants desired to follow the suggestion. Still, it has to be noted that two of these participants were identified as ‘reluctant leavers’ in their actual decision-making (Table 5). This indecision was caused by contradictory aspirations of their partners (both husbands) to ‘always stay and defend’, even though their partners were not present at the interviews.

The varied information needs demonstrated by the interviewees suggests that to cater to a wide audience, an effective wildfire visual warning instrument should supply critical information elements for both ‘self-reliers’ and ‘advice followers’. To further enhance its efficiency and effectiveness, a wildfire mapping tool can be designed to allow for user personalisation by supporting the prioritisation of the information one favours. For example, ‘advice followers’ may choose to prioritise information concerning warning locations and official guidance. In such a case, the map of warning areas and associated action advice could be promoted as the first elements through the warning interface, and hazard-related maps and text could be provided as optional information only shown upon a user’s selection to reduce visual confusion. On the contrary, ‘self-reliers’ may choose to prioritise the presentation of hazard information involving specific locations of the active fire front and wind conditions, and set the map of warning areas and action guidance as optional information. However, further research is required to examine whether such an advanced and interactive warning personalisation strategy is necessary and effective for improving information capturing, risk perception and decision-making by different viewers. Moreover, to assure easy usage of a mapping tool that allows for personalisation by general users requires judicious design of the interactive features.

6.1.2 Subjective ratings of information importance

Given the small sample size of this study, participants’ quantitative ratings of the importance of each information element was interpreted using their descriptive values, rather than through statistical analyses. The quantitative results (Table 6) were generally consistent with the patterns revealed by the qualitative findings. First, most maps providing information on the hazard itself were rated as critical (mean ratings >4.5 on the 1–5 Likert scale) with strong inter-rater agreement (IRA > .7; see the footnote of Table 6 for more details). The maps of current wind (mean rating = 5) and locations of active and contained fire edges (mean rating = 4.9) provided the most imperative information. The map of forecast wind was relatively less important (mean rating = 4.7), as several participants were concerned with its accuracy and uncertainty. Sharing similar problems, the map of fire spread estimation (mean rating = 4.6) was also considered as relatively less important. Furthermore, the maps of burnt area (mean rating = 3.9) and fire origin (mean rating = 3.2) were deemed as not important, and may be excluded from the mapping tool. This is because the map of active and contained fire edges demonstrated in the study also depicted the fire shape and provided more accurate fire location information.

Table 6 Mean ratings of the importance of each information element (on a Likert scale of 1–5, anchored by 1 = not important at all and 5 = critical), and the associated inter-rater agreement (IRA), listed in descending ordered by the mean rating
Full size table

Second, consistent with the qualitative findings, the map of warning areas (warning location) was rated as important (mean rating = 4.3), but less imperative than many of the hazard maps. Third, personalised information, including the map of an individual’s home (mean rating = 4.8) and the calculated distance (mean rating = 4.6), was considered important. During the interviews, a number of participants expressed their preference for such information as it significantly facilitated contextualisation of the fire and warning information. Fourth, most participants regarded the map of closed roads as critical (mean rating = 4.8), as it ‘comes in handy’ for planning evacuation routes, especially when the wildfire is nearby and surrounding roads are affected. In contrast, the map of evacuation centres was not as valued by the participants (mean rating = 3.7), as most identified they would not use the facility. Finally, the detailed action advice communicated in text form was generally rated as important (mean rating = 4.1), but with a higher importance rating by ‘advice followers’ (=5.0) than by ‘self-reliers’ (=3.8), resonating with the previous discussion regarding their divergent information needs. The other non-spatial information element, namely FDR for one’s area, was not viewed as an indicator of the potential fire severity by most participants, resulting in a low mean rating.

Additionally, with regard to the textual descriptors of the mapped information, most descriptors were identified as not imperative except the description of fire control status. As Cao et al. (2016b) identified, the narration of ‘out of control’ provided irreplaceable meaning for people to attain a sense of urgency. Finally, out of the three background maps evaluated, most participants indicated that Google Street map provided sufficient and explicit contextual information (mean rating = 4.7).

6.2 Interactive interface design for information accessing

In addition to the identification of important information for inclusion in a wildfire warning mapping tool to facilitate decision-making, the interview results were also analysed to understand what constitutes an optimal design of an interactive mapping interface for easy usability and information access. Generally, participants in the interviews all demonstrated fundamental capability in using the interactive features of the mapping tool. However, several issues presented themselves throughout participants’ performance of the different tasks, which hindered the access to and understanding of certain information elements by some participants. An analysis of these issues revealed five themes: (a) address search; (b) access to map layers; (c) access to non-visual information via the mapping interface; (d) map scales; and (e) prior education. Specific results for each sub-theme are discussed below.

6.2.1 Address search

The first testing task was to identify one’s home on the map. Upon this request, approximately half of the participants (12/21) found and used the address search bar to complete the task, whilst the others (9/21) attempted to pinpoint their location on the map by manipulating map scale and extent and identifying familiar landmarks. After being prompted with the alternative search option, four of the nine participants who did not originally use the search function expressed favour for the function, whilst the remaining five participants preferred the manual search solution due to lack of trust in the accuracy of the automatic search function.

In addition, six of the participants who liked the search function (N = 16) did not enter their full street address (e.g. omitting street number or suburb name), and five expected a drop-down list of matching options whilst entering their address, a technique employed by Google Maps. As suggested by Nivala et al. (2008), flexible functionalities are needed to cater to varied users’ needs. In this specific context, accurate address searches are critical for risk comprehension and personalisation. Therefore, complete address search may be prompted and mandated by the system through appropriate design to preclude inaccurate mapping of one’s home.

6.2.2 Access to map layers

The prototyped mapping tool allowed users to turn map layers on and off by clicking on the corresponding checkboxes in the map table of contents. Associated legend was also shown in the map table of contents when a map layer was turned on (Fig. 2). A majority (17/21) of the participants instinctively turned map layers on and off to complete the second task in the interviews, i.e. to identify the fire’s location in relation to one’s home, how far it was from one’s home, and how the fire might threaten one’s property. However, with nine optional map layers communicating the wildfire situation, several participants (6/21) did not examine one or more layers during the initial exploration. This is potentially caused by the excessive length of the map table of contents when three or more map layers are turned on, as the legend associated with each visible map layer is displayed. Such a lengthy map table of contents necessitates the use of the scroll bar to access the entire map list, which was not noticed by several of the participants.

Despite most interviewee’s competence in turning on map layers to seek additional wildfire information, only six (6/17) were able to fluently manipulate map layer visibility. Many participants did not think about changing the layers’ visibility when having difficulties in understanding the excessively overlaid maps. Three participants considered turning off certain layers, but forgot the names of the layers and could not identify them in the map table of contents.

These results thus suggest the need for several design principles related to the presentation of the map table of contents to facilitate users’ access to the visual information. First, critical layers should be prioritised in the map table of contents, especially when the list of legend items is likely to become lengthy and a scroll bar becomes inevitable. For a wildfire warning tool, as revealed in Sect. 6.1, certain map layers concerning specific hazard conditions and official guidance should be highlighted, including the map of fire perimeters, current and forecast winds, closed roads, one’s home location, and warning areas. Second, closely related information should be combined as one layer, and indicated using intuitively understandable names. In the current case, the map of burnt area, fire origin, and fire control status (i.e. active and contained fire edges) could be amalgamated as one layer named ‘fire location’. The current and forecast winds may also be integrated into one layer through proper symbology design and legend expression. Finally, spatial information that is not critical to be visualised, such as the fire origin map identified in Sect. 6.1, should be eliminated from the visual presentation.

6.2.3 Access to non-visual information

Five approaches were employed to deliver the diversified text-based information elements through the mapping interface: (1) static text annotation on the map; (2) dynamic text annotation next to the corresponding map feature (accessible by clicking on the map feature); (3) text in the legend of the corresponding map; (4) text in the table of map layers beneath the title of the corresponding fire; and (5) text in a separate information section (accessible through hyperlinks). The information elements presented using each method are specified in Table 1.

It was found that method 1 and 3 were both easy for the interview participants to use, whilst the other approaches were not. First, dynamic annotation in relation to a map feature through the callout box (method 2) was not noticed by a majority (15/21) of participants, despite the change from normal to hand cursor when hovering over the associated map features, which appeared during most participants’ exploration procedure. After a demonstration by the interviewer, most participants liked this presentation approach. However, prominent tooltips are needed to showcase the functionality for novice users. Second, text information displayed under the title of the fire (method 4) showing the personalised distance and updating time (Fig. 2), was largely neglected (19/21). Third, information displayed in a separate information window (method 5) that was not identifiable through the main map interface, may have been difficult to access. In the interviews, such information, including the FDR and detailed action advice, was explained to the participants by the interviewers, rather than being voluntarily explored, meaning that the information was mandatorily accessed. However, several participants forgot where it was presented when they tried to re-access the information provided in the separate information window. Users’ inattention to the text information displayed with certain separation from visual content using method 4 and 5 suggests that users are likely to focus on the visual features and directly adjacent elements (visual or textual) when interacting with a mapping interface. This implies the challenge in designing a balanced map interface to present and facilitate access to large amounts of both visual and non-visual information. One potential solution is to reduce the amount of visual and textual information that needs to be highlighted through the interface by identifying elements that are essential for users’ decision-making, and present such critical information with prominence using the effective methods (e.g. method 1, 2, and 3 discussed above).

6.2.4 Map scale

Following an address search, the prototyped system would centre the map on that location and enlarge the map scale to 1:100,000. Such a scale was deliberately selected to include at least a part of the wildfire information associated with each map layer whilst displaying as much local detail as possible. This means an increment in the map scale by one level would result in potential exclusion of wildfire information in the displayed map extent. For instance, wind conditions were marked by dispersedly distributed arrows, simulating real weather data yielded by observation stations. An increase in map scale would thus carry the risk of not showing any wind arrows on the map. Given this design, the interviews revealed that more than half (12/21; male = 7, and female = 5) of participants examined local details by zooming in on the map. Specifically, six of those participants attempted to confirm the accuracy of the automatic address search by examining the surrounding roads in a scale of 1:25,000–1:50,000, and eight participants sought more specific local context when interpreting wildfire risks in relation to themselves with a map scale of 1:10,000–1:50,000. When map layers such as wind conditions were turned on for the first time on an enlarged map scale, the information was likely to fall outside the borders of the visible map. In such cases, a majority of the participants intuitively reduced the map scale to look for information, whilst four participants did not realise the pitfall of enlarged map scales for information displaying, resulting in failed information access and profound confusion. In addition to the prevalent preference for large map scales, three other participants (3/21) were inclined to examine wildfire information on a small map scale displaying a regional area, despite the fluency they demonstrated in managing map scales. Such small map scales concealed details of both the local context and wildfire information, and hence hindered their understanding. Notably, one other participant (1/21) appeared to get lost whilst inadvertently changing map scales. The remaining five participants (5/21) stayed at the default map scale throughout the information exploration.

The finding that a majority of the participants needed a large map scale to pinpoint their own home on the map and/or interpret the wildfire risk in their local context reinforces the importance of using interactive maps for communicating risk. In fact, several studies (Zhang et al. 2004; Arlikatti et al. 2006) have reported the deficiency of static paper maps in facilitating accurate identification of one’s home in relation to risk areas delineated on small scale local maps. Interactive maps thus provide an effective solution by allowing for flexible exploration of risk information and local geographic context regardless of scales. However, the study results showed that not all residents are capable of fluently manipulating map scales. To enhance the chance of successful information access, a warning map should not only display a proper default map scale that includes all delivered information, but also provide for an agile control of map scales to assist the exploration and interpretation of risk in a balanced local and regional context. In this study, eight participants (8/21) failed to access all information in an adequate manner due to misuse of map scales. These include four participants who preferred large map scales but failed to access all information, three participants who preferred small map scales and did not refer to local details, and one who lost the view of pertinent map extent as a result of incompetent map control skills. Consequently, mapping systems designed for warning communication should provide a prominent shortcut that allows for and promote users’ accessing a default ‘proper’ map scale when necessary. Moreover, when mapped information is beyond the viewed extent, the system could furnish a warning to guide the users back to an appropriate map scale. A more intelligent solution for displaying non-locational information such as wind conditions would be the dynamic placement of map symbols to ensure their inclusion in the viewed map extent at all times.

6.2.5 Prior training

In general, most participants were able to use the interactive features, including address search, information visibility and access controls, and map manipulation tools. Their competence in using these features were enhanced as the interviews proceeded, highlighting in some cases a need for prior training to familiarise users with such a mapping application. For instance, four participants who did not notice the address search function stated that relevant guidance upon initial entry into the system would help. Four participants described a feeling of being overwhelmed by the complex interactivities and ‘did not know what to do’ in the beginning, but felt comfortable accessing the map layers and manipulating the maps after introduction and practice. In an online environment, guided interactions should thus be implemented by providing tooltips, especially for novice users. If widely employed, the wildfire mapping portal could be publicised so individuals can familiarise themselves with the tool prior to a real fire event.

7 Summary and concluding discussion

The current study evaluated the use of a mapping application by residents in the face of simulated imminent wildfire threats for understanding and responding to warnings. The results shed light on how to effectively design map-based wildfire warning instruments to provide necessary and important information content through properly designed interactive mapping interfaces.

In respect to information content, the study results revealed that not all information elements were equally essential for participants’ decision-making. Despite the participants’ diverse wildfire survival plans and response strategies, the following information elements were identified as generally critical: (1) the map of prevailing winds and potentially its forecast change, (2) the accurate map of active fires, (3) the map of closed roads, (4) the personalised mapping of one’s home location, (5) calculated distance between one’s home to the closest fire front, (6) a description of the fire control status (e.g. ‘out of control’), (7) map of warning areas and associated warning levels, and (8) action advice provided by agencies for the designated warning areas. In addition, the map of fire spread estimation was found to be useful in facilitating participants’ assessment of fire arrival time, but further research is needed to identify a more effective way to present this spatial information in order to enhance trust and use.

More importantly, it was found that people’s specific needs for information vary. Specifically, most participants demonstrated a stronger degree of trust in their own risk assessment over suggestions by agencies, and they mainly relied on their own decision-making using the provided hazard information (aforementioned element 1–6). By contrast, a minority of participants were identified as ‘advice followers’ who trusted official warnings and wanted to comply with agencies’ action advice without specific inspection of the risk situation. For this latter group the information regarding warning levels and action advice (aforementioned element 7 and 8) were thus of relatively higher importance. An effective wildfire mapping tool built for the wider audience thereby should deliver all the critical information elements to accommodate users’ varied needs. A more advanced warning communication strategy is to allow for users’ personalisation of information content and/or presentation order. However, further research is required to understand whether such a sophisticated warning approach is necessary for improving warning usage and responses.

With respect to the interactivities of the mapping interface, this study resulted in several recommendations for the effective design of web-based warning maps. First, the results revealed varied users’ preference in searching for their home location on a map, suggesting the need for a system that allows for flexible search functions. Interestingly, a minority of the participants showed a preference to manually locating features by manipulating map scale and extent, and referring to familiar landmarks on the map. However, the result of a manual search approach is subject to the user’s map reading and navigation skills and familiarity with their local geography, suggesting a lack of reliability in comparison with computer aided search approaches. Further research is needed to confirm this user behaviour, understand the underlying reasons, and investigate how to promote automated/semi-automated address searches for this population segment to ensure accurate pinpointing of one’s home. Second, to enhance information access, a map-based warning tool should adopt explicit, concise and intuitive names to represent the critical map layers being visualised. Furthermore, the optimal way to display imperative textual information is to position it next to its relevant visual features. Dynamic presentation of textual information using callout boxes is favourable, but it should be coupled with prominent and visually balanced tooltips to promote access. Third, participants’ interaction with the mapping instrument showed that inadequate control of map scales may lead to significant errors in information interpretation, necessitating the provision of restriction or assistance for controlling map scales, such as by providing a prominent widget for restoring the default map scale, and by dynamically updating the location of wind symbols to ensure its inclusion in the map extent. Finally, as adequate guidance of tool usage may be critical for novice users, such map-based warning applications, if implemented, should be propagated at the beginning of a wildfire season to allow for practice by the residents.

Nevertheless, it needs to be highlighted that as the first of its kind, the current study drew upon a small sample to provide exploratory findings concerning the important content and effective design for a map-based wildfire warning tool. Larger studies should be conducted in the future to investigate whether the current findings can be generalised across the general population. Furthermore, the finding that the majority of the participants were ‘self-reliers’ rather than ‘advice followers’ may be a bias in the sample towards residents with rich fire experience. Specifically, all the interviewees had resided in wildfire-prone areas for more than 3 years and experienced at least one fire in their community or in an adjacent community for which they identified a need to evaluate whether their house was under threat and they needed to make response decisions. Therefore, they should be qualified as a wildfire ‘savvy’ population based on the definition by Dow and Cutter (2000). Still, residents from non-wildfire-prone areas are likely not to be wildfire ‘savvy’, and thus may exhibit a different pattern in their warning information preferences. Further research is necessary to investigate whether there are more ‘advice followers’ among the semi-urban population who face relatively lower wildfire risks.

Moreover, due to the exploratory nature of this study, warning systems constructed based on the current findings should be further evaluated in an attempt to create effective wildfire visual warnings. In fact, researchers have increasingly stressed the importance of an iterative design–evaluation–refinement process for designing optimal computer programs and geovisualisation tools (Nielsen 1993; Robinson et al. 2005). Using the current research method and findings as a guidance, a preliminary visual warning system could be constructed, followed by a secondary evaluative study to further examine the tool’s effectiveness and identify necessary improvements. To adapt to local contexts, emergency managers should also scrutinise the proposed content and functionalities to account for specific cultural and demographic characteristics and emergency management policies. In addition, it is important to construct and evaluate wildfire warning instruments on both computer and mobile devices to assure system versatility.

Finally, it is important to note that emergency managers attempting to employ a visual warning approach should first assure advanced data updating capacity. Even for basic hazard information such as fire locations, timeliness of the spatial data is crucial, as maps provide heightened specificity and accuracy. This is especially important given that most participants expected to refer to the map-based information for close monitoring of the situation and identifying decision triggers. As the warning update interval broadens, maps of the hazard are likely to be increasingly problematic because this creates greater chance for the self-reliant residents to misinterpret their current situation and make erroneous decisions in terms of when to initiate protective actions. Current updating frequency for warnings, ranging from 1 h to half a day (Department of Fire and Emergency Services (DFES) 2015a), thus needs to be enhanced. Otherwise, expected time for the next warning update should be prominently presented on the interface of the warning map to circumvent miscomprehension and improper decision-making.