1 Introduction

Historically, many in the field of fire safety (or protection) engineering assumed that people would panic during an emergency incident [1, 2]. This assumption led to safety managers withholding emergency information during an incident in an attempt to prevent panic behavior developing. In contrast, in situations where information is withheld, human response can be ill-informed, delayed and inefficient, potentially exposing people to more dangerous situations for longer than is necessary (and potentially longer than is safe to do so) [3].

Over the years, this point of view has been replaced with the recognition that people require detailed information as early as possible to inform and initiate a safe and effective response. Although accurate information does not guarantee optimal evacuee behavior, the absence of such information can certainly undermine the evacuee decision-making process. Without appropriate and accurate information, people will often spend valuable time during the event seeking information on the nature of the incident and what they should do in response to it [3]. It is therefore important to provide occupants with sufficient information during an emergency in order to ensure safe and effective human response.

At present, many buildings and building campuses in the United States and abroad are installing mass notification or emergency communication systems to improve communication between the building, or emergency officials, and the public. Until recently, the codes and standards in the U.S. provided requirements only for the application, performance, and installation of emergency communication (or mass notification) technology. Prior to the development of guidance by NIST on emergency communication strategies for buildings, there was little in the way of guidance regarding the content and dissemination strategies for emergency messages. In 2013 and 2014, NIST developed guidance on emergency message creation and dissemination for system designers, building managers, emergency personnel, alarm system manufacturers, and others responsible for enhancing building warning messages and the dissemination of these messages in the event of a building emergency. This guidance was also included in an appendix of most recent version of the National Fire Alarm and Signaling Code (NFPA 72), 2016 edition. The purpose of this article is to present the guidance developed by NIST [3, 4] and provide details on the methods used for guidance development. It should be noted that guidance in this article can also help engineers improve their design of emergency notification and messaging systems, which as a result can help reduce occupant evacuation time.

The article begins with a discussion on emergency communications and various technologies used to disseminate both alerts and warnings. Next, a discussion is provided on how people respond to emergencies. The Protective Action Decision Model represents the decision-making process in human response to disaster events Here it provides a framework for the literature collected as part of this discussion [5]. Next, the method used to create the guidance document is explained, including the steps taken to review the literature collected, generate the findings, and compile the key guidance statements. The guidance statements on alerts, visual/audible warnings and dissemination of warning messages are provided, organized by alert/warning type and dissemination method. Finally, guidance on emergency message testing, including language, readability and fire drills as a means of response testing, is discussed. A schematic of the article content is shown in Fig. 1. The method employed is described both to inform future reviews and allow practitioners a more informed assessment of the results produced from the review.

Figure 1
figure 1

Schematic of article content (Note: methods are included in Sect. 4)

Full size image

The guidance provided in this article is taken from a report published by the National Institute of Standards and Technology [6]. The guidance focuses specifically on message creation and dissemination in the event of fires and rapid-onset events (i.e., events that occur with little or no notice); for example, tornados, flash floods,, terrorism (e.g., chemical, biological, radiological, nuclear), workplace/school/university violence, hazardous material spill/release, or earthquake/landslide or other geological hazard. Guidance development was based on a review of 162 literature sources from a variety of social science and engineering disciplines [6, 7] and the prioritization of the specific findings extracted from each literature source [4]. This 3-year effort was funded by the U.S. Department of Homeland Security, Science and Technology Directorate and the Fire Protection Research Foundation [3].

2 Overview of Emergency Communication Technologies

Emergency communication can be divided into alerts and warnings. Alerts are first disseminated to grab people’s attention and to let building occupants know that a warning message will follow. An alert can be provided audibly, visually, or via tactile means (e.g., vibration). Alerts are an attempt to attract the perception and attentiveness of individuals. Warning messages provide richer information to the building occupants about the severity and nature of the emergency and can be provided via visual or audible means. Warnings are beneficial because they provide additional content to inform the initial response and subsequent protective actions. Both alerts and warning messages are needed for effective emergency communication—to grab attention and then initiate and inform evacuee response [3].

There are multiple ways to provide information to people in an emergency. Various technologies are available to support the procedural strategy in place. The technology used to disseminate an emergency message largely determines the type of information that can be provided and how it might be perceived by the target population. The current technology enables a range of different emergency communication systems; e.g., in-building systems, wide-area systems, notification through individual measures and notification through public measures. These can provide different types of information and reach different populations.

In-building systems are widespread in the built environment. Examples of in-building systems technology include public address systems, alarm bells, strobes and textual signage. However, these technologies address alerting and warning messages in different ways. Some technologies like alarm bells and strobes lack message content; i.e., can only alert the population to the existence of an incident. In contrast, public address systems might include an initial tone (alerting the population of an incident) and a warning message (informing the population of the nature and severity of the incident, and the desired response).

Wide area systems can be applied within a building as well as the surrounding areas. An example of a wide-area system is an exterior or interior (e.g. in a large, open-plan warehouse) alerting system that uses speakers or sirens. As such, a benefit of using sirens is that they are able to reach a large area—i.e., have a wide catchment area; however, they often lack message content, given issues with intelligibility.

Notification through individual measures is when an alert or warning is disseminated to a selected group of people individually, without using a wide area system to alert the entire building population. Examples of this type of notification consist of computer pop-ups, email broadcasts, text and audio messaging by telephone, etc.Footnote 1 This technology system has the ability of being widespread (e.g. a general text message to the population within a particular building); however, occupants can refuse to receive the message or their device may be incapable of receipt or be powered off—message receipt cannot be guaranteed.

Finally, notification through public measures includes alerts or warnings disseminated to the general public, rather than within a building or to selected population. Examples of notification through public measures include satellite television broadcasts, social media or tone alerts over the radio. Television or news broadcasts have the ability of conveying a large amount of information, but also can be easily ignored by people. However, it is important to note that television broadcasts may not reach all intended target audiences if certain channels are not available to all people, people are not watching when the warning airs, etc.

The technologies presented above can also be categorized by the mechanism used in reaching the public; i.e., their ability to push out information or pull people into the information/message. Push technologies are those that do not require individuals to take extra effort to receive the alert or warning message. Alarm bells and textual signage are examples of push technologies. Pull technologies require the individual to deliberately seek additional information to acquire the alert/message. Internet websites are an example of pull technologies. Some technologies, such as short message systems (SMS), can be push or pull technologies depending on whether the targets need to sign up to receive the messages (i.e., opt-in or opt-out systems).

It is important that the emergency communication technology employed is sufficient to meet the procedural needs of the building and the occupant population. That is, the system should reach the occupant population and deliver an alert and warning message capable of imparting the necessary information to instigate and inform the evacuation. It is critical that the technology in place is able to impart the information to the target population needed for the procedure to be employed effectively. It is then critical that those responsible for designing and implementing emergency procedures (and the associated resources) are aware of the capabilities of the technologies available and the information that needs to be shared.

3 Human Response to Emergencies

In contrast to the panic model previously assumed [1, 2], human response to emergencies is better characterized as a decision-making processFootnote 2 in which people receive information from their environment, interpret it according to new and previously held information, and respond based on their interpretation of the picture of the situation formed from this information [5]. This process will be dependent on the conditions faced (and cues received) during the incident in conjunction with the experiences and their personal attributes of those involved. These then combine to influence (inform) the decision-making process and the subsequent action on which the individual decides. A simplified version of this process is shown in Fig. 2.

Figure 2
figure 2

Simplified decision-making process [6]

Full size image

The Protective Action Decision Model (PADM) is the model selected in the report published by the National Institute of Standards and Technology [6] to provide a framework that describes the information flow and the subsequent decision-making that influences the protective actions taken in response to natural and technological disasters [5]. Other similar models include the Communication-Human Information Processing (C-HIP) model [8], the Risk Information Seeking and Processing model [9], and the Precaution Adoption Process model [9]. Consistent with the PADM, the C-HIP model describes the warning process. In both the PADM and the C-HIP models, members of the public encounter a warning message that describes the nature of a hazard and suggests courses of action to avoid injury or death. The basic communication components of both models are used to organize the discussion of public response to emergency warnings. Ultimately, three types of variables interact to determine how the public will react when faced with an emergency warning: (1) attributes of the hazard; (2) warning components; and (3) receiver characteristics [8].

The PADM was deemed an appropriate model for the study in [6] because each step of the process in the PADM served to outline goals that ultimately steered the literature findings in the report. (As will become apparent, it also formed a key component of the derived simplified decision-making, shown in Fig. 2 [6].) With regards to alerts and warning information, the PADM is split into pre-decisional processes and decisional processes. A version of this process is presented below. The PADM is of a sufficient level of detail to capture key components of the decision-making process, while still being accessible to practitioners.

PRE_DEC_1 is the first pre-decisional stage where the individual must perceive or receive the cue(s); e.g., a visual signal must be seen. The second pre-decisional stage, PRE_DEC_2, is where the individual must pay attention to the cue(s); i.e., given that it is possible for the signal to be seen, the occupant actually takes note of the signal. The last pre-decisional stage, PRE_DEC_3, is where the individual must comprehend the cue(s) and the information that is being conveyed; i.e., given that the signal is noted, the information is understood.

DEC_1 is the first decisional stage where the individual must feel that the incident suggested by the cues and/or information is a credible threat. After that, the second decisional stage, DEC_2, is where the individual must personalize the threat (i.e., feel that the incident is a threat to them) and feel that protective action is required; i.e., something needs to be done. The third decisional stage is DEC_3, where the individual searches for what this action might be and establishes options. DEC_4 is the fourth decisional stage where the options identified are assessed (given the information available) and a final action is selected. Finally, the last decisional stage, DEC_5, is where the individual determines whether the protective action needs to be performed immediately.

The three pre-decisional stages of the PADM determine whether external information is processed such that it can inform the decision-making process. In effect, the environmental information is perceived (PRE_DEC_1), attended to (PRE_DEC_2) and sufficiently understood (PRE_DEC_3) for it to affect the decision-making process. Once the information enters the decision-making process, it is assessed. The individual determines whether or not to believe the warning message, given the credibility of the source (DEC_1). If the individual decides the information is credible, the next step is to determine whether the threat is relevant to him/her (DEC_2). Research has shown that a person’s perception of personal risk is highly correlated with his/her response to the disaster [5, 10]. Therefore, the perceived relevance of the information available is pivotal in eliciting a response.

The next decisional stage (DEC_3) requires the individual to search for protective action options. The outcome of this stage is a set of possible protective actions from which to choose in order to meet the current objective. The individual assesses the protective actions and chooses the one to be taken using a process, which is often the process of satisfying, especially in emergencies [3] (DEC_4). Finally, the individual decides how soon he/she needs to perform the protective action (DEC_5). Successfully completing all PADM stages can often be problematic due to incomplete, ambiguous, or contradictory information.

In addition to insufficient emergency information, there are barriers that can delay each stage of the PADM. These barriers may be individually-, socially-, situationally-, environmentally—and/or procedurally-based. It is important for practitioners to be aware of these effects, since they may detract from the effectiveness of the egress system implemented and complicate evacuee decision-making and hence the evacuation process.

First, barriers to PRE_DEC_1 affect how building occupants perceive the emergency information. Factors that can inhibit perception include hearing/visual impairments and situational conditions (such as sleeping, which is especially problematic for children, older adults or those who are drug/alcohol impaired) [11].

Barriers that delay PRE_DEC_2 affects whether an individual attends to the emergency message. These barriers include drug/alcohol impairment, sleep deprivation, intense focus on an existing activity (otherwise known as commitment), and cognitive impairment. The environment could also be a factor; for example, when the building contains an audible or visual distraction that could inhibit attention to an emergency alert or warning message [3]. Stress and anxiety can further inhibit an individual’s ability to focus on emergency information [12, 13]. A degree of stress may help focus in on a particular cue; however, elevated stress levels can eventually distort the decision-making process, both narrowing the range of cues available and influencing to which cues the individual attends.

In the case of PRE-DEC_3, barriers to this step would preclude an individual from comprehending the information received. Factors that can inhibit comprehension include untrained or unprimed individuals, age (i.e., children), non-native speakers (especially those who do not speak the native language at all), individuals with a cognitive impairment, and individuals from different cultural backgrounds [3]. Additionally, echo, reverberation, and extraneous background noise can distort hearing aid transmission for people with partial hearing, interfering with their ability to receive information [14].

DEC_1 requires individuals to believe the emergency information and deem it credible before they can act upon it. A primary barrier to believing that an emergency situation exists is normalcy bias [15, 16]. Normalcy bias is a mental state people enter when facing a disaster that causes them to underestimate both the possibility of a disaster and its possible effects. This may lead to people delaying their response and, in turn, affecting the response of others. It is important that the emergency alert and warning message come from a trusted source so that people are more likely to believe the information [17]. If this is the case, there is a greater probability of issues, such as normalcy bias, being overcome. Additionally, in cases where building occupants experience frequent false alarms or messages, the emergency communication system may lose credibility (DEC_1).

After occupants believe that there is an emergency occurring, they must decide during DEC_2 whether the event will personally affect them (i.e., risk assessment). The main barrier to personalizing risk is optimism bias [18]. This is when individuals believe that they (and/or the people around them) are not personally at-risk even though they are aware of an emergency in the building. Insufficient or inaccurate information can interfere with the personalization of the assessed risk.

After DEC_2 of the PADM, individuals consider one or more options for taking protective action, choose one, and decide when to act. As with the other stages, there are inhibiting factors when identifying, selecting and then taking protective action. For example, individual factors such as economic restrictions can affect whether a person wants to evacuate, if he or she loses pay due to missing work [5]. The environment during the incident can also be an inhibiting factor. For instance, if occupants unexpectedly find a route blocked by smoke, they may not want to pursue that path as a protective action option. Finally, incident-induced factors (such as injury), and innate, pre-incident conditions (such as mobility impairments) can also affect an individual’s ability to take protective action.

Ultimately, there are various ways to enhance the effectiveness of public response from emergency communication systems, which can help in overcoming these inhibiting factors. The successful use of emergency communication technologies will be dependent on working against these barriers and reaching as many of the target population as possible in an effective manner.

Guidance is provided in subsequent sections on how to improve human response to emergency communication systems, via more effective emergency alerts and warning messages; i.e., helping the population to identify the nature and severity of an incident and the response required of them. The following section describes the methods used to obtain findings and create and structure the guidance document.

4 Method for Guidance Development

A literature review has been conducted in order to inform the development of guidance related to the creation and dissemination of emergency alerts and messages [6]. The literature review process followed a series of six steps:

  1. 1.

    Identification of source material and scan of key content

  2. 2.

    Review of source material

  3. 3.

    Identification of subject area

  4. 4.

    Identification of decision-making stage and completion of metric

  5. 5.

    Development of summary

  6. 6.

    Generation of key findings

These six steps were adopted in order to ensure that the source material was systematically reviewed and that the contribution of each source was documented in a consistent manner. The primary objective of reviewing this material was to examine the evidence available on the effectiveness of different communication approaches for both emergency and non-emergency conditions.

Each of the six steps in this process is now described in more detail. Initially, in Step 1, a search on emergency communication with disaster types along with the stages in the PADM was conducted. From there, references of various articles were obtained and reviewed. This source material was collected and skimmed to determine its relevance. A significant amount of material was rejected on the basis that it could not be related to human response to fires and rapid-onset events and/or the decision-making stages. The accepted material was then examined in more detail (in Step 2) to develop an annotation (i.e., shortened description of the study) for each source.

Next, a general template was created for each annotation and the key elements of each source were presented accordingly. Each annotated source listed the following (as shown in Table 1): the source reference, the subject area of the literature in which the source was found (Step 3), the components of the decision-making process that were covered in the material (Step 4), a summary of the source’s findings (Step 5), and a list of the key findings (Step 6).

Table 1 Format for Each Annotation
Full size table

In Step 3, the source material was categorized using one or more of the subject area categories listed in Table 2. Table 2 lists all of the subject areas that were used as a basis for the review: acoustics/audiology, buildings/engineering, crisis management/disasters, disability, ergonomics/human factors, human behavior in fire, illumination/lighting, language, media/communications, psychology/cognition, and standards.

Table 2 Subject Areas
Full size table

The next step, Step 4, involved the identification of the decision-making stage addressed by the source. A simplified version of the Protective Action Decision Model (PADM) [5] was used as a framework for categorizing the reviewed source material. As each source was reviewed, excerpts from the source were highlighted to identify which of the components (or stages) of the decision-making model were addressed by the literature source. These included perception (Pc), attentiveness (At), comprehension (Co), credibility (Cr), personalization (Ps), and action (Ac).

Also part of Step 4, a metric was then completed for each source identifying the components addressed. An example is shown in Fig. 3. In this example, the source material addressed two stages of the decision-making model: comprehension (Co) and Action (Ac). The metric display enabled the relevant subject matter of each source to be more easily identified by the reader.

Figure 3
figure 3

Metric used to identify the decision-making components addressed in each source

Full size image

In Step 5, a summary was developed for each source. This summary included the purpose of the study, a brief explanation of the methods employed, and a discussion of the results. This step also aided the processes required for Step 4, since summary text was highlighted within each annotation, allowing the material to be categorized according to the aspects of the decision-making process that it addressed. Finally, in Step 6, key findings were identified from each of the annotated sources.

An example of a formatted annotation is shown in Fig. 4. The annotation begins with the reference for the source. Under the reference, on the left-hand side, an “E” indicates that the source was found within the subject area of ergonomics/human factors. To the right of the subject area, the decision-making components metric is found. In this example, the source reference addressed the comprehension “Co” phase of the decision-making model (comprehension). Below the boxed information is the summary text for the source. The highlighted information remains, so that the reader can understand how this source was categorized. Finally, the key findings from the source are listed below the summary text. The key findings are also labeled with distinct identifiers—starting with the number of the source, then the decision-making components metric (in this case, Co), and the number of the findings.

Figure 4
figure 4

Example of a formatted source review [7]

Full size image

The selected material included in the literature review was intended to present a representative—rather than exhaustive—view into research and best practices. In an attempt to ensure that key documents were addressed, priority was given to material that was cited multiple times. All material was drawn from publicly available resources, published in English.

After all annotations were completed, the key findings from each annotated source were compiled. Once compiled, the key findings (listed in Appendix A of the original literature review [6]) were reviewed and prioritized using a set of criteria. From this process, the final set of guidance that is included in this article (as well as in [3]) was developed. The criteria used to prioritize the summary report’s findings were: (1) Is the finding relevant to building safety and building occupants? (2) Does the finding relate to emergency conditions rather than normal/non-emergency conditions? (3) Is the finding in agreement with relevant theory and expert opinion on human behavior in emergencies?

Some findings from the literature review were observed to contradict others. The authors attempted to reconcile these contradictions by siding with findings that were most aligned with well-accepted social theory of human behavior in natural or technological disasters and coincided with the balance of evidence. Also, findings that originated from sources found in archival publications received higher priority, in cases of contradicting guidance. There were some contradictory findings in which consensus was not reached. These findings are not included in this guidance document but are discussed in Section 7 of [4], as questions that remain to be answered.

Some topics of interest that were included in the summary report [6] were deemed to be outside of the scope of this article. These include the mechanics of visual lighting systems for signage and the volume necessary for audible alert and warning systems. Both of these topics can improve the perception (i.e., hearing or seeing) of the alert or warning message, and have been covered in some detail in codes/standards on mass notification systems (e.g., NFPA 72 [19]).

The guidance presented in the following sections on emergency communication strategies was selected from the report published from the National Institute of Standards and Technology [6]. The purpose of this guidance is to help improve how occupants receive and process alert and warning information, which can ultimately improve evacuation times. The guidance statements were produced by the approach discussed earlier in this section.

First, guidance on alerts is provided. The alerts section (Sect. 5.1) addresses what should be considered in the design of an alert (and associated procedural measures) to make it more effective. After the alerts section, guidance on the dissemination of warning messages (Sect. 5.2) is provided. Following the section on warning message dissemination, guidance is presented on the creation of warning messages (see Sect. 5.3), including what content to include in the message, how to structure the message and what kind of language/wording should be used. Finally, guidance on visual and aural warnings is provided (Sects. 5.4 and 5.5). The purpose of Sects. 5.4 and 5.5 is to provide instructions on how to improve warnings, with specific guidance statements for visual versus aural messages.

5 Guidance on Emergency Communication Strategies for Buildings

Alerts aid the first two stages of the PADM: perception and attentiveness. In other words, alerts help individuals perceive or receive signals so that they can then pay attention to the subsequent warning message. Warning messages are then required to inform the population of the nature of the incident and the required response. This section provides guidance on how to improve alerts (discussed in Sect. 5.1) followed by warning messages (to be discussed in Sects. 5.2–5.5) in building emergencies in order to prompt safe and effective occupant response in building disasters.

5.1 Alerts

An alert aids perception of the emergency and lets occupants know that they should pay attention. Therefore, alerts should be disseminated to let building occupants know that attention is required as a warning message (to be discussed in Sects. 5.2–5.5) is to follow providing greater insights into the incident and the required response. Some building alarms are only capable of providing an alert and not a warning message. In these situations, the system is then only able to indicate that attention is required given that an incident has occurred (or is imminent), and is unable to provide further information. The individual is then left to determine the need for a response and the response required. Conversely, messages without alerts may be less likely to reach their target audience as occupants will not have been primed for the arrival of the message; i.e.. will be less attentive.

Research has identified specific signals that are more effective at alerting the occupant population in different situations or under different circumstances. For example, a sound or series of sounds can be provided for an audible alert.Footnote 3 Table 3 provides specific guidance on alerts.

Table 3 Summary of Guidance on Alerts
Full size table

Additional methods to alert building occupants to an emergency situation include: tactile methods, mobile alerts, social networks and face-to-face communication. When an alert is selected, it should be tested for success in getting occupants’ attention in the event of an emergency and used as part of building-wide training.

5.2 Warning Message: Dissemination

Research has shown that occupants are likely to seek information during an emergency [3], especially in situations where inappropriate, inaccurate, or insufficient information is provided; i.e., when they do have a sufficient understanding of the situation to make an informed decision. This can delay their movement to a place of safety or even move them closer to the incident itself (in search of more information). Therefore, it is important to ensure that a population receives (and pays attention to) the alerts and warning messages as early as possible. Warning message dissemination should include different technologies and modes to ensure that as many members of a population as possible receive and pay attention to the information provided; e.g., using vibrating pagers and text messages for any hearing impaired individuals and raised characters or Braille on building signage for the visually impaired. Table 4 below provides specific guidance on the dissemination of warning messages to all affected building occupants:

Table 4 Summary of Guidance on Dissemination of Warning Messages
Full size table

Assuming that the message reaches the target population, the information contained in the message then has the potential of influencing the occupant decision-making process—given suitable message attributes.

5.3 Warning Messages: Content and Presentation

Warning messages should follow an alert signal to provide information to occupants about the nature and severity of the emergency and the desired response. Warning messages can be visual or audible and be disseminated through the use of a variety of technologies or through human communication. Regardless of the method used to disseminate the warning message, there are characteristics that can increase their effectiveness at imparting the desired content (information). Well-crafted warning messages can help in all stages of the PADM, especially comprehension, credibility, personalization of risk and taking action. Table 5 presents guidance on warning message content, structure, language, and type. This summary is derived from the original NIST report material [6].

Table 5 Summary of Guidance on Warning Messages
Full size table

In the next two sections, guidance on visual and aural modes of dissemination is provided. Each mode has its challenges and opportunities—being able to address different vulnerabilities in different emergency scenarios. These sections outline the methods that can be used to maximize the effectiveness of both modes.

5.4 Warning Messages: Visual Mode

There are a number of different approaches to visual messaging; e.g., graphical signage, digital signage, etc. Visual messages have different capabilities and limitations to those disseminated aurally. For instance, guidance in this section can affect whether the message is noticed, its readability, whether it can be understood, and its perceived credibility and urgency. Table 6 provides specific guidance on visual warnings:

Table 6 Summary of Guidance on Visual Warnings
Full size table

5.5 Warning Messages: Aural Mode

There are several aural warning technologies; e.g., public address systems (voice communication systems), satellite/AM/FM radio broadcasts, satellite/off-air television broadcasts, and tone alert radios. These have different capabilities and can influence the response of the population in different ways. This section provides guidance for methods that increase the likelihood that an individual will receive or perceive the message, as well as increasing comprehension of the message and the credibility and perception of risk of the event. Table 7 provides specific guidance on audible warnings:

Table 7 Summary of Guidance on Audible Warnings
Full size table

6 Guidance on Emergency Communication Testing

Testing emergency alerts and messages plays a prominent role in determining their effectiveness—in calibrating the emergency communication strategies for the target population. Language and readability tests can help building owners and/or managers test the effectiveness of their warning messages. These tests for warning messages can be applied to electronic as well as non-electronic messages [20]. Response tests can be used to test the effectiveness of both alerts and warning messages. Emergency communication testing should be conducted before any alert or message is shared with the building occupants to provide the opportunity for revision, if needed. This section identifies the methods available to test the effectiveness of emergency communication methods (for both alerts and warnings).

6.1 Language Testing

One way for building owners and/or managers to test the effectiveness of aural or textual messages is by conducting a language test. Language testing is used to evaluate whether an individual understands and/or can correctly interpret the meaning conveyed by the message. Paraphrase and Usability tests, two types of language tests, are described here [20].

Paraphrase testing is used to determine how the participant interprets the meaning of the message, allowing the tester to compare the participant’s interpretation with the actual meaning of the message. To perform a paraphrase test on a particular message, message testers meet with six to nine test participants. As part of this test, the tester divides the message into sections, and then asks the participant to identify what each section of the message means [20]. Additional open ended questions are then asked, such as the following: (1) what would you do if you got this message? (2) what do you think the writer was trying to convey with this message? (3) considering other people you know who might receive this message—what about the message might work for them/what about the message might cause them problems?

Usability tests provide an alternate way to test whether an individual understands the information provided in the message. Usability tests differ from paraphrase tests in that they address typical emergency scenarios and the participant’s experience instead of the textual meaning of the message itself. It is suggested that a minimum of three people participate in a usability study (for each message being tested) to provide multiple data results; however, there is no numerical requirement [20]. Usability test sessions contain an introduction, scenarios and debriefing.

During the introduction, the tester makes the participant comfortable, explains what will happen, and asks a few questions about the person to understand her/his background. Next, the tester gives the participant a scenario in which he or she would receive the message, and then watches and listens as he or she receives and interprets the message. The tester asks the individual to “think aloud” when receiving the message, in order to capture how the participant understands what he or she hears/reads. Finally, debriefing takes place where the tester asks neutral questions about his or her experience reading (or hearing) the message, and follows up about any specific words or phrases used by the participant.

In addition to the usability test method described above, other variations of the usability test are possible. Three examples of variations are described: (1) Together, two participants are asked to “think aloud” about the message at the same time; (2) Several participants located in separate locations are asked to work independently at the same time. This works well if there are several usability test note-takers available to ensure that someone is watching and listening to each participant at all times. After the participants have completed their individual sessions, all participants are brought together for a group-wide discussion on the message; (3) The tester and the participant work together remotely (i.e., the tester is in one physical location and the participant is in another), and they perform the usability study using web-based tools.

6.2 Readability Testing

Another method to test the effectiveness of the warning message is via readability tests. Readability tests are used to measure the reading level of any text (including warning messages) using computed formulas. These types of tests are often used to estimate the number of years of education an individual must have in order to read and comprehend the written material [21]. It is important to ensure that the reading level for emergency warning messages is at an appropriate comprehension level to ensure comprehension in the time available. The suggested reading level for emergency messages is a 6th grade reading level (on average) [22].

During a readability test, the entire warning message or a sample of the message can be tested. The use of multiple syllable words and complex sentences usually raises the minimum education level required to read messages; therefore, readability tests can help message creators when deciding to alter, remove or replace certain sentences or words, when necessary [21].

One example of a readability test is the Flesch-Kincaid Readability Test [23]. The Flesch-Kincaid Readability Test calculates a score based on the number of words, syllables, and sentences in a given piece of text. The higher the score, the easier the message is to read; therefore, a score of 60 or higher is recommended for emergency messages. The formula for the Flesch-Kincaid Readability Test is as follows [24]:

$$ \begin{aligned} & \left( {0.39 \times Average\,number\,of\,words\,per\,sentence} \right) \\ & \quad \; + \left( {11.8 \times Average\,number\,of\,syllables\,per\,word} \right){-}15.59) \\ & \quad = Readability \, score. \\ \end{aligned} $$

Another example of a readability test is the Automated Readability Index. The result of the Automated Readability Index formula is a reading score that corresponds to the minimum grade level expected to be able to read and comprehend the message [25]. The formula is as follows:

$$ \begin{aligned} & \left( {4.71 \times Average\,number\,of\,characters\,per\,word} \right) \\ & \quad \; + \left( {0.5 \times Average\,number\,of\,words\,per\,sentence} \right){-}21.43 \\ & \quad = Grade \, level. \\ \end{aligned} $$

Message testers should recognize the limitations of these readability methods. The quantitative results provided are often “average” values, in that a result for a message equating to a 6th grade reading level does not necessarily mean that all people at a 6th grade reading level and above will be able to read and/or comprehend the message.

6.3 Response Testing

Response testing involves the use of full-scale or real world exercises to test the effectiveness of an alert and/or warning message. Response testing can be performed in either practice drills or actual emergency events. The purpose of response testing is to collect data on how the participants respond to an emergency alert and/or warning message in either the event of an actual emergency or under the guise of an actual emergency.

In the case of warning messages, it is important that response testing takes place after paraphrase or usability and readability tests, so that building occupants are exposed to messages that are as near to ‘final’ as possible. Additionally, response testing studies that occur during a drill or test-setting should be followed by feedback to the building population, including the newly revised emergency alert and/or warning message that will be used in the next actual emergency scenario.

Two examples of response test methods are controlled comparative studies and evacuation drills. In a controlled comparative study, quantitative data are collected on how well the general public (i.e., not necessarily the building occupants) responds to the emergency message in a test scenario (i.e., a drill) or an actual emergency scenario. Public response can be measured by the number of clarifications requested and/or the number of errors resulting from a particular message (e.g., individuals who use an exit not assigned to them or neglect to evacuate in response to the message). Additionally, different versions of a warning message can be disseminated to different sections of a population to assess whether one version of a message is more successful than another.

Fire evacuation drills provide another way to test the effectiveness of emergency messages in a real world setting. NFPA’s Life Safety Code [26] discusses the frequency and methods of emergency evacuation drills for each building occupancy type. Overall, emergency egress and relocation drills should be conducted with sufficient frequency to familiarize building occupants with the egress strategies and address incident scenarios that are as representative as possible of expected real-world conditions.

After each emergency evacuation drill, emergency coordinators should produce and submit a written report of the drill to the designated authorities by a particular time, dependent upon local jurisdiction requirements. Written reports are used to document the procedure and results of the fire drill, including the date, time, participants, location, and egress/relocation results.

Additionally, feedback messages can be provided after a “non-event” has occurred to inform building occupants that the alert signal and warning system operated as planned and to provide the reasons why an event did not occur. Update messages can also be provided during the building incident and can be used to update building occupants on the current situation, including telling building occupants why any information and instructions have changed, so that the new updated message is also viewed as credible. Feedback and update messages can help occupants understand the performance results of the emergency drill, which can ultimately reduce evacuation times.

Table 8 below shows the drill frequencies as well as guidance for each building occupancy type discussed in NFPA’s Life Safety Code.

Table 8 Fire Evacuation Drills for Buildings
Full size table

Similar to any other testing methods, evacuation drills are limited in their ability to assess the effectiveness of alerts and/or warning messages. Response tests, specifically in the case of evacuation drills and the types of scenarios that can be examined, are limited by ethical considerations. In other words, it is not possible to place the evacuating population in undue harm. Repeat drills are also limited (so that each result is only a single instance from the set of possible, unexamined, results), and only represent a single scenario from the range of possible scenarios that could occur. Additionally, message testers should also note that response testing (via drills) that are announced ahead of time may influence the way in which the drill is perceived and can lead to results that have little to do with the warning message itself.

7 Conclusion

The purpose of this article has been to provide guidance to system designers, building managers, and building emergency personnel responsible for emergency communication on how to create and disseminate effective alerts and warning messages using basic communication modes (primarily audible and visual technology, although tactile means are discussed). Guidance is also provided on how to test the effectiveness of these alerts and warning messages.

In this article, emergency communication is categorized into alerts and warnings, and guidance is provided on both. Alerts are intended to grab people’s attention and prime them for subsequent information; warnings provide the building population with information relating to the emergency and their response to it. If alerts are used alone, then the notification system can only indicate that something out of the ordinary has happened; i.e., that attention needs to be paid, rather than describing the nature of the incident, its severity or the occupant response required. If warning messages are also employed, then the system may potentially provide more comprehensive information—according to the design of the message provided. The PADM provides a framework of the decision-making process of human response to emergencies. The steps in the PADM are divided into pre-decisional and decisional processes. The PADM guides our understanding of how the alerts and warning messages can influence evacuee response. It has been used here to frame the review of relevant guidance on emergency communication.

Most of the guidance provided in this article originates from a NIST report written for the U.S. Department of Homeland Security [6]. The method for creating the guidance document is discussed—in order to better identify the strengths and limitations of the approach adopted.

Guidance on alerts, warning messages, audible/visual warnings and dissemination of messages is organized into tables based on communication topic. Each guidance statement is provided with its respective reference source.

Finally, building managers or those responsible for emergency communication can test the effectiveness of their messages by using language, readability and/or response tests. Such tests can improve the desired response of occupants to emergency alerts and messages. Fire evacuation drills are examples of response tests that can be applied to different building occupancy types. It is envisioned that the guidance provided in this article can help to improve the design of emergency communication systems and subsequently ensure that the alerts and messages provided during an emergency effectively prompt and inform the target population in accordance with the procedural design, ensuring they reach a place of safety in a timely manner.