Safety Risk Management of Unmanned Aircraft Systems

Abstract

The safety risk management process describes the systematic application of management policies, procedures, and practices to the activities of communicating, consulting, establishing the context, and assessing, evaluating, treating, monitoring and reviewing risk. This process is undertaken to provide assurances that the risks associated with the operation of unmanned aircraft systems have been managed to acceptable levels. The safety risk management process and its outcomes form part of the documented safety case necessary to obtain approvals for unmanned aircraft system operations. It also guides the development of an organization’s operations manual and is a key component of an organization’s safety management system. The aim of this chapter is to provide existing risk practitioners with a high level introduction to some of the unique issues and challenges in the application of the safety risk management process to unmanned aircraft systems. The scope is limited to safety risks associated with the operation of unmanned aircraft in the civil airspace system and over inhabited areas. This chapter notes the unique aspects associated with the application of the safety risk management process to UAS compared to that of conventionally piloted aircraft. Key challenges discussed include the specification of high-level safety criteria; the identification, analysis and evaluation of the risks; and the effectiveness of available technical and operational mitigation strategies. This chapter also examines some solutions to these challenges, including those currently in practice and those still under research and development.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

ACAS:

Airborne collision avoidance systems

ADF:

Australian Defence Force

ADS-B:

Automatic dependent surveillance-broadcast

ALARP:

As low as reasonably practicable

ALoS:

Acceptable level of safety

ATSB:

Australian Transport Safety Bureau

CAA:

Civil Aviation Authority (United Kingdom)

CASA:

Civil Aviation Safety Authority (Australia)

COTS:

Commercial-Off-The-Shelf

CPA:

Conventionally-piloted aircraft

DoD:

U.S. Department of Defense

EASA:

European Aviation Safety Agency

ELoP:

Equivalent level of performance

ELoS:

Equivalent level of safety

FAA:

Federal Aviation Administration

FMEA:

Failure modes and effects analysis

GCS:

Ground control station

HAZOP:

Hazard and operability analysis

HLSC:

High-level safety criteria

HSE:

Health and Safety Executive (United Kingdom)

ICAO:

International Civil Aviation Organization

ISO:

International Organization for Standardization

LoS:

Line of sight

NAA:

National aviation authority

NTSB:

National Transportation Safety Board

RPA:

Remotely piloted aircraft

SARPS:

Standards and Recommended Practices

SMS:

Safety management system

SRMP:

Safety risk management process

SSP:

State Safety Plan

TCAS:

Traffic Alert and Collision Avoidance System

UAS:

Unmanned/uninhabited aircraft/airborne/aerial system/s (plural same as singular)

UAV:

Unmanned/uninhabited aircraft/airborne/aerial vehicle/s (plural same as singular)

1 Introduction

Unmanned aircraft systems (UAS) are one of a number of emerging sectors of the aviation industry. The potential benefits from the use of UAS have been demonstrated in a variety of civil and commercial applications including crop and infrastructure management, emergency management, search, and rescue, law enforcement, environmental research, and many other applications often described as being too dull, dirty, dangerous, or demanding for conventionally piloted aircraft (CPA). However, as well as benefits, the operation of UAS has associated risks.

Intrinsic to the realization of any system is a finite degree of risk; subsequently, accidents involving UAS will occur no matter how stringent the conditions prescribed or draconian the regulatory oversight provided. One could argue that the only way to assure absolute safety is to prohibit the deployment of UAS altogether. However, to justify this argument, one must also address the philosophical question of what are the risks of not using UAS technologies?

The starting premise of this chapter, and one which is consistent with modern aviation safety thinking (ICAO 2009) is that UAS operations, like CPA operations, are not currently, and never will be, absolutely safe (i.e., have zero associated risks). The challenge for UAS stakeholders is to establish a safety case detailing how these inherent risks can be managed to an acceptable level.

Achieving an acceptable level of risk is a multidisciplinary problem. It requires a balancing of complex social, psychological, technical, political, and economic factors arising due to the following:

• Limited knowledge and resources available to identify characterize, and treat the safety risks associated with a technology

• Subsequent need to make trade-offs between available risk mitigation strategies based on assessments of the associated costs and benefits

• Potentially conflicting values, beliefs, perceptions, objectives, and expectations held by the different stakeholder groups involved in the decision-making process (e.g., those held by the UAS industry, other airspace user groups, and the general public)

• Conditions and environment under which the decisions are made (e.g., hidden political or time pressures)

Achieving a balanced outcome from such a problem space is the objective of the safety risk management process. This objective is achieved through the application of the safety risk management process (SRMP), which can be described as

the systematic application of management policies, procedures and practices to the activities of communicating, consulting, establishing the context, and identifying, analyzing, evaluating, treating, monitoring and reviewing risk. [Definition 3.1, (ISO 2009)]

This chapter explores some of the unique aspects, issues, and challenges associated with application of the SRMP to the safety risks associated with UAS operations.

1.1 Scope

Discussion in this chapter is limited to the safety risks associated with civil UAS operations. There are a variety of descriptions of the SRMP, and these descriptions can differ in their scope, subprocesses, and structure. For the purposes of this chapter, the generalized and domain-independent description of the SRMP provided in ISO 31000:2009 is used and illustrated in Fig. 92.1 (ISO 2009). Some aviation-specific descriptions of the SRMP can be found in references (FAA 2000; ICAO 2009; CAA 2010b).

Fig. 92.1

The safety risk management process, based on ISO (2009)

Establishing, maintaining, and improving safety requires more than the application of an SRMP. The SRMP is conducted as part of an organizational risk framework developed in accordance with a fundamental set of organizational risk principles (ISO 2009). In aviation parlance, these principles and the organizational framework in which the SRMP is applied are part of an organization’s safety management system (SMS) (ICAO 2009). The scope of this chapter does not include the SMS. For general information on the components of the SMS, the reader is referred to the references (ICAO 2009; ISO 2009).

1.2 Aim and Overview of Chapter

The aim of this chapter is to provide existing risk practitioners with a high-level introduction to some of the unique issues and challenges in the application of the SRMP to unmanned aircraft systems. This chapter does not provide a comprehensive description of the SRMP itself. The discussion is intentionally high level in its nature to ensure applicability to a broad range of UAS and their potential concepts of operation.

The structure of this chapter follows the SRMP illustrated in Fig. 92.1. The first step in any SRMP is to establish the context, which is described in Sect. 92.2. This is followed by the risk assessment process. The objective of the risk assessment process is to comprehensively characterize the safety risks associated with UAS operations and, based on this information, determine which of the characterized risks can be tolerated and which of the characterized risks require mitigation (treatment). As illustrated in Fig. 92.1, the risk assessment process comprises the subprocesses of risk identification, risk analysis, and risk evaluation. These are discussed in Sects. 92.3–92.5, respectively. The objective of the risk treatment process (described in Sect. 92.6) is to identify, implement, and evaluate suitable measures to reduce (mitigate, modify, treat, or control) the risk. The SRMP is a living process being a key component of an organization’s overarching SMS. The process of monitoring and reviewing (Sect. 92.7) is pivotal to maintaining and improving the management of the risks. Finally, there is the process of communication and consultation (Sect. 92.8). The communication and consultation process is key to addressing broader stakeholder concerns and those issues that stem from a lack of knowledge of the risks and benefits associated with civil UAS operations.

2 Establishing the Context

Understanding the complexity of challenges to be faced in the safety risk management of UAS requires consideration of the social, psychological, political, and economic factors associated with the broader integration of UAS into society. These factors are identified as part of the context for the SRMP and are commonly overlooked in UAS safety discussions. Establishing the context is the process of “defining the external and internal parameters to be taken into account when managing risk, and setting the scope and risk criteria for the risk management policy” (ISO 2009). This subprocess of the SRMP involves consideration of the “cultural, social, political, legal, regulatory, financial, technological, economic, natural, and competitive environment, whether international, national, regional or local; the key drivers and trends having impact on the objectives of the organization; and relationships with, and perceptions and values of external stakeholders” (ISO 2009).

2.1 Safety Risk Management Process and UAS

The SRMP can be used to support a range of operational, financial, or regulatory decisions concerning UAS. Here, we will focus on those decisions made in relation to the management of the safety risks associated with their operation. In this context, the SRMP provides an accepted and systematic means for providing assurances that the risks associated with UAS operations have been managed to an acceptable level. The SRMP and its outcomes form part of the documented safety case necessary to obtain approvals for UAS operations. It also guides the development of an organization’s operations manual and is a primary component of an organization’s SMS. The SRMP is also used to guide the safety policy, rulemaking, and oversight activities of a national aviation authority (ICAO 2009).

The Civil Aviation Safety Authority (CASA) has released draft guidance material describing the application of SMS principles to civil UAS operators (CASA 2011). The guidance material is believed to be the first of its kind specifically targeted to civil UAS operations. Drawing on ICAO SMS principles and internal CASA policy, the guidance material includes recommendations on how UAS operators should approach the safety risk management of UAS operations (ICAO 2009). Although not a regulatory requirement, CASA actively encourages UAS operators to develop an SMS due to the potential benefits of improved safety and reduced costs.

2.2 The Objective

One of the first steps is to define the objectives of the activity. The general overarching objective is to provide assurances in the safety of a particular UAS operation or organization’s activities. Objectives also need to be defined in relation to the expected benefits of the operation to the different stakeholders involved. For commercial UAS operations, these objectives can often be derived from the corporate and strategic objectives of the organization (e.g., profitability, market growth, reputation). As well as being a goal, objectives can also act as constraints on decisions made throughout the SRMP. All objectives should be clearly defined to ensure transparency in decision-making to help identify potential conflicts in the SRMP.

2.3 Considerations and Constraints on the UAS Safety Risk Management Process

Constraints bound the decisions made within the SRMP and can arise due to a variety of financial, legal, social, psychological, technological, temporal, or spatial limitations or requirements. For example, the national aviation authority (NAA) functions of safety policy, rulemaking, and oversight must be defined in consideration of ICAO Standards and Recommended Practices (SARPS); the safety performance objectives established within a State Safety Plan (SSP); the legal, political, economic, and cultural requirements specific to their respective state; and the internal resources and capability of the NAA to define and execute these functions. Constraints are typically categorized as being either internal or external to the organization. Internal constraints are those that arise due to limits in the capability or resources of the organization or due to the organization’s existing policies, procedures, or objectives. External constraints include existing regulations (e.g., existing civil aviation safety, environmental protection, or workplace health and safety legislation) or other social, cultural, political, or economic expectations held by other stakeholders (including the members of the general public).

2.4 Stakeholders

A stakeholder can be defined as “an individual, group of people, organization or other entity that has a direct or indirect interest (or stake) in a system” (Hull et al. 2011). An interest may arise through the stakeholder using, benefiting from, being disadvantaged by being responsible for, or otherwise being affected by the system (Hull et al. 2011). Stakeholders in the UAS SRMP can include other airspace users, the general public, air traffic service providers, the end users of UAS services or their data products, the aviation safety regulator, landowners, and members of the UAS industry (inclusive of equipment and airframe manufacturers, operators, training, and maintenance organizations). Stakeholders will have their own objectives, information needs, and expectations in terms of the safety performance of UAS. These need to be identified and considered at all stages of the SRMP.

The acceptance of UAS operations requires more than a solid safety case. Understanding stakeholder concerns, the motivation for them and how they influence their decisions in relation to safety, is key to achieving the broader acceptance of UAS operations. Clothier et al. (2008) use the situation faced by horseless carriages in the 1800s as an analogy to the situation being faced by UAS today. As described in Clothier et al. (2008), there are hidden factors concerning the integration of UAS into society that can influence stakeholder decision-making in relation to their safety. UAS are a new user within an existing airspace system. Further, there exist potentially competing industries, whose value and safety performance is already widely known and tolerated by society. These and other factors (e.g., the unemployment of pilots) can manifest as hidden objectives and constraints on the SRMP. Effective stakeholder communication is pivotal to the identification, characterization, and resolution of the potential conflicts that can arise in the SRMP.

2.4.1 Perception

A distinction is often made between those stakeholder assessments of the safety risks that are formed through the use of objective data, expert domain knowledge, models, or formal assessment techniques, and those assessments that are based on the subjective knowledge, beliefs, emotions, values, and needs of the individual. The latter of these types of assessments is commonly referred to as perceived risk. There is a range of factors that influence how different stakeholders appraise and respond to the safety risks associated with UAS operations. Importantly, these appraisals and responses can be different to those they would make for the safety risks associated with CPA operations. These perceptions give rise to different stakeholder expectations in terms of the safety performance of UAS.

At the time of writing, no significant body of research into the perception of the safety risks associated with UAS operations could be found. Clothier and Walker (2006); Clothier et al. (2008) provide limited discussion on factors likely to influence the perception and acceptability of the risks associated with UAS operations. Also worth noting is the survey of air travelers conducted by MacSween-George (2003). This survey attempted to characterize the willingness of people to travel onboard a pilotless passenger aircraft.

In the absence of a risk perception study specific to UAS, general factors taken from existing psychometric modeling studies (Fischhoff et al. 1978; Slovic et al. 1979; Slovic 1987, 1999) are used to hypothesize the public’s perception of the safety risks associated with UAS operations. An analysis of the UAS safety paradigm with respect to the factors of voluntariness of exposure, control of exposure, awareness of benefits, and uncertainty is described below.

Voluntariness. The primary risks of concern due to CPA operations are to the crew and passengers onboard the aircraft. The individuals exposed voluntarily undertake these risks in return for a direct benefit. On the other hand, for UAS operations, the primary risks are to members of the general public overflown who are largely involuntarily exposed to the risks.

Control. The members of the general public overflown by UAS operations are largely unable to influence the level of their exposure. Whereas passengers of CPA have greater control over the level of risk they are willing to tolerate through the number and type of aircraft operations (e.g., gliding, sport aviation, or scheduled passenger flights) they partake in and through choice of a particular air service provider.

Benefit. The knowledge of the benefits of CPA operations (e.g., efficient transportation of people and freight) is broadly understood and widely known. Further, there is a direct and identifiable relationship between the individuals exposed and the benefits they receive. However, the routine operation of UAS for civil and commercial applications has yet to be realized, and as a consequence broader society has limited, if any, knowledge of the benefits. For UAS, the connection between benefits and the individual exposed may not always be identifiable to the individual exposed.

Knowledge and Information. In relation to UAS, there are limited sources of information available to stakeholders. The quality of the information that is available to stakeholders is variable, biased, and often unverified. For example, the movie Stealth™ portrays UAS with unrealistic capabilities. The information available predominantly relates to military UAS operations and their roles in recent conflicts (e.g., as weapons of war). This can create a bias in stakeholder knowledge of UAS. There is also a significant knowledge gradient between stakeholders (i.e., a difference in the amount and quality of knowledge held by the different stakeholder groups). The general public and the NAAs have less personal knowledge that they can use to contrast/verify the information available to them. Whereas the industry stakeholders have much more experience and knowledge relating to UAS operations and their safety performance. This knowledge gradient can lead to issues of trust and in turn higher stakeholder uncertainty in assessments of the risks. Finally, the above factors can lead to lower stakeholder certitude (e.g., belief in their self-knowledge), and potential issues of trust can lead to higher perceptions of the risk. These and other factors give rise to stakeholder uncertainty. The higher the uncertainty, the higher the perception of the risks.

Based on the above factors, it is hypothesized that stakeholder perceptions of the risks associated with UAS operations will be higher than that for a comparable CPA operation. Addressing the issues relating to risk perception requires the development of communication strategies (Sect. 92.8). Psychological factors influence not only stakeholder assessments of the risks but also their appetite for them. It has been proposed that stakeholders will expect UAS to demonstrate a level of safety performance better than that currently expected of CPA operations. If true, this expectation will need to be taken into consideration when defining high level safety criteria (HLSC) for UAS. Most qualitative specifications of HLSC for UAS express a desire for UAS to exhibit a level of risk less than, or equal to, that currently demonstrated by CPA. Some quantitative specifications of HLSC for UAS include a multiplicative factor to account for the hypothesized difference in stakeholder appetite for risk, for example (Weibel and Hansman 2004).

2.5 High-Level Safety Criteria

HLSC are qualitative or quantitative statements describing “the terms of reference against which the significance of a risk is evaluated” (ISO 2009). A review of regulations, regulatory guidance material, and industry position papers yielded a disparate array of qualitative and quantitative statements of the HLSC for UAS. Based on this review, the existing HLSC can be broadly categorized into one of two general categories: acceptable level of safety (ALoS) and equivalent level of safety (ELoS) criteria. These HLSC are not to be confused with equivalent level of performance (ELoP) requirements, which are briefly described in Sect. 92.2.5.3.

2.5.1 Acceptable Level of Safety Criteria

The first category of HLSC for UAS are those defined in relation to an ALoS. Examples of existing qualitative statements of ALoS HLSC are provided in Table 92.1. Although ALoS HLSC avoid many of the issues associated with making a direct comparison to the safety performance of CPA (discussed in the next section), they provide no guidance as to what constitutes an acceptable level of safety.

Table 92.1 Examples of qualitative specifications of the acceptable level of safety criteria for UAS

One approach for qualifying/quantifying acceptable is to base it on the de facto levels of risk determined for other activities (e.g., smoking or riding a bike) or naturally occurring events (e.g., death due to being struck by lightning). For example, the Swedish Aviation Authority use the probability of someone dying in a road accident to guide the setting of ALoS criteria for UAS (Wiklund 2003). Another approach is to directly adopt existing safety criteria specified in the regulation of other industries (e.g., as used for land use planning, space launch activities, and nuclear energy industry).

2.5.2 Equivalent Level of Safety Criteria

The second and most common category of HLSC for UAS qualifies acceptable through reference to the safety performance currently exhibited by CPA. Safety performance is expressed as the level of risk or the potential for harm (i.e., the existence of hazards). These comparative HLSC are widely referred to as ELoS criteria, and some qualitative examples are provided in Table 92.2.

Table 92.2 Examples of qualitative specifications of the equivalent level of safety criteria for UAS

There are a number of critical assumptions that need to be considered in the use of ELoS HLSC. Firstly, there is the foundational assumption that risks as tolerated or accepted in the past (i.e., those associated with CPA operations) provide a suitable basis for judging the acceptability of future risks associated with a different technology (i.e., those associated with UAS operations). Many of the factors discussed in Sect. 92.2.4 would challenge this assumption.

Secondly, ELoS criteria require a mechanism for making comparisons between the different categories of CPA and of UAS. For example, CASA states that HLSC for UAS should be defined in relation to CPA of equivalent class or category (CASA 2002). A range of mechanisms for making such a comparison have been proposed and include those based on similarities in the maximum takeoff weight of the aircraft, the maximum kinetic energy of the aircraft under different failure modes (JAA/EUROCONTROL 2004) or in the expected number of casualties (Grimsley 2004). For some UAS, it is not possible to establish an equivalent type within the CPA fleet on the basis of a similarity in the attributes of the aircraft alone. This issue is clearly illustrated in the comparative histogram plots of the UAS and CPA fleets presented by Clothier et al. (2011). Even if equivalence in terms of a similarity in aircraft attributes can be made, such attributes may not account for the differences between the safety risk profiles associated with the two different aviation concepts. These differences are discussed in the risk assessment subprocesses of Sects. 92.3, 92.4, and 92.5.

A range of measures, reference data, and approaches have been used to quantify ELoS criteria and some examples are provided in Table 92.3. There are a number of issues associated with the use of these measures. Firstly, most of the ELoS HLSC were determined through an historical analysis of CPA accident and incident data. As discussed by Clothier and Walker (2006), this quantification approach can be sensitive to the period over which the historical analysis is conducted and the type of CPA activity considered. Averaging over a historical period does not reflect trends in the safety performance of CPA operations or the infrequent nature of the events being characterized. Further, the averaged/aggregated measures do not account for peak risks that can occur due to geospatial or temporal concentrations in aviation activity or variations in the level of exposure of different subgroups within the populations exposed to the risks (e.g., the level of risk to pilots and aircrew compared to the level of risk to members of the general flying public).

Table 92.3 Examples of quantitative specifications of the equivalent level of safety criteria for UAS

To ensure a more comprehensive management of the risks associated with UAS operations and to be consistent with the safety risk management of other industries (see HSE 2001b), Clothier et al. (2011) propose that the specification of HLSC for UAS includes measures indicative of the individual and societal risk, in addition to the measures of group/collective risk that have been previously proposed. Further, it is recommended that the HLSC for UAS be defined based on the peak risks associated with CPA operations as opposed to averaged values.

Irrespective of the measures used or where the baseline level of safety is set (e.g., equivalent to that of CPA or not), there is the inherent difficulty of verifying that a system or operation actually satisfies the HLSC.

2.5.3 Equivalent Level of Performance Requirements

The ELoS criteria described in the previous section should not be confused with the requirement for an ELoS as described in (FAA 2009, 2011b; Wolfe 2009) which are referred to here as ELoP requirements. Whereas ELoS criteria are expressed in terms of levels of safety or of risk, ELoP criteria are typically expressed in terms of equivalence to the following:

• Existing regulations, standards, or procedures for CPA (e.g., design or operating requirements)

• Functions or functional performance (e.g., UAS must demonstrate a sense and act function equivalent to the see and avoid function provided by a human pilot).

  ELoP requirements are not HLSC but lower-level requirements mandated to control (or mitigate) the risks associated with UAS operations (discussed further in the Section 92.2.5.3). Satisfying an ELoP does not necessarily give rise to an ELoS. The use of ELoP as de facto safety criteria requires assumptions to be made in relation to the nature of the relationship between system performance (e.g., reliability) and the level of risks to different entities of value (e.g., the potential damage to people and property).

2.6 Summary

Establishing the context defines the inputs, desired outputs, and the boundaries and constraints on decisions made throughout the SRMP. It is important to note that obtaining a public license for UAS operations must take into consideration a broad range of issues. The integration of a new technology into society is subject to a wide range of broader social, political, cultural, and economic considerations. For example, one of the primary concerns identified in the survey of air travelers conducted by MacSween-George (2003) was the potential unemployment of pilots. A search of mainstream media sources reveals numerous articles identifying a broad range of public concerns including privacy, noise and public disturbance, and the potential misuse of UAS by drug traffickers or terrorists. Such concerns can be as significant as those issues relating to their safety. Further research is needed to characterize the safety criteria for UAS and to better understand different stakeholder concerns, perceptions, and expectations. In the interim, guidance can potentially be found through exploring the safety risk management of other new technologies, such as genetically modified foods, nanotechnologies, stem cell research, nuclear power, and the use of automation in the rail and shipping industries.

3 Risk Identification

The objective of the risk identification process is to identify how the system can fail, how these failures and conditions manifest as hazards, and the potential undesired outcomes that can result from the occurrence of the hazards. The identification of a specific combination of these three components describes a risk scenario. The set of all risk scenarios can be defined through the identification of the set of hazards, and for each particular hazard the associated sets describe the following:

• 1. The different conditions, failures, and events contributing to the occurrence of the particular hazard

• 2. The potential types and levels of consequential outcomes associated with the occurrence of the particular hazard

The set of all scenarios identified with a given activity is described as the risk profile. By way of general introduction, the high-level UAS and CPA risk profiles are illustrated in Figs. 92.2 and 92.3, respectively. Illustrated in Figs. 92.2 and 92.3 are the primary and secondary hazards and their potential consequential outcomes to people and property. Not shown are the conditions, failures, and events contributing to the occurrence of the hazards. The profiles, and the tools, data, and techniques that can be used to identify and characterize them are described in the following subsections.

3.1 Risk Identification Tools

A range of techniques can be used to identify and characterize the risk scenarios associated with UAS operations. The CAA categorizes these techniques into historical (e.g., a review of accident and incident data), brainstorming (e.g., elicitation of knowledge from domain experts), and systematic (e.g., formal tools and processes) techniques (CAA 2010b).

A typical starting point for any risk identification process is a review of existing accident and incident data. Such a review can provide general insights into the key hazards and their likely consequential outcomes and, depending on the scope and quality of the investigative reports available, the factors contributing to their occurrence. Some notable examples of UAS accidents and incidents are provided in Table 92.4.

Table 92.4 Notable accidents and incidents involving UAS

There is limited data on UAS accidents and incidents. The majority of publicly available data relate to military UAS operations primarily because of the limited amount of nonmilitary UAS activity to date (a product of the current regulatory environment) and that mandatory reporting of accidents and incidents involving nonmilitary UAS has only recently come into force (refer to Sect. 92.7.1). Seldom does a review of accident and incident data provide a comprehensive identification of the potential hazards and their outcomes. This is particularly the case for UAS, where there is limited data available and the primary hazards are inherently rare events. Further, the ability to identify the complexity of factors contributing toward the occurrence of an accident or incident is often restricted by the method and quality of the records available. Incidents occur more frequently than accidents. Incidents provide valuable information as precursor or lead indicators for accidents; however, less information is typically available in incident reports due to the limited amount of resources available to investigate them. There is also a bias in the data toward military UAS operations, and therefore, when using this data, it is important to consider some of the differences between military and nonmilitary UAS operations. For example, the potential differences:

• Between the design and operational philosophies adopted for military and nonmilitary UAS (e.g., trade-offs made between survivability and mission risk vs. public, and personnel risk)

• Between the environments they are operated in (e.g., natural environment, mix and types of other airspace users, and electromagnetic environment)

• In how they are managed within the airspace system (e.g., procedures for separation, the situational awareness available to air traffic control, the UAS operators and other airspace users, and the type of services provided)

• In the nature of the missions performed (e.g., low-level flights, maneuver, and mission profiles)

• In their hazards (e.g., for military UAS, there are unique hazards associated with the carriage of ordinance, self-protection systems, and payload self-destruct mechanisms)

These and many other differences can give rise to unique sets of risk scenarios for military and nonmilitary UAS operations. Although a valuable input to the risk identification process, UAS accident and incident data should not be used as the sole means for risk identification. This data should be complemented by other risk identification techniques to ensure a comprehensive identification of the risks. References (SAE 1996; FAA 2000; FAA and EUROCONTROL 2007) describe a number of tools, and that can be used in the identification and analysis of aviation safety risks. A domain-independent review of over 100 different risk identification and analysis techniques can be found in Stephens et al. (1997). Commonly used risk identification and analysis tools are provided in Table 92.5.

Table 92.5 Some common risk identification and analysis tools

3.2 The Identification of Hazards

The specification of a risk scenario starts with the identification of the hazards. A hazard is a state or condition that has the potential to cause loss to something of value. ISO31000:2009 describes the analogous concept of a risk source, defined as an “element which alone or in combination has the intrinsic potential to give rise to risk” (ISO 2009). Prescriptive definitions of hazard can be found in ICAO (2009); DoD (2010a).

3.2.1 Primary Hazards of Concern Associated with UAS Operations

A primary hazard is one that has the potential to directly cause harm. Some definitions of primary hazard include the additional condition of immediately (Dalamagkidis et al. 2008); however, such definitions preclude primary hazards that have delayed effects or require long-term exposures (e.g., radiation, psychological losses, exposure to carcinogens, or damage to ecosystems). With respect to the operation of UAS in the civil airspace system and over inhabited areas, the primary hazards of concern are well known and common to those for CPA operations. As described in JAA/EUROCONTROL (2004), Clothier and Walker (2006), and Dalamagkidis et al. (2008) and as illustrated in Fig. 92.2 these hazards are the following:

• (A) A collision with a CPA (situated on the ground or in the air) and the potential harm caused to people onboard the CPA (e.g., incident 2 in Table 92.4)

• (B) The controlled or uncontrolled impact with terrain or objects on the terrain (such as people or structures), for example, incidents 5 and 6 in Table 92.4

Fig. 92.2

Illustration of the high-level risk profile associated with UAS operations

3.2.2 Secondary Hazards of Concern Associated with UAS Operations

Secondary hazards of concern are those that can occur as a result of a primary hazard. Some of the secondary hazards associated with the primary hazard A above, include the potential harm caused to people:

• 1. On the ground due to falling aircraft or debris from a midair collision (e.g., the falling debris described in incident 2 in Table 92.4)

• 2. On the ground due to falling aircraft or debris from a near midair collision (e.g., incident 8, Table 92.4, where wake turbulence caused the loss of the UAS)

• 3. Onboard the CPA due to evasive maneuvers performed in order to avoid a collision with a UAS (while either of the aircraft is in the air or on the ground)

Some of the secondary hazards associated with the primary hazard B above include the potential harm caused to people on the ground due to the following:

• 1. Release of hazardous materials (e.g., chemical payloads, composite materials, or ordnance) following an impact with terrain or an object on the terrain

• 2. Progression of fires, the collapse of buildings, motor vehicle accidents, or other hazards arising as a result of the UAS coming to earth (e.g., in incident 3 of Table 92.4 there was the potential for an explosion or fire had the UAS damaged critical components of the oil refinery)

As can be observed in Figs. 92.2 and 92.3, the primary and secondary hazards identified within the UAS risk profile also exist within the CPA risk profile. However, not shown are differences in the failures and conditions contributing to the occurrence of these hazards and in the types and levels of consequence associated with their occurrence.

Fig. 92.3

Illustration of the high-level risk profile associated with CPA operations

3.3 The Contributing Failures and Conditions

There are a variety of ways in which the hazards illustrated in Fig. 92.2 can eventuate. The specification of a risk scenario includes identifying how a particular hazard can occur. A hazard is typically the result of a series of active failures in combination with latent conditions that involve all components of the system (i.e., the interaction of the components of man, machine, and organization) and the interaction of the system within its operating environment. Some key techniques for identifying these failures and conditions include FMEA, HAZOP, fault tree analysis, human factors studies (discussed below), and anticipatory failure determination.

High-level guidance on common factors contributing to UAS mishaps can be found in studies of existing accident and incident data. For example, some frequent causes of mishaps reported by the U.S. Department of Defense (DoD) are summarized in Table 92.6.

Table 92.6 Percentage of mishaps attributed to different failure mode categories, from OSD (2003)

3.3.1 Unique Components and Functions

There are some obvious differences in the design and operation of UAS when compared to CPA. For example, a communications link for command and control is a critical component of the safe operation of UAS particularly in the absence of aircraft autonomy (i.e., a remotely piloted aircraft). Other unique components of an UAS include the ground control element, flight termination systems, and devices used in the launch and recovery of the air vehicle. The existence of these components can create unique hazards and contribute toward the occurrence of the primary hazards illustrated in Fig. 92.2. Consideration of such components (and their failures) is not captured in existing CPA risk identification studies.

3.3.2 The Importance of a “Systems” Mentality

UAS are more than an aircraft. Consideration of the individual components of the UAS in isolation of the other components of the system and its environment would fail to provide a comprehensive identification of the risks. An emergent property is one which is not determined solely from the properties of the system’s parts but which is additionally determined from the system’s structure and behavior (Thomé 1993). These emergent properties and the boundaries and constraints on them are all potential sources for active failures or latent accident-producing conditions. For example, the UAS system has the property of line-of-sight (LoS) communication range. LoS range is an emergent property, arising due to the interactions between the system and its environment. Specifically, it is a function of the state of the air vehicle (e.g., antenna attitude), the properties of the communications system (e.g., frequency and minimum permissible signal to noise ratio for a given bit error rate), the ground control system (e.g., geographical position), the mission (e.g., the flight path), and the environment (e.g., weather, terrain, and ambient radio frequency environment). Together, these properties interact to define the maximum LoS range of the system at a given time. Exceeding this range can contribute to the occurrence of a hazard (i.e., a loss of command and control, which for an RPA, could lead to a mishap).

3.3.3 The Human Element

Despite the relocation of the pilot, the human element still has a significant contribution to the safety of UAS operations. A clear example of this is incident 7 in Table 92.4 (refer to associated accident report). An analysis of U.S. DoD operations recorded over the 10-year period ending in 2003 (Tvaryanas et al. 2005) found that 68.3 % of the 211 mishaps reviewed involved operations or maintenance organizational, supervisory, or individual human factors. References (Manning et al. 2004; Williams 2004; McCarley and Wikens 2005; Tvaryanas et al. 2005; Hobbs 2010) provide further analysis and discussion of the contribution of the human element to UAS accidents using a variety of modeling frameworks. Common human factors identified in these studies include crew resource management, decision-making, situational awareness, human machine interface design, training, task load, and fatigue. Psychological issues can include the apparent risk-taking behavior of UAS operators due to the absence of a shared fate between the operator and the UAS; issues of operator trust, awareness, and dependency on automation; issues associated with a handover between remotely located operators; and issues relating to the simultaneous control of multiple UAS. It is important to consider human factors in all aspects of a UAS deployment and not just its launch, operation, and recovery. For example, Hobbs and Stanley (2005) identify the personnel issues of complacency and a model aircraft culture in the maintenance of UAS; such factors can contribute toward the 8 % of U.S. DoD UAS accidents that were the direct result of maintenance errors (Tvaryanas et al. 2005). For some UAS, much of the maintenance can be performed in the field during an active deployment (e.g., change of payloads, replacement of wings, minor repairs). Maintenance in the field can be subject to additional time pressures (e.g., push for readiness for next deployment), poor working conditions (e.g., exposure to the environment), and the need to make decisions and actions without access to all the necessary information or tools (e.g., arising due to poor logistics and operational planning).

3.3.4 The Operation and the Environment

It is important to consider how failures can eventuate through the interaction of the UAS and its operational environment. Many of the hazards arising from the natural environment are common to CPA and are well known, for example, storms and bird strikes (e.g., incident 3 in Table 92.4). However, for UAS, the detection of these conditions can be difficult as the operator is not located onboard the aircraft, and even if it is detected, many UAS do not have the same resilience to them as CPA (e.g., the absence of anti-icing systems or bird strike protection).

A single UAS type can be used for a wide range of applications. The potential failures and conditions need to be investigated for these different operations and environments. For example, the low-altitude operation of UAS in the vicinity of structures creates a number of additional challenges over UAS operations in relatively clear areas. For example, large structures can impede communications, create turbulent environments, and degrade navigation performance through increased multipath and a reduction in the number of visible GPS satellites.

3.3.5 Software

Most nonmilitary UAS make use of Commercial-Off-The-Shelf (COTS) consumer-grade software that is often provided without warranty or assurance. Without such assurances, it can be extremely difficult to assess the likelihood of encountering latent errors or undesired behavior. Often, the dependability of software can only be gauged through extensive experience in its use under a variety of conditions. Configuration control is also particularly important for those systems using COTS software. Small bug fixes and auto updates to operating systems can introduce new latent conditions and significantly change the stability and behavior of the software system as well as its performance under existing conditions. Software considerations should extend to include any electronic databases (e.g., publicly available digital elevation maps), firmware, operating systems, and applications used during flight or prior to and after flight (e.g., flight planning, software, and documentation control systems). In addressing software-related risks, there are two separate, yet often confused, considerations. Firstly, there are risks associated with the behavior of algorithms and, in the case of UAS, the validity of autonomous behavior. The latter is particularly of concern when the level of autonomy increases (Parasuraman et al. 2000). The second consideration relates to the implementation of the algorithm and is addressed by standards such as DO-178B (RTCA 1992).

3.3.6 Security

Security threats are a subcategory of hazards. More specifically, they are hazards that arise, either directly or indirectly, through the intentional disturbance of the safe or normal operational state of the UAS. Most often, these disturbances originate from objects external to the system, which exploit the interfaces between the UAS and its environment (e.g., interference, jamming, or the overriding of control via communications links or physical access to the ground control station (GCS)). The security of the UAS should take into consideration:

• The type of radio control gear, voice, and data links used for communication between all components of the system (including ground personnel and air traffic control)

• Whether the links are vulnerable to intentional or unintentional interference and whether the loss of this link has a safety impact for different phases of the operation

• The type of information conveyed on these links and its criticality to the safety of the operation of the aircraft if corruption, disruption, or spoofing occurs

• Whether the sender or recipient of the information on these links needs to be verified or not (e.g., incident 10 described in Table 92.4)

• The location and physical security of the GCS and any launch, recovery, communications relay, maintenance, and storage sites

• Whether software security, such as firewalls and antivirus programs, is installed and used

• Policies in relation to access to the Internet and the transfer of media via removable storage.

3.3.7 The Criticality of Failure Modes

Firstly, flight critical failures are no longer restricted to the aircraft; one must also identify those flight critical failures that exist in the GCS and communications components of the UAS. Secondly, what is considered a catastrophic failure for CPA may not necessarily be catastrophic for a UAS, and vice versa. For CPA, the assignment of criticality to a failure is based on an assumed exposure probability of one (i.e., there is always at least one person onboard; thus, someone is always exposed, see Fig. 92.3). For UAS, the exposure probability is a complex function that depends on where the UAS is operated. In some cases, the exposure probability may approach zero (e.g., those UAS operations restricted to uninhabited areas and in segregated airspace). In such cases, the failure criticality can potentially be assigned to a lower severity category (e.g., major or hazardous), and this assignment should be based on the potential impact of the failure on the ability of the UAS to remain in its predefined operational area. On the other hand, some failures for UAS may have a higher criticality due to the absence of the additional protection provided by a pilot onboard. Thus, adopting existing CPA failure criticality assignments for UAS must be treated with caution.

3.4 Assessing the Potential Consequences

The final component of the specification of a risk scenario is the identification of the potential consequential outcomes. Explicitly linked to the concept of hazard are the concepts of loss, harm, or consequence. For example, the definition of hazard provided by ICAO (2009) includes a specification of the types of consequential outcomes to be considered:

Any real or potential condition that can cause injury, illness, or death to personnel; damage to or loss of a system, equipment or property; or damage to the environment. pp. 4–1 (ICAO 2009)

As can be observed in Fig. 92.3, the risks associated with CPA operations include consideration of the potential harm to people onboard the aircraft in addition to those onboard other CPA or on the ground. An analysis of worldwide accidents involving conventionally piloted commercial jet aircraft over the period 2001–2010 reveals that more than 95 % of all fatal injuries were to people onboard an aircraft (Boeing 2011). Therefore, for both of the primary hazards associated with CPA operations, the consequences of principal concern are those to the passengers and crew onboard the aircraft and, secondarily, to the population of people external to the aircraft (e.g., those living in the regions overflown). For UAS, there are no people onboard the aircraft, and the primary risks are instead to those entities of value considered external to the system. Consequently, the primary types and spectra of consequential outcomes associated with UAS operations are different to those associated with CPA operations.

3.4.1 Domains of Consequence

There are a variety of potential consequential outcomes associated with the occurrence of a hazard. For example, MIL-STD-882D (DoD 2010a) defines loss in terms of damage to people, equipment or property, or the environment. These types of loss describe the different domains of consequence. Typically, the primary domain of consequence is that of physical harm to people, with secondary domains being the potential loss registered to equipment and property (inclusive of the air vehicle), the environment, the organization (e.g., financial, reputational, capability, market, or mission losses), clients, the broader industry, and the losses registered to other less tangible values held by society (e.g., culture, trust).

Distinctions are often made between consequences of the same type. For example, the risk management of CPA operations makes a distinction between those operations with fare-paying passengers onboard and those without. Such a distinction is made due to social and psychological factors that influence the general public’s perception and acceptance of risk (e.g., the assigned value, dread, fear, control, voluntariness of exposure). For UAS, similar distinctions will need to be made in relation to the primary entities of value at risk on the ground. For example, a distinction is often made between third-party casualties (e.g., a member of the public) and a first-party casualty (e.g., personnel supporting the deployment of the UAS). Similar distinctions are made between damage to property and the damage to hospitals, schools, residential areas, historical or culturally significant sites, etc.

3.4.2 The Spectra of Consequences

A qualitative or quantitative spectrum of consequence needs to be defined for each domain of consequence identified. Take, for example, the consequence domain of people. The associated scale of loss could be defined from no injury to multiple fatal injuries. As shown in the studies by Clothier et al. (2010) and Fraser and Donnithorne-Tait (2011), there can be categories of UAS which are unable to cause significant and direct physical harm to other aircraft or people or property on the ground. For these categories of UAS, the losses associated with secondary domains of consequence (e.g., organizational, financial, or environmental) or those losses arising due to the occurrence of secondary hazards (e.g., ensuing bushfires or downstream losses due to damage to critical infrastructure) are likely to be more significant in the evaluation and management of the safety of their operation.

3.5 The Set of Scenarios

The outcome of the risk identification process is a set of characterized scenarios. This set is seldom complete as there will always be unknown hazards or failures and conditions that can give rise to existing hazards. It is important that the hazard identification process is periodically reviewed to make use of new knowledge, information, or identification techniques (refer to Sect. 92.7). A hazard log should be maintained to record and track any new scenarios identified during the course of operations and should form a valuable input to any review of the risk assessment. Finally, the endeavor to ensure the set of scenarios is as comprehensive as possible, coupled with the use of conservative assumptions in their characterization, can lead to the specification of unrealistic scenarios. There are limited resources available to treat the risks associated with the identified scenarios; therefore, it is important that all scenarios be subject to a test of plausibility.

4 Risk Analysis

The third step in the SRMP, Fig. 92.1, is an analysis of the risk. Risk analysis describes the process of characterizing the nature and level of the risk for each of the identified risk scenarios. A measure of risk is expressed through the combination of assessments of the consequence and the likelihood of occurrence of the given scenario.

4.1 Assessing the Consequence

A qualitative or quantitative table is often used to group and rank the different types and levels of consequence associated with the identified risk scenarios (examples shown in Table 92.7). An assessment of the consequence for a given risk scenario is made by mapping its potential outcomes to one of the consequence levels defined within the table. As there can be more than one consequential outcome associated with the occurrence of a single-risk scenario, a mapping is typically based on the worst possible outcome identified.

Table 92.7 Examples of existing consequence/severity classification schemes

4.2 Likelihood of Occurrence

A range of formal techniques can be used to assess the likelihood of a scenario occurring (e.g., the risk assessment tools described in Table 92.5). Some high-level models characterizing the primary risk scenarios illustrated in Fig. 92.2 have also been proposed, for example (Weibel and Hansman 2004; Clothier et al. 2007; Lum et al. 2011; Lum and Waggoner 2011).

Assessments can draw on a range of information sources including incident and accident data, aircraft activity data, component reliability data, and expert knowledge. Assessment can also use existing models used in other application domains (e.g., space vehicle launch and reentry, motor vehicle accident studies, munitions, debris modeling, generic human injury models, and CPA airspace collision risk models). The output is a qualitative or quantitative assessment of the likelihood of realizing a given risk scenario. Depending on the tools and modeling approach used, this assessment can be used directly in the assessment of the risk or mapped to a likelihood scale or classification scheme (Table 92.8).

Table 92.8 Examples of existing likelihood/probability classification schemes

4.3 Assessing the Risk

A range of qualitative and quantitative scales have been used to describe levels of risk. For example, MIL-STD-882D (DoD 2010a) assesses risk on the qualitative ordinal scale: low, medium, serious, and high. The component measures of consequence (Sect. 92.4.1) and of likelihood (Sect. 92.4.2) then need to be mapped to one of these levels of risk. A risk matrix is the most common method for illustrating this mapping, and an example of which is provided in Fig. 92.4. ICAO (2009) also provides an example of a risk matrix.

Fig. 92.4

Example of a risk matrix as per MIL-STD-882D (2010)

4.4 Uncertainty

A particular issue in the safety risk management of UAS is managing uncertainty in the risk assessment process. Uncertainty can pervade all stages of the SRMP. Uncertainty influences the level of risk perceived by stakeholders (i.e., the higher the perceived uncertainty the higher the perceived risks) and their preferences for risk treatment (i.e., a preference to treat those risk scenarios with a higher degree of associated uncertainty). Uncertainty arises through a lack of knowledge and information available in the risk assessment process, differences in the level of knowledge held by stakeholders (leading to issues of trust), and a lack of transparency in the SRMP. An effective communication and consultation process (Sect. 92.8) is key to addressing the uncertainty of stakeholders. However, managing the uncertainty in the assessment process is particularly difficult for those UAS that employ COTS equipment with limited or no information on their reliability. A defensible starting position is to attempt to establish the boundaries on the assessment of the risks as opposed to an estimate of the point value of the risk. The upper boundary on the risk can be estimated by propagating the assumption that all systems and components will fail. An estimate on the lower boundary can be made by adopting a less conservative assumption based on the best available data. As the SRMP is iterative and ongoing, these initial and conservative assumptions can be revised as more experience and data becomes available. An introductory paper on the types of uncertainty and how uncertainty pervades the SRMP can be found in Zio and Pedroni (2012).

5 Risk Evaluation

Risk evaluation is the process of comparing the results of the risk analysis with the HLSC to determine whether the risk for a given scenario is tolerable (ISO 2009) or whether further measures need to be undertaken to reduce the risk. There are a range of decision-making frameworks that can be used within the risk evaluation process; these include the as low as reasonably achievable, globalement au moins Équivalent, or minimum endogenous mortality frameworks used in the Netherlands, France, and Germany, respectively. Discussion in this chapter is limited to the as low as Reasonably practicable (ALARP) evaluation framework, which is advocated by ICAO (2009) and has been widely used in the management of a broad range of risks in the UK, the USA, and Australia.

5.1 The ALARP Framework

The ALARP framework is intended to represent safety decisions made in everyday life (HSE 1992, 2001b). There are some risks that people choose to ignore and others that they are not prepared to entertain irrespective of the benefits associated with them. In addition, there are those risks people are prepared to take by making a trade-off between the benefits of taking the risks and the precautions required to mitigate them (HSE 2001b). These three types of decision scenarios are the basis for the development of the ALARP framework. Referring to Fig. 92.5, the ALARP framework comprises

Fig. 92.5

The ALARP risk evaluation framework

A Region of Broadly Unacceptable Risk – Except under extenuating circumstances, risks that fall within this region are generally considered unjustified regardless of the benefits associated with the activity. Such activities would be ruled out unless further action can be undertaken to reduce the risk (HSE 2001b). This region corresponds with the notion of a de manifestis level of risk, which is based on the legal definition of obvious risk (RCC 2007). It is defined as the level of risk above which a person of ordinary level of intelligence intuitively recognizes as being inherently unacceptable (Fulton 2002; RCC 2007).

A Region of Tolerability – This region describes those risks which are considered tolerable, specifically those situations where there is “… a willingness to live with a risk so as to secure certain benefits and in the confidence that it is being properly controlled. To tolerate a risk means that we do not regard it as negligible or something we might ignore, but rather as something we need to keep under review and reduce still further if and as we can” (HSE 1992). As described in HSE (2001b), risks that fall in the region are considered tolerable if and only if the:

• Risks have been properly assessed (e.g., assessments based on the best available scientific evidence or advice), and the results are used to determine appropriate measures to control the risks.

• Residual risks are not unduly high (e.g., above the de manifestis level) and are kept to level as ALARP.

• Risks are periodically reviewed.

A Region of Broadly Acceptable Risk – Risks within this region are “generally regarded as insignificant and adequately controlled” (HSE 2001b). There is no distinct line demarcating tolerable risks from broadly acceptable risks; instead, it has been described as the point at which “the risk becomes truly negligible in comparison with other risks that the individual or society runs” (HSE 1992). Obtaining a broadly acceptable level does not mean the pursuit for the reduction of risks to ALARP should be abandoned. As described by the UK Health and Safety Executive (HSE), “duty holders must reduce risks wherever it is reasonably practicable to do so or where the law so requires it” (HSE 2001b).

The Concept of ALARP – A risk is considered ALARP if the cost of any reduction in that risk is in gross disproportion to the benefit obtained from the reduction Determining that risks have been reduced to a level as ALARP involves an assessment of the risk to be avoided, of the sacrifice or costs (e.g., in money, time, and trouble) involved in taking measures to treat that risk, and a comparison of the two to see if there exists a gross disproportion (HSE 2001a). General discussion on the cost-benefit process that needs to be undertaken and some guidance on the meaning of gross disproportion can be found in references (HSE 2001b,a; CASA 2010; Jones-Lee and Aven 2011).

De Minimis Level – Some specifications of the ALARP framework include a specification of the de minimis level of risk. The de minimis level stems from the legal principle de minimis non curat lex (the law does not concern itself with trifles) (Paté-Cornell 1994; Fulton 2002; RCC 2007). It is often used as a guide for determining when risks have been managed to a level that could be considered below concern.

A Scrutiny Level – Some implementations of the ALARP framework feature a scrutiny line, which is often used to put newly assessed risks in context with risks that have been tolerated or broadly accepted in the past. Often, the scrutiny level represents the de facto risks for a similar activity/industry.

It is important to note that the meaning of ALARP and its implementation in law can change between states (an important consideration when it comes to the risk management of international UAS operations). The description of ALARP provided above is consistent with its implementation in those countries that adopt common law (e.g., the UK, the USA, Australia, Canada, New Zealand). Ale (2005) provides an example of some of the issues that can arise due to the application of safety decision-making frameworks such as ALARP within different legal systems.

There are psychological, social, and practical difficulties in the specification and sole use of quantifiable criteria within the ALARP framework. This has lead to the use of qualitative frameworks that focus on demonstrating that all reasonably practicable measures have been undertaken to reduce a risk as opposed to making quantifiable comparisons of the assessed risks to specifications of the de manifestis, de minimis, or scrutiny levels. The results from comparisons of assessed risks to HLSC ultimately translate to requirements on design; hence, a quantifiable specification of HLSC within the ALARP framework is most desirable. When introducing a new technology into society one cannot avoid the commonly used litmus test of a comparison to similar and existing risks (as often made by the media or by members of the public). In this case, the ELoS HLSC (as described in Sect. 92.2.5) should be represented as scrutiny lines within the ALARP framework. Further research is needed to explore the psychological, social, and practical implications relating to the representation of the quantitative HLSC for UAS in the ALARP framework. There can also be general issues associated with the application of ALARP specifically to new technologies such as UAS, and these are discussed in Melchers (2001).

5.2 Evaluating the Risk

The ALARP framework is represented in a risk matrix by assigning the levels of risk, and hence cells of the matrix, to the different regions of the ALARP framework. This assignment is often illustrated through the use of a graduated color scale (e.g., refer to the corresponding colors used in Figs. 92.4 and 92.5). Refer to Figs. 5–4 and 5–5, pp. 5–8/9 of ICAO (2009) for another example of a representation of the ALARP framework within a risk matrix. Each risk scenario can then be mapped to one of the regions within the ALARP framework. Whether or not a particular scenario requires treatment will depend on the ALARP region it is mapped to (as described in Sect. 92.5.1).

6 Risk Treatment

For those risk scenarios that are not tolerable, measures need to be undertaken to reduce (mitigate, modify, treat, or control) the residual risk to a level considered as ALARP.

6.1 Prioritization of Treatment

The scenarios requiring treatment need to be prioritized due to a practical limit on the resources available to treat the risks. This prioritization is usually based on the level of unmitigated risk, with those scenarios having a higher level of risk given a higher priority for treatment. However, there are other factors that can influence the prioritization of scenarios, for example, the prioritization of scenarios:

• Based on the nature of their associated consequences. For example, the apparent public aversion to accidents with a higher-level consequence, psychological factors (e.g., fear or dread), or those scenarios that have prolonged or sustained consequences

• With a high level of uncertainty.

It is important to note that the treatment of some scenarios may be mandatory irrespective of their risks (e.g., due to environmental protection or workplace health and safety regulations).

6.2 Determining Available Mitigation Options

The first step is to determine a list of all possible treatment options. Guidance on potential mitigation strategies can be found in regulatory materials (CASA 2002; FAA 2011b) or by reviewing the safety cases used in the approval of existing operations. In general, risk mitigation strategies reduce the risk through the following:

• A. Removing the hazard altogether

• B. Reducing the likelihood that a hazardous event occurs

• C. Reducing the level of potential consequence associated with the occurrence of an hazardous event

• D. Sharing the retained risk with other organizations

• E. Combinations of the above

6.2.1 Risk Mitigation Strategies for Midair Collision

A range of strategies can be used to mitigate the risks associated with the hazard of a midair collision between a UAS and a CPA. Some example strategies are summarized in Table 92.9. The strategies in Table 92.9 are classified based on how the primary reduction in risk is achieved, specifically (1) through elimination of the hazard, (2) through a reduction in the likelihood the hazard occurs, or (3) through a reduction in the consequence given the occurrence of the hazard. Category 2 mitigation strategies are divided into the subcategories of the following:

• A. See – strategies that provide the UAS with an awareness of its air traffic environment

• B. Be seen – strategies that provide other airspace users with an awareness of the UAS

• C. Staying away – UAS operational strategies that reduce the likelihood of encountering other aircraft

• D. Services – third-party air traffic separation services that provide situational awareness and separation management services to the UAS and/or other air traffic

• E. Strategic – ongoing strategies that improve the effectiveness or proficiency of the UAS crew in managing the risk of midair collisions or build a general awareness of UAS operations

Table 92.9 Examples of existing midair collision mitigation strategies

Fig. 92.6

An example mitigation technology: the INSITU Pacific Mobile Aircraft Tracking System with communications, primary radar, and ADS-B In (Wilson 2012) (Image courtesy of Dr Michael Wilson)

Fig. 92.7

An example mitigation technology: the Australian Research Centre for Aerospace Automation (ARCAA) electro-optical sense-and-act system fitted onto the wing strut of the ARCAA flight test aircraft (Lai et al. 2012) (Image courtesy ARCAA)

Fig. 92.8

An example mitigation technology: the INSITU Pacific ScanEagle^TM on launcher with high-visibility markings and strobes (Image courtesy INSITU Pacific Ltd)

The subcategories of A and B comprise technological and operational strategies that help to provide an alerted see-and-avoid environment and can be further categorized based on whether the additional situational awareness is achieved through active transmission or not. Some mitigation strategies can be assigned to more than one category, and it is important to note that some of the proposed mitigation technologies are concepts still under development; their suitability as effective mitigation strategies has yet to be determined.

6.2.2 Risk Mitigation Strategies for an Impact with Terrain

A range of strategies can be used to mitigate the risks associated with the hazard of a controlled or uncontrolled impact with terrain or objects on the terrain. Some example strategies are summarized in Table 92.10. The example approaches summarized in Table 92.10 are classified based on whether the reduction in risk is achieved through (1) the elimination of the hazard, (2) a reduction in the likelihood the hazard occurs, or (3) a reduction in the consequence given the occurrence of the hazard. Category 2 mitigation strategies are further divided into the subcategories of operational, technological, and strategic. The strategies summarized in Table 92.10 are in addition to those that improve the airworthiness of the system (e.g., the adoption of sound engineering design practices, fault-tolerant design principles, the certification of software to high levels of assurance, the implementation of quality control in manufacturing processes, increasing the depth and frequency of preventative maintenance cycles, completion of preflight checks, and procedures and policies for crew management, training and licensing).

Table 92.10 Example strategies for mitigating the risks of a controlled or uncontrolled impact with terrain or objects on the terrain

6.3 The Selection of Mitigation Strategies

ICAO (2009) evaluates mitigation strategies on the basis of their effectiveness (in terms of risk reduction), associated costs and benefits, practicality, whether they create new problems (e.g., introduce new risks), and other factors such as whether they stand up to scrutiny, the acceptability to other stakeholders, whether they are enforceable or durable, and whether the residual risks can be further reduced.

6.3.1 Effectiveness and the General Hierarchy of Mitigation Strategies

The effectiveness of a mitigation strategy is measured in terms of the magnitude of the reduction in risk achieved. The most effective strategy is to eliminate the hazard, followed by those strategies that reduce the severity of the hazard or the likelihood of its occurrence. The third most effective strategies are those that employ engineering controls preventing the mishap from occurring, followed by warning devices, and procedures and training (DoD 2010a).

6.3.1.1 Effectiveness of Midair Collision Avoidance Mitigation Strategies

The most effective strategy is to segregate UAS from other airspace users; however, due to issues of practicality and cost, this is not always a viable treatment option. Those safety cases that are primarily based on the situational awareness of other airspace users (e.g., be seen, Table 92.9) or strategies that reduce the level of consequence given the occurrence of a mishap are the least effective and, on their own, are not likely to provide an acceptable safety case. Reducing exposure (e.g., staying away, Table 92.9) in combination with other see and be seen mitigation strategies is likely to provide the most effective approach for managing the risk of a midair collision. In assessing the effectiveness of the different strategies, consideration should be given to the following:

• Types of airspace users that are likely to be encountered and their:

  • – Resilience to damage due to a collision with the particular type of UAS (e.g., bird strike protection of transport category aircraft)

  • – Observability to the different sensing or awareness approaches that could be used (e.g., radar cross-sectional area)

  • – Equipage (e.g., whether they have radios or transponders onboard)

  • – Ability to detect the UAS

  • – Ability to maneuver

  • – Typical operating speeds (e.g., determination of closing speeds and time to react)

  • – Conditions of right of way

• Operating conditions (e.g., instrument meteorological conditions vs. visual meteorological conditions) or the operational profile flown (e.g., variation in radar clutter performance with altitude)

• Geographical volumes over which protection or awareness needs to be provided

• Temporal changes (e.g., use of strobes during the day vs. at night) and the duration of activity (e.g., effectiveness of ground observers for extended missions)

6.3.1.2 Effectiveness of Mitigation Strategies for Managing the Risks to People and Property on the Ground

The most effective mitigation strategies for mitigating the risks to people and property on the ground are those that reduce the following:

• Probability of a flight critical failure or human error occurring (e.g., through fault-tolerant design, maintenance, crew resource management, and training)

• Exposure of people and property to the hazard. Specifically the operational mitigations in Table 92.10 that restrict UAS operations to uninhabited areas or avoid/minimize the overflight of densely populated regions, critical infrastructure or culturally sensitive sites

Automated emergency flight termination systems and recovery systems are effective but not for all failure modes (e.g., typically only those for failure modes where there is still a degree of control over the flight path of the UA). Least effective are those strategies that rely solely on the general public being sheltered, wearing personal protective equipment, or emergency equipment and procedures employed following a mishap.

6.3.1.3 Effectiveness of CPA Mitigations

It is important to note that a risk control strategy that is effective for CPA may not be effective for UAS. For example, a number of studies have been conducted to evaluate the effectiveness of ACAS as a means for self-separation, collision avoidance, or situational awareness for UAS (FAA 2011a). These studies identified a number of technical and operational issues, which have a significant impact on the effectiveness of ACAS as a midair collision avoidance system for UAS.

6.3.1.4 Evaluating Layers of Mitigations

Seldom will a single mitigation strategy be sufficient for a risk to be considered tolerable. In evaluating the cumulative effectiveness of multiple mitigation strategies, it is important to consider the potential for one strategy to degrade the effectiveness of another, thus forcing a reevaluation of the residual risks. The selection of strategies should ensure coverage of the complete spectrum of risk scenarios (e.g., implementing strategies that are only effective under visual meteorological conditions) and how the mitigation strategies, in isolation and in combination, can be overcome or can fail. The selection of strategies should try to ensure that these failure modes are not common to all of the strategies adopted.

6.3.2 Practicality

The practicable feasibility of mitigation strategies needs to be considered in relation to the physical and performance limitations of the system. For example, there are fundamental limits in relation to the maximum takeoff weight, payload volume, and power available to support mitigation systems onboard an unmanned aircraft. Similarly, there are fundamental limits in relation to the maneuverability, speed, range, endurance, glide performance, or ceiling of the UAS.

6.3.3 The Costs and Benefits Associated with Treatment Options

Costs should be considered in relation to a broad range of stakeholder groups (e.g., existing airspace users, air service providers, NAAs, the UAS industry, and ultimately, the general public) and include the indirect costs beyond those immediately associated with the occurrence of an accident (e.g., beyond the compensation for loss of life, the damage to property, and fines). Take for example, the mitigation strategies of (a) the use of redundant flight critical systems and (b) the equipage of a collision avoidance system. These mitigation options can result in increased:

• Platform costs due to the direct added cost of the collision avoidance system or the use of duplicate subsystems and the increased costs incurred in the engineering design, manufacture, and quality control processes

• Operational costs due to additional personnel training (e.g., in the operation of the collision avoidance system)

• Through-life costs due to additional maintenance

• Mission costs due to reductions in the following:

  • – Performance of the system (e.g., the extra weight and drag and its impact on endurance, range, speed, or ceiling)

  • – Ability for the UAS to support payloads (e.g., less weight, volume, and power available for payloads)

  • – Subsequent ability of the UAS to meet mission objectives

• Market cost due to a reduction in the number of serviceable clients

• Reduction in benefits with respect to foregone downstream benefits to end users and the broader society

As well as costs, there can be indirect benefits associated with the implementation of mitigation strategies. For example, improving the overall reliability of the UAS can lead to the benefits of a lower platform attrition rate, a reduction in insurance premiums, an increase in availability, and an increase in customer trust and in turn repeat business. These costs and benefits, along with the direct benefit of a reduction in the risks, need to be factored into the determination of ALARP.

6.3.4 Other Factors

Mitigation strategies for UAS should be assessed to determine whether they introduce new risks and whether these introduced risks warrant treatment or whether they outweigh the benefits of the employing the mitigation strategy altogether (e.g., explosive flight termination systems). The selection of mitigation options can also be guided by secondary objectives, values, and constraints held by different stakeholder groups. For example, the FAA (2011b) explicitly preclude treatment options that reduce the operational freedoms of other airspace users (e.g., the designation of airspace specifically for use by UAS). Another example is the military preference for passive midair collision avoidance systems due to the requirement to reduce the observability of military UAS operations. External constraints can include applicable standards and regulations (e.g., existing aviation safety, environmental protection, or occupational health and safety regulations) or constraints imposed by insurance providers.

6.4 Summary

The selection of mitigation options for UAS is a complex decision-making process. Mitigation options must be evaluated in terms of their effectiveness, costs, benefits, practicalities, and other factors to determine whether their implementation is reasonably practicable or not. This decision process is guided by the ratio of the costs and benefits associated with pursuing the different options for mitigation. A risk is considered ALARP if this ratio is in gross disproportion, a concept which is subjective and variable. Finally, a determination of ALARP does not make a risk tolerable. For every scenario, a decision must be made by the organization as to whether it is willing to retain the residual risk in return for the benefits of the operation. Authorization should be obtained at two stages in the treatment process: (1) at the point of approving the selection of mitigation strategies and the decision to retain residual risks and (2) to verify that strategies have been implemented as described. Typically, the delegation of authority is dependent on the level of residual risk that is being retained.

Currently, operational mitigation strategies (e.g., restrictions on the flight of UAS over populous areas) are central to obtaining operational approvals. Mitigation technologies like sense-and-avoid and automated emergency landing systems, are currently under development and showing much promise. These mitigation technologies will reduce the need for restrictions on UAS operations and will be key to the uptake of UAS in a greater number of civil applications. These technologies also have the potential to greatly improve the safety of CPA operations.

7 Monitor and Review

Risk is dynamic. Key to maintaining and improving the SRMP is a process to monitor and review the SRMP in response to changes in the risk. Risks evolve with changes in the organization, technology, and operations performed and in the natural, social, regulatory, and political environments. Further, there can be opportunities to improve the safety risk management of existing activities if new information, assessment tools, or treatment options become available.

7.1 The Importance of Accident and Incident Recording

One of the primary triggers for an ad hoc review of the safety risk management of an activity is the occurrence of an accident or incident. Accident and incident data are a valuable source of information that can be used to identify new risk scenarios and update risk assessments. Most importantly, an analysis of accidents and incidents provides organizations with the opportunity to evaluate the effectiveness of their mitigation strategies and to put in place new measures to further reduce the risks.

The definition of accidents and incidents and the conditions for their reporting depend on the particular state in which the accident occurs. The National Transportation Safety Board (NTSB) in the USA defines an unmanned aircraft accident as the following:

“an occurrence associated with the operation of any public or civil unmanned aircraft system that takes place between the time that the system is activated with the purpose of flight and the time that the system is deactivated at the conclusion of its mission, in which: (1) Any person suffers death or serious injury; or (2) The aircraft has a maximum gross take-off weight of 300 pounds or greater and sustains substantial damage.” p. 600, 49 CFR §830.2 (GPO 2010)

Mandatory reporting of accidents involving UAS in the USA only formally came into force in October 2010 [amendments to title 49 CFR 830 (GPO 2010)]. FAA accident and incident reporting requirements were in force prior to this date and were mandated under the conditions of a certificate of waiver or authorization (FAA 2011b). Annex 13 to the Chicago Convention was amended in November 2010 to include the investigation of accidents and serious incidents involving international civil UAS operations but only for those UAS with design and/or operational approval (ICAO 2011).

7.2 Triggers for Review

The occurrence of an accident or incident as a trigger for a review is a reactive approach to safety management. A proactive strategy does not wait for an accident or incident to occur in order to trigger a review of the SRMP. Reviews can be periodic or triggered by certain conditions (e.g., a change in operations, operating environment, regulations, applications, operational types, business activity). Identified risks need to be continually reviewed to ensure that the level of risk has not changed, that mitigations are still effective, that stakeholder expectations are still being satisfied, to determine if new options for risk mitigation are available, or to determine whether there is new information or tools available that can be used to improve the assessment of the risks. Reporting mechanisms should be established that allow the organization to identify and track emerging risks.

7.3 Tracking Safety Performance

Measuring and tracking the safety performance of an activity or organization is part of the overarching SMS or SPP established by the organization or NAA, respectively. In most cases, accidents are extremely rare events, and hence, a proactive safety performance management strategy is needed. Such a strategy attempts to estimate the safety performance through the use of a variety of safety performance indicators or measures of lead indicator events (e.g., recording and tracking the number of breaches in policies or procedures, issues detected as part of the preflight inspection of an aircraft, as opposed to counts of accidents).

8 Communication and Consultation

The risk communication and consultation process is described as the “continual and iterative processes that an organization conducts to provide, share or obtain information and to engage in dialogue with stakeholders regarding the management of risk” (ISO 2009). Communication and consultation is key to avoiding potential conflict in the safety decision-making process, for ensuring that stakeholder concerns are being addressed, and for reducing uncertainty in the decisions and outcomes. This process is undertaken at all stages of the SRMP. Key to addressing issues of trust and uncertainty is ensuring transparency in the SRMP to the different stakeholders. Both the outcomes from the SRMP and the SRMP itself need to be communicated to stakeholders. It is also important to note that the different stakeholders will have different information needs. The right information needs to be communicated to the right stakeholder and in a method and manner that is acceptable and comprehensible to them. Finally, communication and consultation is a bidirectional process. Eliciting domain knowledge from stakeholders can significantly improve the SRMP by reducing uncertainty and ensuring a more comprehensive management of the risks. Expert domain knowledge can be used at all stages in the SRMP (i.e., risk identification, analysis, evaluation, and treatment).

9 Conclusion

This chapter has highlighted many of the unique issues and challenges associated with the application of the safety risk management process to UAS. These issues and challenges can be technical, operational, economic, political, and social in nature and can influence all facets of the safety risk management process. Some sections of this chapter pose more questions than they do answers, highlighting that there is still much to be learned. The area of greatest need is in developing an understanding of the broader perceptions, beliefs, and expectations of society and how these factors influence decisions in relation to the safety of UAS operations. The challenges and issues discussed in this chapter are, in general, not unique to UAS. Challenges of a similar nature will need to be addressed in the safety risk management of other emerging aviation sectors such as reusable space launch vehicles, personal air vehicles, and hypersonic aircraft. It is hoped that the general processes developed and the lessons learned in the safety risk management of UAS will help to pave the way for these and other emerging and highly beneficial aviation sectors.

While this chapter has highlighted many issues, it is important to note that UAS are being safely operated in civil airspace today. In Australia, an approval to operate is obtained through the presentation of a suitable safety case to CASA, a safety case underpinned by a safety risk management process. Addressing the issues identified in this chapter will be pivotal to reducing the uncertainty in these safety cases, for ensuring consistency in the regulation of the industry, and for supporting the definition of more prescriptive safety regulations.