Flood Mapping from Multi-Sensor EO Data for Near Real-Time Infrastructure Impact Assessment: Lessons Learned from the 2017 Spring Flood in Eastern Canada

Abstract

Spring 2017 flooding in the Canadian provinces of Ontario and Quebec was caused by a number of consecutive record-setting rain events combined with melting snow from early April to mid-May. The event significantly damaged residential infrastructure by flooding approximately 2500 Quebec residences in 146 municipalities, forcing mass evacuations and declaration of a state of emergency. The International Charter on Space and Major Disasters was activated shortly after on May 6, providing near-real time earth observation data from a range of sensors through Charter member agencies. Upon activation, the Emergency Geomatics Services (EGS) at the Canada Centre for Mapping and Earth Observation (CCMEO), Natural Resources Canada (NRCan), produced flood maps from Canada’s RADARSAT-2 and Charter satellite imagery to provide up-to-date emergency situational awareness. Previous flood extraction methods that relied on manual thresholding of single bands were deemed suboptimal to map open water in a timely and efficient manner from all data received through the Charter. In addition, previous methods ignored flooding beneath vegetation, which produced a large underestimation of flood extents on vegetated floodplains. Work that was ongoing in the previous year to develop reliable surface water extraction methods from multiple sensors for floodplain characterization were quickly adapted and put into operations during the activation, enabling rapid flood map production from RADARSAT-2, Sentinel-1 and TerraSar-X, among other sensors. This chapter describes the methodology and presents successes, challenges and lessons learned from the 2017 EGS activation for flooding in Ontario and Quebec, Canada.

1 Introduction

The Emergency Geomatics Services (EGS) is a section within the Canada Centre for Mapping and Earth Observation, Natural Resources Canada, responsible for providing geospatial intelligence during natural disasters including flooding. EGS maps floods to help mitigate impacts on people and infrastructure by providing an overview of current flood extents that allow emergency responders including the military to prioritize and deploy help where it is needed most. Mitigation measures include shutting down power and gas transmission lines, erecting sandbag walls to protect buildings and other infrastructure, as well as mass evacuations to safeguard people from flooding.

On 23 April 2017 EGS was activated by Public Safety Canada for flooding caused by snowmelt and heavy precipitation in communities located on the Ottawa River between the Quebec–Ontario provincial border and Lake of Two Mountains, Canada. As record amounts of precipitation continued to fall into May 2017, the activation area expanded to include the Ottawa River from west of Ottawa to east of Montreal as far as Lac St-Pierre further downstream (Fig. 12.1).

Fig. 12.1

2017 flood activation area from west of Ottawa to Lac St-Pierre

At the outset of the 2017 flood, EGS was in the midst of changing its operations; from the way, it extracted surface water extents from satellite imagery to the incorporation of citizen geographic information to improve its flood map products. This chapter describes lessons learned amid this transition during EGSs’ 2017 springtime flood activation in Eastern Ontario and Western Quebec. Data received by EGS used to generate accurate and consistent near real-time flood information is listed, including satellite imagery, as well as airborne and ground-based reference data used to improve and validate flood maps. The transition that was taking place within EGS to update surface water mapping methods is described as this coincided with the 2017 activation. Due to the challenges faced by processing vast quantities of satellite imagery received during the event, EGS decided to adopt the new methods due to improved mapping efficiency and consistency. The rationale for this transition including advantages of the new methods are explained.

1.1 Background

In the year prior to the 2017 activation, EGS had made significant progress on automated surface water mapping to extract both open water and flooded vegetation extents from optical and radar satellite data. This work was ongoing within Public Safety’s floodplain characterization programme with the goal of generating useful information for EGS, including historical surface water maps to help precisely delineate floodplains as well as calibrate satellite flood models for operational mapping. This new methodology for automated flood mapping was demonstrated using historical optical satellite imagery from the Landsat mission obtained through the USGS in Olthof (2017).

This methodology was subsequently adapted to single and dual-polarization radar imagery from RADARSAT-1 and 2 and work began to generate historical surface water maps over the Saint John River floodplain to compare with optical data with the eventual goal of seamlessly integrating information derived from both sensors (Olthof and Tolszczuk-Leclerc 2018). The overarching objective of this work was to produce a robust, accurate, automated and efficient methodology to map surface water from a range of sensors for both operational and historical flood mapping with minimal user intervention. The large data volumes of satellite imagery that exist in historical archives necessitated the transition from manual methods to full automation, while efficiency is needed since EGS has set a goal to publish maps within 2–4 h of satellite data reception to provide current emergency situational awareness. The 2–4 h timeframe includes several time-consuming steps from data download, processing, quality control including editing when necessary, to map production and publishing.

At the request of Public Safety Canada, the International Charter on Space and Major Disasters was activated on 6 May 2017 for flooding in Southern Quebec. The Charter members consist of 17 space agencies from around the world who provide free imagery from 34 satellites to relief organizations working in areas affected by natural or manmade disasters. Upon activation, the methodology was applied to data from different sensors received through the Charter in addition to Canada’s own RADARSAT-2. The majority of data were from radar satellites, providing the ability to detect water through cloud during near continuous cloud cover that was present in the weeks of flooding. Fortunately, the methodology proved to be sufficiently reliable to be adapted to other sensors that include different radar wavelengths, polarizations and spatial resolutions, requiring specification of only three sensor-specific parameters to account for these differences.

1.2 Data

Satellite, airborne, ground-based and ancillary data layers were integrated to provide near-real-time flood maps that were quality checked on a best effort basis prior to release to government agencies and the public.

1.2.1 Satellite Imagery

EGS tasks Canada’s RADARSAT-2 satellite to image flood-prone regions through acquisition plans prior to the springtime flood season and gets priority tasking during an emergency activation. EGS tasks RADARSAT-2 dual-polarization imagery in standard or wide-mode to ensure sufficient coverage over the affected region. Through the Charter, EGS received both optical and radar imagery from a number of different satellite sensors with detail ranging from sub-metre to 250 m resolution and coverage from tens of square kilometres to thousands (Table 12.1).

Priority needed to be given to the best available scenes due to the volume of data received through the Charter. From April 23 to May 24, 112 scenes were evaluated for map production and publishing, the majority of which were discarded due to excessive cloud cover in optical, high wind in radar or some other quality issue. Of the 112 scenes, 41 were processed to some degree and 22 were fully processed and published as maps simultaneously on the Federal Geospatial Platform (FGP) used to disseminate flood products internally to government and Open Maps to the public.

Table 12.1 Satellite provided by the International Charter on Space and Major Disasters to the Emergency Geomatics Services during the 2017 flood.

1.2.2 NASP

From May 7–16, 2017, Transport Canada’s National Aerial Surveillance Program (NASP) aircraft were tasked to acquire oblique pictures of the flooding along approximately 600 km of the waterfront from Pembroke, Ontario eastward to Quebec City. Four separate flight zones were defined, with flights rotating through each flight zone over the 10-day period. Pictures captured every 5 s during flights totaled nearly 14,000 images taken during the 11-day period. These pictures were used to populate an online interactive web map through ArcGIS Online to distribute information and help improve and validate flood maps as they were generated from satellite imagery (Fig. 12.2).

Fig. 12.2

Open water and flooded vegetation mapped from RADARSAT-2 near Rigaud, Quebec, confirmed by NASP oblique airphotos

1.2.3 CGI

Prior to the activation, a beta version Citizen Geographic Information (CGI) application was released to select trusted users to acquire near real-time in-situ flood information. The application notifies users of a satellite overhead about to acquire an image, at which time the user is requested to take a geotagged picture of flooding and upload it to a central geodatabase for analysis. The image metadata also includes azimuth, enabling co-registration of features in CGI pictures with features in the satellite image. When available, this information is used to verify flood extents in satellite products and better tailor products by correcting obvious errors before release on the FGP and Open Maps.

1.2.4 LIDAR HRDEM

Canada’s new High-Resolution Digital Elevation Data Model (HRDEM) LIDAR-derived elevation data were made available to EGS prior to their official release to the public over portions of the flood-affected area where available. The complete dataset includes elevation, slope, aspect and shaded relief for a digital terrain model available at 2 m spatial resolution in 20 km × 20 km tiles with floating-point precision. Tiles covering urban areas where CGI data were acquired were used to improve flood mapping in locations where remote sensing has traditionally had difficulties mapping water due to obstructions including buildings.

1.2.5 Water Occurrence

A water occurrence map was obtained from a global product generated from 32 years of historical Landsat data by the European Commission (Pekel et al. 2016). Water occurrence is also referred to as inundation frequency because it maps the percentage that water has been observed at each location in satellite imagery. It is generated by overlaying a time-series of water maps and for each pixel location, counting the number of times water is detected divided by the total number of valid observations. These maps have been produced at a global scale to support applications including water resource management, climate modelling, biodiversity conservation, food security and flood response mapping.

2 Emergency Geomatics Services New Operational Flood Mapping Methods

2.1 Open Water Mapping

The new EGS flood mapping methodology performs multi-channel supervised machine learning classification of available radar polarizations to map open water, and then bright threshold region growing from open water to map flooded vegetation (Fig. 12.3). Smooth open water is a specular surface that produces a dark appearance in radar imagery caused by single bounce of incident radiation away from the sensor. Previously, EGS mapped open water using the traditionally accepted method of single polarization interactive dark image thresholding even when multiple polarizations were available (Bolanos et al. 2016; Li and Wang 2015). While thresholding has been shown to perform well for open water under ideal conditions, water surface roughness due to ice that is sometimes present during the spring flood season as well as waves can increase backscatter to a level where a single threshold value cannot reliably separate water from land (Henry et al. 2006). Automated methods used to determine optimal threshold values are compromised by these factors, while manual thresholding can better tune the value to minimize errors of omission and commission. Even still, a significant amount of post-processing is often required to reduce errors to an acceptable level (White et al. 2014).

Fig. 12.3

New methodology put into operations for the 2017 flood activation

Information contained in multiple radar polarizations can help to reduce errors in open water extraction; however making use of this information requires an approach other than single band image thresholding. The supervised multispectral classification has long been used in terrestrial remote sensing applications, but has seen limited use for surface water mapping from radar. One challenge is that supervised classification approaches require spectral signatures for each class to train a classifier, for example, signatures representing land and water in the case of water extraction. The use of standard spectra to classify land and water will not achieve an optimal classification result for several reasons. First, the spectral variability of water in radar due to wind and ice can cause confusion between water and land. Second, spectral signatures of land also vary due to the presence of several land cover types that change through time because of vegetation phenology, moisture and atmosphere. An additional complicating factor is that in order to perform traditional multispectral classification such as minimum distance or maximum likelihood, separate signatures must be sampled for all land cover types present in the scene.

To deal with these limitations, an automated open surface water extraction methodology that is an extension of one already developed in Olthof (2017) was implemented. The approach makes use of recently available inundation frequency products from historical Landsat data (Pekel et al. 2016; Olthof 2017) to sample all scene-specific signatures representing every land and water class present in the image. Land signatures are extracted where inundation has never occurred (0% occurrence) based on historical inundation maps, while water signatures are extracted where water has always been observed (100% occurrence). Sampled, scene-specific signatures are used to train the C50 (See5) decision tree machine learning algorithm that classifies the entire image into the water versus land. Machine learning is used because it does not assume a statistical distribution and can handle vast amounts of training data to precisely tune the classifier.

2.2 Flooded Vegetation Mapping

A significant amount of flooding in Canada occurs in vegetated areas, which has lead to considerable research to improve flooded vegetation mapping in recent years. Flooded vegetation presents a challenge to remote sensing because the signal must penetrate the vegetation layer to sense water below. While optical remote sensing has a limited capability to detect flooding beneath vegetation and can do so reliably only during leaf-off in the early spring or late fall, radar is able to detect water beneath leaf-on canopies under many conditions. Radar is an active sensor that is side-looking, causing double bounce of the incident beam first off horizontal water surfaces beneath the canopy, and second off vertical trunks and stems acting as corner reflectors before returning to the sensor (Hess et al. 1990). Leaf size, wavelength and incidence angle are all factors that affect water detection beneath vegetation from radar, with longer wavelengths relative to leaf size (Townsend and Walsh 1998) and shallower incidence angles (White et al. 2015) generally providing greater signal penetration through the canopy.

Double bounce off flooded vegetation causes a high-intensity return to the sensor, producing a bright signal in radar imagery particularly in like-polarization channels (e.g. HH). Bright target thresholding has been used in the past to map flooded vegetation; however, this approach was never operationalized by EGS because it leads to significant commission error caused by the presence of other corner reflectors such as buildings and rock outcrops in the scene. Post-classification editing in a GIS environment can be performed to help reduce commission error, but this relies on ancillary data layers that may be out of date or contain errors. Manual editing has also been done to improve flood products but is both time-consuming and subjective in the absence of reliable ancillary information. A robust flooded vegetation extraction methodology independent of ancillary data requirements is preferred for emergency flood mapping for product quality, improved processing speed and efficiency.

To minimize processing time and commission error, the new methodology maps flooded vegetation by iteratively region growing from open water into adjacent bright intensity areas characteristic of double bounce using a bright threshold value criterion. Region growing has the advantage over thresholding the entire image by assigning only adjacent pixels to the flooded vegetation class, thereby reducing commission errors caused by double bounce elsewhere in the scene. Double bounce off buildings adjacent to open water was corrected using the urban class from a recent Landsat land cover of North America (Latifovic et al. 2017), which was generated including information from Canada’s road network layer. Once the first region growing using a bright threshold value is complete, a second region growing is performed using a dark threshold value from open water and flooded vegetation to help infill dark areas contained within bright flooded vegetation and better connect nearby areas of open water. Both instances of region growing stop when either no additional adjacent pixels meet the threshold criterion or after a set number of iterations. Open water and flooded vegetation are subsequently improved using morphological operators to infill small land areas entirely surrounded by water, followed by filtering and sieving to reduce both errors of omission and commission.

Because the methodology relies on signatures extracted from each scene to classify open water, and only user specification of radar polarization, bright and dark threshold values are needed to map flooded vegetation, the approach is sensor-independent. As new charter data was received during the activation, tools were modified to accept the number of polarizations available to work on single polarization (e.g. TerraSAR-X), dual-polarization (e.g. Sentinel-1) and quad-polarization (e.g. RADARSAT-2) (Fig. 12.4). Where multiple polarizations are available, specification of the polarization to use for flooded vegetation region growing must be specified. Generally, the polarization with the best contrast between flooded vegetation and surrounding water and land is selected. Where available, like-polarization provide the best contrast, for example, HH in RADARSAT-2.

Fig. 12.4

Example of multi-sensor flood mapping using the new operational methodology from RADARSAT-2 (interior white box), and TerraSar-X (outside), showing the consistency in mapping open water and flooded vegetation between sensors

2.3 Urban Flood Mapping

A major shortcoming of the approach applied to radar data was its inability to accurately map open water in urban environments . The presence of buildings causing shadow made dark water target detection impossible, while confidence that any detected dark targets were water was low. Further, urban areas located on the water were often mapped as flooded vegetation due to the presence of corner reflectors such as road and buildings producing a high-intensity return. Post-classification improvements were made with land cover data to remove flooded vegetation in urban settings; however, this also removed any real water that was detected.

As Citizen Geographic Information (CGI) data submitted during the event became available during the activation, tests began on using it in combination with newly available LIDAR HRDEM data to improve urban flood mapping. A total of 394 observations were submitted across three flood-affected regions including Lac St-Pierre, Montreal and Lake of Two Mountains, and Ottawa-Gatineau between March 29 and April 14, 2017. The use of LIDAR data for flood mapping requires a local water depth measurement, which can be estimated from CGI data assuming the observer is taking pictures at or near the flood perimeter and can be verified by looking at the pictures themselves. Once an elevation along the flood perimeter has been established from CGI, the full local flood extent can be mapped by filling adjacent pixels below the established flood elevation at the flood perimeter, as shown in Fig. 12.5 using 80 observations submitted in Gatineau 6 April 2017. The extent to which infilling can be done varies with the degree of local variation in the geoide. For example, if a single elevation threshold were applied to a LIDAR DEM from Ottawa to Montreal, flooding would be under predicted in Ottawa and simultaneously over predicted in Montreal due to the slope of the geoide from Ottawa to Montreal that causes water to flow in that direction. While the extent of urban flooding is considerably less than the distance between Ottawa and Montreal, CGI data acquired in Montreal’s West Island should not be used to estimate urban flooding in the East End. Therefore, local CGI data can be used to estimate local residential/urban flooding from inland to the nearest adjacent permanent water body, but not between separate, distant locations.

Fig. 12.5

Use of Crowdsourced Geographic Information (CGI) data acquired on 6 April 2017, and a new LIDAR DEM to improve urban flood mapping in Gatineau, Quebec, combined with a RadarSat-2 flood product from 7 April 2017

2.3.1 Limitations and Lessons Learned

Currently available global historical dynamic surface water maps used to train machine learning have known limitations due to input optical Landsat imagery that is the only medium resolution data available with its spatial and temporal coverage. First, optical data does not penetrate cloud and therefore, maximum water extents during cloudy, rainy events including a significant percentage of floods such as the 2017 event are not fully captured. In addition, optical remote sensing has a limited capability to detect water beneath vegetation, while flooded vegetation comprises a high percentage of the overall flood area on many floodplains in Canada. Lastly, because the product is global in scale, it is not optimized to Canada’s specific geography or seasonality. All of these factors generally lead to water omission errors. Nevertheless, the new flood mapping methodology is robust enough to overcome these limitations and Pekel’s occurrence map proved very useful during the 2017 activation.

The availability of historical radar imagery from Canada’s RADARSAT mission going back to 2008 combined with the advantages cloud penetration and the ability to map flooded vegetation suggest that separate dynamic surface water maps should be generated from these data over Canada. While current global products contain error, machine learning used to classify open water is known to be very robust to errors in training data (Ghosh et al. 2016; Olthof and Tolszczuk-Leclerc 2018). Therefore, even if global maps omit the water, errors in training data extracted using these maps should be relatively small compared to the overall training sample size. The activation season was also considered successful using a global historical map product as input, further supporting the use of Pekel’s map. However, the more extreme the flood event, the more errors will be introduced when relying on a product that omits water in favour of land. Currently, EGS has no sense of the level of improvement that can be achieved with better historical information. For this reason, work will continue to generate Canadian products tuned to our geography and seasonality from historical optical and radar imagery over select floodplains in Canada. With support, the historical RADARSAT archive can be exploited for this purpose as is ongoing along the Saint John River, NB. Finally, the complete historical and operational surface water mapping approach is iterative as current flood products are used to update and improve historical inundation products, which will then in turn be used to produce flood products next season. As scenes are added to the time-series of water maps, both historical and near-real time water maps should continue to improve.

In addition to falsely detecting flooded vegetation by region growing in urban areas adjacent to open water, a second limitation of the methodology relates to the local topography of riverbanks. Where riverbanks are steep and roughly parallel to the SAR orbit, flooded vegetation will also be falsely detected due to a high backscatter caused by steep local incidence angles. Using ancillary datasets such as slope and aspect based on look direction can help reduce these errors by removing flooded vegetation on slopes above a certain grade. Work is still needed on the best approach to integrate digital elevation data to improve flood maps, particularly in areas of flooded vegetation. Factors such as the quality and resolution of the DEM in relation to the resolution of satellite imagery and decisions on whether to incorporate the DEM directly into region growing, to use it to create a processing mask of flat areas where surface water can pool, or to use it in post-processing still need to be considered.

Editing of flood maps is still required to remove either false detections or those that are not of interest for public safety. However, false detections appear to be relatively rare, and those detections that are not of interest to public safety could be relevant to other potential users. Enhancing flood products with ancillary information to target these users may be an avenue worth pursuing to make flood maps of value to a wider audience. For example, flooding is consistently detected in agricultural fields, while visual observations confirm that this is a common occurrence during the flood season. These detections are ephemeral and are currently being removed from flood maps, but they may be relevant to agricultural production as tilling and seeding are delayed by standing water in fields. Agriculture Canada and crop insurance companies may be interested from a crop inventory perspective to know if crops will be planted late, how much and where. The intersection between flood maps and land cover may be sufficient to characterize different types of flooding in terms of longevity and impacts on different stakeholders.

EGS is generally most active in the response phase of the emergency management cycle that includes mitigation, preparation, response, recovery. However, a request was made to generate maximum 2017 flood extent polygons over the activation region to help evaluate flood risk. Due to the nature of emergency mapping, accuracy assessments are performed on a best effort basis within the 4-h period that the EGS has to generate and distribute its flood extent polygons. Nonetheless, accurate maximum flood extent maps are considered a valuable product to the emergency management and land planning communities during the recovery phase. EGS undertook work to generate a maximum flood extent product that spans all of the affected areas that were mapped at least once by our services. To achieve this, the highest resolution products generated closest to the timing of peak flood were selected, reviewed using all available datasets and merged, including urban flood products in Gatineau and along Pierrefonds Boulevard on the Island of Montreal generated from CGI and the LIDAR DEM. Each product was initially assessed separately against very high-resolution satellite imagery (Worldview, Pleiades) medium resolution optical satellite imagery (Sentinel-2, Landsat), crowdsourced field observation (photography and surveys) and oblique aerial photography acquired by NASP before publishing. Each flood extent polygon was edited and its accuracy improved where errors were noted compared to reference data.

A more formal accuracy assessment of the final maximum flood extent product was subsequently conducted using visually interpreted NASP oblique airphotos as a reference. Interpretation was performed by putting placemarks in Google Earth at locations interpreted as either flood or land at the time of the NASP photo acquisition. Because assigning points in open water under normal non-flood conditions would artificially inflate classification accuracy, reference points were located near the flood margin with flooded points interpreted as locations that were inundated in NASP photos but were interpreted as land in Google Earth during baseline, non-flood conditions. Land points were interpreted just beyond the flood margin as permanent land before and during the flood event. A total of 793 points were collected in this manner from a subset of 1475 photos acquired from west of Ottawa to east of Lac St-Pierre from 7 May 2017 to 16 May 2017. The maximum flood extent is a mosaic of different flood products acquired on different dates that may not correspond precisely to the acquisition dates of co-located NASP. Consecutive dates checked in many locations showed consistent flood margins, suggesting differences in the timing of a few days between NASP and satellite acquisitions likely had a minimal effect on overall accuracy but may have led to error in some cases where flood margins changed significantly over a short period of time. Despite these potential issues, an overall accuracy of 86.2% was obtained for this product using the assessment methodology outlined above. The result is a validated complete and uniform maximum flood extent product shown in Fig. 12.6.

Fig. 12.6

Maximum flood extent of the activation area generated by combining flood maps of different sub-regions

3 Conclusions

The 2017 EGS flood activation was considered a success by clients, stakeholders and management at CCMEO. EGS operations adopted new methods to extract open water and flooded vegetation from multiple sensors that were developed the previous year on historical satellite data, which proved to be reliable and robust on a range of sensor data received through the International Disasters Charter. Feedback received from Public Safety Quebec was extremely positive, stating that new products were significantly improved over ones previously generated by EGS during the 2011 Richelieu River activation, particularly in areas of flooded vegetation that were mapped operationally for the first time. Rather than relying on global inundation products from optical data, work on historical inundation frequency integrating radar data should continue for floodplain characterization to better tailor products to the Canadian context and assess the level of optimization that is still possible. Finally, EGS envisions enhancing our flood products through ancillary data integration to further improve them and to make them more relevant to a wider set of users and stakeholders.