Keywords

1 Introduction

Sensornet is the Internet of Things Platform of the Emilia-Romagna Region, collecting data and information from thousands of objects distributed across the territory, building in time a digital map of the reality we live in (Fig. 1).

Fig. 1.
figure 1

Sensornet platform [1].

Full size image

Whether they are generated by inclinometers for landslide monitoring, rain gauges or inductive coils for the traffic monitoring, data generated by the sensors define a snapshot of a reality made of continuously updating information, allowing a better knowledge of what is happening in cities and in territories [2].

Usual rain gauges, like the ones integrated on Sensornet platform, are not so effective during intense precipitation since their operating principle is based on mechanical tilting parts, which makes their measurements unreliable during this type of phenomena. Besides, their measurements provide limited spatial resolution, since they only represents the immediate proximity of the measurement spot, as well as limited temporal resolution.

Lastly, given the rapidity of this type of phenomena evolution, traditional real-time monitoring systems are not sufficient for an anticipated recognition of critical scenarios.

For the purpose of dealing with heavy rain events and with the possible consequent flooding events, that are affecting not only Italy, but all regions in Europe in the recent years [3], it is therefore necessary to update and integrate the current hydropluviometric monitoring infrastructure both through the use of innovative technologies, able to exploit the pervasiveness of cellular networks, and through the use of hydrological simulation models, able to foresee in advance water rivers level status.

The first part of the article describes the purpose of the project starting from the analysis of the environmental problems from which it was born (hydrogeological risk due to extreme rainfall) (Fig. 2), and the description of the currently available methodologies for their mitigation.

Fig. 2.
figure 2

Parma flash-flood October 2014 [1].

Full size image

The second part illustrates the innovative use of the beforementioned instruments within Sensornet platform, which derives both from their novel and unitary implementation and from their integrated and synergic use.

The last part summarizes the results, giving evidence of the achievement of the intended purpose.

2 The State of the Art

The microwave rainfall monitoring system is an innovative, but already tested technology [7], exploiting the microwave links used in commercial cellular communication networks.

Radio signals propagate across microwave links from the transmitting antenna of a base station to the receiving antenna of another base station.

Rain-induced attenuation and, subsequently, path-averaged rainfall intensity can be retrieved from the signal’s attenuation between transmitter and receiver by applying, almost in real-time, a rainfall retrieval algorithm.

A distributed monitoring system can be developed by using received signal level data from the massive number of microwaves links used worldwide in commercial cellular communication networks.

Many studies are currently addressing, not only the problem of monitoring critical environmental phenomena in real-time, but also that of predicting them.

Some of them are not based on forecasting system, but rather on real-time monitoring systems [8] or on certain percutaneous parameters of the phenomena [10].

In other cases, the forecasting system is based on the integration of the predictive algorithm in physical sensor networks and is strictly bound to all the problems and limitations associated with this type of solution [9]. In other cases, the predictability aspect of the phenomenon is only mentioned as one among many others involving in the management of critical events [11].

The preparation and reaction to such disruptive phenomena can increase the resilience of the territory in short time as early warning of hazardous conditions and in medium term as territorial planning and preparation to emergency response.

The management of extreme rain events can not, therefore, solely rely on traditional real time monitoring systems, but must also include the use of new and distributed technologies, like microwaves one, and new forecasting systems based on hydrological simulation models and meteorological modeling.

3 RainBO Life

The analysis of climate variability over the municipality of Bologna, as resulted from the BlueAp LIFE project (Bologna Local Urban Environment Adaptation Plan for a Resilient City 2012–2015), reveals important changes observed in the main climatological variables.

During the last two decades, years with intense precipitation have been frequently registered in Bologna, having an important impact on the city and its citizens.

The quantity of precipitation shows a slightly negative trend during winter, spring, and summer and a positive trend during autumn, over the period 1951–2011.

With regards to seasonal extreme of precipitations, the dry days index presents a positive tendency over 1951–2011 period, more intense during summer.

Analysis performed on intense precipitation time series evidence a slightly positive trend of the frequency of days with intense precipitation (based on 90th percentile as a threshold) in all season, except on spring.

The flooding risk of small water courses is a major problem in several urban areas (especially in Italy): the constant growth of urbanization, with the consequent decrease of soil permeability and loss of space for river and stream beds, is leading to increased flood hazard and vulnerability; in such conditions, severe rainfall events over steep catchments of limited area can produce dramatic consequences; in addition, ongoing climate changes are likely to increase the occurrence of severe precipitation events, thus increasing flash flood hazard.

Historical and recent records report that the urban areas of Bologna located beneath the highland are prone to severe flood events caused by small water courses. The most severe event occurred in 1932, when rainfall of 134 mm within a few hours caused flooding of a large urban area, including a portion of the Ravone catchment area.

Another severe flood event occurred in the Bologna area in 1955, while in 2002 a further flood event affected several small municipalities nearby. In all of these cases, the recorded hourly peak intensity exceeded 50 mm/h.

Despite its relevance, the risk of flooding of small water courses in urban areas is often underestimated and few measures are taken for prevention and mitigation [4].

The high level objective of RainBO LIFE project (2016–2019), that is a follow-up of BlueApp one, is the improvement of knowledge, methods and tools for the characterisation and forecast of heavy rains potential impact due to the hydrological response, not only of medium and large basin, but also of the small ones and for the evaluation of the vulnerability of assets in the urban areas.

RainBO project aims to achieve its goal developing an advanced infrastructure based both on traditional monitoring systems and on innovative ones, such as cellular communication network monitoring, and implementing an early warning system based on hydrological models driven by fileds of precipitation for the forecasting of flash floods in small and large basins.

Since RainBO monitoring system is based on Sensornet platform, its goals have been achieved through the incremental integration, within the platform, of new “virtual” sensors associated to estimated data calculated from the attenuation of microwaves links and to forecast data from hydrological simulation models and meteorological modelling.

4 The Microwave Rainfall Monitoring Network

The microwave rainfall monitoring network exploits the Microwave links (MWL) used worldwide in commercial cellular communication networks. Along such links, radio signals propagate from a transmitting antenna at one base station to a receiving antenna at another base station. Rain-induced attenuation and, subsequently, path-averaged rainfall intensity can be retrieved from the signal’s attenuation between transmitter and receiver by applying, almost in real-time, a rainfall retrieval algorithm.

The algorithm chosen for this purpose is the RAINLINK retrieval algorithm developed from Aart Overeem and Remko Uijlenhoet from Wageningen University (Wageningen, the Netherlands) and Hidde Leijnse from the Royal Netherlands Meteorological Institute (KNMI) in 2015. The code is an open source R package (https://www.r-project.org) available for free download on GitHub (https://github.com/overeem11/RAINLINK).

The implementation of the algorithm for the specific needs of the Italian test areas, as well as code debugging and improving, is done by MEEO, one of the partner of RainBO LIFE project [13], in close contact with the original authors.

The great advantage of this technology is related to the high resolution (15 min and 1 KM2) of 2D rainfall maps to monitor and forecast rainfall events even on geographical area not covered by traditional gauges network but equipped with mobile telecommunication antennas.

RainBO adopted as rainfall estimation method the RAINLINK algorithm, which is already suited for a midlatitude flat land like the Netherlands and is easily portable to the Pianura Padana. The algorithm capabilities over the hilly areas of the Appennino have been investigated and results obtained are encouraging.

The limit of this technology is substantially related to the data availability from the telecommunication operators.

So far, microwave links data for RainBO project are provided from Lepida and Vodafone telecommunication operators. The data consists in near real time receiving power readings, collected at time intervals of one (Lepida) or fifteen (Vodafone) minutes. The algorithm works with 15-min maximum and minimum received powers, from which signal attenuation are calculated and, through the link’s pathlength information, the instantaneous path-averaged rainfall depths are estimated.

Data are representative of the link between two points in space, as the first internal output of the algorithm is the rainfall estimation on the midpoint between the transmitter/receiver, derived from the power attenuation data measured on the antennas.

The ordinary kriging interpolation is applied on these middles points of the links on a 1 km by 1 km grid, taking into account the other values and obtaining more reliable map of rainfall data, on a 5 km by 5 km.

High density network coverage is needed for best performances. The interpolation grid adopted by RainBO project is the one already in use at Arpae for the meteorological interpolated maps and for the limited area models.

This choice was done to simplify the validation of microwaves links data through many comparisons with data, radar and ERG5 (interpolated rain gauges) ones, by carrying out qualitative and statistical analysis. The deliverable C2_5 of RainBO project describes this activity and its results in detail.

The Algorithm evaluates the signal attenuation along the link and assigns the related rainfall intensity to its average point that we call, in this project, “Microwaves Virtual Sensors”. In particular “Microwaves Virtual Sensors” map published in RainBO are based on data provided by Lepida. Rain gauge data provided by Vodafone cannot be published as project output for data confidentiality, but they were fundamental for the comparison between the microwaves maps and those obtained from the traditional systems, radar and ERG5, for the purposes of the algorithm validation process.

5 Hydrological Models

5.1 Medium and Large Basins: Random Forest Method

Following the flooding of the Baganza river in Parma on October 2014 (Fig. 2), caused by heavy rains, which flooded several neighborhoods southwest of the city, the Civil Protection Agency of the Emilia-Romagna region required ArpaE the development of a hydrological simulation model to be able to recognize in advance the probability of overcoming the three alert thresholds fixed for the main rivers of Emilia-Romagna region: Warning (threshold 1), Pre-alarm (threshold 2), Alarm (threshold 3).

Hydrological modeling for medium and large basins is based on a statistical method, Random Forest, which uses decision trees.

The Random Forest model, applied to hydraulic modeling, provides the probability of overcoming the alert thresholds of some observation point of the medium and large basins, for the next 6–8 h, depending on the dynamics of the river.

In particular, the Random Forest hydrological model gives the following forecast data:

  1. 1.

    Probability of not exceeding threshold 1

  2. 2.

    Probability of exceeding threshold 1

  3. 3.

    Probability of exceeding threshold 2

  4. 4.

    Probability of exceeding threshold 3

5.2 Small Basins: Criteria 3D Model

Forecasting models of heavy rainfall initiating flash flood from small basins are different from other models (e.g. from large basins or waterways).

The size of the small basin results in very rapid response times to heavy rainfall.

In other words, the time interval between the start time of the precipitation and the span peak can be reached in less than two hours, which in reality would make the prediction of the event very difficult and therefore the alert system.

For this reason, it is considered essential to develop a hydraulic simulation model for small basins, and the installation of specific measuring points, allowing hydrometric observations to its validation.

Criteria3D is a three-dimensional hydrological model, which also simulates water infiltration into the soil, developed by ArpaE-SIMC of Emilia-Romagna region.

The model was developed starting from the study of the river Ravone, which is one of the creek that from the hills south of Bologna goes down to the city (Fig. 3).

Fig. 3.
figure 3

Ravone creek [1].

Full size image

In this basin, all those critical and valuable factors that are present in the hilly waters, such as the effects of the strong anthropization that currently characterizes the end of the valley and the crossing of the city, are also present (Fig. 4).

Fig. 4.
figure 4

Ravone’s suture and measure point [1].

Full size image

Figure 5 shows the good result of the test of simulation of the water level at the stream gauge of Ravone in the event of 2015-02-05.

Fig. 5.
figure 5

Criteria3D simulation of the water level at the stream gauge of Ravone in the event of 2015-02-05 [1].

Full size image

6 Meteorological Modeling Cosmo-Lami

The numerical meteorological model limited area Cosmo-Lami [5], called Lami for brevity, carried out in consortium between national civil protection department, USAM Air Force, Harp ArpaE Piemonte and ArpaE Emilia-Romagna, provides numerical forecasts with a spatial resolution at 7 km and 2.8 km and temporal validity respectively three and two days.

Predictions based on this model are carried out twice a day, at 00 and at 12 UTC, at the supercomputing Cineca center, in accordance to a contract with the IdroMeteoClima Service and the Department of National Civil Protection.

Data are provided in grid format, GRIB [6], and each file contains forecast data for various meteorological parameters, including precipitation, either at the surface or close to it, at an hourly or three-hourly in dependence on the parameter.

The data relating to the meteorological modeling can be used both at the level of maps to have, for example, an overview of the precipitation forecast, but also at the numerical level to have, for example, the detail on hourly precipitation provided on a given grid cell, as in the case of Ravone creek, which constitutes the basic information for the prediction of the hydrometric level corresponding starting from the product simulations scenarios resulted from the 3D hydrological model for small basins.

Data from meteorological modelling, limited to the Emilia-Romagna region, are distributed GRIB format on the open date platform of ArpaE of Emilia-Romagna.

7 Advanced Environmental Monitoring and Early-Warning System Integration

The implementation of an advanced infrastructure for environmental monitoring and of an early-warning system is based on incremental integration within Sensornet platform of new “virtual sensors”, using the same data model initially defined for physical ones.

Unlike real sensors, virtual ones are not associated with observed measurements, but with estimated data starting from other measures observed related to them, as in the case of rain estimates through the attenuation of radio signals, or starting from simulation models hydrological systems, which provide a future estimate of the river level at the observation point.

The concept of “virtual sensor” allows, indeed, to integrate information from non-traditional systems, georeferencing them with respect to the same reference system and synchronizing them over time.

The integration of these new “virtual sensors” into the Sensornet platform has been accomplished in a simple and immediate way, using the same data model defined for the physical sensors, without the need for any extension or specialization and providing the platform with an enhanced monitoring infrastrucuture and a new feature crucial for recognition and generation of early-warning reports.

The integration of sensors data coming from different monitoring systems in Sensornet is realized through a federated approach whose main advantage is to preserve the investments made on already existing systems and to protect the technical, technological and organizational autonomy of the individual systems and of their owners.

The architecture implemented in Sensornet platform provides an interconnection middleware between the different data sources and the central system, acting as a data collector from different sources and a data normalizer towards the central system, as shown in Fig. 6.

Fig. 6.
figure 6

Flow of collection, standardization, storage and access of the data [1].

Full size image

It consists of a series of atomic modules for data retrieval from individual sources and of their manager, which oversees their activation and coordination.

Each module contains the access rules and the required commands for retrieving data from a specific source or database and for storing them in a standard format on the centralized database.

In order to acquire data from heterogeneous sources and use them in a contextual and correlated mode, a standardization process is necessary.

The creation of a standardized data stream is one of the added values offered by Sensornet platform, which transforms the data from the different sources into a single standard format, regardless of the technologies, the interfaces, formats and data type of the sources [2].

The integration of data form cellulars communication networks and the hydrological simulation models and meteorological modelling data in Sensornet has been achieved through the definition of “virtual” sensors, which, unlike the real, are not associated with physically measured data, but to the forecasted ones provided by the models.

This type of solution allowed to completely integrate these new types of “virtual” sensors with the real ones, while maintaining the consistency of data and their modeling within the Sensornet platform.

7.1 Microwave Rainfall Monitoring Network Integration

In addition to the data coming from the conventional real-time monitoring system of ArpaE, mainly consisting of regional hydrometers stations to measure temperature, rain and hydrometric levels, Sensornet platform also integrates those belonging to Microwaves Rainfall Monitoring Network.

The new “virtual Microwaves sensor” has been defined in Sensornet platform to allow the integration of data related to microwaves links, as well as a new “Microwave” module data acquisition.

Microwave data supplied by Lepida, cover the Apennini region of the Emilia Romagna. At present time, data corresponding to the metropolitan areas of Bologna and Parma (i.e. the test site of RainBO project) have been integrated, for a total of 53 sensors.

The integration of all Lepida Wireless Network Links, corresponding to a hundred additional sensors, is expected by the end of this year.

Figure 7 shows the “virtual Microwaves sensors” currently integrated on Sensornet Platform.

Fig. 7.
figure 7

Microwaves virtual sensors integrated in sensornet platform.

Full size image

7.2 Water Hold Capacity Data Integration

The Water Hold Capacity index provides the maximum amount of water the soil can hold, given the current conditions of soil moisture.

It is one of the input parameters to identify the hydrological scenario for small basins. The WHC index for the Ravone creek test case is daily estimated through a mono-dimensional restriction of the Criteria3D water balance model and it is calculated on the first 35 cm of soil. It becomes zero when the soil is above the field capacity and is no longer able to retain water. To compute this index, the model uses as input soil, crop and weather data (temperature and precipitation) that Arpae collects for all the Emilia-Romagna region.

The use of an estimated value of soil moisture instead of a measured one has the advantage that it can be estimated on all the similar sites of the region and it is not affected by lack or failures of sensors nor by local peculiarity

As shown in the graph relating to Fig. 8 the soil drainage capacity is higher in summer than in autumn or winter (when rain is more frequent). The water status of the basin is updated every 24 h

Fig. 8.
figure 8

WHC virtual sensor graphic.

Full size image

7.3 Forecast Data Integration

Sensornet early-warning system integration is based on the integration of forecast data into the platform through the configuration of new virtual sensors based on hydrological simulation models and meteorological modelling.

7.3.1 Lami Virtual Sensor Integration

The new “virtual Lami sensor” has been defined in Sensornet platform to allow the integration of the GRIB data related to the modeling of the weather, needed for the 3D hydrological simulation model for small streams. In the case of the Ravone stream, as in most small streams, the size of the basin is contained in a single cell of the reference grid, whose data can then be represented by a sensor placed within the cell itself.

From the datum for cumulative rain, collected from GRIB data at the beginning of each run for each cell, it is possible to calculate the corresponding precipitation per hour. Once a precipitation threshold has been established, it is possible to determine when a precipitation starts and when it finishes, to infer the duration as well as the peak and the accumulated of the corresponding event.

The algorithm to calculate the significant parameters of a rainy event, starting from the data for the cumulative hourly precipitation, can be described as follows:

  • starting from the GRIB data (the cumulative hourly precipitation from the beginning of the run), the hourly precipitation is extracted with simple subtractions;

  • the hours in which rain is expected and when it is not (0–1) are calculated according to the established threshold (normally 0,2 mm);

  • depending on the distribution of 0 and 1, it is estimated when a rainy event starts and when it ends;

  • at this point it is possible to calculate the duration, the accumulated (constitutes from all the hourly precipitation included in the event) and the maximum intensity relative to the event (peak).

The storage of the main parameters of a rainfall event, calculated as it has been described, is made by defining four corresponding measures associated with each “virtual” Lami sensor:

  • hourly precipitations

  • event

  • cumulative hourly precipitations

  • peak

The integration of the new type of Lami virtual sensor inside the Sensornet platform, indeed, required the implementation of a new GRIB data acquisition and processing module, the definition of a new type of Lami sensor, to which the four measures previously described are associated, and the configuration of a new Lami type sensor at the Ravone stream, identified by the coordinates of the corresponding grid cell.

The integration of other sensors related to the meteorological forecasts at another stream, simply requires the configuration of another Lami type virtual sensor associated with the coordinates of the corresponding grid cell.

Currently, eight more virtual Lami sensors have been integrated in correspondence of the cells adjacent to those of the Ravone basin, to take into account rain forecasts around the basin.

Figure 9 shows an example of graphing data for the Lami sensor defined for the Ravone cell (lat 44.46 and lon 11.31) provided at 00:00 on 30/06/2017 and valid for 72 h later.

Fig. 9.
figure 9

Example of graphic data of Lami virtual sensor [1].

Full size image

In the example shown, the expected rainfall is below the defined threshold (normally 0,2 mm) and is therefore not a source of rain for a rainfall event.

7.3.2 Random Forest Virtual Sensor Integration

The new “virtual” Random Forest sensor has been defined to allow the integration the data related to hydrological modeling for medium and large basins.

The integration of the new type of Random Forest virtual sensor inside the Sensornet platform required the implementation of a new data acquisition and processing module of the .csv files (provided by ArpaE every 10 min on a ftpServer, hosted by LepidaSpa), the definition of a new type of Random Forest sensor, to which the previously described four probabilities are associated, and the configuration of two new Random Forest type sensors in correspondence of Parma and Baganza rivers, identified by the station ID of the corresponding hydrometer.

The integration of other data related to hydrological modeling for medium and large basins, simply requires the configuration of others Random Forest type virtual sensor associated to corresponding .csv files.

Figure 10 shows a graphical representation data for the Random Forest sensor defined for the Baganza river provided at 14:40 of 18/07/2017 and valid for 6 h later.

Fig. 10.
figure 10

Example of virtual sensor Random Forest data graphication [1].

Full size image

In the example, the probability of the river level of not exceeding threshold 1 in the point of observation is equal to one, while the probability of exceeding threshold 1, threshold 2, and threshold 3 is equal to zero.

7.3.3 Criteria 3D Virtual Sensor Integration

The predictions of the hydrometric level of the Ravone stream, are based on the following three sources of information:

  • observed weather data (temperatures and precipitation) that feed a water balance model (Water Hold Capacity)

  • forecast rain in the next 72 h, from Lami meteorological modelling

  • hydrometric level of Ravone in the observed point during calibration and validation

The forecasting process includes the following phases:

  • The water status of the basin (WHC) is updated every 24 h

  • Every 12 h, the platform receives the LAMI +72 predictions on the cell in which the basin falls (and the 8 neighbouring cells); precipitation events are identified and in particular the cumulative and maximum hourly intensity.

  • The predicted precipitation values are updated based on the data observed by the Paderno rain gauge.

  • To take account of any LAMI underestimation on the maximum intensity, maximum intensities increased by + 33% are also considered possible.

  • These values are passed to the estimation equation of the maximum drainage waterway producing a statistical distribution of the forecast

The algorithm must be performed on the 9 cells (the one referring to the Ravone basin and the eight adjacent ones) and must be performed both for the peak value of the event estimated for each cell, from Lami Virtual Sensor, and for the same values increased by 30%, for take account of possible underestimation.

The output algorithm is a statistical distribution of the possible 18 water levels, associated with the peak time of the expected event. In particular, the algorithm outputs the 5 percentile values: 5°, 25°, 50°, 75° and 95°, in addition to the time when the peak is expected.

Figure 11 show an example of graphic output.

Fig. 11.
figure 11

Example of graphic data of Criteria 3D VirtualSensor.

Full size image

8 Results

The use of “virtual sensors” and “microwave links” technology allowed the integration into Sensornet of dozens of measurement points, without the addition of any more infrastructure compared to those already existing in the territory.

The geographical and temporal integration of the “virtual sensors” with the real ones, allows their overlap and comparison.

Figure 12, for example, shows the correlation between rain gauges and microwaves estimates.

Fig. 12.
figure 12

Example of correlation between Raingauge and MW-Links estimates values.

Full size image

Figure 13 shows the correlation between soil moisture sensor and WHC index. In particular, it can be noted the ratio of inverse proportionality of the two parameters and the perfect correspondence between their derivatives.

Fig. 13.
figure 13

Example of correlation between soil moisture sensor and WHC index.

Full size image

With regards to hydraulic risk, the possibility of early detection of extreme precipitation events and their effects on the river level allows to recognize in advance critical scenarios and to support their management or, vice versa, to give evidence of the absence of critical conditions.

Figure 9, for example, shows the rain forecast for the next 72 h on Ravone’s cell.

The graph gives evidence of the expected rainfall and its cumulated level, whose duration and intensity are not sufficient to generate a significant event to report.

Figure 10 shows that the probabilities of the level of Baganza River to not exceeding threshold 1 is one, while it is equal to zero the probability that it exceeds any of the three defined alert thresholds, for the next 6 h.

Integration of forecast model data is the right prerequisite for the creation of an early-warning system that allows recognition and signaling of critical thresholds over with an anticipation that depends on the simulation model used.

In particular, the integration of forecast data related to the hydrometric level for medium-sized basins and rainfall events for small ones allows to identify critical scenarios in advance with a margin of some hours in the case of medium-sized basins and up to a few days for those small ones.

The concept of “virtual sensor” made it possible to describe estimated and forecast data with the same model defined for those observed, creating an advanced and integrated monitoring and early-warning infrastructure.

9 Conclusions

The management of events of intense precipitation and the risk of flooding connected to them is an ever more current topic, which requires new actions.

The risk of these events can be mitigated through the upgrading and integration of monitoring infrastructures suitable for this type of phenomena, but also through the development of early-warning systems [12] that allow their forecast with a significant margin of advance.

In this sense, hydrological simulation models and metereological modelling play an extremely important role and their continuous development is fundamental in order to increase their level of reliability.

The availability of a platform, like Sensornet, able to integrate all the available, observed, estimated and forecast data, allows to realize an advanced and integrated monitoring and early-warning infrastructure for extreme precipitation measurements and water rivers level forecast, for the purposes of mitigating the hydraulic risks of the territory.