Real-Time Monitoring of Small Reservoir Hydrology Using ICT and Application of Deep Learning for Prediction of Water Level

Abstract

There has been growing interest to monitor these reservoirs’ hydrological parameters and estimate risk of flooding in the downstream of reservoirs using modern sensing and simulation technologies. A study was carried out to develop a low-cost system to monitor the hydrological parameters of reservoirs using information and communication technologies (ICT) incorporated with a deep learning model to predict the water level using the long short-term memory (LSTM) algorithm. Using the LSTM, the future water level was estimated from the series of observed current and past water levels and rainfall data in addition to the forecasted information on future rainfall events. The system is intended to be used to estimate possible amounts of water storage during the future rainfall event so that it could be used to reduce the risk of downstream flooding while ensuring enough water storage for irrigation owing to its ability to determine an appropriate amount of water release from the reservoir prior to rainfall events. For testing and evaluation of the performance of the developed system, a reservoir located in a tributary of the Yamato River Basin was selected in Nara Prefecture, Japan. The developed system successfully monitored hydrological parameters of the reservoir and predicted the water level changes. However, it would require more training data to increase its prediction accuracy. The advantage of the model using LSTM compared to traditional hydrological models is that it may require a smaller set of hydrological parameters and its accuracy could be increased with an increased number of training and testing data even though the basic structure of the model remains the same.

7.1 Introduction

A reservoir is an artificial pond to collect water from the catchment area where there are frequent deficits in precipitation or river water. There are more than 150,000 reservoirs in Japan (Matsuno et al. 2019). Seventy per cent of them have been built before the eighteenth century and many of them have deteriorated. Recently, occurrences of natural disasters, such as torrential rains and earthquakes, often bring about floods and the collapse of reservoirs (Japanese Ministry of Agriculture, Forestry and Fisheries 2018). As shown in Fig. 7.1, the annual number of heavy rains has increased over the past decades (Japanese Meteorological Agency). Flooding and collapse of reservoirs caused by these disasters have given rise to an increase in the number of secondary disasters in the downstream basins. In particular, as shown in Fig. 7.2, 73% of the causes of such damage and 98% of the causes of such collapses are heavy rains (Japanese Ministry of Agriculture, Forestry and Fisheries 2018).

Fig. 7.1

(Modified from the data by Japanese Meteorological Agency. https://www.data.jma.go.jp/cpdinfo/extreme/extreme_p.html)

Yearly number of precipitations

Fig. 7.2

(Created from the information by Japanese Ministry of Agriculture, Forestry and Fisheries. https://www.maff.go.jp/j/nousin/bousai/bousai_saigai/b_tameike/attach/pdf/index-58.pdf)

Trend of reservoir’s damages in Japan

There has been an increasing interest in monitoring small reservoirs’ hydrologic parameters and predicting the risk of local floods using modern sensing and simulation technologies. For example, Tanihara (2008) created a simulation model estimating an embankment breach of irrigation tanks based on catchment area, spillway and freeboard and rainfall data. This model estimates inflow and outflow volumes and water level and, based on these estimations, predicts flooding risks. However, there is a limitation in accurate prediction with this model as the initial loss was not accounted for when using the general runoff estimation. Generally, there are many advantages in making a prediction model for large dams or major rivers. But for small reservoirs, the model is a relatively high cost when considering O&M costs for development of a particular water level prediction model. More recently, Hitokoto et al. (2016) predicted a river’s water level and a reservoir’s water level utilizing the deep learning technique. Although there are researches utilizing deep learning on drought prediction (Agana and Homaifar 2017), urban water level prediction (Assem et al. 2017) and daily reservoir inflow forecasting (Bai et al. 2016), the authors found very few published research on water level prediction of small irrigation reservoirs.

In this paper, we describe developed low-cost monitoring systems to acquire the hydrologic information utilizing information and communication technologies (ICT), which is coupled with a model that predicts the future water level applying the long short-term memory (LSTM) algorithm as one of the deep learning techniques.

7.2 Material and Methods

7.2.1 Study Site

A survey was conducted at the Takayama Reservoir in Takayama Town, Ikoma City, Nara Prefecture, and at the Kaerumata Reservoir located in Ayameike-Minami Town, Nara City, Nara Prefecture (Fig. 7.3). There are approximately 4300 reservoirs in Nara Prefecture, the majority of which are found in the Yamato Plain. The Yamato Plain region, which occupies two-thirds of the prefecture's agricultural land area, has been plagued by a shortage of water due to low annual rainfall and the absence of large rivers and lakes. The outflow from the Takayama Reservoir flows to the Tomi River, a tributary of the Yamato River basin. The Kaerumata Reservoir flows through the Oike River to the Akishino River, which also belongs to the Yamato River system. Both reservoirs are mainly used for irrigation so that water is released during the irrigation period from early May or late April to mid-September. The specification of the Takayama Reservoir is provided in Table 7.1.

Fig. 7.3

Location of Takayama and Kaerumata Reservoirs

Table 7.1 Specification of Takayama and Kaerumata Reservoirs

7.2.2 Water Level and Weather Data Monitoring

The sensor was set to monitor water level and water temperature of the reservoir every ten minutes (GSC-01A, Geotech Service) and show them on the web page through the remote data logger (HOBO RX3000, Onset) with the communication channel, SORACOM. Atmospheric pressure, atmospheric humidity, air temperature, solar radiation, precipitation, wind velocity and wind direction were also measured every ten minutes using the KOSEN weather station. A solar panel was attached to the sensing system for electricity supply. Water level data was stored as a CSV data file, while the weather data was stored as a JSON format so that it could be later converted to a CSV data file. Combined data accumulated from July 2018 to September 2019 was used for training and testing of the water level prediction model. The Python program was used to organize and combine the water level and weather data (Fig. 7.4).

Fig. 7.4

Weather station (left), data logger (upper right), water level sensor (lower right)

For the Kaerumata Reservoir, we installed a different set of IoT devices to monitor real time water level, water temperature, rainfall. In addition, a web camera was set up to observe the reservoir water online. These data were sent to the server once every hour through the LTE mobile line. METER’s CTD-10 was used as the water level and water temperature sensor, and METER's ECRN-50 was also used as the rain gauge. The data observed by these was integrated and quantified using Atmel's microcontroller ATMEGA328P-PU, and the data was stored and sent to the server using RS Components’ single board computer Raspberry Pi 3B+. Regarding the ATMEGA328P-PU, we designed the circuit board to connect to each sensor and to communicate with the Raspberry Pi 3B+, as shown in Fig. 7.5.

Fig. 7.5

Designed circuit board for connecting ATMEGA328P-PU

These devices are readily available and can be purchased through online shops in Japan and can be assembled even with a limited knowledge of electronics. Figure 7.6 shows the schematic of the IoT system flow. The solar power supply was used for the system that ensured the supply of sufficient power to run the sensors, web camera and communication devices (Fig. 7.7).

Fig. 7.6

IoT system flow for the Kaerumata Reservoir

Fig. 7.7

Installed devices and solar panel for the Kaerumata Reservoir

7.2.3 Water Level Prediction Model Using LSTM

The water level prediction model using the deep learning technique was developed using Python with neural network libraries such as TensorFlow and Keras. Deep learning, a kind of neural network, imitates the neural transmission of living things. By assigning a threshold and weight to each unit from a huge number of input and output data groups, it can be expressed similar to a person identifying an event with a lot of information. In recent years, deep neural network models with deeply complicated neural network layers have been successful in fields such as image processing and pre-language processing and their effects are accepted in the fields of hydrology and agricultural engineering (Li et al. 2016; Taniguchi et al. 2019; Xudong et al. 2019).

A neural network is composed of an input layer, hidden layers and an output layer, while past water level, precipitation and discharge are considered as inputs to predict the future water level of the reservoir, as shown in Fig. 7.8.

Fig. 7.8

Schematic of deep learning

The model was developed by applying long short-term memory (LSTM) algorism for prediction of the reservoir’s water level as it is suitable for handling the time series data. LSTM is considered as a kind of recurrent neural network (RNN) that integrates past hidden layers with present learning data (Fig. 7.9). RNN is a deep learning method often used for natural language processing and time series data analysis. In this study, we treated water level fluctuations as time series data and assumed that future water levels could be predicted by RNN. Simple RNNs, on the other hand, had a problem of disappearing gradients in the past due to long-term memory difficulty and an increase in learning volume. LSTM was adopted to solve these problems because long-term past memory is considered to be significantly involved in learning water level prediction. With LSTM, the vanishing gradient problem encountered in the ordinary RNN is solved because the long temporary dependence vanishes every time the model learns a new.

Fig. 7.9

Schematic of long short-term memory (LSTM) algorithm

7.2.4 Layer Setting for LSTM Model Development

Table 7.2 shows the selected parameters and layer setting of the developed model.

Table 7.2 Model parameters and layer

7.2.5 LSTM Model Evaluation

The difference between predicted and actual water level data was examined by applying Nash–Sutcliffe efficiency (NSE) and per cent bias (PBIAS) for the evaluation. NSE is the normalized statistic to determine the relative magnitude of residual variance for comparison with the variance of measured data that is often used to assess the performance of hydrological models. NSE is expressed as:

$${\text{NSE}} = 1 - \left\{ {\frac{{\mathop \sum \nolimits_{i = 1}^{n} \left( {Y_{i}^{{{\text{act}}}} - Y_{i}^{{{\text{pre}}}} } \right)^{2} }}{{\mathop \sum \nolimits_{i = 1}^{n} \left( {Y_{i}^{{{\text{act}}}} - Y^{{{\text{mean}}}} } \right)^{2} }}} \right\}$$

(7.1)

where ${Y}_{i}^{\mathrm{act}}$ expresses the ith actual water level, ${Y}_{i}^{\mathrm{pre}}$ expresses the ith predicted water level and ${Y}^{\mathrm{mean}}$ expresses the mean of all data. PBIAS estimates the deviation of data expressed as percentages that is shown in Eq. (7.2):

$${\text{PBIAS}} = \left\{ {\frac{{\mathop \sum \nolimits_{i = 1}^{n} \left( {Y_{i}^{{{\text{act}}}} - Y_{i}^{{{\text{pre}}}} } \right)*\left( {100} \right)}}{{\mathop \sum \nolimits_{i = 1}^{n} \left( {Y_{i}^{{{\text{act}}}} } \right)}}} \right\}$$

(7.2)

7.2.6 Reservoir Monitoring System and Water Level Prediction Model

PHP, JavaScript, MySQL and Python were used to develop a website to display the hydrologic data of the reservoir in real time. The prediction model was also incorporated into the website to show the future water level with given forecasted precipitation events.

The accuracy of the deep learning model depends on the quality and amount of training and testing data. In the system, the model can automatically update these data by incorporating observed data into the program in real time. It was set to use 70% of the observed data for the training and 30% for the testing. The schematic diagram for building the system is shown in Fig. 7.10.

Fig. 7.10

Flow of developed system

The rainfall intensity data obtained from KOSEN weather station and the water level data obtained from GSC-01A are saved in HOBO RX3000 and sent automatically to the cloud server. The LSTM model uses the data for prediction of future water levels. At the same time, the observed water level and rainfall data are displayed graphically, together with predicted water levels on the web screen.

7.3 Results and Discussion

7.3.1 Collection and Display of Water Level and Weather Data

With the developed system, the water level of the Takayama Reservoir was successfully monitored. Every ten minutes water level data displayed on the web page of HOBOlink (see Fig. 7.11). The data file was exported as a CSV file from the page to the developed system every hour. The weather data, such as rainfall, ambient temperature, relative humidity, solar radiation, wind speed and wind direction, were also successfully obtained every ten minutes and was shown on the web page of the KOSEN system (Fig. 7.12). The data from HOBOlink and KOSEN was automatically acquired via FTPS communication and API, respectively.

Fig. 7.11

Display of water level on web page of HOBOlink

Fig. 7.12

Display of weather data on web page of KOSEN system in Japanese

The water level from HOBOlink and the precipitation data from KOSEN system were merged to visualize the relationship between water level and precipitation. The gradual water level changes with changes in climatic variation are shown in Fig. 7.13. The precipitation events from 1 July 2018 through 31 December 2018 shown in this figure indicate the response of the reservoir’s water level with rainfall events. An overflow was observed in July as the water level surpassed the spill level of the Takayama Reservoir. The overflow–precipitation relationship is shown in Fig. 7.14.

Fig. 7.13

Relationship of water level and precipitation

Fig. 7.14

Relationship of overflow and precipitation

7.3.2 Reservoir Monitoring by Web Camera

In the Kaerumata Reservoir, a web camera was installed for surveillance of the reservoir water and surroundings that could be useful, especially during heavy rain events, to observe the situation of the reservoir without going to the site. Figure 7.15 shows the location of the installed camera in the reservoir and an image taken and transmitted to the server.

Fig. 7.15

taken from the camera

Location of web camera and an image

The images can be seen from the mobile phone communication tools, Slack and Line. Figure 7.16 shows the screen display on the Slack application sending the imagery data every hour.

Fig. 7.16

Reservoir images displayed on the screen of Slack during the night time

7.3.3 Water Level Prediction Component of the System

The water level prediction model was developed using the aforementioned monitoring data for training and testing of the model. Using the LSTM algorithm, the future water levels of one to 11 h were estimated from the series of observed current and past water levels, rainfall and the forecasted precipitation data. The forecasted precipitation data was received from a commercial weather forecasting service. The model was developed to estimate possible amounts of water storage during a future rainfall event. It could also be used to reduce the risk of the reservoir’s collapse and downstream flooding that occurs with the collapse, while ensuring sufficient water storage for irrigation owing to its ability to decide an appropriate amount of water release from the reservoir prior to rainfall events. The observed data from 1 July 2018 to 31 December 2018 was used for learning and testing of the model.

The performance of the developed model, as indicated by the relation between the epochs of the model, i.e. the number of times to learn an absolute error with observed values, is shown in Fig. 7.17. The abscissa of Fig. 7.17 is for the number of epochs. This figure indicates that the error decreases as the epochs increase. The prediction model became accurate sharply when the epoch was around 30.

Fig. 7.17

Relation of epochs and absolute error of water level prediction

Prediction of this model was then compared with the observed data obtained from 15 January 2019 to 21 January 2019 as shown in Fig. 7.18. Figure 7.18 shows that the model could respond to precipitation and predict water level, although it reacted too immediately after the rainfall event. On the other hand, as shown in Fig. 7.19, the difference between observed and predicted water level is about 1 cm.

Fig. 7.18

The comparison of predicted and observed water level

Fig. 7.19

Difference between observed and predicted water level

Figure 7.20 expresses an accuracy of the model, when it compared observed water level with predicted water level data after an hour and after 11 h. Nash–Sutcliffe efficiency (NSE) and per cent bias (PBIAS) were used for model evaluation. Table 7.3 shows the results of the model evaluation. Both NSE values significantly surpass the standard value. Generally, a model is judged as satisfactory if NSE is close to 1 and low values of PBIAS indicate accurate model simulation (Moriasi et al. 2007). Here, both NSE and PBIAS values show satisfactory levels of prediction.

Fig. 7.20

Accuracy of predicted water level after an hour and after 11 h

Table 7.3 Results of NSE and PBIAS values estimated

7.3.4 Visualization of Past, Present, and Future Water Level and Rainfall

The water prediction model was incorporated into the monitoring system for its application to water management of the reservoirs. Figure 7.21 shows the web screen displaying the current observed water levels, future water levels from after an hour to 11 h with forecasted rainfall events in the Takayama (upper figure) and Kaerumata (lower figure) Reservoirs.

Fig. 7.21

Display of current and predicted future water levels

It should be noted that, even in the same rainfall event, the prediction results differ due to the difference in the period of training data used in the model. Accuracy is increased with the increased training data period. Consequently, long-term monitoring data has an advantage in the construction of accurate DNN models.

7.4 Conclusions

The water level sensor and the weather station were set in the Takayama and Kaerumata Reservoirs to monitor water level and the other hydrologic parameters. The system was able to obtain the data from a remote location at relatively low cost. However, there are issues that were realized during the process of the development, such as power supply when installing in the area, wireless data transmission cost, versatility and interchangeability of sensing devices.

The developed model predicted the water level changes using past water level and precipitation data but requires more training data to increase its accuracy. Both NSE and PBIAS meet the criteria of prediction. The advantage of the model using LSTM is that it may require a smaller set of hydrologic parameters than the conventional one and the accuracy could be increased with an increased number of training data, even though the basic structure of the model remains the same. On the other hand, the quality of the model outcomes may depend on the availability of training and testing data.

By combining the monitoring system and the prediction model developed in this study, it would be easy to use the model for reservoir water management to decrease the risk of flooding in reservoir downstream and also for irrigation.