Keywords

3.1 Introduction

Limited access to safe water, sanitation, and increased demand threatens human health, environmental sustainability, and economic growth, while leading to poor water security and supply. The increasing pressure on water resources is the result of global drivers such as rapid population growth, industrialisation, inefficient water management, water pollution, climate change, serious imbalance between supply and demand, but also poor water data (World Bank 2018). Therefore, water security, reliable water supply, as well as environment and ecosystem protection are directly dependant on the level of efficiency, integration, and sustainability of water resources management.

Data and information are denoted as the “life blood” of a research with the purpose of efficient water management and decision making. Hence, data is the foundation for well-informed decisions and sustainability that is required by all decision makers. Water data, or in other words measurements, is a key element in water resources management projects and planning. Water information is generated from water data through a process of analysis, integration, and interpretation. In other words, water information is the processed and synthesized form of water data. The generated water information should be combined with infrastructure and institution in order to reach an efficient water resources planning and management for both the current and uncertain future periods (Cantor et al. 2018).

Different water management disciplines, such as the flood response, drought management, water supply systems, reservoir operations, water sanitation, groundwater management, and irrigation supply require incorporation of different types of water information for better decision making and water planning. Thus, decision makers and water organizations need to access water information with proper spatial and temporal resolution (Cantor et al. 2018). Additionally, provision of sound data with proper quality is essential as it can affect the decisions and management strategies. However, there have been debates among organizations and water sector participants that some provided data do not meet the need of water users. Additionally, some water information is not shared openly. Despite the development of technology and modern measuring devices, there are still many gaps in available water data, and the quality of some water information is also still unacceptable (BOM 2017). Therefore, unmet data needs can interfere the efficient and sustainable water management.

Figure 3.1 indicates the data life cycle (NSF DataONE project 2017) and the required steps from the data need evaluation to the data application. The main nine steps of the data life cycle are as follow:

Fig. 3.1
figure 1

Data life cycle (Adopted from NSF DataONE Project 2017)

Full size image

  • Data need evaluation and planning: needs of decision makers are fully evaluated;

  • Data collection: observations are performed;

  • Data quality assurance: checks and inspections are required to analyse the data quality;

  • Data description and documentation: data should be accurately described using the metadata standards and guidelines;

  • Data preservation and archival: data should be preserved in a specific archive (e.g., data service and data centre);

  • Data discovery: both useful and relevant data information should be obtained;

  • Data integration: data from different sources should be combined;

  • Data analysis: data should be interpreted; and

  • Data release planning: data should be described, managed, and made accessible.

Globally, water data and information is essential for sustainable planning, policy, management, and prediction. Additionally, water data provides decision makers with proper information about water demand, weather patterns, hydrologic modelling, infrastructure needs, climate change, and extreme events. This book chapter examines the important role of water data and information in integrated water resources management. It also provides an in-depth understanding of different water data types, sources, challenges, processing, and dissemination, which are essential for sustainable planning, policy, management, prediction, and decision making. Thereafter, examples of two international information systems (WHYCOS and GRDC) and a national water portal (BOM) are proposed to indicate how water data organisation can enhance climate trend analysis, water information products and services, and sustainable protection of water resources systems through regional and international cooperation. Finally, conclusions summarizes focal points of this work, and recommendations are proposed to help stakeholders discover and overcome limitations of current water data.

3.2 Water Data Type

Water data is a set of information that can address issues associated with physical, environmental, social, economic, ecological, biological, chemical, cultural, and political parameters in disciplines such as water use, availability, and accessibility. Additionally, water data can consist of a complex mix of different formats, including spreadsheets, satellite images, geospatial databases, and photographs (Laituri and Sternlieb 2014).

Generally, water data can be classified in two main categories: (1) Primary data and (2) secondary data. Primary data indicates the collection of raw data, including the data collected through measurements by technician and/or automated sensors. Measurements of water quality parameters, streamflow and precipitation are examples of primary data. Secondary data are obtained from models, specific methods, and lab experiments under controlled condition. Typically, models and methods apply primary data from different sources to compile and analyse them for estimating a variable. For instance, precipitation pattern, streamflow condition, irrigated area, and crop data can be combined through a model or method to estimate the agriculture water demand (Ziman 2016). Climatological models, hydrological models, processed remote sensing data or satellite images are some examples of secondary data (Laituri and Sternlieb 2014). In addition to these classes of water data, 12 extensive groups of water data can be developed based on disciplines or sectors where water data is applied (BOM 2017). Detailed classifications of water data with respective examples are shown in Table 3.1.

Table 3.1 Water data types together with data sources and examples
Full size table

All the stated data are essential for efficient decision making in different water disciplines, including the sectorial water management, integrated water sector planning, climate change adaptation, global and regional reporting, as well as operational and emergency management. Additionally, the type of required data for a specific project relies on the type of activity and plans. For instance, a flood inundation project will not require the same data as a peak flood prediction project. Additionally, the temporal and spatial resolution of the required water data should be specified for each project. For instance, the hourly river flow is generally applicable to peak flow prediction, and estimation of flash flood peak flows requires even higher temporal river flow resolution (e.g., minutes), while the daily river streamflow is required for analysis of river flow due to changes in land use pattern. Thus, the researchers and managers are required to consider the required data type and resolution from the beginning of a project.

3.2.1 Water Data Source

Different distinct types of water data are required to determine water quality and quantity to achieve efficient integrated water resources management. Water data is obtained through different sources and methods. Basically, the water data comes from four sources: (1) measurements; (2) models; (3) remote sensing; and (4) administrative institutes (BOM 2017). Each of the stated sources will be discussed in the next four sections.

3.2.1.1 In Situ Measurements

Water resources management constitutes a data-driven discipline, which involves consistent measurements of water quantity and quality. The measurements are mainly concerned with monitoring meteorological and hydrological variables (BOM 2017). In order to measure water variables, including physical, chemical, and biological parameters, specific methods and instruments are applied. For example, water level can be measured using different techniques such as a float and shaft encoder, a radar or an ultrasonic level sensor (NIWA 2019), turbidity by determining the intensity of light scattered by suspended particles in the water column (EHMP 2006–07), and biological contamination using E. coli concentration as a marker (Edberg et al. 2000). In addition to the variables which come from direct measurement, some other variables are calculated according to a mathematical transformation of a direct measurement. For instance, river discharge is computed by multiplying the water area in a channel cross section by the water average velocity in that cross section (BOM 2017).

In situ measurements can yield the most reliable water data, provided that the measuring devices are precise and properly calibrated. The restriction of direct measurements is the high cost of purchasing, installing, calibrating, fixing, and maintaining the measuring devices and equipment. Additionally, regular monitoring should be conducted periodically to obtain the proper data length and spatial coverage, and therefore, constant financial support and effort are required to manage the long and intensive monitoring programs (BOM 2017). Due to the discussed high costs, many poor countries suffer from improper and inadequate water data, which can directly affect the water management planning and strategies. This issue necessitates the provision of financial support by international agencies and organizations to improve the water monitoring networks in poor countries dealing with critical water supply and management problems.

3.2.1.2 Remote Sensing

In 1957, the Sputnik 1 was the first artificial earth satellite that placed into orbit. This satellite had significant impact on humanity’s perception of space, and announcing a new era if earth observation. After launching the Sputnik 1, several systems (e.g., Television and InfraRed Observation Satellite (TIROS 1), Nimbus 1 etc.) were launched in the following year to weather and climate conditions. Such satellite missions led to dramatic advances in remote observations and measurements (McCabe et al. 2017). Remote sensing refers to the process of inferring surface parameters, which are derived from measurements of the emitted/reflected electromagnetic radiations, from the land surface (Schmugge et al. 2002). The application of remote sensing in hydrological sciences has provided new datasets with high temporal and spatial resolution for continuous water resources observations, which addresses the low resolution and expensive in situ measurements. Figure 3.2 indicates the earth observing system applied in hydrological sciences. The stated system consists of several components, signals of opportunity (e.g., cars, mobile phones), Doppler radar, mobile rovers, smart phone and citizens science (e.g., simple image capturing, plug-in and Bluetooth technologies), cell signals (e.g., antennas), unmanned aerial vehicle (UAV) (e.g., drones), research balloons, airborne vehicles, CubeStas, high definition (HD) videos, satellite missions (e.g., Landsat 1–8), sensors aboard the International Space Station (ISS), and geosynchronous meteorological satellites (McCabe et al. 2017).

Fig. 3.2
figure 2

Earth observing system (Adopted from McCabe et al. 2017)

Full size image

There are two types of remote sensing instruments: i) active and ii) passive. These instruments are mounted on satellites, drones, vehicles, aircrafts, and ground-based structures to measure different water data parameters, such as soil moisture, evaporation, water temperature, rainfall rate, sediment concentration, flood inundation area, rainfall distribution, surface topography, wind speed and direction, cloud composition, and air humidity (BOM 2017). Active sensors provide their own source of energy for emitting radiation towards the investigated target, and then measure the time of arrival radiation reflected from the target. The majority of active sensors are able to perform in the microwave portion of the electromagnetic spectrum. Hence, these sensors can potentially penetrate the atmosphere under rough conditions. Laser altimeter, radar, Lidar, Ranging instrument, Scatterometer, and Sounder are examples of active sensors (NASA 2019).

On the other hand, passive sensors detect natural energy (light wavelength) which is emitted or reflected from the observed scene. The common source of radiation measured by passive sensors is the reflected sunlight. Passive sensors include various types of spectrometers and radiometers. These sensors are able to perform in the infrared, visible, thermal infrared, and microwave portions of the electromagnetic spectrum. The Accelerometer, Hyperspectral Radiometer, Imaging Radiometer, Sounder, Spectrometer, and Spectroradiometer are some examples of passives sensors (NASA 2019).

Remotely sensed data provides continuous spatial coverage and are typically presented in gridded form. The most temporally regular remotely sensed data set is obtained from satellites (BOM 2017). Since its advent in the 1960s, satellite remote sensing has been widely used as a complete source of information. Satellite sensors can measure almost all components of hydrological cycle, such as precipitation, evaporation, surface water, soil moisture, lake and river levels, surface and sub-surface water storage, and snow storage (Sheffield et al. 2018). However, satellites have relatively coarser spatial resolution than in situ measurements due to their large distance from the earth in relation to the resolution of the sensor and the attenuation effect of the atmosphere between the sensor and the observation point. Low earth orbiting satellites present much higher spatial resolution with lower temporal resolution (BOM 2017).

The only issue that should be carefully considered in remote sensing is the requirement of significant information technology infrastructure for data management. Additionally, complex image processing tasks are needed to provide the data with suitable format (BOM 2017). In summary, remote sensing tools play an important role in improving water resources management through provision of large spatial and high temporal resolution.

3.2.1.3 Models

In order to tackle the shortcomings and costs associated with in situ measurements and also to address the poor spatial support of in situ point measurements, different types of models and methods can be applied to generate water data. Modelling approaches are useful tools to address poor spatial support of in situ point measurements. For instance, required spatially distributed water surface elevations for flood magnitude, extent and timing estimations are derived from hydrodynamic models. Therefore, models and methods are applied for different purposes. For example, different rainfall-runoff models have been developed for generating the runoff data in ungauged catchments. Conceptual rainfall-runoff model applications such as Nedbor-Afstromings Model (NAM) (Makungo et al. 2010; Faiz et al. 2018; Zhen-lei et al. 2019), Hydrologiska Byråns Vattenbalansavdelning (HBV) (Engeland and Hisdal 2009; Osuch et al. 2019), and The Hydrologic Engineering Center Hydrologic Modeling System (HEC-HMS) (Gumindoga et al. 2017) are examples of model applications, which converts the rainfall to runoff in a defined catchment. In the field of water resources engineering, data-driven approaches have gained popularity for different water modelling purposes (Kim et al. 2013; Jahandideh-Tehrani et al. 2019; 2020a). Artificial neural network (ANN), support vector machine (SVM), and random forest (RF) are few examples of data-driven approaches, which have been extensively applied in the area if water related modellings such as ground water level prediction (Daliakopoulos et al. 2005; Tsanis et al. 2008; Baudron et al. 2013), rainfall-runoff modelling (Daliakopoulos and Tsanis 2016; Berezowski and Chybicki 2018; Jahandideh-Tehrani et al. 2020b), reservoir operation (Niu et al. 2019; Ahmad and Hossain 2019), flood prediction (Zhou et al. 2019; Ahmadalipour and Moradkhani 2019).

Some models are also used to extrapolate to past (e.g., paleoclimate) and future conditions, such as climate models. Future precipitation, evaporation, wind, and temperature can be generated using different climate models, such as Hadley Center Coupled Model (HADCM3), Norwegian Erath System Model (NorESM), and The Fifth Version Model for Interdisciplinary Research on Climate (MIROC5) (Randall et al. 2007). In order to summarize a large amount of information, specific approaches such as system dynamics (SD) are applied to analyse and interpret information. Using flow diagrams and feedback loops facilitate the analysis and conclusion for complex information (Ahmad and Simonovic 2004; Jahandideh Tehrani et al. 2014; Christias et al. 2020). In summary, models and methods are used for tasks, such as making predictions, data assimilation, and interpreting the large amount of complex information. The main limitation of applying models is the uncertainty resulting from models due to prior assumptions and scale issues. In water resources engineering field, analytical techniques (e.g., Mellin transform technique, and Fourier transform technique) and approximation techniques (e.g., first-order variance estimation (FOVE), probabilistic point estimation (PE) methods, and Monte-Carlo simulation) are the two main uncertainty analysis categories that can be applied to address hydrologic, hydraulic, structural, and economic uncertainties (Tung 2011). Many researchers have discussed model uncertainty analysis and proposed methods to reduce the amount of uncertainty. For instance, Wagener and Wheater (2006) investigated the sources of uncertainty in a rainfall-runoff model by focusing on 10 catchments. They identified that the uncertainty is related to local modelling (process of selection and calibration of the local model structure) and regionalization procedures. To increase the accuracy of model results, many studies have focused on analysing uncertainty and reducing the vulnerability. Her et al. (2019) evaluated the associated uncertainties with multi GCMs in runoff and precipitation projections. In terms of adaptation planning for uncertain climate change impacts, evaluating the robust adaptation decisions to climate change uncertainties was studied by Dessai and Hulme (2007). They concluded that water resources are sensitive to regional climate response uncertainties in climate change impacts. Also, Fletcher et al. (2019) developed a new planning framework to evaluate climate uncertainty over time for flexible planning strategies. In summary, despite the wide applications of models in improving water data, research is still required to improve the accuracy of the model results.

3.2.1.4 Administrative Institutes

Two types of data, including the water rights data and administrative data (mentioned in Table 3.1) are not obtained from measurements or models. Data such as water rights, water pricing, water infrastructure inventories, water demand in different economic activities, water ownership, and basin borders are basically recorded by water management agencies as part of their business purposes. Additionally, such data can be collected by conducting household and business surveys (BOM 2017).

Water rights data and administrative data play a critical role in developing, proposing, and evaluating the water management strategies and policies as well as decision making. For instance, water system governance, which refers to the structure and administrative process, should direct and control operations, decision making, legislation, and finance as part of their tasks. Therefore, administrative data is required in order to determine operational standards and water regulations.

A large number of agencies and organizations are involved in the establishment and administration of water rights in scale of regional, national and international scopes. In Europe, the Water Framework Directive (WFD) is the main foundation of European Union (EU) water policy that aims to ensure sustainable water use and protect aquatic systems through applying legislations to all surface waters (rivers, lakes, transitional and coastal waters) as well as groundwater (Carvalho et al. 2019). In Australia, the Department of Natural Resources and Mines is involved in water system administration in all states (Productivity Commission 2003). While different countries/regions/continents have their own administrative institutes to dictate water related policies (e.g., water pricing, water ownerships, river borders, etc.), there are also international foundations, which deal with global water issues. For example, World Water Council is an international multi-stakeholder organization that enhance political action and awareness to deal with critical water issues through developing partnerships between countries (World Water Council 2020). Pacific Institute, Clean Water Action, and Water.org are other examples of global organizations that aim to solve water conflicts, promote legislation for water protection, and deal with water crisis.

3.3 Key Characteristics of Water Data Type

Different types of water data have their own characteristics and use. In other words, each type of water data should be applied for a specific purpose of water resources management, such as flood management, water quality management, sediment control, and water security management. The key characteristics of water data together with examples are listed below (BOM 2017):

  • Parameter type (e.g., river flow data for reservoir operation; water demand data for irrigation management)

  • Measurement location (e.g., water quality data for upstream of water offtakes for drinking water supply; groundwater level data for heavily utilized aquifers)

  • Spatial coverage (e.g., flood level data for flood prone areas; national scale water assessment)

  • Spatial density (e.g., rainfall distribution for rainfall pattern assessment)

  • Temporal frequency (e.g., daily or monthly river data for regional water security assessment; high frequency river and rainfall data for flood forecasting)

  • Longevity of measurements (multi-decadal data of continuous rainfall for rainfall IDF analysis; historical river flow data for flood frequency and magnitude analysis)

  • Latency (e.g., basin time concentration for rainfall-runoff modelling; real-time rainfall and river level data for flood forecasting)

  • Precision (e.g., water quality data should be precise in order to confirm safe drinking water supply; water pricing should be assessed carefully to prevent probable issues between stakeholders).

3.4 Challenges of Water Data

In most countries, the nation’s water data arrangements cannot satisfy the need of water sector participants, and the processes of data collection are limited. There is an increasing need for systematic data collection in order to improve monitoring and ensure informed decision making (IWRM 2019). Over the next three sections, basic challenges of accessing proper and accurate water data will be discussed.

3.4.1 Poor Quality of Water Data

There are many gaps in the available water dataset, and the quality of some water data is poor. Generally, the available water datasets are heterogeneous, dispersed, and poor in quality (BOM 2017). Dispersion and gaps in water data are mainly the result of poorly calibrated measuring devices as well as inefficient equipment maintenance. Apparently, water management will be challenging without good water data. Therefore, policy makers and managers are unable to make sound decisions as the provided water data are unreliable and contradictory (BOM 2017). Additionally, many poor countries, which are facing financial problems, are not equipped with modern and efficient measuring devices and techniques, which leads to poor measurements, and subsequently poor quality datasets (INBO 2018).

Lack of homogeneity in water data causes poor data management as the data is useless, and massive investments on water infrastructure and data collection is wasted. Heterogeneous data mainly occurs when private organizations follow their own specific guidelines and procedures for data collection, and there is no general adopted rules for all data collector organizations (BOM 2017).

3.4.2 Lack of Integrated Water Portal Systems

One of the main water data challenges in many countries is the lack of information about the available water data, and also not openly shared datasets. Basically, there is no single established organization/tool/website to provide potential users with information about the different available water types, and characteristics of the water date sets. Detailed metadata, i.e. information on “how the data is produced”, “how the data can be accessed”, “how the data should be used”, “who to contact”, and “what the licence of the data is” are not provided by institutions in many countries. Thus, researchers and water data users require considerable time, effort, and investment to identify stated information.

Therefore, data users and researchers need to spend time on basic data gathering and transformation rather than scientific analysis. This problem is noticeable when data is collected by multiple agencies for a specific task purpose. For instance, the required meteorological data models and formats for hydrological purposes (e.g., groundwater and catchment modelling) are different from the meteorological data formats for atmospheric science analysis. As a result of this data difference, a hydrologist should spend significant amount of time on learning the file format and visualization tool used by atmospheric science community. To address this problem, data should be communicated between scientific sub-disciplines systems through standard protocol/web services (Goodall et al. 2008). In this context, organizations such as European Flood Awareness System (EFAS), Global Flood Awareness System (GLOFAS), European drought observatory (EDO), Watershed Index Online (WSIO), United States Geological Survey (USGS) Water Data, BOM climate data online, European Space Agency (ESA), Copernicus Space Component (CSC) and National Aeronautics and Space Administration (NASA) have made efforts to provide users with documented, homogeneous, and well-organized data set, which can be accessed through web services, and indicate datasets in the form of maps, tables, and graphs.

3.4.3 Limited Access to Data

According to the Integrated Water Resources Management (IWRM) principle, the public should participate in decision making processes. In order to pave this way, water institutions and organization should make the water data and information available and accessible to the public. However, much water data is not accessible by the public, or the water data type is not understandable by people with no expert knowledge. Thus, to ensure the participations of the public, water data should be disseminated in an informative and comprehensible way (INBO 2018). In addition to the public, openly shared and freely access to most water data is limited for water data users, particularly researchers. Regarding the fact that in many countries, funds for data collections come from private sectors, institutions are mainly reluctant to share the data freely. Additionally, many national organizations have no tends to share the collected data with the neighbouring countries or other international organizations. This situation is observed particularly when the countries are facing critical water scarcity, and the countries under such conflicts deny publishing information due to security risks (INBO 2018).

Additionally, poor countries, which are facing financial problems, are unable to build efficient water information systems, manage water data, and develop efficient web portals and tools for data sharing. As a result, international organizations and agencies play an important role in providing financial support as well as remote monitoring to deal with the water problem in poor countries (INBO 2018).

3.4.4 Big Data Problems

Regarding the increased amount of water data, several challenges can appear for big water data management. Big water data is associated with increased data volume, variety, and velocity. The amount of measured and processed data is growing rapidly. Additionally, the growing formats of data can make data management challenging. Considering the increased automation in data measurement process, the biases, abnormalities, and noise are also increasing in raw water data. As a result, a suitable and massive validation process is required to treat raw water data. Citizen science data is another aspect of big data complexity. Citizen science refers to the participation of the general public in the process of research design and data collection, which leads to the collection of large volumes of data (Buytaert et al. 2014). In the field of climate modelling, the spatial and temporal dimensions of data are intensive as such data indicates complex process of atmosphere and ocean circulation. Therefore, such climate data requires high storage space and large computer processing power (Cuntz et al. 2007). High volume data with long time periods can lead to high potential of data loss or intrusion. Generally, with high increase in the amount of data, many challenges are coupled with the process of big water data management.

3.5 Data Processing

Data processing refers to the manipulation of data obtained from in situ measurements or remote sensing, before they can be efficiently stored or made available (Hughes 1998). Raw data should be processed and transformed into formats to be identified and applied by researchers, data users, and decision makers (INBO 2018). Regarding the cost, effort, and high volume involved in data collection, proper data processing tools and approaches should be adopted (Hughes 1998). Many tools are available for data processing, including spreadsheets and databases, database management tools, geographic information systems (GIS), Extract-Transform-Load (ETL) tools, modelling tools, and statistical data analysis tools. However, the selected tools and approaches should meet the requirements of the data users and researchers: What data format is required? What is the purpose of using the data? What kind of research will be conducted? And which organization will use the data? (INBO 2018).

3.5.1 Water Data Processing and Analysis

The main purpose of data processing is to convert raw data into information, which meets the need of researchers and decision makers. Key points of water data processing are (Hughes 1998):

  • Data encoding and digitizing;

  • Quality control, consistency checking, and error checking;

  • Data storage resolution;

  • Data calibration and stability;

  • Data storage and retrieval system.

The stated key points will be discussed below:

Data encoding and digitizing: Converting the measurements to digital is the first step of data processing. Hand-written field sheets or automatically penned charts mounted on a rotating drum are simple forms of data collection, which is still applied in many areas, particularly for collecting rainfall, evaporation, and other meteorological variables. Next, this data should be transferred to a computer database. Lost or damaged data is the main drawback of this manual data recoding. Therefore, modern telemetry, remote sensing, and other modern instruments have been developed to record the data automatically, particularly collecting the “real time” data (Hughes 1998). The vast quantities of recorded data (in situ measurements and remote sensing data) should be transformed into the cloud services in order to be stored and classified and then be accessible through portals and web services (McCabe et al. 2017).

Data quality control: Data collection corresponds with issues such as missing data, recorded data over the incorrect time, accumulated data, instrument failure, etc. Additionally, most databases are not able to identify poor and incorrect data and inform data users of data inconsistency (Hughes 1998). Missing data processing, logical error detection, repeated data processing, abnormal data detection, and inconsistent data processing are main aspects of hydrological data quality control. Prediction methods (e.g., recurrence neural network (RNN), and support vector machine (SVM)), optimization methods (e.g., particle swarm optimization (PSO)), and statistical methods (e.g., adaptive boosting, and statistical control) are some examples of data driven approaches for data quality control (Zhao et al. 2018).

Data storage resolution: Time resolution is of great importance in water data, particularly, the data used in hydrological analysis. For example, required time step for analysing the rainfall intensity in temperate climate is different from the arid and semiarid climates. Normally, the rainfall data with a resolution of one hour should be proper in temperate climate, while short-term variation of intensity is more important in arid and semiarid areas, and hence, minute should be the proper time step in arid areas (Hughes 1998). Therefore, disaggregation methods, such as Hyetos based on Bartlett-Lewis process and Marcov chain model should be applied in order to break down the data into shorter time steps (Koutsoyiannis 2003), which is considered as an important task of data processing.

Data value accuracy: Accuracy of observations is a numeric value that quantifies the data measurements accuracy based on the closeness to the standard value. The stated value accuracy measures the associated uncertainty with data measurements, which is the result of errors in bias and precision. The value accuracy can be estimated using the knowledge of instrument accuracy and measurement method as well as statistical analysis of the records from repeated measurements (Horsburgh et al. 2008). Additionally, some instrument calibration process (either manual calibration or using table and equations) is required to ensure accurate measurements. For instance, the stage-discharge relationship is commonly used to convert water level measurements of a flume to flow rate or discharge. In the case of a flooding event in a river, calibration process is necessary as riverbank erosion and massive sediment transport occur over flood events (Hughes 1998).

Data storage and retrieval system: As describe in Sect. 4.4, big data requires massive amounts of data to be stored. Metadata catalogues and data storage systems help data users to identify the available datasets and check their characteristics (Hughes 1998). Within this context, recorded observations should be stored with sufficient supporting information (e.g., the location, date and time and type of variable that was observed) about the observations to enable data interpretation and cross-dimension data retrieval and analysis (Horsburgh et al. 2008). The databases should provide users with helpful information on the type measurements, data length, instrument details, the latitude and longitude of measurements, data quality, missing data, time zone, etc. Additionally, an efficient database is able to allow user to access and download the data in different formats, such shape files, text file, and excel files. It is also necessary to design a frequent backup system for the databases in order to prevent data lost in a case of system failure (Hughes 1998). In the scope of data storage and retrieval, Internet of Things (IoT) technology, which has been developed rapidly in recent years, is considered as a key approach in big data processing. Within this context, cloud computing is a new generation of computing architecture that supports big data applications and growing data processing (Fan et al. 2018). As an example of such applications, the cloud-computing services have been applied to watershed management by enabling watershed partnerships for focusing primarily on decision-making activities (Sun 2013). Bürger et al. (2012) also developed an intuitive Web interface to create an integrated hydrologic simulation platform that enables users to access scientific software through a web browser.

3.5.2 Water Data Visualization

After the data process step and data preparation, the next step is to present the data in an understandable and efficient way. Regarding the recent improvements of digital culture and information technologies, large datasets become complex to understand. Therefore, visual presentation of data has been developed, which presents data in the form of a graph, map, video, animation, hierarchy, etc. Data visualization can also improve understanding of existing trends, patterns and correlation in the datasets that might not detected in text and number-based data. As a result, stakeholders and decision makers can easily identify the important issues of water resources management to focus on as well as adapting new policies. In the scope of data visualization, the quality of data presentation is important as a poor or incorrect type of data visualization may cause misleading and hinder smart decision making. The visualization tool should be selected based on specific criteria, including i) how data viewers interact with data, ii) the information that should be obtained by the viewers, iii) the function of the data, and iv) the data composition. The three main types of water data presentation are as below (INBO 2018):

  • Maps: They indicate the geographical distribution of different parameters, such as evaporation, rainfall, temperature. Maps can be produced using GIS software.

  • Key figures: They are generated from data processing and document analysis. Key figures are understandable by public.

  • Factsheet: They show a regular summary of data analysis, information, and graphs on different topics such as global warming, climate change, and drought trend.

3.5.3 Water Data Dissemination

After collecting, verifying, processing, and storing the water data, the next important task is to release and disseminate the data according to the target group (e.g., the general public, researchers, and decision makers) (e.g., before and after a flooding event). Data dissemination consists of four main steps (SEQwater 2014):

  1. (1)

    Data value identification: The first step is to evaluate the data value and information asset to decide on the price of the water data. The data value is conducted by surveying business system, water companies, and stakeholders about their needs and interests. Some data can be accessible free of charge considering the expenses of data preparation process and company’s needs and interests.

  2. (2)

    Data assessment: Over the second step, the data suitability for release should be assessed based on relevant guidelines, policies, and legislations. Additionally, depending on the target group (e.g., scientific, non-expert purposes), data visualizations, data use instructions, and data use policies should be defined.

  3. (3)

    Data publication: Over the third step, data should be published based on priority factors such as public interest, stakeholder feedback, company and institutes need, and economic values. Additionally, given the type of requirements of different target groups (e.g., the general public, researchers, and decision makers) and required time of hydrological events (e.g., before and after a flood event, before and after a dam construction), data dissemination should be specified.

  4. (4)

    Data management: Over the last step, data quality should be improved with regard to data user’s feedback.

In order to release the water data, different strategies and digital tool are available. Below is the list of different water data dissemination plans (INBO 2018):

  • Web portals and websites: data resulted from monitoring programs is delivered through portals and website. Basically, portals provide organized data, and water data is classified based on different criteria such as collected data from open stations, water data type (e.g., water level, streamflow, and rainfall), collected data from closed station, river type, time periods, water data unit (e.g., mega litre and cubic meters), and measured interval (e.g., hourly, daily, and monthly).

  • Factsheets: guidelines, water issues (e.g., global warming and climate change), water data change trend analysis (e.g., temperature trend, major flood event trend and evaporation change trend), monitoring programs (e.g., coastal erosion monitoring), management programs (e.g., integrated sub-basin management plans) can be published through digital books.

  • Social media networks: In addition to portals, websites, and digital books, social media networks also play an important role in water data dissemination. Social media networks increase people’s awareness of available water data and establish contact with internet and data users. Additionally, social media are effective way of transferring news and water data updates to users.

  • Smart apps: smart and free apps are favourable, particularly among the public. Smart applications can potentially target different water users and improve interaction. For instance, “Ma Cons’eau” is a useful free application, which can estimate the people’s water consumption through providing a set of simple questions. The amount of water consumption is calculated based on the water price of resident locations.

3.6 Practical Examples

Many organizations, systems and programs have been developed to provide high quality data and services of weather, climate, hydrological, and environmental fields for the purpose of integrated water resources management. Therefore, multiple water data, information, and knowledge should be shared to build cross-functionality between stakeholders.

One example of the water data system is the World Hydrological Cycle Observing System (WHYCOS), launched by the World Meteorological Organization (WMO) in 1993 to improve sustainable development in delivery and use of hydrological data as well as promoting regional and international cooperation. WHYCOS aims to enhance the sustainable protection, use and management of water resources systems as well as support decision makers through reliable and data provision. In terms of regional water resources management, HYCOS promotes data and information products, such as flood forecasts and warning data. At global scale, the WHYCOS International Advisory Group (WIAG) was introduced to provide support for policy guidelines and future development of the projects (WMO 2019).

Another example of global water data organization is the Global Runoff Data Centre (GRDC), which was established three decades ago. GRDC is an international archive of streamflow data, which aims to enhance analysis of global climate trends and evaluate environmental impacts and risks. The Global Runoff Data Base (GRDB) at GRDC was built on an initial dataset collected in the early 1980s. GRDB is a unique tool for obtaining river streamflow data at daily and monthly time step from over 9,500 gauging stations in 161 countries.

In terms of national scale, the Bureau of Meteorology (BOM) in Australia has developed the Australian Water Resources Information System (AWRIS). As shown in Fig. 3.3, the purpose of AWRIS is to receive and manage water data and information as well as supporting and disseminating different water information products and services.

Fig. 3.3
figure 3

The role of AWRIS in water information in Australia (Adopted from AWRIS Information Sheets 2019)

Full size image

AWRIS is able to store and manage information about groundwater levels, river flow, the quality of water in rivers and aquifers, water trades, water use and restrictions, and water volume in storages in a central database. Next, the stored data is checked for quality and standards to ensure that the stored data is consistent with the current available data. Over the next step, the Australian Hydrological Geospatial Fabric cast the stored water information in a spatial context through encoding the spatial connections between Australia’s hydrological features (e.g., rivers, lakes, aquifers, dams, channels etc.). As indicated in Fig. 3.3, the final step includes analysing, interpreting, and integrating the water data. AWRIS delivers variety of water data, reports, and forecasts, such as real-time water reports, regular national water resources assessments, improved flood warning systems, and seasonal streamflow forecasts. The main advantage of AWRIS is that water users are able (i) to identify water resources condition in Australia through integrated viewpoint, (ii) to prepare benchmark reports using quality and transparent data, (iii) to check the details of standards, (iv) to control the quality of applied water data, and (v) to easily use the data for planning, analysis, reporting, and modelling (AWRIS information sheets 2019).

3.7 Summary

Limited water resources and increased demand for the resources leads to growing challenges facing water resources systems. This growing pressure on water resources is the result of many issues, such as rapid population growth, poor water data, inefficient water resources management, and climate change. Therefore, the need to enhance water security, sustainable water management, reliable water supply, and environment protection necessitate the integrated water resources planning and management.

The chapter underlines that water data and information are considered as the key components of water-based research, which provide decision makers with proper information in different water disciplines. Water data can be classified into two main categories, primary data and secondary data, where primary data is the collection of raw data, and secondary data obtained from models, lab experiments, and specific methods. Additionally, water data types can be grouped based on disciplines/sectors that water data is used. For instance, hydrometric data, meteorological data, and groundwater data are applied in hydrology services, meteorological services, and geological and mining institutes, respectively. Different distinct water data is obtained from four main sources: (1) measurements; (2) models; (3) remote sensing; and (4) administrative institutes. The chapter also discuss the limitation of water data, including poor quality, lack of integrated water portal systems, limited funds for data provision, and big data problems. In order to efficiently store and apply data, water data should be processed and analysed through key steps: (1) data encoding and digitizing; (2) quality control, consistency checking, and error checking; (3) data storage resolution analysis; (4) data calibration and stability; and (5) data storage and retrieval systems. After the data processing steps, water data should be visualized and presented in an understandable way to find our existing climate trend, pattern and correlation for the purpose of adapting new policies and management strategies in the realm of water resources. Thereafter, water data should be released and disseminated through web portals and websites, E-books, media and social networks, and smart apps to be used by decision makers, researchers, and policy makers for efficient water management planning.

In general, the outcomes of the present chapter will benefit different water disciplines sectors through provision of discussion on the role, current condition, and importance of water data and information in water resources management.

3.8 Recommendations

Regarding the important role of data provision in integrated water resources management, it is recommended:

  1. (1)

    To provide public easy access to water data with the purpose to serve many different contexts: In order to improve decision making strategies, enhance national and international research studies, and ensure efficient water resources management, different water data sources and types should be highly accessible. Additionally, free provision of some data sources can potentially increase people’s awareness about the responsibility and importance of different water sectors. As a results of this increased awareness, public tends to have interactions and collaborations with experts, which leads to efficient water supply and management.

  2. (2)

    To improve collaboration and engagement between water data system or portal developers and users: Identification of the intended purpose of data provision is a fundamental need for achieving efficient data use. Therefore, before providing the data, sound understanding of the need of stakeholders and data users is essential to ensure the relevant data collection.

  3. (3)

    To develop a single global data system: Despite the availability of many different sources, portals and websites for data users, some researchers have challenge to identify the most accurate and efficient source of data for their studies. In order to address this issue, multiple sources of water data at different spatial and temporal resolutions should be consolidated in a single global data system.

  4. (4)

    To provide data in different formats and resolutions to target different decisions making strategies: Based on the type of decision-making strategy, research project, and water management strategy, different temporal and spatial resolutions of data should be provided. Hence, an efficient water data system should provide resolution flexibility in data as well as ensuring the quality and integrity of data.

  5. (5)

    To propose and build a novel water data system for water data format conversion: Many researchers and water data users should convert the format of the collected water data into the required input data format of different water software packages. The process of data format conversion is time and money consuming. Therefore, the generation and provision of different water data formats help users to save time in running water software and models.

  6. (6)

    To develop efficient water data systems, which are able to generate and update information from water data automatically: Water data systems should be improved in order to automatically generate some illustrations (maps, figures, and tables) from the input data. Additionally, the new collected data should automatically update the water data system by providing updated illustrations and information.

  7. (7)

    To develop more accurate data gap filling methods: Some data, such as future meteorological data under climate change, should be estimated. There are also gaps in some data due to different reasons, such as defective measuring device. In order to inform decision making and researchers with proper data, the stated limitations and gaps should be addressed.

  8. (8)

    To integrate different data sources to ensure quality data: Various organizations provide water data based on different standards of data measurements, processing, and collection. Water organizations and sectors should integrate water data system by satisfying specific level of standards in terms of data quality, processing, measurements, and documentations.

These recommendations will help water data providers, modellers, and researchers to detect the limitations of current water data and to identify room for improvements.