Keywords

1 Introduction

The pressure on water resources will increase worldwide along this century due to the increase of population and economic development. Population levels are expected to increase at the fastest rate in emerging regions that are already undergoing a strong urbanization process. Investing in the water system’s efficiency should be the first option to allow the cities avoiding water scarcity, as traditional water resources will no longer be guaranteed in enough quantity and quality. In that context, water utilities will have to be as efficient as they can to avoid any waste of water and energy. Water networks will be increasingly complex and may also treat and supply non-drinkable sources such as reclaimed wastewater. The good news is that recent technology breakthroughs such as the Internet of Things and artificial intelligence will allow achieving that higher standard of efficiency.

In this chapter, it is first discussed the water loss control in distribution systems. Related concepts such as sectorization, water audits, pressure management and proactive leak detection programs are addressed.

Then it is assessed the transition of water networks toward truly smart water systems. Thus, the architecture and functionalities of a smart water network are described with the aim to explain how such technologies can increase resilience against extreme climate events, improve the asset managing of water infrastructure, and operate the network efficiently to reduce leakages, pipe bursts and energy waste.

Finally, as smart water systems generate an incredible amount of data, the management of that data becomes a crucial issue. Statistical tools and algorithms allow processing that data efficiently, so it is reviewed techniques about the statistical treatment of data to improve its quality, as well as recent studies about pattern recognition and modeling in smart water systems.

2 Water Loss Control in Distribution Systems

Not all the volume of treated drinkable water that enters a water system will be finally registered by consumers’ meters. That “lost” volume of water, which represents lost revenue to the water utility company, is due to real losses (leaks in the pipes and mains) and due to apparent losses (unauthorized consumption and also inaccuracies and errors committed by the metering system). The water loss indicator is the percentage of the total water losses over the amount of treated drinkable water that is supplied to the system. This indicator gives a good idea about the efficiency of a water utility company. Utilities from countries like Japan, Germany, Australia and New Zealand have water loss rates around 10% and even lower. In Brazil, water losses in distribution systems are on average 40.8% [1]. In the USA, water losses in distribution systems range from 7.5% in South Carolina to 31.2% in Philadelphia, with most utilities performing around 20% [2]. Water distribution systems in small villages typically have higher rates of water losses (around 5% or 10% higher) than big cities [2]. The main reason is that the lower concentration of population makes water loss interventions less cost effective. Water utilities that supply small villages have limited human and financial resources to operate and maintain networks that frequently cover large areas with sparse consumers. Independently of the size of the water distribution system, water losses represent a direct loss of financial revenue as well as a waste of two increasingly scarce environmental resources: water and energy. At a planning level, reducing water losses will contribute to reduce or stabilize future water withdrawals and therefore guarantee water supply.

Water loss control programs are fundamental in the management of water systems. Their aim is to plan a cost-effective strategy to control excessive losses, based on identifying the most viable interventions in the water system, and implementing them. As shown in Fig. 12.1, a water loss program comprises three essential parts: water audit, intervention, and evaluation of results.

Fig. 12.1
figure 1

Flowchart of a water loss control program

A water loss control program is a continuous iterative process. It starts with a water audit, in which the utility tracks the volumes of water that flows from water sources to customers, in order to calculate water losses and performance indicators. At the end of the water audit, opportunities are identified and a strategy that consists of a series of interventions is set. The best interventions are selected according to their technical and economic viability. Interventions typically lie within one of these three groups: pressure management, proactive leak detection, or infrastructure replacement. After the intervention is completed, its results should be evaluated by measuring and verifying the actual water savings, or other type of economic and operational improvements. The results may also justify adjusting the management plan if needed. The revenue recovered from water loss intervention can be re-allocated to fund additional water loss control activities. Periodically, other iterations of the water loss program are started, with the aim to achieve the best cost-effective interventions.

The existence of a flow metering system is essential to conduct a water loss control program. During the water audit stage, it will provide input data for the water balance and for calculating water losses and later on the evaluation of results. It will be crucial to verify the water savings achieved by the interventions. The flow metering data of a water network comes from two main sources: in-line flow meters (turbine flow meters or the more modern ultrasonic meters) that provide the utility with operational data, and the customer meters, which provide information about billed metered consumption. In order to guarantee sufficient reliable data to conduct the water audit it is important to split and monitor the different supply zones or sectors that comprise the network.

The water network can be split into sectors to better monitor the flows that enter and leave each sector, which are often called District Metered Areas (DMAs). Monitoring those flows is particularly interesting late night, when urban water consumption is at its lowest. During nighttime (for example, from midnight to 5 h) flow is at its minimum and pressure rises to its maximum. Historical consumption data of that sector can be used to calculate the share of flow for a particular night that is due to actual consumption and what amount is due to leakages. Also, the minimum water flow registered at night within the different sectors can be compared, in order to identify those sectors with an abnormal consumption level which is indication of a higher incidence of leaks. That comparative should carefully take into consideration the different pressure levels, age of the pipes, and population density of the sectors. Sectorization is very useful to monitor the rise of leak rates in each sector and keep water losses under acceptable levels. However, sectorization also has costs (logging equipment and data retrieval system, construction of meter chambers), therefore, water network managers have to calculate the economic level of sectorization for the network, as well as the optimum size of the sectors to be monitored and which ones are to be prioritized.

Besides sectorization, customer metering is also a key part of the overall metering system of a water network, and an important source of information to elaborate the water balance and calculate losses. Therefore, it is important to be aware that customer meters have inaccuracies that tend to under-meter the authorized consumption of water, which has an impact on the apparent losses that are to be estimated. Customers’ water meters are subjected to inaccuracies, which altogether can represent almost 20% of the total volume of water losses in a distribution system. This is the case in Brazil, where real losses are 72% of the total, while apparent losses represent the other 28%, being 19.6% customer metering inaccuracies and 8.4% unauthorized consumption [3]. The main cause of those metering inaccuracies is that water meters have limitations at measuring low fluid flows. In addition, a series of effects accumulate over time and contribute to the eventual under-registration of the flow: wear of mechanical parts, improper sizing or type of meter for the customer usage, improper installation, or aggressive water quality. Positive displacement and multi-jet meters are accurate for low-to-moderate flow rates of households and commercial users, and therefore are widely used by utilities to measure consumption. Those types of meters have a minimum measuring threshold, typically around 10 L/h, so they do not measure trickle or drip flow, which can represent and important share of the total volume that passes through the meter. Turbine flow meters, consisting of a turbine immersed in the fluid, are less accurate at low flow rates although do not create apparent losses as they are used to monitor flow across an entire sector instead of measuring individual user consumption. There are other types of water meters that perform much better at low flow rates, such as droplet counter devices, compound meters, or non-mechanical devices (electromagnetic and ultrasonic water meters) but these are too expensive to be used at residential and commercial levels. An interesting and economically viable solution may be the installation of pulsed valves before the positive displacement or multi-jet type water meter. Those valves interrupt the passage of flows below the minimum measuring threshold and accumulate volumes of fluid that are later released as a flow that the meter can measure [4].

Metering inaccuracies represent a volume of water that is actually supplied to customers, and in that sense this type of losses is preferable to real losses (leakages) as they do not pose a direct threat to water resources or water quality. However, meters at the consumption points are the cash registers of water companies, and thus high metering inaccuracies have an impact on the economic sustainability of a water utility company.

Water Audit

A Water Audit involves the review of data recorded by the utility’s process meters at different points of the distribution system and at the consumption points (the billed metered consumption). That data will serve to conduct a water balance and calculate water losses, with the aim of identifying the final destination of all the water that is put into the system. The IWA (International Water Association) and AWWA (American Water Works Association) propose a “top-down” water balance methodology that traces the flow of water from its sources to customer properties, as show in Table 12.1.

Table 12.1 Water balance [5]

Utilities can use methodologies and software to compile a standardized water audit, for example the IWA/AWWA Water Audit Method [5]. The Excel®-based AWWA Free Water Audit Software (FWAS) is an interesting tool in concordance with that methodology [6]. Other specific data management software that can be downloaded freely is the CUPSS—Check Up Program for Small Systems, an asset management program created to assist small systems in water loss management as well as in operation and maintenance scheduling [2, 7].

Once the data from the metering system is gathered, it must undergo a statistical analysis to eliminate outliers and to assess the uncertainties of the data, mainly the under-metering of mechanical meters and the systematic data handling errors. Then the water balance (Table 12.1) can be conducted to calculate water losses as per the following steps:

  1. 1.

    First, calculate the total water losses as the water input minus the authorized consumption (the billed consumption, registered by the water meters).

  2. 2.

    Then estimate apparent losses, which are a sum of (I) unauthorized use, (II) customer metering inaccuracies, (III) systematic data handling errors.

  3. 3.

    Finally, calculate the real water losses (leakages and overflow of tanks) as the difference between water losses and apparent losses.

  4. 4.

    The result obtained for the real water losses should be validated though component analysis of real losses and a “bottom-up” analysis of real losses using district metered area (sectorization) and an analysis of minimum night-time flows.

The results of the water balance, based on metering data and estimates, should complement a Geographic Information System (GIS) linked to a detailed inventory of the water system (metering points, pipe sizes and types, reservoirs, list of water lines with quality complains and high incidence of detected leaks).

The main output of the water audit is the calculation of water losses through the water balance and further validation analysis. In addition, the water audit calculates a variety of performance indicators to assess the efficiency of the water utility, such as: Real losses (% or gallons/service connection/day), Apparent losses (% or gallons/service connection/year), Infrastructure Leakage Index (ratio), Non-Revenue water by volume (%), Energy indicators (kWh/m3), and so on.

Once the water losses and the indicators are calculated, the next step is to determine the monetary valuation of existing water losses and the current level of expenditure to manage them. Each water system and utility has to track its various costs, which are to be considered in the assessment. In every water system, there will always be a small rate of water losses as completely eliminating them would be uneconomical. Therefore, the economic assessment should calculate the economic level of leakage that is appropriate, in which the utility does not spend more on loss control activities than the value of the benefit it expects to recover from the loss reduction. That economic level of loss will serve to establish performance targets. Those outputs will serve to plan a cost-effective strategy to control excessive losses. Figure 12.2 summarizes the inputs and outputs of a Water Audit.

Fig. 12.2
figure 2

Inputs and outputs of a water audit

The water audit should guarantee that only cost-effective water loss intervention strategies are pursued, and identify among them the most technically and economically viable strategies. The effect of those interventions can be modeled before their implementation, via the use of water network modeling and analysis software such as EPANET or KYPipe [8, 9]. A hydraulic model considers the system’s operating parameters (flow rates, pressures, water quality, pipe sizes and age, etc.) and, once calibrated against field reality, can simulate how the system would behave after the implementation of the measures. After selecting the most appropriate intervention, the last step is to formulate an implementation plan and assess and allocate resources for it.

The IWA—International Water Association points to four basic strategies that can be used to reduce losses [5]:

  1. 1.

    Proactive leakage detection.

    Aims to reduce any backlog of leaks and keep losses from unreported leaks at an economic optimum. Within this strategy, it should be determined the economically optimum leak detection frequency.

  2. 2.

    Improve the speed and quality of repairs.

  3. 3.

    Pressure management.

    A rule of thumb is that 10% reduction in pressure results in 10% reduction in real Losses (leakages) [10]. Within this strategy, how much the pressure can economically and technically be reduced should be determined.

  4. 4.

    Infrastructure rehabilitation and replacement.

    Besides water savings, the direct and embedded energy savings by infrastructure replacement should be assessed. For example, the decrease in pipe roughness, bigger pipe diameters, and pipe retrofit resulting in fewer leaks will also result in energy savings.

    Proactive leak detection, and improving the speed and quality of repairs are strategies that can be implemented in the short term. Pressure management and infrastructure rehabilitation involve greater investment and should be considered in the economic long term. In the following section, leak detection and pressure management are discussed in more detail.

Leak Detection

There are three categories of leakages [10]:

  • Background Leakages—Continuously running small leaks that are undetectable with conventional leak noise detection equipment. Precisely, background losses represent the volume of water that is lost through small leaks and weeping joints at flow rates that are too small to be detected using detection equipment. This type of losses will flow continuously until they are found by chance during some other maintenance work or until they gradually increase to the point where they become detectable.

  • Unreported Leaks—Leaks with moderate flow rates that escape public knowledge and are only identified through the active leakage control work of the water utility.

  • Reported Leaks—Large leaks that are reported by customers, traffic authorities, or any other outside party due to their visible and/or disruptive nature.

Figure 12.3 illustrates the three types of leakages, together with the strategies to minimize their occurrence.

Fig. 12.3
figure 3

Types of leakages and strategies to minimize their occurrence [11]

The leak detection techniques can be classified into four groups:

  • Visual survey. These are reported leakages that surface above pipes or that are located in valves, hydrants, meters and other exposed parts of the network.

  • Acoustic leak detection survey. This is the most common and inexpensive leak detection method. A trained operator uses listening devices in all available fittings on mains and service connections, as well as geophones above the pipes, searching for leak noises. The instrumentation ranges from simple mechanical listening sticks and rods to sophisticated geophones with digital filters.

  • Step testing. This involves temporarily sectorizing parts the network. Small sections of the water distribution system are isolated and then have their supply measured during the minimum nighttime flow period though portable flow meters.

  • Leak noise logger survey. Noise loggers are installed, permanently or temporarily, on pipe fittings such as valves and hydrants. They can periodically send “packages” of data automatically via a radiofrequency emitter, or log data in an internal memory until an operator arrives.

Leak detection equipment has greatly evolved in recent years. Acoustic equipment has incorporated electronic advances to assist the operator in detecting and to precisely locate leaks. In bigger size transmission mains, where leakages represent higher economic loss and can even disrupt the supply, more expensive instrumentation has been developed such as above-surface radar or sensors inserted into the pipes. Pipe inspection through automated vehicles (robots equipped with cameras or sensors) is still not common in water networks, as is the case, for example, in the oil and gas industry, where PIGs (Pipeline Inspection Gadgets) regularly inspect the pipes and perform various maintenance operations. However, currently there are sophisticated mobile sensors for the inspection of water networks, such as free-floating inline acoustic leak detectors or streaming cables. The first of these consists in a small plastic ball that houses acoustic sensing equipment in its core and travels along the length of the pipe with the flow, recording the acoustic profile of the pipe. In the latter case, the sensor travels across the pipe pulled by a small parachute at its head, transmitting an acoustic signal through the streaming cable to a trained operator on the surface. Both instruments inserted into the transmission main have location sensors together with the acoustic sensors so they can pinpoint its exact location in the pipe when a leak signal is detected. Table 12.2 summarizes the set of equipment available for leak detection, for both small and big diameter pipes.

Table 12.2 Leak detection equipment

Water Loss Reduction Through Pressure Management

Water leakages depend on the water’s pressure in pipes, according to the expression:

$$\frac{{L_{1} }}{{L_{2} }} = \left( {\frac{{P_{1} }}{{P_{0} }}} \right)^{N}$$

It expresses the relationship between changes in leakage (L) due to changes in pressure (P). The above expression is known as the “Fixed Area and Variable Area Discharge” theory [12]. The exponent in the pressure-leakage relationship is known as the N value. It generally ranges between 0.5 and 1.5, but an average N value of 1.0 and a value of 1.5 for background can be assumed [13].

Pressure reduction will reduce leakage due to two effects. First, as leakage flow is directly related to pressure, reducing pressure will reduce both background and leak flow rates. That’s why pressure management is particularly effective in minimizing background losses or widespread small leaks across the system. In addition, a second effect is that burst frequency rates will also be reduced due to reduced stress on the pipe network [10]. Therefore, pressure management is also a valuable strategy to inhibit water main bursts from occurring and improving the lifetime of the water distribution system infrastructure.

Pressure management involves adjusting pressure to an optimum level of service, in which sufficient and efficient supply is guaranteed to legitimate consumers while at the same time pressure transients and excess pressure is avoided, therefore reducing leakages, pipe breaks and energy waste. There are three main strategies to achieve an effective management of pressure in a water network: split the network into different pressure zones, pressure control valves and pump controls.

Pressure Control Valves include various types of valves such as relief, reducing, sequence, counterbalance, and unloading valves. Pressure reducing valves are the most used to maintain reduced pressures in specified locations of hydraulic systems. They use a spring-loaded spool to control the downstream pressure to preset values, which are adjusted by a screw on top of the valve. When the downstream pressure increases to the valve setting, the spool moves and partially blocks the outlet port, thus controlling the flow that passes through the valve and this keeps pressure below the desired level.

Pressure can also be managed by varying the flow of water that is pumped throughout valves, recirculating a part of the flow, or controlling the pump rotation via variable frequency devices. The most energy-efficient solution is the use of variable frequency devices. These electronic devices can control the water flow by controlling the frequency of the alternated current and voltage that is supplied to the pumps’ motors, and therefore the speed to which motors rotate. Modern variable frequency devices can act as programmable logic controllers so they automatically vary the rotation of the pumps, and therefore, the flow of water that is pumped, to maintain a preset value of pressure in the network.

Pressure management is commonly used by water utilities worldwide. It has also been used as a water conservation tool under emergency circumstances. For example, the city of Cape Town pioneered extreme water pressure reduction or “throttling” resulting in an intermittent supply, during the water crisis of 2014–2018. Meanwhile the water utility of São Paulo, Sabesp, made the installation of pressure control valves a key point of its strategy to face the 2013–2015 drought. Both strategies were essential in the successful response of those cities to the water scarcity threat they experienced. For instance, in the case of São Paulo the rate of water losses dropped from 35.8 to 30.6% after the widespread installation of pressure control valves [1].

There are some potential concerns with pressure management. First, minimal water pressures must be maintained to satisfactorily meet customers’ varying water demands. According to the typical water demand profile of urban consumers, pressure drops to minimum levels during the central hours of the day, as demand (flow) is at its maximum and, during those hours, shortage could occur if pressure is too low. During nighttime, pressure typically rises as demand drops to its minimum. Therefore, it is important to develop a pressure management strategy that takes into consideration the effect of the water demand profile and varies pressure accordingly. Another potential concern is to provide sufficient pressure for firefighting flows, in accordance with local norms. Finally, an excessive low pressure could cause the backsiphonage or backflow of contaminants. If there is a link connecting a source of pollution and the potable network, and the latter is at lower pressure, there will be a net force toward the potable supply and polluting substances will tend to enter the potable network. Hydraulic systems of buildings count on backflow preventers such as air gaps and atmospheric vacuum breakers to avoid any link or channel (cross-connection) to potential sources of pollution. More serious is the case of underground leaky pipes, in which low pressure can allow possible intrusion of microbial and chemical contaminants.

3 The Transition Toward Smart Water Systems

Water poverty already affects an important share of low income urban and rural population in the forms of limited, time-consuming and unsafe access to the resource as well as a high incidence of waterborne diseases. As quality drinking water becomes increasingly scarce, it will be more unevenly distributed among society [14]. Universalizing access to potable water and sanitation by being efficient and avoiding waste of resources may be the most important challenge of water networks in future years.

It is also noticeable that most undeveloped parts of the world will undergo a strong urbanizing process in the next decades. In that sense it will be crucial to develop decentralized, low cost, and remote-controlled water systems capable of supply clean drinking water to a growing urban population, which is often located in risk or polluted areas such as slums or marginal lands with little infrastructure.

Other challenges of water networks in the actual context are:

  • Conserve as much water as is economically viable, as traditional water resources will no longer be guaranteed in enough quantity and quality.

  • Increase resilience against extreme climate events (droughts and floods).

  • Manage an increasingly complex network that may also treat and supply non-drinkable sources such as reclaimed wastewater and rainwater.

  • Manage the different pressure levels of the network sectors efficiently to reduce leakages, pipe bursts and energy waste.

  • Asset management of an aging water infrastructure, while managing limited budgets.

  • Minimize the awareness, location and repair time of leaks.

Overall, the water industry has a huge need to be more efficient. But recent technology breakthroughs have finally arrived to the water supply sector. Most water networks have already experienced a digital transformation, with an increasing number of sensors and automation in its processes. Automation components, sensors, telemetry and computers have been constantly reducing its price during the past two decades. Internet connection is widespread, affordable and reliable. This has led to a point where is viable to take another step and start a transition from “digital” to “smart”. Thus industrial processes are experiencing a transition toward “Industry 4.0” and electricity networks are becoming smart grids capable of absorb increasing amounts of intermittent renewable energy while dealing with the new challenge posed by the rising of electric cars. The emergence of the IoT (Internet of Things) allows engineering smart buildings and even smart cities that can better manage their infrastructures, water and energy supplies or traffic flows. Similarly, water networks are undergoing transformations that will disrupt the way a water system is operated and managed, and will increase its efficiency to pace with the new standards required by the global water scarcity context. Besides water scarcity and regulatory requirements, the evolution toward smart water networks is also driven by market dynamics, or the constant need to adopt the technological advances to reduce costs, and the need for solutions to more cost-effectively manage billing and customer management, leakage rates, and energy consumption.

Water, energy and gas companies started to use smart meters to record and transmit consumption data and to generate the monthly bills. It soon became clear that the automation of the reading and billing process saved a lot of time and money when compared to the previous situation, where a human worker had to walkthrough visiting all and every of the consumption points and annotate the meter readings [15]. The new walk-by/drive-by systems, consisting in a operator walking or driving through the streets while uploading the data from the meters, were eventually outpaced by advanced metering infrastructure, defined as two-way, fixed radio or cellular communications network. The meters now send their readings directly to the network, where they can be stored in the cloud. The consumer can also have access to that data to, for example, monitor its own consumption or get data about tariff periods or the quality of the water that is receiving [15].

Smart water consists in a group of emerging technological solutions that help water managers operate more efficiently and, in a smaller scale, help consumers tracking and managing their water usage. Those solutions are based on leveraging the Internet of Things, Big Data, and analytics. IoT offers the technological background (hardware and software) for data acquisition, transmission, and operation. The visual display of a myriad of data from the water network helps the water managers to monitor, optimize and take real-time decisions. Meanwhile, Big Data allows going a step further toward system intelligence. First, cloud-based information technology allows storing very large data sets. Secondly, through the application of artificial intelligence, the value of data can be can be maximized as those data sets can be analyzed by computational models to reveal patterns and trends. As a result, the water system becomes capable of gathering and analyzing data and also communicating with other systems. Most of the operation of the water system (pressure management, flow control, etc.) can be automated and optimized.

Following, it is discussed the architecture, key elements and functionalities of a smart water system.

Architecture

The concept of Internet of Things (IoT) refers to a network of physical sensors embedded with electronics and software that allows getting real-time data of any measurable parameters such as level, flow, pressure, temperature, noise correlations, etc. and make them available online. Figure 12.4 illustrates the flowchart of information across the different components: sensors, actuators, controller, data server and interface devices.

Fig. 12.4
figure 4

Architecture of a smart network (IoT)

The microcontroller is the core of the system and acts as a data concentrator, receiving real-time data from the sensors and sending “packages” of information to the web server [15]. The web server makes the data available online, where it can be stored, processed and analyzed. The system can be limited to only receive and display the information to human managers that will make the decisions, or can have some degree of system intelligence that allows operating some parts of the network automatically. Whether is human supervised or the system’s algorithms, the microcontroller will receive the control orders (set points for variables such flow or pressure) and operate the components of the system (pumps, valves …). That is, a set of microcontrollers receive the sensor readings and transmits the data, and then receives the commands and operates the system. The “brain” of the water network (human or AI) communicates with the controllers to monitor the system and perform remote controlling of it.

Visual Display of Information

One of the most basics functionalities of a smart water system is to monitor the network. The system is fed at regular intervals with data from the sensors deployed across the network. That information can be displayed within a map interface that is linked to a Geographic Information System (GIS), in which each sensor node can be identified through geolocation. Figure 12.5 shows an example of such a map.

Fig. 12.5
figure 5

GIS map of a water network [16]

The aim of such interfaces is to visualize data in ways that make it easier for engineers to identify anomalies. Interfaces should be easy to view and to navigate, quick to update and represent a sufficiently broad context that allows extracting conclusions. Any operation variable exceeding threshold values should be automatically highlighted. In addition to that, the monitoring system could allow to set alarm threshold values, in order to automatically send SMS or emails to the water management team.

Smart Metering

A smart meter is an electronic device that records one or multiple parameters and communicates the information to the system. A smart meter can be configured to track multiple parameters at the same time, such as static pressure, flow, temperature or chlorine levels.

Every part of the network, from hydrants and valves to pipes and meters can be equipped to collect—and in some cases analyze—data. It is now possible to monitor nearly every asset of a utility, which should conduct studies to determine the level of metering that is economically viable. Automatic Meter Reading (AMR) that record flow, particularly the ones used to record and bill water consumption, represent the biggest share of new-acquired smart meters by water utilities. Smart meters equipped with pressure sensors and noise correlators are slowly increasing their presence. Following, Fig. 12.6 shows a battery-powered water meter that can measure flow, pressure and temperature.

Fig. 12.6
figure 6

Courtesy Khrone Messtechnik GmbH [17]

Electromagnetic bulk water meter.

The above electromagnetic bulk water meter can be used for leak detection, quality control and pressure management systems in water networks. For metering the water volume supplied to small consumers, two types of smart meters are widespread: pulsed water meters and non-intrusive ultrasonic meters. The latter ones are non-intrusive, more accurate and also more expensive. The information registered by both types of smart meters is processed by the water utility but it can also be accessed online by the consumer, which views it in the form of graphics and consumption reports, as Fig. 12.7 illustrates.

Fig. 12.7
figure 7

Courtesy Badger Meter, Inc. [18]

Ultrasonic (left) and pulsed mechanical (right) flow meters, provided with wireless transmitters.

Fixed acoustic correlators are an example of a device that records and also analyzes the data before transmitting it, in order to report assertive data to the water utility. They evolved from noise loggers that used to be installed in valves boxes and other parts of the network and then picked up after some days of recording in order to retrieve and analyze the data with a noise correlation program. State-of-the-art systems, such as the ones showed on Figs. 12.8 and 12.9, are smart sounding systems capable of analyzing the acoustic signal themselves. They are equipped with proprietary filters that periodically compare the acoustic signal collected by the sensor to baseline acoustical signatures at each location. When an acoustical anomaly is identified, the meter sends a data file to an analysis module. That module automatically requests additional correlation data from surrounding meters and performs multiple correlation combinations to accurately target the location of the acoustic anomaly. If the leak is confirmed, the device sends a notification to the utility. In addition, such system is linked to a GIS that displays the information over a map of the network and enables an operator to view and manipulate a correlation graph. This greatly reduces the awareness time needed to detect a leakage, but also its location time, as the graphical data improves confidence in the exact location of the leak. Water managers can better schedule their field investigations and allocate the limited resources of the utility. Figure 12.8 shows a smart leakage detection system, consisting in acoustic correlators developed to be installed in fire hydrants. Traditional caps are replaced with “smart hydrants” equipped with the device. Those smart hydrants conform geolocated nodes that communicate with a central data collection hub.

Fig. 12.8
figure 8

Courtesy Echologics and Mueller Co. [19]

Smart hydrant cap

Fig. 12.9
figure 9

Courtesy Echologics and Mueller Co. [19]

Monitoring water mains through smart acoustic correlators linked to GIS.

When searching for locations to place noise correlators, fire hydrants are preferable than underground chambers or valve boxes because they offer a less aggressive environment as well as stronger and more stable radio signal due to the above-ground location of the antenna. Figure 12.9 shows another type of smart acoustic correlators linked to GIS that search for leakages in water mains.

This type of non-intrusive monitoring platforms can be installed in a chamber or in any other location of a water transmission main where access to the acoustic signals of the water main can be obtained. They are typically placed 500–1000 m apart [19]. Data files are recorded at user-specified intervals and wirelessly uploaded for analysis. Each monitoring platform consists of a data processor, communication hardware and a battery power source. At assigned times that can be specified by the user, the collected data is uploaded to a server where advanced algorithms interpret the data, search for leak signals and generate reports. That is, data is uploaded to the cloud to be analyzed by software that automatically sends reports and sends alarms in case of leak events. A customized information interface is created for the utility to facilitate the visual display of data and the interpretation of the reports.

Communication Protocols

The widespread option for data transfer from permanently deployed battery powered communicating sensors has been the 2G mobile communications networks, in the forms of SMS for very small data volumes and GPRS for larger data transfer. Nowadays, 2G networks are on their path to obsolescence and being decommissioned to free up spectrum for higher capacity 4G. But cellular standards that could replace 2G such as LTE-M and Narrow Band IoT (NB-IoT) for 4G are still in their early stages. As they evolve during next years, sensors and actuators that were deployed early in the smart water network, and use 2G, could become incompatible with new communication protocols.

Mobile internet is evolving from 2G and 3G to 4G and 5G with some uncertainties about which would be the prominent communication protocol for the next years. Meanwhile, emerging communication protocols for low power remote sensor applications have arisen. They are called Low Power Wide Area Network Protocols (LPWAN), and the two main examples are Sigfox and LoRa. These are proprietary systems (closed source or non-free software) which generally operate in license-free spectrum of frequencies. Smart water utilities have to choose between mobile communications or the new proprietary LPWAN solutions. That is an important issue for the convergence of infrastructure communication i.e. the interoperability of communications infrastructure across a single water network and also across the different networks of water, gas and electricity.

As water utilities are increasingly investing in telemetry and smart system, they will have to deal with the issue of choosing a communication protocol that is affordable, doesn’t become obsolete too soon or causes incompatibility between older and newer components. Maybe a reasonable strategy is to stick to mobile communications and work with 2G and 3G while, at the same time, preparing a roadmap in the expectation of moving to NB-IoT for new and replacement sensors in around 5 years’ time [20]. System managers of water utilities should assure that the network is flexible enough to accommodate that transition.

Other key aspects about communication protocols are:

  • The storage of large datasets will rely on cloud-based information technology.

  • Cybersecurity will increasingly be a high priority. Protection against hacking incidents should be considered.

  • Most sensors deployed on the water distribution network are battery powered. Batteries and costly, as well as the visiting the sites to replace spent ones. Protocols that make a smart data transmission can increase the life span of sensor batteries. LPWAN, for instance, offers low transmission and network costs and meter battery life-spans of 15–20 years.

  • In addition to the above, it is important to notice that real -time transmission of data is not always necessary for the system to perform optimally. It is important to use battery power sparingly and only when it’s needed, which means that the sensor communicates only when the information will be really useful. If the sensor has to be always on and frequently sending back data, the battery will be spent much faster. A more convenient configuration for data transmission in most sensors may be to give real-time priority to any alarm that could be generated, as well as the ability to respond anytime if asked, but otherwise deliver only a small but representative dataset each day.

New Business Models

Smart metering has allowed the development of new business models such as online prepaid water consumption. This system requires the installation inside the home or business of an internal unit that acts as the customer’s interface. The customer will insert in it a “Smart Card” and information regarding the credit status, the volume of water remaining, tariffs and potential leakage alarms will be displayed.

Customers purchase credit at vending points using their smart card. At that point, data from the meter that was stored on the Smart Card is uploaded and sent to the utility headquarters. When arriving home and inserting the Smart Card into the internal unit, the internal unit will communicate, through wireless radio, with a external meter box that houses a water meter and a control valve. The solenoid valve will open and allows the costumer to use the volume that has purchased.

This business model offers advantages to both customers and water utilities. For the first ones, it offers a tool for water usage management and more control towards its expenses with water consumption. For the utilities, guarantees payment prior to water distribution and removes the need for a billing department or meter readers. Figure 12.10 shows an overview of such a system.

Fig. 12.10
figure 10

Courtesy Universal Metering Ltd. [21]

Overview of a smart system for prepaid water consumption.

Asset Management and Scheduling of Repairs and Replacements

Physical assets like pipes, valves, pumps, wells, hydrants, treatment facilities and other infrastructure tend to deteriorate over time. That increases the cost of operation of the water network and eventually causes disruptions on supply. The main aim of asset management in the water utility sector is to achieve long-term sustainability and deliver consistent service in a cost-efficient manner [22]. Asset data is crucial to make informed, data-driven replacement and pro-active maintenance decisions that extend asset service life and reduce system failures. Figure 12.11 shows the five core components of asset management.

Fig. 12.11
figure 11

The five core components of asset management [23, 24]

A smart water system relies on remote sensors that gather data and applies system intelligence to monitor and operate the network. Thus, it is easy to implement some asset management functionalities within that architecture. For example, a sensor that measures and transmits flow and pressure data in a valve can easily be adapted to also transmit data about wear or vibrations in the valve. The analysis of the data from a leaking detection system can also serve to detect which pipe sectors tend to deteriorate and burst more. By managing strategic assets according to their performance, utility managers can make better decisions about repairing, replacing or rehabilitating aging assets, and also develop an effective strategy for long-term funding. The evolution toward smart water networks will allow switching to maintenance based on continuous condition monitoring of assets, which as the following advantages:

  • Create more accurate inspection schedules for predictive maintenance.

  • Prolong the operational lifespan of assets with preventive service, maintenance and workload allocation.

  • Perform timely corrective maintenance and repairs. In particular, rapid event detection reduces the awareness time of emerging problems, speeds up resolution times and gets ahead from eventual customers complains.

  • As any failure in a component is rapidly detected and fixed, reduces the risk of escalation to a catastrophic failure that may also damage other parts of the system.

  • Manage risk more efficiently and improve the response to emergencies produced by asset failures.

  • Forecast the future needs for funding and prioritize investments that meet the system’s actual requirements.

In sum, maintenance based on continuous condition monitoring of assets requires considerable capital investment in sensors and IT but reduces the overall cost of maintenance, extends asset life and, as rapidly detects leakages and malfunctions, also improves the quality of customer service.

Electric motors, smart devices, and pressure reduction valves (PRVs) are good examples of applications for continuous condition monitoring. Electric motors that power key components such as pumps can be provided with sensors that measure vibration, temperature, electrical parameters or running hours. Sensors will generate an alarm at any moment if temperature or vibration rise above threshold levels. If not, they will just keep logging the measures and send a report periodically that will serve for data-driven predictive maintenance.

The estate management of a large number of smart devices is new challenge that has arisen in smart networks. Thus, continuous condition monitoring applied to smart devices serves to identify issues such as loggers that are struggling to communicate or batteries that are nearing end of life.

Finally, a pressure monitoring system involves to monitor valve condition, flow, pressure, and transients across the network while optimizing energy efficiency for pumping operations. Pressure sensors located in pipes, valves and critical points collect maximum, minimum and standard deviation pressure values in each measurement interval. Data is used to operate the system in a way that minimizes water loss and energy consumption, but also to prevent failures through preventive maintenance and setting alarms when pressure reaches threshold levels. PRV condition monitoring typically uses 3 pressure readings, one from upstream, another downstream and the third from a control space. The monitoring system can even identify some causes of failure: maintenance required, if the valve is bypassed or incorrectly setup, or if it is unable to fully close or open. The system’s engineers receive that pre-diagnosis together with the raw data from the monitoring system, which assists them in providing a quick response to the alarm.

Water Loss Reduction

A smart water system can reduce water losses by three ways. The first one is by manage pressure more accurately. The second is by improving asset management, because predictive maintenance and data analytics help preventing pipe burst and failures before they happen. The third way through which a smart water system reduces water losses is by reducing the time needed to be aware of and locate new leaks.

The total run time of a leak (T), which is the time passed since it occurs until it is repaired, has been found to be a crucial factor influencing the volume of water lost to leakage [10]. T (total run time) is the sum of A (awareness time), L (location time) and R (repair time):

A—Awareness time of leaks is the time passed since a leak occurs until it is detected. In other words, is the time needed for the operator to become aware that a leak exists. In urban areas a visible break in the mains will not take more than 24 h until it is reported but most leakages can only be found through active leak detection. In those cases, the awareness time depends on the utility’s implementing or not an active leakage control program.

L—Location time is the time taken to pinpoint the source of the leak once the operator is aware of its existence. Location time is short for reported breaks that are visible. For unreported leakages, it depends if it has been detected through acoustic equipment or though night flow monitoring. Acoustic leak detection allows pinpointing the leak almost at the same instant that it is detected. Differently, regular night flow monitoring allows the utility to be aware of the new leak in a general area, but might take additional time to locate its exact location.

R—Repair time is the time to repair and stop the leakage flow, once the leak position has been identified.

Location time is difficult to reduce in the cases where night flow monitoring has indicated a general area that is leaking. In those cases, inspection crews equipped with acoustic instrumentation should be sent to the area and even the best team will take its time until pinpointing the leak. Repair time depends on trained crews that use the appropriate materials and are equipped with the adequate tools to safely repair leaks quickly and securely. It can be improved in the initial stages through training and equipment. Once repair crews are properly equipped and trained on a variety of fix techniques, so they achieve an excel level of expertise, it becomes increasingly difficult to reduce repair time. Meanwhile, awareness time can be reduced by active leakage search, by night flow monitoring, or by monitoring both flow and pressure and detecting “abnormal” values that are not coincident with the ones expected in the absence of leakages. There is more opportunity to improve awareness time at a lower cost, in comparison with location and repair time. To illustrate this situation, imagine the volume of water that is lost in a leakage of a size of 10 m3/day. Let’s suppose that a water utility manages to reduce the awareness time from 25 days to only 4 days, while maintaining the same time needed for location and repair. Figure 12.15 shows how such reduction of awareness time, which is technically achievable in most cases, would be translated in lesser volume of water lost during the leak event (Fig. 12.12).

Fig. 12.12
figure 12

Effect of the reduction of the “awareness time” on the leak loss volume

In particular, awareness time could be greatly reduced by smart metering of pressure and flow, combined with artificial intelligence (AI). The supervision of the data from the smart meters allows detecting any disruption of “normal” values of flow and pressure that may be caused by leakages, allowing their early detection. But that task is time-consuming and requires a good knowledge of the behavior of the water system, which means that water utilities should divert high skilled human resource and allocate it for data monitoring and interpretation. The advances of AI, in particular mathematical modeling, data storage and processing speed allows delegating the monitoring and interpretation of the real-time, remote data to computers that automatically compare the received data to the network’s hydraulic and water balance models. Human operators and engineers can still analyze data records to reach conclusions and improve the system’s operation, as well as develop and calibrate the mathematical models. It is noticeable that through the advances on machine learning and pattern recognition, computers will progressively be capable of developing and calibrating the models by themselves.

Resilience

The emerging water crisis is not exclusive of regions that face a rapid urbanization in developing countries. In the past two decades, a series of extreme drought events have affected heavy populated, wealthy areas: Melbourne in the 2000s (Australia’s Millennium drought), California (2011–2017), São Paulo (2013–2015), Cape Town (2014–2018), parts of Texas (2018) and Brasilia (2018). For the first time at such large scale, those populated areas have been subjected to rationing and threatened with lack of supply. Water stressed megacities such as Los Angeles, Istanbul, Delhi, Beijing or Seoul have also shown vulnerability in their water supply in recent years. Among the above-mentioned cities, Cape Town in South Africa and São Paulo in Brazil offer two relevant study cases of resilience under urgent water scarcity threat, as both cities succeeded in reducing its water consumption, mainly through the control of the operation pressure and through demand management.

Unusually extreme and lasting droughts must be addressed through making a more efficient use of water resources. That can be achieved by (I) the use of “alternative” resources such as reclaimed wastewater, rainwater and desalination and (II) improving water conservation in industry, agriculture and cities. The latter, particularly reducing water losses in the network, is where a smart water system excels.

Water scarcity is going to be a main issue and most cities will have to improve their resilience with regards to it. The other side of the coin is extreme floods. In that sense, the architecture and functionalities of smart water systems allow re-engineering storm water management practices and urban infrastructure planning [25,26,27,28]. The concept of smart water can be applied to sewer systems, in order to conduct a continuous condition monitoring for improving infrastructure maintenance, to monitor sewer flooding and also to apply system intelligence and make optimum use of the existing infrastructure. An ideal application is to prevent sewer overflow.

An extreme rain event can cause an overflow of stormwater and sewage into creeks and rivers, polluting them and threatening lives and infrastructure. If it is raining heavily in a part of the city and the sewer system of some districts is overload, a smart sewer system can make use of the available capacity in other sectors where it hasn’t rained. It would use sensors to monitor flows levels and operate valves and gates to divert the excess flow to other parts of the system that can handle it. A smart sewer system maximizes the use of existing assets and reduces the need to build large sewers and storage tanks. In sum, real-time sensing technologies can provide watershed-scale monitoring to produce quality data to drive decision-making, more rapidly and assertively. Smart sewer solutions introduce more flexibility in existing collection networks, which maximize storage and retention capacity of existing infrastructure through real-time control systems.

Automation of Pressure Management

Pressure management is vital to maintain water losses under acceptable levels. In addition, it can generate energy savings in the pump stations. A smart water system should enable remote control and automatic optimization of pressure across the network. Fed with real-time pressure and flow data from sensors located in control points, the system intelligence would calculate the optimal values for pressure and flow and operate PRVs and pumps. To operate the system and adjust those two variables, there are two main components: electronic actuated PRVs and variable frequency devices that control the rotational speed of pumps. Figure 12.13 illustrates a remote controlled PRV.

Fig. 12.13
figure 13

Courtesy i2O Water Ltd. [29]

PRV control.

As can be seen in Fig. 12.13, a PRV used to remotely control pressure relies on a controller that communicates with the system and an Advanced Pilot Valve (APV). The PRV is typically a globe type diaphragm actuated pressure reducing valve with considerable diameter (>50 mm). The controller receives flow data from upstream the valve, as well as pressure data from sensors located upstream, downstream and also in a control point located under the influence zone of the PRV. Pressure and flow data are processed by an algorithm that automatically determines the optimal control curve that should be applied. The algorithm gets refreshed with new data inputs, so any changes in supply, demand, headloss, etc. over time are incorporated automatically into an updated control curve without the need for any manual analysis or intervention [29]. The controller can adjust pressure in the control point, or automatically deliver target pressures at another critical point. The results of state-of-the-art systems for automatic optimization of pressure can be very satisfactory, as Fig. 12.14 illustrates.

Fig. 12.14
figure 14

Courtesy i2O Water Ltd. [29]

Pressure curves before and after automatic optimization of pressure.

Though the implementation of a system as the one described above, the water network obtains a set of new functionalities:

  • It can be used just for remote control, implementing a control curve that human engineers have manually calculated. The controller would control pressure by adjusting it at any moment to a fixed outlet value.

  • Under automatic optimization mode, not only the control task but also the calculation of the pressure is delegated to the system itself. The algorithms would determine the optimal control strategy to achieve a targeted minimum control point pressure. The system automatically adjusts for flow-related headloss and delivers smooth and accurate control. Depending on the level of artificial intelligence involved in the algorithms, the system could learn and adapt to seasonal, cultural and growth-related changes in demand over time.

  • If an event is scheduled to happen (for example, a cultural or sportive event that is going to have a significant influence in water consumption), its requirements in terms of flow and pressure can also be scheduled in advance using a simple calendar function. That will guarantee water supply whether the preferred control strategy is remote control or automatic optimization of pressure.

  • Constraints on how control is exercised in extreme circumstances can be set to ensure that the network is protected.

4 Data Management in Smart Water Systems

Without an effective management of the data provided by the network’s sensors, it can be easy to get buried under an ever-growing mountain of information [30, 31]. Statistical tools and powerful algorithms are crucial to process that data efficiently. The firsts provide data quality control and reduce instances of “dark data” so the data that serve as input for the algorithms are more reliable. Meanwhile, algorithms can analyze computationally large data sets to reveal patterns and trends, and allow for more effective decision-making.

Statistical Treatment of data

Basic concepts of learning from data structure, inference and forecasts based the proposal of many methods that inspired statistical and machine learning disciplines. About 20–25 years ago, with the advent of powerful and user-friendly software, statistical and machine learning tools emerged focusing on supervised and unsupervised learning that started to be used not only by statisticians and computer scientists but also by a broader community [32]. Many of these methods started to be applied to residential water demand management relying on data generated by processes related to smart water data. As data-driven methods, the challenge is to transform the meter data into actionable insights that trigger decisions and measures.

With or without preconception of the data under study, exploratory data analysis (EDA) as the first step of any learning can reveal a fundamental understanding of the data and its structure allowing, for instance, the identification of water consumption patterns and variations over time, and outliers that can be identified as excessive water users or leaking. An outlier can occur due to extreme variation or incorrect measurement caused, for instance, by corruption of sensor installation or operation or by the data gathering process. The unveiled information can be applied to empower the customers for changing their habits and for a real-time action, and to build an integrated decision support system that allows smarter building management and a smarter water service. Based on a diverse quantity of published papers, a comprehensive discussion evolving data gathering, water end uses characterization, user modeling, leakage detection and pipe bursts, water distribution sectorization is presented by Britton et al. [33], Cominola et al. [34], Gurung et al. [35] and Nguyen et al. [36].

Database can be subject to a diverse graphical analysis to find probabilities of fixture use, peak hours of use, fixture flow rates and efficiency levels, and per capita correlations per number of bedrooms and bathrooms using various sample selections. The dynamics of water consumption data on different time scales can be evaluated by using time series plots. Box plots can be used to understand data distribution and spread and to represent location statistics (minimum, maximum, median, 1st and 3rd quartiles, interquartile range—IQR, and whiskers). They can also be used to identify outliers. However, when data are skewed, the inclusion of a measure of skewness in the determination of the whiskers can avoid that erroneous data are declared as outliers [37].

Supervised, semi-supervised and unsupervised techniques can also be used to outlier detection, as presented in Table 12.3. They consider the knowledge of normal and anomaly classes, the knowledge of normal class for labored instances and no information of classes in the training data set, respectively.

Table 12.3 Summary of outlier detection approaches (based on Lhango et al. [38])

The missing values treatment is a key step in the interpretation and modeling of data from water metering systems. Their causes can range from configurations in measuring equipment (e.g. calibration, measuring range), data transmission, or even equipment maintenance.

Missing-data imputation, removal of missing observations or even maintenance of missing values will depend on the limitations of the techniques to be applied for the system modeling. While artificial neural networks for the classification of residential end-uses and ARIMA models for water demand prediction require complete data, models based on decision trees (e.g. random forest, boosted trees) or generalized linearized models can handle missing values as a separate category in the estimation of its parameters. An important framework about missing-value imputation methods is presented in Bertsimas et al. (2018) [39], where different datasets and percentage of missing values are treated and compared as optimization problems.

The analysis of the missing values distribution, such as time of day, days of the week or even the values that precede the missing values may reveal undesirable patterns in the application of missing data imputation algorithms for missing data not being missing at random (MAR) or missing completely at random [39], that is, not being independent of the observed values.

Pattern Recognition and Analytics

An early example of a modeling technique to predict leakage rates from the network data is the Background and Bursts Estimates (BABE) model, developed in 1993 [2]. BABE is a statistical component analysis model that estimates the current leakage conditions (real losses and minimum night flows) and shows how much leakage savings the utility can achieve if they improve their active leakage control activities.

A number of mathematical and statistical techniques have been used to develop models for predicting water consumption, pressure, leakages, or pipe bursts and components failures. Patterns of change over time in some variables like flow and pressure are predictable, as Fig. 12.15 shows. Therefore, the comparison between the current values measured by sensors and the ones that would be expected by the historic dataset, or a statistical or mathematical model, can reveal possible failures and malfunctions in a monitored area of the network.

Fig. 12.15
figure 15

Weekly profiles of flow and pressure values in a monitored node [16]

A trend topic nowadays is how to convert the large database generated by the smart systems in tools to solve the efficiency and sustainability concerns. Wang et al. [40] present a comprehensive review about this problem where it was identified three key application areas for the database provided by smart meters: load analysis, load forecasting, and load management. One more item could be added to Wang ‘s list, the fault detection, which applies the historical data in order to improve the smart system itself. In order to provide an overview about this topic, Table 12.4 presents a list of works published in the two last decades addressing each one of the mentioned four points.

Table 12.4 A brief review of data management of smart water systems by application area

5 Conclusion

Smart water technologies are capable of significantly reduce water losses, improve the asset managing of water infrastructure and increase resilience against extreme climate events. This set of technologies are mature and ready to help water utilities to operate more efficiently. They are also crucial for urban areas to better preserve their limited water resources.

Rural areas and small villages are also prone to implement smart water infrastructure. The initial costs are higher than a traditional water network, but it is possible to implement a smart network with a limited set of features at an affordable cost. A rural water supply or irrigation system with smart online metering and automated pumping, for instance, won’t be much more expensive than a traditional one, and the increase in the initial costs could be recovered in a few months with the reduction in leakages, pipe bursts and energy waste. Another characteristic of a smart water system is scalability, which means that, as the network grows, the system can be easily adapted to a wider user base than originally intended. Thus, a smart water system that was initially deployed with a limited set of features could later be upgraded with more automation, or with the capacity of monitoring more parameters. Also, as the water network expands, an increasingly large area can be monitored without varying the core architecture of the system. A higher initial cost could be a setback for many smart water projects in rural areas. Other typical obstacles are the lack of internet access and the lack of local IT specialists. The latter can be mitigated by investing in capacity building among the local population. Mobile collaboration, in which experts assist remotely to the workers that are on field, could also be very useful for solving maintenance issues in rural water systems. Besides higher cost and technological complexity, the benefits and functionalities of smart water systems make the challenge worthwhile for cities of all size, from large urban areas to small villages.