Keywords

1 Problems Related with Reconstruction of Infrastructure and Public Utility Objects Following Natural Disasters

Natural disasters are almost always accompanied by significant damages to property and human casualties. Major natural disasters may result in a complete destruction of the region’s economy and take dozens (or even hundreds) of lives [1]. Thus, according to a humanitarian organisation Christian Aid (https://www.christianaid.org.uk), among all natural disasters in 2019, seven caused more than USD 10 billion of damage each [2]. The largest devastating impact of the disasters was felt in the USA, Japan, China and India. The greatest damage of USD 25 billion was caused by forest fires in October–November in the U.S. state California. Experts usually take into account insurance as well as other direct compensational payments to people and companies affected by disasters when calculating the amount of total losses. Two second greatest damages of USD 9 billion and USD 15 billion, total losses reaching USD 24 billion, were caused by Typhoon Faxai and Typhoon Hagibis correspondingly in September and October in Japan.

Statistically the number of man-made disasters hardly differs from the number of natural disasters [3], however, natural disasters cause considerably greater damage to both economy and population. It is worth mentioning that there is a clear tendency of a decreasing occurrence of man-made disasters, the reasons behind this are improved safety protocols and regulations, introduction of new technologies and automation of security systems, etc. [4].

As for natural disasters, the damage they cause increases steadily [5]. Even such minor and casual things as rain or snow may turn into serious problems, for instance, flooding or snowdrift.

The task of public services, rescue and maintenance teams is to reach as quickly as possible those locations, where people may be in need of first aid. Their task also encompasses restoration of main infrastructure objects and, if possible, complete revival of consumption levels, namely, assurance of steady public services supply, functioning hospitals, schools, nurseries, public transportation and other public, logistical and infrastructure objects.

Post-disaster recovery process is rather complex and non-trivial even for typical phenomena. International organisations invest a lot of effort into development of international assistance mechanisms for countries and regions affected by disasters. Thus, the 2015 Sendai Conference saw the presentation of the Framework for Disaster Risk Reduction [6] aiming to generalise the experience of the Global Facility for Disaster Reduction and Recovery (GFDRR) as well as various partner organisations, including the World Bank, United Nations Development Programme, European Union and governments of separate countries within the organisations. Even though the precision of data in the document was not guaranteed and its conclusions were meant as recommendations, this document was an important step towards creation of formalised criteria and principles of damage assessment, reduction of consumption levels as well as reduction of needs during natural disasters and their consequences; these criteria and principles should enable quantitative assessment of needs related to post-disaster recovery and development of universal strategies of impact management. Development of such assessment is a regular procedure, however, every country relies on its own preferences and understanding of the situation, which does not provide an objective perspective for other countries. Moreover, experience of the past decades shows that development of an adequate assessment requires not only absolute values of disaster impact presented in, for instance, monetary equivalent, but also understanding of the level of the country’s commitment to implement recovery measures and accordance of these measures with the further development plan. As a result, all over the world there is a high demand for elaboration of universal approaches adopted by the international community. This approach helps to formulate the vision of the recovery, develop the strategies, define the priorities and finalise the plans, it also includes the description of financing, realisation and monitoring of the recovery process.

One of the problematic issues is motivating other countries to provide humanitarian aid. Humanitarian aid may take the following forms [7]:

  • financial aid – provision of financial assistance, letting the government of the affected country spend the money on its sole discretion. Such assistance may also be targeted or addressed;

  • food aid – in some cases provision of food assistance is more important than financial aid, enabling the population to survive;

  • technical aid – provision of various specialised technical means, accessories and consumables, etc. It is not uncommon that disaster impact management requires the use of some kind of specific equipment that is available only to a limited number of countries with other countries possessing only small amount of it or not being able to afford it completely;

  • technological aid – provision of expertise and technologies to manage the disaster impact and to revive the economy and improve the environment;

  • political aid – in the modern world of fierce competition, sanctions, protectionism and aggressive politics coming from both other countries and corporations, a country, which has lost its competitive advantages because of natural disasters, may not be able to recover without the help and protection of major political players;

  • qualified experts and labour force – provision of highly qualified teams of technicians, rescuers or pilots may considerably reduce the time required for the first aid and safe lives.

One should also keep in mind the psychological aspects. Given a sharp drop of consumption level, one needs time to re-adjust and get accustomed. Unfortunately, not everyone is ready for such changes, which, as a rule, take place abruptly. Considering additional stress related to the proximity of death, loss of relatives and friends, sense of helplessness and despair, there might appear spontaneous outbreaks of violence, which could then turn into mass pogroms and acts of civil disobedience, magnifying overall negative impacts. 15 years ago at the end of August 2005 Hurricane Katrina hit New Orleans [8]. The city of approximately half a million of people (484,674 in April 2000) had to be nearly entirely evacuated. Katrina had left in her wake what one reporter called a “total disaster zone” where people were “getting absolutely desperate” [9]. Apart from that, there were the New Orleans marauders. Violence, killing and plundering became normal for New Orleans. The federal government was forced to deploy the National Guard and military units with combat experience in order to prevent the spread of violence and protect the remaining civil population and members of rescue and repair teams. Officials repeatedly made harsh statements against marauders, whereas news from New Orleans terrified the whole country [10]. 15 years have passed, but there are still some city districts remaining unrecovered.

Another situation of mass hysteria and insanity is exemplified by the following incident. On 4 August 2020 in the port of the city of Beirut, an explosion took place. It resulted in more than 200 deaths, 6500 injuries, USD 10–15 billion in property damage, and leaving an estimated 300,000 people homeless [11]. Help came from numerous countries, international organisations and private individuals. However, as early as 6 August the city was shaken by protests, as a result of which hundreds of people were affected; protests not only increase the number of the affected, but also hinder the conduction of the repair works.

A person finding him/herself in a critical situation and under severe stress may act unpredictably and aggressively. Therefore, the task of the country as a whole as well as local authorities in particular is to safe the citizens and ensure at least minimum level of consumption as soon as possible.

Steady supply of electric power is required to re-start the functioning of most of public utilities, service sector and production. Access to the source of steady and available electricity defines the consumption level and the quality of life; formally it is quite possible to meet the basic needs of food and shelter without using electric power (in a critical situation), but no society is able to function and maintain an acceptable consumption level without it over a long period of time. A drastic drop of consumption level means lack of food, medications and personal hygiene items, etc., resulting thus in massive human casualties.

Even during emergencies electric power has to be delivered to hospitals, first aid units, temporary shelters (flophouses), dining facilities and many more. For that reason, consumption level of electric power is generally taken as one of the indirect indicators of current consumption level and economic situation generally. Following the quarantine measures in order to prevent the spread of Covid-19 taken by numerous countries of the world, one of the parameters indicating the stop of the production and general economic processes was the level of electric power consumption. Analytical agencies report a considerable slowdown in the growth rate of global electricity consumption within 2019 (+0,7% compared to 3%/year between 2000–2018) [12]. The forecast for 2020 predicts up to 10% reduction of electricity consumption in some areas of the world [13].

The task of public services, rescue and technician teams is to reach as soon as possible the rubble, where there might be people trapped, restore main infrastructure objects and, if possible, ensure stable delivery of public utilities. Therefore, one may define the time and the number of technician teams as critically limited resources; such technician teams often have certain specialisation, namely, fire and ambulance services, natural gas, road repair services, power system object maintenance and other teams. Each of the abovementioned specialisations requires specific equipment and means that such teams are hardly interchangeable.

Time required for restoration may be reduced through improved management efficiency and quality of repair work organisation. Clear coordination of various technician teams, real-time situation monitoring and application of methods and algorithms enabling optimal solution finding may all contribute to the decision-making process and relieve extra burden from the responsible individuals.

2 Sequencing of an Action Plan in Case of Natural Disasters

As a rule, local authorities are in charge of reconstruction management. The work plan may be presented as a sequence of actions listed on Fig. 1.

Fig. 1.
figure 1

Reconstruction work sequence

Full size image

Local aims are often set depending on the experience of the authorities that are in charge of the reconstruction works in the region. This approach accelerates the decision-making process and increases the management flexibility, decreasing, at the same time, overall efficiency and coordination of decisions that are to be made. However, experience of various international organisations should not be disregarded. They were established in order to reduce the risks of disaster occurrence and deal with their consequences, these international organisations conduct information activity, assist in assessing the degree of disaster preparedness and, in worst cases, provide humanitarian help. They also assess disaster risks, develop measures, protocols and methods of risk reduction and publish current dynamics. The organisation of natural disaster management should be based on international experience and make the most of modern technologies as well as computing capabilities of analytical centres.

Figure 2 illustrates the sequence of a decision formation and making when aligned with analytical and coordination centres.

Fig. 2.
figure 2

Modified sequence of reconstruction work

Full size image

Its main feature is constant update of data on the region’s state, one may assume that information on malfunctions, breakdowns, accidents, traffic jams and other issues arrives constantly with varying delays. The analytical centre makes a decision based on the current amount of information, risks and neural network findings concluded from previous emergencies. Once local aims are set for every available technician team, control is then taken by the coordination centre, its main purpose is to provide information support to the team so that it has access to any needed consultation and information on peculiarities of repair works or routes.

Once the repair works are done, information is transmitted to the analytical centre, where the decision is made to send the technician team to the closest point based on the complete picture of repair works time minimisation.

Figure 3 presents a city map including objects that require repair (icon with crossed tools) and places, where routes had to be blocked (triangles with exclamation marks). The number of technician teams is smaller than the number of breakdowns, therefore all repair tasks are to be distributed in such a way that the total repair works time is minimised. Total time includes travel time, time spent on the repair itself and time to report task completion as well as receiving the new one.

Coordinates of breakdowns as well as information on the possibility of reaching corresponding locations are taken as input data to start the analysis. Most probably, city plan as well as all possible routes are prepared in advance, information on actual situation has to be added timely. “Thatch your roof before the rain begins”. Unfortunately, not everyone is guided by this principle.

Fig. 3.
figure 3

Example of mapping the places that require repair and road blockage

Full size image
Fig. 4.
figure 4

Presentation of information as a graph

Full size image

In essence, information is presented in the form of a graph (Fig. 4) with its vertices set either as breakages (white circles on Fig. 4) or route ramifications (black circle). When data is presented as a graph, this task takes the form of the travelling salesman problem with a number of restrictions.

3 Ways of Obtaining Input Data

Another aspect, which is at the same time the weakest point of the suggested model, is the system of obtaining up-to-date information. In order for the analytical centre to start functioning as described above, there must be available data on the exact location of breakages and routes that technician teams will be able to take. Most promising ways of acquiring these data are ground inspections, satellite surveillance and aircrafts. Let us examine strengths and weaknesses of every one of them.

Ground inspections. In other words, this is actual presence at the site or traffic jam. This is the most secure, but also the most costly way. This approach may lead to unreasonable waste of time and resources, disorientation and other related problems when it is not coordinated from one centre. Given efficient management, precise coordination and other ways of obtaining information, this approach is an indispensable way of checking, correcting and clarifying the information received. It is also worth mentioning that some kind of data may only be acquired through personal presence and special verification.

Satellite surveillance. Ultra-high quality imaging is required to correctly determine the status of routes and locations of alleged breakdowns. Ultra-high spatial resolution is considered the pixel size range of 0,3 m to 1 m. Satellite data are easier to process and analyse thanks to wide-area coverage, no flight distortion and availability of rational polynomial coefficients. Current market of ultra-high quality imaging encompasses more than 20 satellites belonging to different countries and operators. They all have their own advantages and solve particular tasks. Let us look at some of them [14]:

  • TripleSAT 1–4 spacecraft – is a Chinese spacecraft constellation launched into orbit by 21AT company in 2015 (1–3 satellites) and 2019 (4th satellite). Their distinctive feature is that they are in the same orbit at the distance of 4 s from one another and can therefore take images of the same territory with small time difference;

  • SuperView-1 spacecraft – is an orbit constellation of Chinese satellites with ultra-high spatial resolution of 0,5 m – these function for the benefit of civil consumers;

  • Kompsat-3A spacecraft – is a spacecraft constellation belonging to South Korea. SIIS (Smart Eyes in the Space) company owns three satellites with ultra-high resolution;

  • WorldView-3 spacecraft – is the first hyperspectral commercial satellite with ultra-high resolution of 0,3 m. WorldView-3 films up to 680 thousand sq. km per day enabling up-to-date data with the highest spatial resolution. Furthermore, this is the only satellite with 30 different spectral channels, including 8 SWIR (short-wavelength infrared). It also has the highest geo-positioning precision of 1–2 m without using reference points.

Cost of imaging is USD 6–10 per sq. km, which is several times cheaper than aerial imaging. As can be seen, even data from commercial satellites may suffice. Considering the fact that there are numerous military satellites in the Earth orbit, and some countries are willing to share their data in case of emergency, satellite surveillance may become an important source of information.

However, there is also a range of disadvantages that question the appropriateness of satellite surveillance:

Disaster zone imaging time – usually disaster zones spread over hundreds, if not thousands of square kilometres taking several days in order to film them, which makes it an unacceptable delay. Furthermore, the higher the quality, the smaller the area of filming, which results in longer waiting time;

Weather conditions – clouds, fog, smog, and other factors, which may distort the images or make imaging not possible;

Time to reach the orbit – satellites are constantly moving, they will therefore require time to reach the necessary imaging location.

  • Aerial survey. An alternative to satellite surveillance is imaging from an aircraft. There are several types of aerial survey [15]:

  • Low oblique aerial photography – an aerial imaging with the camera axis set parallel to the ground surface (or forms a small angle). Low-altitude aerial photography is considered imaging at an altitude of 10 to 150 m above ground level.

  • High oblique aerial photography – imaging parallel to the ground surface at an altitude of 150 m above ground level and more.

  • Short-range oblique aerial (view) photography – an aerial imaging with the camera axis set at an angle of 25–65° to the ground surface, at an altitude of 10 to 150 m above ground level.

  • Long-range oblique aerial (view) photography – at an angle of 25–65° to the ground surface at an altitude of 150 m above ground level and more.

  • High vertical (vertical, topographic, nadir) photography –an aerial imaging with the camera axis forming an angle of 90° (or close to it) to the ground surface, in other words, vertically down, at an altitude of 150 m and more above the ground level.

If there are enough aircrafts to launch, they may be able to provide and regularly update actual information on the emergency area and enable the start of repair works.

Figure 5 shows an interaction scheme between the operator and the object taking the images, in this case the focus is on the use of an unmanned aerial vehicle (UAV). A UAV transmits the data via communication towers or satellites to the server, where the interaction with the operator takes place. The operator checks the data received and transmits control signals through available UAV communication channels. The scope of the transmitted data is difficult to estimate, because a UAV sends not only images and video, but also additional information in order to ensure the correct flight.

Currently, solutions are being developed in order to reduce the volume of the transmitted data by means of applying artificial intelligence elements to process the data and make corresponding decisions. During a 4–6-h flight there are about 2 terabytes of data generated [16]. In order to make a decision based on the collected data, it is necessary to process this whole information and retrieve relevant parts.

Fig. 5.
figure 5

Interaction scheme between an unmanned aerial vehicle and operator

Full size image

Figure 6 shows the application scheme of artificial intelligence elements at various stages of obtaining information.

Fig. 6.
figure 6

Interaction scheme between an unmanned aerial vehicle and operator with the application of artificial intelligence elements

Full size image

Let us look closely at every stage:

  1. 1.

    Obtaining and processing image and video data. Recorded data do not always contain useful information, the purpose of this data is not mapping, but checking the accessibility of the route and the state of certain infrastructure objects, for instance, integrity of power lines and utility poles. Therefore, transmitting large amount of data is not required, what is, however, necessary is analysing the image and forwarding the information on the state of objects. It should also be possible to transmit the image depicting the identified anomaly, be that a breakage or blockage on the route, for further verification. This will undoubtedly reduce the amount of data transmitted and allow for faster data collection and management decisions.

  2. 2.

    Aircraft control, this area of the artificial intelligence application is already considered standard practice. In an emergency situation, the UAV must be able to independently solve tasks related to flight support, respond to objects, select modes, type and altitude of survey to ensure the best operating conditions for image recognition and analysis algorithms. In this case, the operator is only required to adjust the control and assign the initial targets.

  3. 3.

    Third point is applying AI to process the obtained data and solve the tasks related to work planning, route selection and decision making under conditions of risk and uncertainty.

4 Approaches to Image Processing and Data Acquisition

Deep Learning, as a research area of machine learning [17], is recommended as the basis for the approach to recognise such heterogeneous and non-trivial objects as roadblocks, power line or utility poles breaks. Deep Learning encompasses multiple layers of an artificial neural network, where the current layer takes the output of the previous layer as input. This methodology applies non-linear transformations and high-level model abstractions. It was first known as hierarchical learning [18] and was used to identify images. Deep Learning generally considers two main factors: non-linear processing over several layers or stages and learning with or without supervision [19].

Controlled or uncontrolled learning is related to creation of a controlled or uncontrolled system.

Figure 7 presents an example of a trained neural network functioning, blockages and suspicious areas are marked on the image. The image was taken after the hurricane, as can be seen, fallen trees blocking the road in three places were marked with red rectangles, if we analyse suspicious areas, they turn out to be either shadows from houses or trees or small debris scattered along the road.

The idea is that the UAV control system takes a separate picture of every suspicious area and forwards it to an operator for additional verification.

Fig. 7.
figure 7

Example of an obtained city image and identification of blockages or suspicion of blockages

Full size image

The neural network can learn from previously made decisions and consequently adapt to current situations. Once areas of blockages have been identified, information on their location will be superimposed on a route plan or graph, as shown in Fig. 8 as can be seen, the system is able to process an image, identify areas of blockages and their coordinates, generate additional requests concerning suspicious areas to the operator, and provide initial data to solve the task of repair work planning.

Fig. 8.
figure 8

Marking the route plan.

Full size image

In order to make the operator’s work more comfortable and avoid replication of incorrect information, different approaches to user authentication are proposed [20]. It is important to understand that working under stress creates unique conditions, and the user simply may not have the time or opportunity to undergo standard authentication.

Apart from locating the blockages, it is necessary to verify the integrity of power lines and utility poles. Figure 9 and Fig. 10 illustrates how a neural network identifies poles and power lines.

Fig. 9.
figure 9

Power transmission line recognition result

Full size image
Fig. 10.
figure 10

Poles recognition result

Full size image

Images were recognised by a neural network with the Mask R-CNN architecture. Unlike object detection, segmentation determines the silhouette object, which allows to check their integrity more accurately. Unfortunately, available images of power lines taken by UAVs did not suffice to train the neural network, and it was therefore decided to use images from terrestrial photography.

Now there is no standardised learning base for such a neural network. Therefore, a small dataset was formed from internet images, and each image was labelled by VGG Image Annotator (http://www.robots.ox.ac.uk/~vgg/software/via/).

Fig. 11.
figure 11

Loss graph for recognising poles

Full size image
Fig. 12.
figure 12

Loss graph for recognising power transmission lines

Full size image

Figure 11 and Fig. 12 illustrate a reduced recognition loss with decrease in the number of epochs for poles (Fig. 11) and power transmission lines (Fig. 12).

It is possible to carry out electric power system reengineering based on the acquired data on its current state [21], improving thereby the overall efficiency of the system.

5 Creation of a Repair Work List

The next stage is distribution of tasks between technician teams. The formulated task is known as the travelling salesman problem. There are numerous mathematical methods to find both an exact and an approximated solution to the problem. Among the methods providing an exact solution, best known are brute-force search and branch-and-bound algorithms).

Main disadvantage of these methods is their high temporal and capacitive complexity, which is important to keep in mind if there is a large number of points. Heuristic methods which, in fact, simplify the brute-force search method, thereby reducing the working time, list the following [22]:

  • genetic algorithm

  • ant colony optimisation

  • nearest neighbour algorithm

  • Clarke-Wright algorithm

  • nearest insertion algorithm

  • cheapest insertion algorithm

Unfortunately, none of the widespread algorithms may be applied in their usual form. The Clarke-Wright algorithm is the most acceptable method to solve our problem. However, conditions to solving the task are to be outlined:

  1. 1.

    Large number of objects – when it comes to consequences of natural disasters, the number of locations requiring repair may list thousands of items; for instance, Fig. 8 illustrates the area of about 0,3 sq. km, but even within such a small radius there were 3 critical zones and 4 places of suspected blockage identified. Therefore, one should look closely before choosing an algorithm with quickly increasing working time;

  2. 2

    Uncertainty, risks and dynamic information updates – numerous objects may require additional verification, data on many objects may not be proceeded yet, but information keeps updating. The algorithm should be able to distribute the tasks among technician teams basing on lacking information and taking into account those risks that might be hidden in not yet inspected areas. For example, the further away from the epicentre of destruction, the less likely it is that the route will be blocked.

  3. 3

    Lack of consistency in the implementation of the optimal strategy – in continuation of the previous point, the system allocates work at a specific point in time, based on known information and risk assessments, the optimal (or acceptable) strategy is selected. However, as the technician teams move or complete their tasks, the information is updated, which makes the optimal strategy subject to major changes. Repeated changes to the strategy may lead to a highly inefficient renovation sequence, causing harm and increasing the overall renovation time.

  4. 4

    Variety of technician teams – in this paper we mention only two different types of teams that are involved in solving the task: to repair electric power equipment and to clean the routes. Actually, the number of specialised teams may reach dozens, making not only the task of work allocation, but also the task of image processing a lot more complicated, as it is then necessary to determine the nature of work and recommend a corresponding team.

  5. 5

    Interconnection between the repair works – the work of some teams affects the capabilities of other teams, which makes it difficult to find the best strategy.

6 Conclusions

The paper formulates and justifies the task of creating a list of tasks for technician teams of varying specialisations.

It describes the problems that public services have to deal with in the aftermath of natural disasters. Examples show that the main task of public services is to restore as quickly as possible minimum living standards, otherwise, the situation may grow out of control.

A model for developing a sequence of actions in dealing with the consequences of natural disasters was proposed. This model implies the use of systems for the automated allocation of work to technician teams and collection of up-to-date information. This will reduce the time spent on organisational measures and coordination of reconstruction work.

The strengths and weaknesses of different ways of acquiring data on the extent and nature of damage have been discussed. In particular, the paper looks at terrestrial photography, satellite surveillance and aerial survey. It should be noted that sometimes, in order to obtain information, it is helpful to use not only images in the visible range, but also infrared, ultraviolet, radiological, radioactive, and ultrasound etc. This will bring more information and more correct conclusions.

Neural network models trained on the principle of deep learning have been proposed.

The first neural network is responsible for determining traffic congestion. It allows to find the optimal routes. In the paper, existing solution were used to demonstrate the neural network operation.

The second neural network is responsible for recognising poles and determining their integrity. The proposed network is capable of recognizing poles with an accuracy of about 90%.

The third neural network recognises power transmission lines with an accuracy of about 80%. To create the second and third neural networks, the Mask R-CNN architecture was used with retraining the last CNN layer.

When formulating the task of compiling a list of repair works we compared it to the travelling salesman problem. However, standard methods do not solve this task due to various limitations and peculiarities, including: limited time, uncertainty, dynamics, specialisation of teams, variety of tasks, etc. It is also recommended to use the following parameters to compile the list: coordinates of the repair site, possible routes, repair time, travel time, time to eliminate blockages, and number of available teams.