UAV Survey of Bridges and Viaduct: Workflow and Application

Abstract

In this paper a workflow for bridges and viaducts aerial survey through Unmanned Aerial Vehicle (UAV) is presented.

Actual methodologies for bridge inspection and survey are described focusing on the use of UAV and 3d photogrammetry as a game changer to speed-up the process for the extraction of relevant data. In this context, a workflow for the complete survey of bridges, from data gathering, elaboration, presentation of results and automatic extraction of geometrical data is presented. The presented workflow was applied to a highway viaduct “Annunziata” located in a seismic risk zone in the city of Reggio Calabria. The application of this workflow allows a complete 3d reconstruction of the viaduct, with the extraction of the structure’s geometry for future analysis and remote inspection using a web-based platform.

1 Introduction

Infrastructure maintenance and monitoring, with particular attention to bridges and viaducts, is an actual problem that western country has to face. These critical infrastructures are highly exposed to seismic risks. The first document and regulation about the maintenance activity to be performed on infrastructure and bridge’s “Retrofitting guidelines for Highway Bridges” was emitted in the US Federal Highway Administration (FHWA) in 1983; while the first research program, financed by FHWA, to investigate and evaluate the seismic risk assessment of bridges started in 1992. The output of that research was released on 1995, “Seismic Retrofit Manual for Highway Bridges” and updated until today in the “Seismic Retrofitting Manual for Highway Structures: Part 1 Bridges” (Buckle et al. 2006) Seismic Retrofitting Manual for Highway Structures: Part 2 Retaining structures, slopes, tunnels, culverts and roadways” (Power et al. 2004).

In Europe the Eurocode 8 part 2 contains a document for “Design of structure for earthquake resistance: Bridges” (Holst et al. 2011) and the evaluation of seismic risks, but the code for assessment and retrofitting of structures limits their analysis only on existing buildings (Eurocode 8 part 3 “Assessment and retrofitting of buildings” (Holst et al. 2011). In Italy, designs regulations are contained in “Norme tecniche per le Costruzioni” NTC2018 (Ministero delle Infrastrutture e dei Trasporti 2018). Moreover, “Civil protection Department” (DPC) has activated research in collaboration with Italian University about “Evaluation and reduction of seismic risk of existing bridges”. The main objective is to develop a procedure to evaluate the structural condition of the existing bridge in order to reduce the risks.

In Italy “Union of Italian Province” (UPI) has developed a recent report (Unione Province Italiane 2018) about the actual condition of Italian infrastructure focusing on Bridges and Viaducts that have exceeded their life cycle (almost 50 years). The report was the result of the investigation requested by the Italian Minister for Transportation (MIT) after the collapse of Morandi Bridge in Genova (2018). Italian provinces have to manage almost 100.000 km of roads with 30.000 bridges, viaducts and tunnels. The status of these bridges is reassumed in the next graph (Fig. 1):

Fig. 1.

Status and distribution of Italian Province bridges and viaduct

The estimated cost for the monitoring of 14.000 bridges is about 566 million and estimated costs of intervention for actual bridges is 2.7 billion. This count excludes the intervention on the regional and national highway, managed by public or joint venture (public-private) company Autostrade per l’Italia (ASPI) (4200 bridges and viaducts) and Anas (13.000 bridges and viaduct). Each of this company has its own monitoring system and standard operation manual to ensure maintenance and control. Actual methodology for monitor and control of bridges are presented in the next paragraph. The major damages on bridges and viaducts due to external forces such as seismic loads can be divided according to super and substructure: major damages and cause of collapse are in fact concentrated on deck and piers (Pinto et al. 2009): deck doesn’t have anti-seismic resistance function and major cause of collapse are essentially due to hammering between adjacent span and losses of support. Piers, responsible for supporting the deck and resists to different forces, can collapse due to flexural ductility defects, shear resistance and inadequate design of beam/pier joint. Adequate monitoring and survey techniques are necessary to understand the structural health of the construction.

1.1 Infrastructure Survey Techniques

Structure from motion coupled with the use of UAV represent the latest and significant advance in digital surveying, thanks to the possibility to acquire information in a cheap and fast way, and their non-invasive characteristics guarantee the acquisition without any contact with the object/area to be surveyed. The use of photogrammetry in surveying and monitoring spread in recent years and is growing rapidly, thanks to the numerous advantages compared with more traditional survey techniques. The choice of the survey technique to be used by the surveyor is related to the expected results and different factors such as: (i) data accuracy and precision needed, (ii) intended usage of the captured data, (iii) constraints such as time and budget for the operation and (iv) expertise and availability of both hardware and software for data acquiring and processing. Compared with traditional techniques, such as Total Stations, GPS (Global Positioning System), LIght Detection And Ranging (LIDAR), airborne laser scanning (ALS) and terrestrial laser scanning (TLS), photogrammetric algorithms coupled with UAV survey can offer truly 3d information with reduced labor cost and capital expenditure (Mader et al. 2015). Moreover with careful use of ground control points (GCP) this technique can rival other digital survey methods for spatial accuracy, and with the use of more precise onboard Global Navigation Satellite System (GNSS) navigation (e.g. Real Time Kinematic GNSS) the spatial accuracy can be improved to centimeter precision without GCP (Gerke and Przybilla 2016; Cryderman et al. 2014) (Table 1).

Table 1. Survey techniques comparison

With careful application, the delivered results in terms of accuracy can be compared to the best achieved with any other topographic surveying method, both direct or indirect (Marcus and Fonstad 2008). From the other side, limitations are represented by the dependency on external ambient light condition (Marcus and Fonstad 2008; Gienko and Terry 2014), the high computational power needed to elaborate data and the impossibility to elaborate live data on field in order to understand attributes that the point cloud will have. Moreover, software used for point cloud analysis and elaboration actually is in its infancy.

1.2 Photogrammetric Algorithms for 3d Reconstruction: SFM-MVS

Photogrammetric principles and algorithms allow, as discussed, the reconstruction of the 3d scene starting from different images acquired respecting stereographic criteria. Quality of photogrammetric reconstruction is influenced by Sensor size, resolution, photo acquisition parameters, image format acquisition, stabilization. The well-known computer vision algorithm Structure from Motion (SFM) (Micheletti et al. 2015) is the most reliable and utilized algorithms for the generation of a valuable 3d model from 2d imagery. SFM algorithms identify matching features in a collection of overlapping digital images and calculate the camera location and orientation from the differential position of multiple matched features. Based on these calculations overlapping imagery can be used to reconstruct a “sparse” 3d point cloud model of the acquired scene. Later the model is refined to a much finer resolution using Multi-Stereo-View methods, producing high-quality, dense, 3D point clouds of a scene/area with minimal financial cost. The use of this computer vision algorithms to become relevant in geoscience thanks to the emergence of affordable commercial user-friendly software coupled with rapid developments of UAVs platform.

2 Methods: Aerial Survey Using UAV

The combination of Unmanned Aerial Vehicle (UAV) and computer vision algorithms presented makes this combined solution the perfect inspection platform for infrastructure surveying, bridge and viaducts inspection and monitoring. The first level of application can be represented by photographic dataset acquired according to structure segmentation. A more precise level of acquisition involves the 3d reconstruction and virtual asset inspection using Virtual Reality. The main advantages related to the use of these technologies are summarized in (i) the possibility of reaching inaccessible zones in reduced time and (ii) gather high detail of structural components with camera zoom (iii) use of remote piloting (Behoind Visual Line Of Sight operation) (iv) setting-up standard and automatic flight plan for data gathering associated with different scenarios and (v) ensure regular service during inspection process and (vi) repeatability of inspection process during time. Moreover, with tailored camera and systems (vii) non-invasive deformation monitoring it’s applicable (Yoon et al. 2018), (viii) creation of dynamic database and (ix) creation of 3d model for a virtual tour and remote collaborative inspection. Applying computer vision images analysis (x) automatic finding of defects and deterioration and (xi) extraction of geometrical characteristics to perform structural analysis.

From the other side different challenges and open point in the acquisition phase must be faced: (i) environment complexity (presence of obstacles, vegetation near the structure) for flight, (ii) presence of river near the infrastructure, (iii) complex structure, thins parts and occlusion requires manual flight or dedicated UAV for confined space inspection, (iv) weak or not reliable GPS signal under the bridge.

Moreover, the main challenges in data analysis are represented by (i) 3d point cloud segmentation, (ii) extraction of key information according to tasks, (iii) visualization and sharing of acquired data and models (iv) no possibility to verify the quality of the data during the acquisition process.

As discussed before, the use of UAV technology in infrastructure surveying recently spread from 2013. Different applications and case studies have been presented in last 5 years (Ham et al. 2016; Khaloo et al. 2018; Hackl et al. 2018; Chen et al. 2018; Morgenthal and Hallermann 2014; Escobar-Wolf et al. 2018; Lovelace 2015). However, due to the different disciplines involved in this application and to the recent and new technology used, there is not a standard methodology and workflow for data acquisition and analysis.

The use of this technology it’s not yet available as a standard inspection platform and it’s task dependent. Moreover, the competence needed for acquisition and data analysis involves the different field of science and requires different knowledge in aeronautics, civil engineering, electronics, computer vision and 3d graphics. The technician involved it’s only a pilot but should have different specialization. Acquisition techniques depend on different factors such as Level of Detail required, payload and sensors, and data analysis and extraction of crucial characteristics from a large dataset (e.g. 3d point clouds or terabyte of images) are not yet standardized. In this paragraph a methodology for the standardization of the bridges inspection process through UAV survey is presented (Fig. 2):

Fig. 2.

Workflow for UAV photogrammetry bridge survey

Three main phases for the bridge survey and 3d reconstruction using UAV can be identified: first planning and acquisition phase according to the area, structure and task specification. The main issue is represented by the setup of the flight acquisition plan. In the second phase, the acquired data (e.g. photo or video) are elaborated to extract a measurable and classified 3d point cloud of the infrastructure. In the third phase, the extraction of the relevant characteristics is performed with the use of a web-based platform to visualize and analyse the results, allowing the possibility to perform virtual inspection of the scene and extract key information for future analysis.

3 Annunziata Viaduct: Case Study

3.1 Annunziata Viaduct, A2 Highway Reggio Calabria

The methodology for survey with UAV and extraction of geometrical features was applied to a highway bridge located on the A2 “Autostrada del Mediterraneo” in the city of Reggio Calabria, Italy (Fig. 3). The viaduct, built on 1970 upon the “Annunziata” river, is a simply supported, beam viaduct made of pre-stressed reinforced concrete with 9 short-spans of 29 m, and a total length of 254 m (in curve). Curvature radius is 150 m and the medium height of the bridge is 25 m a.s.l. The viaduct has a simple structure and static schema. No vegetation or other obstacles are present around the object, so free and pre-programmed flight are possible without issues.

Fig. 3.

Aerial view of highway viaduct Annunziata, Reggio Calabria, Italy

The infrastructure, part of the A2 highway and managed by a public-private company, ANAS S.p.A, is located in south of Italy and for this reason exposed to high seismic risk according to Italian INGV (National Institute of Geophysics and Volcanology). The strategic position makes this viaduct fundamental for the entire highway, linking north and south part of the city, allowing circulation of vehicle and truck outside of the city. In case of collapse the entire highway will be interrupted with high risk and consequence on vehicle circulation and on emergency response. The viaduct deck is composed by standard module of 29 m with 4 beams and 3 crosses in pre-stressed reinforced concrete (Fig. 4).

Fig. 4.

Deck structure

The two decks (one per each direction) are sustained by a couple of piers with a common foundation (Fig. 5a, b). Piers are made of rectangular section of 2.50 m × 1.60 m and pier cap dimensions are 8 m × 3 m.

Fig. 5.

(a) Bridge structure and deck from left side, (b) piers foundation

Several superficial cracks are present on the structure (Fig. 6), as sign of lack of maintenance operations. Moreover, water infiltration from deck to piers, due to lack of adequate gutter, represents a serious issue for structure.

Fig. 6.

Superficial cracks on piers (a, b)

The airspace around the viaduct is classified by ENAC (Ente Nazionale Aviazione Civile) as CTR (Controlled Traffic Region) and non-critical operation are allowed for UAV with operating take-off mass less than 25 kg, up to maximum height of 70 m above ground level (AGL). Regulation in Italy are defined in “Regolamento Mezzi Aerei a Pilotaggio Remoto” by ENAC. The national regulations have integrated the EASA Drone Regulatory Framework, active in the European Union (Fig. 7).

Fig. 7.

Controlled space for UAV operation in the survey area

Visual line of sight (VLOS) flight is allowed at a maximum distance of 200 m, with manual or automatic flight. In the area of interest, to avoid collisions and delimitate the operating zone, a virtual geo-fence was created to allows UAV operations in the limited space area to accomplish local regulation. With the software limitation, the UAV can fly only inside the virtual area (Fig. 8). The airspace around the viaduct is occupied by low altitude buildings and two cranes in the right side, and vegetation in the right side. The operation has taken into consideration the presence of these obstacles. The viaduct height is 25 m a.gl. and the maximum flight height is 70 m a.g.l.

Fig. 8.

Geo-fence around Annunziata Viaduct for delimitation of aerial space for survey

3.2 Annunziata Viaduct Data Acquisition

The aerial survey of the Annunziata Viaduct was executed in the early morning with cloudy weather to avoid direct sunlight in the acquired images and optimizing the dataset for 3d reconstruction process. The workflow explained in the previous paragraph was applied in order to plan the survey, the acquisition and elaboration process. The survey was performed by an authorized UAV pilot for non-critical operation.

Definition of Mission’s Objective, Area and Tasks.

The mission objective was the complete acquisition of the Annunziata viaduct with centimetre accuracy and the extraction of the geometrical feature of the structure.

Definition of UAV Type and Payload.

The aerial survey was performed using a commercial quadrotor UAV from DJI (DJI, Shenzen, China), Mavic Pro, whose specs are summarized in the following table (Table 2):

Table 2. DJI Mavic Pro characteristics

This low-cost UAV has onboard GPS and waypoint navigation with front collision sensors, allowing the possibility to execute automatic waypoint missions. The combination of these characteristics makes this platform compliant with local regulation and suitable for viaduct survey operation.

Mavic Pro UAV has a fixed payload with Sony camera sensor. Camera characteristics are reported in Table 3.

Table 3. Payload camera sensor

Definition of the Flight Plan.

The first acquisition plan was set-up to obtain a low detail 3d model in order to use it as input for a more detailed acquisition plan in the 3d space. The acquisition plan in a two-dimensional environment was set-up and executed using Pix4d (Pix4d Inc. Lausanne).

The first mission for the area acquisition was set up using 6 different circular mission (to cover the entire area) at 50 m a.s.l. capturing images every 5° (for a total of 72 photos per circle) as summarized in the next table (Fig. 9 and Table 4):

Fig. 9.

Sparse point cloud of circular mission elaborated with SFM algorithm

Table 4. Circular mission for area acquisition

To reconstruct the scaled and georeferenced 3d model a GNSS survey of the area where performed acquiring different Ground Control Point (GCP) on WGS84 reference system, evenly distributed on the area (Table 5):

Table 5. Acquired ground control point

The processed 3d model, reconstructed with SFM-MVS algorithm, was imported in the UGCS software (Universal Ground Control Software) (SPH Engineering, Latvia) and used to plan a specific mission for detailed and automatic acquisition of the area (Fig. 10).

Fig. 10.

3d model imported in UGCS plan software

Two different side missions for 3d model acquisition were executed to ensure (i) 80% overlap between images and a (ii) Ground Sampling Distance (GSD) less than 1 (cm/pix). Mission parameters are defined in Fig. 11, flight pattern for 3d flight execution are presented in Fig. 12:

Fig. 11.

Flight parameter for side acquisition

Fig. 12.

3d side scanning of viaduct

The use of 3d planning tool allows operation’s repeatability to perform regular inspection and acquisition on a defined time-basis.

4 Results

The obtained photographic dataset (1039 photos, 2,5 GB) was elaborated using Agisoft Metashape (Agisoft LLC, Russia) using SFM-MVS reconstruction process. Previously, the images obtained have been optimized to improve the contrasts and the light/shadow ratio to highlights details. After the elaboration process, the reconstructed sparse point cloud obtained consists of 46.000 points, and dense point cloud obtained was composed of 32 billion points. In Fig. 13 the obtained 3d model of the highway bridge is represented:

Fig. 13.

Annunziata Viaduct 3d model

The obtained model was used for the extraction of the relevant geometric information of the structure, and to perform a virtual and collaborative inspection with an online platform.

Extraction of Structure Geometrical Characteristics.

To extract relevant information from the surveyed model a methodology for semi-automatic extraction of geometry is presented. The use of this procedure allows to extract shapes from the structural parts, and automatically insert this data into a pre-defined spreadsheet. The classified structural parts are then transformed from point cloud into a 3d mesh object using Screened Poisson Surface Reconstruction (Kazhdan and Hoppe 2013). The entire workflow of the developed methodology is synthetized in Fig. 14: after the photogrammetric survey using Drone (1), and the creation of mesh as previously described (2), the ad-hoc instruments integrated with a simplified User Interface (UI) automatic transcribe data into a spreadsheet (point 3) using Rhinoceros and Grasshopper (McNeel, North America); point 4 is a file for data swap used to avoid non-compatibility of extraction algorithms with Visual Basic Marco and point 5 allows the transcription of information for analysis on the spread-sheet file.

Fig. 14.

Extraction of geometrical features

All the components are programmed ad-hoc. The basic principle used is the definition of two cut-plane XY and YZ to define the structure resistant section. The bounding box, as volumetric element around the object, was created to intersect cut-plan inside the box and the object. Subsequentially the cut-planes are setting up in XY and YZ. The user can define the position of the cutting plan in % compared with height, offset distance from cutting-plane and a total number of cutting plane as shown in Fig. 15:

Fig. 15.

XY and YZ plane to extract geometrical feature of piers

A preventive verification of the planarity and closure of the polyline is executed. If the polyline it’s not close, the algorithm will approximate the closure. The developed module was used both on piers and deck to extract the geometry and automatically insert on the spreadsheet.

3D Inspection and Measurable Model on a Web-Based Platform.

The reconstructed 3d point clouds are huge data and information difficult to manage and share (Wimmer and Scheiblauer 2006; Scheiblauer et al. 2014). To enable a simple and effective visualization, the possibility of analysing and inspect the acquired assets with a collaborative approach web-based framework was used. In the entire survey process, the platforms for visualization information are fundamental for collaboration and sharing (Eschmann and Wundsam 2017). Potree (Schuetz 2016) is a free open-source WebGL based point cloud renderer for large point clouds. This platform allows the online visualization and share of the obtained 3d point cloud, converted into a light HTML file using LasTools (Hug et al. 2012). The classified 3d point cloud was uploaded to the web viewer (Fig. 16) and shared online on a dedicated web server.

Fig. 16.

Web-interface for visualization and collaborative inspection

Moreover, the online platform allows different interaction and measurements to gather information from the uploaded model, for remote users and inspector access to the surveyed 3d model. With the use of the online platform, it’s also possible to verify the automatically extracted geometry and take manual measurements of the structural parts.

5 Conclusions

In this paper a methodology for inspection of bridges and viaduct is presented. Survey and modern techniques to acquire spatial data and information are discussed with particular attention to the use of photogrammetry combined with UAV to acquire spatial data.

The methodology was applied to a case study located on a highway bridge in Reggio Calabria, to acquire detail and information with centimetre accuracy. Finally, a platform to present and share surveyed model between client and different stakeholder’s is discussed. The obtained results compared with blueprint confirms the survey’s quality and the possibility to automatically extract the geometrical feature for future structural analysis. Moreover, the obtained 3d model was uploaded on a web platform to allows remote inspection.