Static and Dynamic Monitoring of Bridges by Means of Vision-Based Measuring System

Abstract

Civil structure static and dynamic monitoring is a key activity for both safety and maintenance purposes. In this paper the use of cameras to monitor a bridge response to train transits is being considered and results are presented and compared with a reference measurement, provided by a laser interferometer. A camera is a non contact measurement device, having mainly two advantages: first of all the effort required to prepare the set-up is very low, because the camera is fixed in the proximity of the structure and the targets to be mounted on the bridge (if required) do not need any cable connection for both power supply and signals. Secondly cameras can measure the displacements of multiple targets in the field of view. The leading idea of the paper is to use image acquisition and processing (Pattern Matching and Edge Detection) not only to measure the displacement of a few targets but also to grab images from a wide structure portion in order to recover displacements of a large number of points in the field of view. The extreme final solution would be having wide area measurements with no targets, to make measurements really easy, with evident advantages, but also with some drawbacks to be fully comprehended.

9.1 Introduction

Static and dynamic testing of structures is at the same time a fundamental and critical operation, mainly due to the fact that this type of testing is required for both safety and maintenance purposes. In the case of bridges, the need to periodically assess the structure stability and runability makes this need relevant. Together with the regional railway company of Lombardy (Ferrovie Nord Milano), it has been decided to consider a 50 m long steel trussed bridge, as a laboratory bridge, to be used for testing vision-based motion monitoring techniques to get the best bridge response estimation and to gain awareness of the limits related to this approach. The considered structure is an old steel trussed bridge crossing a river; it allows good access from both sides, therefore responding to the requirements of being a sort of ‘laboratory’ bridge in which any change in the measurement parameters are possible to obtain without too difficult operation. Trains travel on the bridge at a rather low speed, and also dynamic phenomena involving the whole bridge are confined to the low frequency range. The main goal of this work is to define the performances of vision-based displacement measurement techniques in terms of sensitivity, resolution, uncertainty, in the harsh field environment, which might strongly deviate from the behaviour defined during laboratory testing.

In this work vision approaches have been applied together with more traditional measurements: among these a single-point laser interferometer has been adopted as the reference, at mid span, to measure the effects of train pass by in terms of structural static and dynamic displacements (also the bandwidth definition is among the aims. The known advantages of camera-based measurement devices, that makes them attractive for the application of bridge monitoring, include: remote contactless monitoring (i.e. no critical problems in setting up the measurement system, especially in the case of bridges with frequent train passages), possibility to perform a multipoint measurement with a single camera, quick and simple measurement device set-up (no transducers nor cables on the structure), no need of access in critical areas, especially below the bridge to fix transducers. The mentioned aspect are even more attractive in the case of bridges crossing rivers or deep valleys.

An additional advantage in using a camera for displacement measurements is that every row or column in the pixel matrix can be considered a sensor on its own: so the complete camera corresponds to a very high number of parallel sensors, allowing for distributed sensing or giving the chance to exploit this redundancy to improve measurements reliability. Due to this reason, cameras are usually referred to as ‘dense’ sensors. However, cameras show the common problems of relative displacement transducers and have a restricted frequency bandwidth.

9.2 State of the Art

Thanks to the development of digital cameras, to the growth of computer processing capabilities and to the new image processing libraries, the use of vision-based measuring systems to measure the vibration of targets has become popular in the last years, with particular reference to the vibration monitoring of civil structures.

One of the issues in this kind of measurement is the type of target used for the measurement itself: in most the cases planar black and white targets are being attached to the structure in order to improve the measurement technique reliability and to reduce result uncertainty of the [1–5]. In other circumstances, the natural texture of the structure under investigation can be used for the measurement [6–8]. The main advantages of the markeless case are that the preparation of the set-up is much faster and that it is not necessary to have access to the structure to be monitored, no permissions, no target maintenance.

In [1] and [9] bridge vibration measurement is carried out using fit-to-the-purpose targets fixed to the structure; in this case the target is constituted by one black circle in white background, while in [2] a planar target with four circles is used. In [3] cross-shaped targets are used and the viewing system is equipped with an additional reference system, which decreases the sensitivity of the camera basement to ambient vibrations. Two types of targets are used in [4]: ring-shaped and random ones; in this case multiple target are measured contemporarily with a single camera, allowing multi-point measurements. In some applications active targets are used; for example in [5] LEDs are used for suspension bridge vibration monitoring. Markerless solutions are proposed in [6, 7] and [8], to monitor cable vibrations in cable-stayed bridges also in the case of power head transmission lines. In [10] vibration measurements obtained through image acquisition and processing are used to develop a modal analysis of a simple structure.

9.3 Design of the Vision-Based Measurement System and Testing Layout

In vision-based bridge monitoring a compromise must be looked for between two needs: on one side a wide field of view can theoretically allow to get the whole dynamic deformed shape, but this fights against the need to get a reasonable resolution in terms of pixel/mm. If the will to get the complete bridge deformed shape pulls towards the need of having a wide view, there is the serious risk that this choice impacts on resolution, so that the peak to peak vibration amplitude is confined within a few pixels, thus worsening the signal-to-noise ratio. Due to this reason, in the past, most dynamic measurements have been performed on small structure portions, trying to increase contrast, developing fit-to-the-purpose targets to be fixed to the structure, being rather close to the target and assuming that motion is rigid, to rely on the redundancy offered by the grabbed images to increase the reliability of results: in fact a single image allows to measure the motion of multiple targets, therefore in case of rigid motion the average of the displacements estimated by different targets allows to obtain a more robust motion estimation.

The steps faced in this paper are to which extent it is reasonably possible to move from 1D measurements to 2D dense measurements along a bridge span by using cameras. Once fixed the limits, this method can be really attractive, as it works with no sensors on the bridge, no power supply, in a simple approach manageable by whichever worker. This allows to get more dense checks (since the measurement setup can be easily assembled and disassembled), dramatically changing all the approaches to maintenance and testing.

The chosen test bed is a 50 m long steel trussed railway bridge crossing a river close to a village named Erba (Fig. 9.1a).

Fig. 9.1

Tested structure (a), laser interferometer at mid span (b) and camera framing the bridge (c)

Trains run on bridge at low speed during the whole day, approximately every 30 min. In this work, vision based measures will be exploited in order to quantify the bridge sag during the trains pass-by: a movie of the structure during every train crossing has been acquired and subsequently analyzed in order to estimate the vertical displacement at different points along the bridge main span. The movies are acquired by means of a standard user-consumer digital camera (Canon Legria HF 21, 1920 × 1080 px, 25 fps, Fig. 9.1c) to prove that no special hardware is required for these measurements.

The upper and the lower horizontal beams of the structure are connected by 13 vertical trusses on each side (Fig. 9.2, where a schematic sketch of the structure is reported). These trusses represent the points where the bridge vertical displacement will be measured. All the analyses will be carried out by two different image processing codes: a pattern matching and an edge detection algorithm. On every vertical truss, two targets were fixed, both printed on A4 sheets (210 × 297 mm).

Fig. 9.2

Measuring layout

The first target (Fig. 9.3a) is composed by a couple of round blobs and it will be used to both fix the scaling factor (i.e. px to mm conversion) and track the pattern matching sequence. Local scaling factor is computed for every measuring point in every movie, knowing the physical centre to centre distance of the two circles and estimating, by means of standard blob analysis, their distance (in px) in the first acquired frame of every movie. The second target (two black lines, Fig. 9.3b) will be analyzed by an edge detection algorithm. Only data extracted by the evaluation of the tilted line edge will be presented in this work: edge detection algorithms, in fact, have proven to work better when the edge is not aligned to the grid of the camera sensor [11, 12]. In order to provide a low uncertainty reference to these measurements, a laser interferometer (resolution = 2.56 μm) has been placed at the bridge mid span (Fig. 9.1b). The displacement measured by the interferometer can be assumed equal to that of target 7 in Fig. 9.2 and it will be used to compute the measurement uncertainty associated to the vision-based approach. By varying the portion of the bridge framed by the camera (i.e. the zoom level and the distance between the camera and the structure), it has been possible to study the relationship between the scaling factor and the obtained measurement uncertainty.

Fig. 9.3

Targets fixed on the bridge: (a) for pattern matching and (b) for edge detection analyses

9.4 Experiments

In this section the results of the Erba bridge tests will be reported. The first series of tests aims at verifying the measurement performance in frontal view, i.e. with the camera placed at a side of the bridge, with the optical axis normal to the train direction and pointing several targets fixed onto the steel structure (Fig. 9.1c). The second series of tests has been carried out to verify the measurement performance in case a camera is being used to detect the bridge displacement without any target fixed to the structure, relying on the structure natural texture. These tests cannot be considered a real and complete innovation in case of controlled laboratory conditions: a smaller number of reported cases relates to field measurements. It must be noted that these tests are being performed under uncontrolled environmental conditions and are applied to a civil structure monitoring, where the image analysis must deal with the specific requirements of this practical application. In comparison with other similar works about real structures [8, 9], this one is focused on the uncertainty qualification of these techniques applied under uncontrolled environmental conditions.

9.4.1 Frontal Measurements with Target Fixed to the Structure

These measurements have the purpose to qualify the results obtained by the image analysis when the camera is placed in front of the bridge side. As explained in the previous section, several targets, one every 3.65 m, have been placed on the structure (Fig. 9.3). The tests specifications are summarized in Table 9.1: for every test the value of the px/mm ratio is given for the targets acquired by the camera. During test 1 the camera was near the structure and the zoom parameter has been set to have only one single target enclosed in the field of view. Under these conditions the highest image resolution in terms of px/mm ratio has been achieved, but for just one measurement point. All the further tests have been made by increasing the number of acquired targets so that a wider description of the vertical displacement of the bridge deck could be obtained by a single video. However, increasing the target number has the drawback of decreasing the image resolution in terms of the px/mm ratio, which means to worsen the measurement accuracy. For this reason, the presented results will consider tests with no more than five targets framed by the camera; further increase in the mm/px scaling leads to unacceptable measurements uncertainty.

Table 9.1 Frontal tests: framed targets and local scaling factor (px/mm)

The aim of these tests was to evaluate how the accuracy of the estimated displacements is affected by the target number and consequently by the image resolution. This evaluation has been carried out on a real big structure and with uncontrolled environment conditions, which is the situation in which the whole process is expected to operate for structural monitoring purposes. For every target the pattern matching technique is applied to the two blobs, whereas the edge detection is applied to the tilt line and a mean value of its position at every frame is extracted (Fig. 9.3). In this way it is possible to estimate the target displacement as a function of time.

Figure 9.4a shows the results obtained for test 1 (measurement point #7), where x axis is time and y axis gives the vertical displacement at mid span. The bridge vertical displacement, due to the train passage, which is between 7 and 8 mm can be appreciated. The green line identifies the laser displacement measurement which can the considered the reference for the camera qualification. The red and blue lines represents the displacements estimated with the edge detection and pattern matching techniques respectively. The figure shows a good correspondence among data obtained with the three techniques. A direct comparison in the time domain between the displacement measured by the camera (the edge and the pattern analysis) and the reference (the laser) is given in Fig. 9.4b, where the maximum discrepancy is around the absolute value of 0.4 mm (this is the method uncertainty, as data are the same).

Fig. 9.4

Displacements measured at point 7 (a) and discrepancies between camera and laser (b)

In Fig. 9.5 the moduli of the spectra (1–12 Hz) of the bridge sag computed starting from camera based measurements are compared with the one of the reference interferometer. Both edge detection and pattern matching are able to provide a reliable estimation of the dynamic displacement of the structure: the peak frequencies are correctly identified and their amplitudes are properly estimated. An overall slight overestimation of the structure vibrations can be notice in all the harmonic components characterized by low levels of vibration: this is a consequence of the lower signal to noise level of the camera based measurements with respect to the reference interferometer.

Fig. 9.5

Moduli of the spectra of the bridge sag computed at point 7: test 1

The same evaluations have been made for the other tests, but a direct comparison with the reference signal is possible only for target #7 (Fig. 9.2) because no reference transducers are available for the other tested bridge sections. However, it should be noted that the camera is mounted with the optical axis almost normal to the bridge, therefore, in each test, the scaling factor is nearly the same for all the targets; thanks to this consideration we can assume that the validation for the measurement at point #7 is valid also for the other targets. In order to give a complete description of what can be measured with a single video, all the estimated displacements of test 5 are shown together in Fig. 9.6. In this case data are obtained by the pattern matching algorithm applied to all the five targets. The results give a description of the bridge sag at different positions along the deck, and data are consistent with simple models of a beam supported at both ends, not being possible a direct check with a reference measurements for all points.

Fig. 9.6

Displacements measured from point 03 to point 07 in test 5 applying pattern matching algorithm

It can be qualitatively noticed a higher noise level in the single point measure, as a consequence of the lower px to mm scaling factor. The same conclusions can be derived by the comparison of the signal spectra estimated by the camera with respect to the one measured by the interferometer at point #7 (Fig. 9.7): as in the previous example, the camera based approaches are correctly able to identify the peak frequencies but a higher noise level can be notice as a consequence of an increased framed area.

Fig. 9.7

Moduli of the spectra of the bridge sag computed at point 7: test 5

Bringing attention to the measurement qualification, the Fig. 9.8 summarizes the results obtained from the tests, for the position of #7; in the figure the RMS value of the discrepancy between the reference signal and the displacement estimation is shown. The evaluation has been performed only for the time record corresponding to the train transit. The values are plotted as a function of the image resolution in terms of the mm/px ratio. The data quantify the trend of the measuring uncertainty with respect to the scaling factor. The results, as expected, show an increasing trend.

Fig. 9.8

RMS of the discrepancy of target p07 as a function of resolution

Figure 9.9 shows the RMS dynamic component (the standard deviation) and the RMS static component (the mean value) which are related to the RMS value by the formula:

Fig. 9.9

Mean and standard deviation of the discrepancy of target p07 as a function of resolution

$$ RMS=\sqrt{{\frac{1}{N}\mathop{\sum}\nolimits{\Delta^2}}}=\sqrt{{(\mu {{{(\Delta )}}^2}+\sigma {{{(\Delta )}}^2})}}. $$

where μ(Δ) and σ(Δ) are respectively the mean and the standard deviation of the discrepancies between camera and reference signal in correspondence of a fixed target.

The analysis of the mean and standard deviation trends help to understand the RMS behaviour in Fig. 9.8. The standard deviation shows a clear linear increasing trend up to about 5 mm/px resolution value, whereas the mean takes random values between −0.20 and 0.20 mm. This means that the uncertainty linked to the dynamic component of the bridge vibrations is strongly affected by the image resolution, whereas the uncertainty of the static measure of displacement due to the train mass depends on the uncontrolled biased errors (mainly uncertainty in the mm/px ratio).

Finally, in Fig. 9.10 the mean and the standard deviations of the discrepancies between the interferometer and vision data are shown in the case of no train on the bridge. The goal of this analysis is to analyse the uncertainty in this condition and compare it with the measuring uncertainty obtained during the train transit. As it can be seen in Fig. 9.10, the values and the trend of the data match well with those of the previous figures. The conclusion is that the measurement uncertainty does not depend upon the vibration condition experienced by the bridge; on the contrary, it is only due to the image resolution and to errors in the calibration process; it is consequentially possible to estimate the accuracy of camera based measurements by simply framing the unloaded tested structure, without a real need for calibrated external reference.

Fig. 9.10

Mean and standard deviation of the discrepancy in correspondence of no train transit

9.4.2 Targetless Analysis

The analyses presented in the previous paragraph were based on the presence of fit-to-the-purpose targets at the measuring points. This approach presents certain advantages but it requires to access the structure before the test execution, which cannot always be guaranteed in real applications. The proposed techniques can work even without any target, relying on the natural texture of the structure: the analyses of the resulting performances in the targetless conditions will be presented in this paragraph.

Targets on the measuring points present two main advantages: on one hand, a framed known geometry makes the computation of the mm/px scaling factor straightforward; on the other hand, it is possible to design sharp edges and high contrasted patterns, basically increasing the signal to noise ratio of the measurement process.

In order to present an unbiased comparison with respect to the analyses performed using targets, only the second aspect (i.e. the possibility to obtain sharp and high-contrast target) will be investigated, maintaining the previously computed scaling factor.

The comparison has been performed only close to point #7, near one of the five test targets (Fig. 9.11), where the laser interferometer acquires the bridge sag; in this test the camera frames the bridge with a scaling factor of 0.108 mm/px.

Fig. 9.11

Five patterns test, target-less analyses: first frame of the movie and tracked features

Two different features have been extracted and tracked in the movie (Fig. 9.11). The first is a horizontal strip due to the presence of a white tape stuck to the bridge. This feature represents a less intrusive pattern with respect to that analyzed in the previous paragraph. This feature has been tracked with both the edge detection and the pattern matching algorithms. The extracted pattern dimension is 18 × 83 px and edge detection data have been averaged on the same width (18 px). The second feature is a riveted section of the bridge. Due to the lack of evident edges, only pattern matching can be applied for this analysis, on the 57 × 58 px pattern of Fig. 9.11.

In Fig. 9.12 the discrepancies with respect to the data recorded by the interferometer during the train transit are graphed and the previously presented synthetic data are reported. Considering the first pattern (tape), a slight increase in the standard deviation of the discrepancies, with respect to the data obtained from the target, can be noticed for both edge detection (from 0.31 to 0.36 mm) and pattern matching (from 0.33 to 0.35 mm) analyses. Considering the small differences in the measurement performances with respect to the previous presented data, this approach can still be considered attractive in cases where either the presence of the operator on the structure has to be minimized or the surface presents inherent textures comparable to the tested one.

Fig. 9.12

Five patterns test, target-less analyses: discrepancies with respect to the interferometer data

The third curve describes, conversely, a more critical situation: the rivet plate connecting the two beams is characterized by very low gradient in its textures and this results, in the analysis, in a higher measuring uncertainty. The standard deviation of the discrepancy is about 0.82 mm (more than twice the one obtained with the tape), with peaks that reach 2.5 mm on an 8 mm displacement.

9.5 Concluding Remarks

Static and dynamic monitoring of civil structures represents a crucial step in serviceability and health assessment. Vision-based measuring techniques are in these years gaining more and more importance, due to their advantages in terms of density of measured points and static and dynamic capabilities. In theory, nowadays image processing algorithms allow to appreciate displacement fields with an accuracy that can reach a few thousandths of pixel, but in real applications the performances strongly decreases due to external uncertainty contributions, such as lighting conditions, sensor noise, camera vibration and scaling factor estimation. As a consequence their application results in a compromise between field of view and accuracy. In this work, the relation between measurement uncertainty and spatial resolution is studied in a real environment. Two different state of the art image processing algorithms, edge detection and pattern matching, are exploited to estimate the vertical sag of a railway bridge subjected to train transit. The collected data are compared with a low uncertainty reference, a laser interferometer transducer, in order to quantify the vision-based measuring uncertainty.

At first, the displacement is measured in correspondence of fit-to-the-purpose targets mounted on the structure: the root mean square of the discrepancy between camera-based measurements and reference transducer shows an approximately linearly increasing trend with respect to the setup scaling factor. The RMS of the discrepancy has been decomposed into its two components: the standard deviation, representing the random component of the discrepancy and characterized by a deterministic trend if plotted against the scaling factor, and the mean discrepancy (bias), that, on the contrary, does not show any evident trend. Furthermore, it has been proven that the measuring uncertainty estimated in static condition (no loading of the structure) is able to correctly quantify the measuring system uncertainty, suggesting an easy way to estimate such a parameter in an on-the-field application.

In the last part of the paper the displacement of the bridge deck is measured without using the targets but relying only on the natural texture of the bridge. In this conditions the measurement reliability is strongly affected by the structure texture contrast, however, in favourable conditions, small differences in the measuring performances are found with respect to the measurement with target mounted on the bridge. The markerless approach can still be considered attractive in cases where either the presence of the operator on the structure has to be minimized or the surface presents inherent textures comparable to the tested one.