Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

In several applications of structural monitoring and/or structural health monitoring (SHM), it is extremely challenging to attach to a structure sensors able to quantify the response to environmental conditions and to the geometrical constraints. This is mainly made for checking that the assigned serviceability requirements are satisfied. Non-contact experimental techniques have been developed as an alternative approach [3, 4, 11]. Non-contact devices are also suitable for the experimental analysis of properly reduced scale models: to adopt conventional sensors in such experiments could result unfeasible since the added sensor’s mass can affect the behavior of the models. For this reason, there is a growing interest in developing alternative techniques for measuring movement without allowing contact with the structure [8]. Moreover, non-contact sensor could be more durable since it doesn’t endure the same hazard of the structure. Among them vision-based monitoring systems are currently proposed [7].

A vision-based positioning system is adopted in this paper for in-plane SHM and image processing algorithms are applied to optimize its performance. Velocity and displacement measurements are based upon tracking the object motion between sequences of images. Displacement measurements based on vision system provides a good accuracy and a good robustness although a “sight-on-line scene” is required. The implemented system covers a medium range field and takes advantage of well-developed and still developing processors and digital image processing technologies. On one hand, the fast development of processors provides the designer the flexibility to develop their image acquiring and processing system according to their application scene thanks to a great increase of the image processing ability of the hardware. On the other hand, the image processing algorithm and the theory for computer vision have been well-developed in the last several decades. It is worth mentioning that Intel has developed three cross-platform libraries to support real time computer vision: Intel®; Integrated Performance Primitives (Intel®; IPP), Open Source Computer Vision (OpenCV) and Image Processing Library (IPL) [6]. In the present study the off-the-shelf image processing software Image Pro Plus 6.0 [9] is used. Image-Pro Plus 6.0 performs the image enhancement using powerful color and contrast filters and other spatial and geometric operations. It can trace and count objects manually or automatically and measure object attributes such as: area, perimeter, roundness etc. The collected data can be viewed numerically, statistically or in graphic form (histogram and scatter gram). The features isolate an area of interest from the rest of the image: it can be extracted by spatial tools, or by using segmentation tools that extract features by color or intensity value. Images from multiple fluorescent probes can be composed.

2 Vision-Based Monitoring Systems

2.1 Hardware and Software

A monochrome camera SV642 (Table 1) is employed to capture the motion of a three-stories frame, which is mounted on a shaking table. It provides 640 by 480 resolutions at 204 frames/s (fps). The sizes (vertical V and horizontal H) of view can be calculated according to the focal length f, the work distance W and the sensor sizes (v and h).

$$\displaystyle{ \frac{f} {W} = \frac{h} {H} = \frac{v} {V } }$$
(1)

The captured image sequences are sent to a PC through a PIXCI digital frame grabber for PCI bus. The software “Image Pro Plus 6.0” is used to process the image sequence. When one measures the displacement based on a vision system, the unit for the displacement of the object is the pixel, which should be multiplied by the scaling factors to obtain the real displacement. Assuming the size of the target is (L x , L y ) which corresponds to (x pixels, y pixels) on the image, the scaling factors (S x , S y ) are calculated according to

$$\displaystyle{ S_{x} = \frac{L_{x}} {x} \,,\;S_{y} = \frac{L_{y}} {y}. }$$
(2)
Table 1 The parameters of the SV642 camera
Full size table

2.2 The In-Plane Measurement Method

Velocity and displacement measurements using images are based upon tracking the object motion between sequences of images. The proposed image processing methodology comprises four steps:

  1. 1.

    Preprocess: it is the process of manipulating an image so that the result is more suitable than the original for a specific application, such as contrast enhancement.

  2. 2.

    Calibration: it is employed to determine the transformation matrix that correlates the image coordinates and their respective actual coordinates. After one ensures that the camera optical axis is perpendicular to the structural plane, there remain a distortion of the image which may be caused by the optical system (radial and tangential lens distortions, perspective distortion) and the electronic system. These distortions were estimated for less than 0.5 pixels on the basis of the observation of the test image and could be reduced to 0.02 pixels by employing a suitable image processing algorithm [10]. In order to obtain the coordinate transformation matrix, which establishes the correlation between the actual coordinates and those in the image, it is necessary to apply an adequate calibration method. It is performed by identifying the (u, v) coordinates of the calibration points selected in the image and inserting the respective actual coordinates (x, y) (2D). This step is carried out by the calibration routine. The calibration can be performed in one of these two ways: (i) extrapolation and (ii) interpolation, as illustrated in Fig. 1a, b. In the first approach, the calibration is performed on the basis of the known dimensions of the structure or through the application of markers with pre-established distances in the structure itself. The advantage of this procedure is that it facilitates the acquisition of the actual data. However, extrapolating the measurements outside the region used for the calibration can result quite imprecise. In the second way, the calibration is performed through markers defining the borders of the region where the movement of the structure occurs. Therefore, any configuration of the structure will always be within the calibration region. Thus, the results obtained through this method are expected to be more precise than those obtained by simple extrapolation. The disadvantage of this second method is that it is necessary to impose markers outside the structure, which in some cases can result difficult [8].

    Fig. 1
    figure 1

    Forms of calibration: (a) extrapolation and (b) interpolation

    Full size image
  3. 3.

    Segmentation: it is the process of assigning a label to every pixel in an image, such that pixels with the same label share certain visual characteristics. Thus, it partitions a digital image into multiple segments and simplifies the representation of an image into something that is more meaningful and easier to analyze. It is typically used to locate objects and boundaries (lines, curves, etc.) in images. Several general-purpose algorithms and techniques have been developed for image segmentation, such as thresholding, histogram-based methods, edge detection, watershed transformation method, etc. [5].

  4. 4.

    Object representation, description and recognition. To recognize objects, one must have an internal representation of an object suitable for matching its features to image descriptions which means to describe or represent the object through certain features. There are three kinds of techniques to represent the objects [2]: appearance-based, intensity contour-based, and surface feature-based techniques. Shape represent techniques, as that of appearance-based techniques, include contour-based (perimeter, compactness, eccentricity, etc.) and region-based (area, Euler number, eccentricity, geometric moments, etc.) options [12].

3 Feasibility of Vision-Based Displacement Measurements

In this section, an experiment using SV642 is carried out to analyze the feasibility of a displacement measurement vision-based system. A three-stories frame is mounted on a shaking table which moves at the frequency of 2. 4 Hz, with the amplitude of 4 mm. The dimension of each story is 60 × 30 × 3 cm [1]. The first and third stories are braced. The optical axis of the SV642 camera is perpendicular to the front plane of the frame. The sample rate is 120 frames/s and 1,242 frames are recorded (i.e., 10.3 s). Two white markers T1 and T2 are installed at the supported nodes on the second story, while further two markers T3 and T4 are installed on the first story, as shown in Fig. 2.

Fig. 2
figure 2

Photographs accounting for the initial position at rest. On the right side four markers are outlined

Full size image

According to Eq. (2), one gets 1.685 mm/pixel as the scale factor S. Thus the obtained horizontal displacements shown in Fig. 3 are derived and the resolution turns out to be 0.16 mm (0.1 pixels).

Fig. 3
figure 3

The track of: (a) T1 (left-top); (b) T2 (right-top); (c) T3 (left-bottom); (d) T4 (right-bottom); (The unit of vertical coordinate is the pixel)

Full size image

The movements of T1 and T2 are consistent each with the other, while the movements of T3 and T4 are consistent each with the other. They are quite similar to the shaking table sine wave since the stiffness of the first story is largely increased by the presence of the cross-brace. On the movement of T1 and T2, one can distinguish two mode frequencies: one is 2.4 Hz (the excitation), the other is 0.41 Hz (the system). If one compares the movement between T2 and T4, a 180∘ phase difference can be discovered. Focus now the attention on the movement of T1 (Fig. 4): one obtains for the maximum displacement of T1 the value 16.85 mm (10 pixels) and the instant when the maximum shift occurred: it corresponds to the digital image n∘175 which is shown in Fig. 5.

Fig. 4
figure 4

The maximum displacement of T1

Full size image
Fig. 5
figure 5

The instant when maximum shift occurred

Full size image

The vertical direction displacements are shown on Figs. 6 and 7. The vertical displacement of T3 and T4 is similar and is nearly 0 due to the fixing cross-brace. But the vertical displacement of T1 and T2 cannot be ignored. Its peak-peak amplitude is around 0.7 pixels and corresponds to 1.18 mm. One can also distinguish two frequencies from Fig. 6: 4.8 and 0.41 Hz.

Fig. 6
figure 6

The vertical displacements of T1 and T2

Full size image
Fig. 7
figure 7

The vertical displacements of T3 and T4

Full size image

Figure 8 shows the absolute values of the difference of the horizontal displacement in T1 (and T2) at time t i and t i−1. Since the movement is uniformly sampled, these values are directly proportional to their velocities. One can see That the velocity of T1 coincides with the one of T2. The absolute value of the difference of the horizontal displacement of T3 (and T4) (Fig. 9) is relatively small, since the two markers are rigidly connected to the supporting system. But the absolute distances of T3 (and T4) suffers a serious noise.

Fig. 8
figure 8

Absolute values of the difference of the horizontal displacement in T1 (and T2) at time t i and t i−1

Full size image
Fig. 9
figure 9

Absolute values of the difference of the horizontal displacement in T3 (and T4) at time t i and t i−1

Full size image

4 Conclusions

This paper introduces a vision-based view of structural monitoring. The procedure is illustrated for in-plane measurements. The feasibility of a vision-based measurement system is analysed with reference to a three-stories frame which is mounted on a shaking table. Displacement and velocity signals are adequately inferred and a high resolution is achieved. The errors in accuracy could be magnified by different focal lengths, different flickering illumination, different background colours and different work distances. The influence of different camera orientations is a further aspect to be considered and will be the object of future work.