The Measurement of Construction Error Technology for Deformable Modular System by Image Processing

Abstract

Modular structures are growing into a promising field in the construction field. However, the error between bolting, which occurs during on-site construction of a modular system, is one of the biggest drawbacks of a modular structure and is a serious problem that must be resolved. If this error keeps occurring, it can weaken the strength of the overall modular structure and can cause further damage. Therefore, precise measurement of displacement errors is required to reduce construction errors of modular systems. In this paper, we propose a measurement technique based on vison-based images without using displacement meter such as LVDT. This technique is consist of raspberry-pi cameras which is much more economical than displacement meters. Through image processing algorithm, displacement and angle can be measured. As a result, it is judged that the error rate for bolting can be minimized during construction of a modular system by measuring an error range of up to 0.24% through this technique presented in this paper.

1 Introduction

Modular system is that all parts like frames, plumbing, and insulations are manufactured in advance at facilities so that those are assembled fast in site. It can make the 80% of construction process manufactured and reduce the construction period up to 50% (Kim & Lee, 2011). This systems are mainly constructed with the steel structure as a basic unit frame. Also bolt joint strength in modular system directly affects the overall system after it combined. The system is quite effective system for low-story buildings in constructive side but it also can be a drawback for high-story buildings in structural side. To deal with this weakness, it is required enough capacity for internal strength in modular joint. Furthermore, new research of modular joint has been studied (Choi & Kim, 2014). The modular joints are used for connecting other modular with bolts so discorded bolt holes are led to construction error. It is very difficult to make two holes straight in sites. Therefore, it is suggested that the nominal bolt diameter is generally used 2 mm lager than bolts (Kulak et al., 2001).

There are two possible errors in modular system. One occurs in manufacturing and the other occurs in delivering, which are manufacturing error and construction error by deformation of modular, respectively. As the precise of construction in modular system required, it is important to minimize the error (Shin et al., 2017). The construction error by deformation of modular can be diminished by using fastener or preventing impact load. But manufacturing error is inevitable when consisting modular system, even though quality management is implemented. Reducing this construction error is mandatory for maximizing its advantages and increasing its possibility of use. In order to reduce the construction error of such the modular system, a precise displacement measuring method taking into account the characteristics of the modular are required. Also, the most general error in modular system is the difference of bolt holes arrangement. The error of tightening bolts affects directly the joint strength of its modular system. If the error occurs in site, it delays the construction period because it disturb quick installation of modular frame. Therefore, a necessity to reduce construction errors has emerged by applying image processing technique to the modular construction process. In this study, the image processing method that can find construction error by measuring deformation of the modular system is introduced and verified through the experiments.

1.1 Image Processing Review

A variety of image processing techniques have been developed by many researchers. Wang et al. (2018) suggested the algorithm using long pulse thermography in order to evaluate the damage of the composites. Vision-based image processing method has been frequently used to find crack or deformation on the structures (Kim, 2016; Ni et al., 2019). Lee and Shinozuka (2005) made a technique that chases target trace and finds the target in real time. This method was applied to bridges which hardly measure displacement with displacement device such as linear variable differential transformers (LVDT). Potenza et al. (2017) also detected the damage on steel bridge by mobile robots and UAV. Moreover, other image processing technique applied to high-rise building. Park et al. (2010) suggested new type of method, which were vision-based image processing. This method is based on shear deformation so that the displacement of top story in a building can be easily obtained. Also, Choi et al. (2011) presented a new concept of technique using image processing named dynamic displacement vision system (DDVS), which applies mm/pixel coefficient (MPC) to calibrate the distortion of video images captured by camcorder. To prove the DDVS algorithm, it was used to measure dynamic displacement in shaking table test.

1.2 Scope

This study aims to verify of image processing method by measuring the deformable variations in modular system based on video image. This technology is consist of two units, which are image filming unit and image analysis unit. At image filming unit, raspberry-pi cameras are used to film. 4 cameras are located on the each spot where the column is. After capturing image, it is sent to the image analysis unit by wireless data communication. At image analysis unit, MATLAB program is the main data processing tool. Using MATLAB program, the captured image is used to be measured the error of arrangement of bolt hole.

Two experiments are implemented for this technology. First, an experiment was conducted to verify the image processing algorithm. Secondly, a reduced modular specimen was set up in a small laboratory-level space, and the arrangement of bolt holes was simply set up with L-shaped plate to conduct measuring experiment of 2-D displacements. The reason why L-shaped plate was used is to alternate other modular unit so that it is checked how many angle is rotated on each modular joint. At this test, the bolt-to-bolt distance (d_ij), the edge distance from bolt (u_ix, u_iy), and the angle (θ_i) of each corner were measured so that the reliability of the algorithm was confirmed by comparing the measured data and the actual distance. Finally, the experiment was carried out in the realized modular transport process and the error rate of the measured data is to be checked. By confirming this technique, it aims to minimize errors in the actual construction site by checking errors in manufacturing.

2 Concept of the Measuring Algorithm

2.1 Scale Model Experiment

It is common to use a crane to transport the modular unit after fastening it with a lifting frame of the same size as the unit. The same lifting method is used in manufacturing plants and construction sites. 4 cameras for capturing the bolt arrangement were installed at the corner of the lifting frame. If one camera had been used, the focal length became much farther than original length in order to shoot all bolt arrays, which affects the distortion of the image or the accuracy of the bolt array measurement. Therefore, the technology, as shown in Fig. 1, was configured to measure the bolt arrangement of each column and the bolt distance between columns by arranging each camera for each column.

Fig. 1

Concept of modular system

For the algorithm review, as shown in Fig. 2, it was made into a scale model and constructed in a shape that can be photographed horizontally. The image filming unit is equipped with four cameras capable of photographing on the edge of base plate, the target (called Marker), instead of the real bolt hole, marked with 4 black circles was photographed. 4 camera modules were connected with raspberry-pi and the target was shot at the same time with the cameras. The images captured by each camera are transmitted through wireless data communication to a single board computer (Raspberry-pi), and the bolt arrangement and overall dimensions are measured using the image processing algorithm. Figure 3 and 4 shows the actual test set-up and devices of the image filming unit. The distance between camera and bolt hole was set to 200 mm and one bolt hole can be captured in one camera module (Fig. 2).

Fig. 2

Concept of measuring displacement of modular system

Fig. 3

Test set-up of scale model

Fig. 4

Raspberry-pi and Camera module

2.2 Specimens

To verify the image processing algorithm, the target, named ‘Marker’, was made with 4 black circles on a white background as shown in Fig. 5. 4 black circles mean 24 mm of bolt holes where M22 bolts can be fastened at each corner of a modular unit. Utilizing one bolt hole (black circle), distance between bolts (d_ij), distance between bolt and edge of Marker (u_ix, u_iy) and corner angles (θ_i) were measured. Four specimens were set. The reference specimen is BH-0 which has no variables. Additional variations are set as the name of specimens go from BH-0 to BH-3. B₁ keeps its position, B₂ keeps moving to the down for 1 mm, B₃ for 1 mm to the right and B₄ for each 1 mm to the left and down simultaneously. The BH-2P was also set by reducing the corner angles from BH-2 specimen, as shown on Table 1.

Fig. 5

Detail of target (marker)

Table 1 Exact size of each specimen

2.3 Image Processing Algorithm

First, a shared folder was created to transmit the images acquired from the raspberry-pi to a computer for analysis in real time. Using raspberry-pi cameras located at each corner of the base plate, shot bolt holes images located at four corners and saved them in a shared folder. Afterwards, the four corner images saved in the shared folder were loaded on MATLAB for analysis and made scale images grey. The center coordinates of the circle were extracted and the radius length was obtained using a circular Hough transducer algorithm (Atherton & Kerbyson, 1999). At this time, the obtained center coordinate and radius length were in pixel unit so these were required to convert into mm unit for comparison with the actual length. In order to obtain millimeter per pixel (mm/pixel) in each image, the diameter (mm) of the black circle is divided by the diameter of the pixel unit extracted from the image. If the obtained mm/pixel value is different from other mm/pixel value detected in other black circle, find the mm/pixel value again. In the following order, measure the distance between the centers of each of the four camera lenses and the black circles extracted from each image as shown in Fig. 6. By multiplying the mm/pixel value of each image obtained previously, X_i distance between the center of the camera lens and the bolt hole can be measured. In each image, after measuring X_i distance, add X₀ distance between the cameras lens. X₀ is 210 mm which is initially set. Finally, the distance (d_ij) between each bolt hole can be measured. Additionally, although the distance (d_ij) between the bolt holes matches the actual distance, the edge distances (u_ix, u_iy) and the entire outer frame could be significantly different. If so, it can be difficult to fasten the modular units during construction. Using the algorithm to prevent this, after found the boundary of Marker, distance (u_ix, u_iy) between the edge and each circle are measured. The total length of the frame (D_ij) can be obtained by adding the d_ij distance and the measured distance (u_ix, u_iy) between the corner and the bolt hole, except BH-2P. Because this algorithm can only measure in X or Y direction. Also, using 4 edge of Marker obtained previously, an angle (θ_i) of each corner can be measured. Figure 7 is an algorithm that measures the distance and angle using image processing.

Fig. 6

Find distances between two black circles

Fig. 7

Algorithm of suggested image processing

2.4 Result of Algorithm Verification Test

Table 2 shows the whole test results of image processing. In the case of the distance between bolts (d_ij), maximum difference was 0.46 mm and the distance between the corners of the frame (D_ij) showed a slightly larger than other measurement distances, which maximum difference of 2.86 mm occurred. In measuring the D_ij distance, it is difficult to extract the boundary of the edge accurately due to the distortion of the camera lens. That is why the D_ij error is larger than d_ij error is. In the case of edge distance (u_ix, u_iy), maximum error occurred at B1, which was 1.42 mm. But other edge distances showed the difference 0.5 mm below. It is believed that the conversion factor (mm/pixel) could influence the when the unit conversion of the edge distance. The angle (θ_i) of the frame edge was 1.41° from the B3 edge (θ₃) of the specimen (BH-2P) whose angle was adjusted, showing the greatest difference. The error rate at this was 1.55%. Figure 8 shows the difference between the real size (RS) and the image processing size (IS). Figure 9 shows the actual shape (RS) and the image processing result (IS) as relative errors. The distance between the edges of the frame (D_ij), which showed the greatest absolute difference, showed an error within 1%, and the edge distances (u_1x, u_1y) showed a maximum error of 3.16%. The rest of the variables showed an error within 1.5%. In addition, the specimens except for the BH-2P specimens have an error rate of 0.14%, which is estimated to be nearly accurate.

Table 2 Results of image processing

Fig. 8

Numerical difference (IS-RS)

Fig. 9

Error rate of the algorithm

3 Experiment of Deformed Modular Unit Measurement

3.1 Experimental Plan

Image processing algorithm was reviewed and verified in scale model test. Full scale of experiment for deformed modular unit were conducted. As shown in Fig. 10, it is similar to the actual modular size but it was manufactured in a size of 4000 mm × 2000 mm × 1500 mm that can be tested in a laboratory. For the transport of the modular unit, the lifting frame was manufactured in same size of the modular unit. The lifting frame and the modular unit were fastened with eye bolts and six chains (1500 mm) in a same way at construction site. 4 raspberry-pi cameras were installed at each corner of the lifting frame and the distances between the each camera was 1840 mm (Y₀) for the short side and 3840 mm (X₀) for the long side. The variations in this experiment is shown in Fig. 11. The L-shaped steel plate located at each corner is 160 mm × 80 mm (t = 6 mm) with two M22 bolt holes (24 mm). The experiment was conducted while the No.4 L-shaped plate is moving diagonally or rotating its angle in anticlockwise on the outer vertex. When the plate changed diagonally, the specimens were named BH-C and the other specimens were named BH-A when its angle was changed. Exact size of specimens show in Table 3.

Fig. 10

Experiment for deformed modular unit

Fig. 11

Upside view of modular unit

Table 3 Exact size of specimens for deformed modular unit test

3.2 Algorithm for Deformed Modular Test

The concept of algorithm used in this experiment is as same as the previous experiment using a raspberry-pi and MATLAB. Each image captured on the raspberry-pi cameras is transmitted to a computer for analysis through wireless data communication. If the transmitted images are distorted, they are calibrated by MATLAB to increase the accuracy of image analysis. The images are cropped in 514 × 346 pixel to make the plate on the center of the image for effective interpretation. Find the two bolt holes in each cut-out image and measure mm/pixel through the actual distance (mm) and the distance on the image (pixel). If the bolt hole is not found, the existing RGB color map is changed to the HSV color map. Then, convert the image into black and white when value is in average. Convert back to RGB and find the center of the bolt and mm/pixel value. The reason why especially brightness in average among hue, saturation and value is that the bolt hole could not be found when modular is. Although the error rate is the least among them as shown in Fig. 12, brightness (value) is changed to average before it converts back to RGB to find the bolt hole. The distance (X_i or Y_i) between the center of the lens and the center of the bolt is measured. Next, it is multiplied by mm/pixel value to convert the distance into mm unit. Finally it is found that the distance between two holes by adding X₀ (or Y₀), as shown in Fig. 13. Also, the corner angle is measured, using the slope of the connected line between the centers of the two bolts in each image.

Fig. 12

Hue, Saturation and Value (HSV) error rate

Fig. 13

Find the distance between two bolt holes

3.3 Algorithm Test Results

Table 4 shows image processing size of test results. In short side of the modular unit (D₁₂, D₃₄), the maximum displacement error occurred, which was 1.81 mm at BH-A-10. In long side (D₁₃, D₂₄), up to 0.75 mm is differe°nt with the original length. Also, 1.5 mm error occurred in diagonal length (CD₁₄, CD₂₃). So it is judged that these errors can be acceptable because extra bolt hole is 2 mm. Besides, in angle case, maximum error occurred 0.84° at θ₃. As shown in Figs. 14 and 15, it was confirmed that the maximum error rates of D_ij and CD_ij are 0.08% and 0.1% for BH-C and BH-A, respectively. On the other hand, in case of θ_i is 0.26% and 0.24% respectively. It is confirmed that the error rate could occur three times more than error rate of the D_ij and CD_ij. The reason for the greater error in angle measurement is predicted to be due to various influences such as camera distortion, accuracy of bolt center coordinate extraction, or distance between camera and the plate. If the cameras were rotated before the test, the distance between bolt holes would be more lager as much as sine (θ₀) (θ₀ is the angle of the center axis of camera to the target). Positively, when comparing the actual distance and the measured distance by image processing, it is judged that the reliability of this technology is verified by comparing with each error rate.

Fig. 14

Error rate of BH-C specimens

Fig. 15

Error rate of BH-A specimens

Table 4 Exact size of specimens for deformed modular unit test result

As shown in Fig. 16, average and standard deviation for total specimens (BH-C and BH-A) can be confirmed. No.4 plate was a moved and rotated as a variation so that the distances and angle measured at No.4 plate have large standard deviation. The maximum difference of standard occurred at CD₁₄. When the deformed modular test was conducted, variations were set up in person. Figure 17 shows that the averages of variations are less than 0.4 mm, putting up with the error cause of the plate installation. So, numerical difference between IS and the other IS in BH-C-0 (reference) are more reliable than the difference (IS-RS).

Fig. 16

Statistical result of │IS-RS│

Fig. 17

Statistical result of │IS-IS (BH-C-0)│

4 Conclusion

This paper suggests that the algorithm of measuring the deformation of the bolt array of the modular system using image processing and the practicality by using the actual modular is verified. The accuracy of the algorithm was confirmed through the comparison analysis of the actual length and angle (RS) and the data of image processing (IS). For length variations (D_ij and CD_ij), maximum error rate was confirmed 0.1%. Also, maximum error rate for the rotation was confirmed 0.26%. This results is based on the 4 raspberry-pi cameras is assumed that the axis is straight to the target. Through this analysis, it is believed that an image processing technology can be used at the manufacturing plant and site if a communication network is capable of transmitting captured image. In addition, as a result of considering the maximum displacement error and error rate for each variable through two experiments. Also, through the average and deviation numerical difference (IS-RS), the reliability of the image processing technology is confirmed. Therefore, it is ensured that this technology can be used as a solution that can measure the self-deformation of a modular in a modular manufacturing plant or construction site. Finally, it is considered that it can have a great influence on the reduction of the construction period.