Keywords

1 Introduction

The digital images captured at a construction site have increased dramatically in recent years. Furthermore, convenient, inexpensive, and quick access to these images sparked a wave of research into data collection and construction site monitoring [1, 2]. Videos and images allow project managers to monitor building sites more efficiently, enhance communication among stakeholders, and better track progress data [3, 4].

Many definitions have been proposed for image processing (IP). It is usually accepted that IP involves manipulating a digital image to generate a second image, while computer vision (CV) can extract the information from that image automatically [1, 2]. When discussing the adoption of IP in the construction industry, it is common to associate it with CV-based technologies and building information modelling (BIM), as illustrated in Fig. 1

Fig. 1
A venn diagram for integration of monitoring technologies. Three technologies are illustrated. Image processing, C V based technologies, and B I M.

Integration of monitoring technologies for construction monitoring

Full size image

The merging of IP and CV-based technologies promise to reduce traditional information gathering and report preparation time [2, 5]. This gives managers more time to focus on decision-making activities and to take corrective actions in real-time [5]. Construction managers rely on fast and accurate information to make successful decisions during the control process on the job site [5]. IP has increased the probability of construction site automation monitoring [6, 7].

IP studies have proved to be cost-effective and efficient for automated progress monitoring, resource tracking, and material classifications by extracting information directly from digital images [8,9,10]. This paper discussed the applications of IP in the construction industry, challenges it faces in adoption, and opportunities for more development. To achieve the objectives of this study, a review of the literature was conducted to select the relevant studies related to the applications of IP for monitoring construction projects.

2 Applications of IP in Construction

The applications of IP in the construction industry have increased dramatically recently among researchers due to the great improvement in specifications of capturing devices, ease of use of these devices (no high skills are required to capture images), high storage databases, and accessibility of the Internet [5,6,7]. The mechanism for reaping the benefits of IP in construction monitoring (CM) applications, applications of image processing regarded to construction progress monitoring (CPM), resources tracking (workers and machines), material classifications, productivity analysis, and structural element recognition are discussed in the next subsections.

2.1 The Mechanism of Image Processing in Construction Applications

The first aim of digital IP is to improve pictorial information for human interpretation, and the second is to process image data for transmission, representation, and storage for autonomous machine perception [2, 11]. The IP may be divided into low, medium, and high-level processes as shown in Fig. 2.

Fig. 2
An illustration of three levels of image processing. Low level, medium level, and high level. In high level, the input is attribute and the outputs are monitoring, tracking, and navigation.

Levels of image processing in construction

Full size image

There are several procedures used for IP implementation in construction sites: image acquisition, general processing, feature extraction, object detection, and image segmentation [12].

2.2 Image Acquisition

Image capture can be done by various devices like digital cameras, smartphones, monocular cameras, surveillance cameras, unmanned ground vehicles (UGV), and unmanned aerial vehicles (UAV) [6, 7]. The selected device for capturing has to be suitable for capturing images from the construction site because the construction sites have a uniquely dynamic environment where multiple machines and workers are working simultaneously, which causes occlusions and affects the captured scenes (for example, surveillance cameras fixed on construction sites are limited in view and could not cover the whole site) [6, 7]. It is also necessary to calibrate the selected device, which includes calculating the primary point location, radial distortions, and tangential distortions, which may be accomplished using a checkerboard or by utilising established camera calibration toolboxes [3, 4].

2.3 General Processing

Like image cropping, video clipping, key frame selection, and converting red, green, and blue (RGB) to other colour spaces (HIS, HSV, CIELAB, or grayscale) [12]. It is a vital phase in image and colour enhancement so that the end result is more appropriate than the original for certain applications. The goal of enhancement techniques is to draw attention to hidden elements or to highlight certain features of interest in an image [12].

2.4 Feature Extraction

Several techniques involve employing algorithms to detect features such as shapes, colours, and edges in images [12]. Extracting the needed features is critical to decreasing the number of resources without losing any significant information. Feature extraction is a stage in the dimensionality reduction process that involves reducing and sorting an initial collection of raw data into more manageable categories to simplify the subsequent processes [12].

2.5 Object Detection

Object detection is a part of image processing and computer vision that concerns itself with detecting instances of semantic items of a certain class (such as individuals, machines, construction elements, or equipment) in captured images and videos [12]. To produce meaningful results for object detection in construction, several algorithms typically use deep learning or machine learning to extract the required objects [12].

2.6 Image Segmentation

It is a technique used in digital IP to divide an image into various portions or regions, frequently according to the pixels’ characteristics, and it might include separating background and foreground pixels or grouping regions of pixels based on shape similarity or colour [12].

2.7 Progress Monitoring

The state of CPM must be identified and corrected in real-time to prevent construction overruns (time and cost) [13, 14]. CPM is defined as the collection, recording, and reporting of progress information to prevent overruns and take corrective actions in real-time [5]. The traditional methods for progress monitoring need manual data collection from construction sites and schedules by construction engineers [5]. As a result, traditional techniques are time-consuming, expensive, and error-prone. To overcome these difficulties, IP and CV technologies were used to boost automation levels in CPM recently by several researchers [3, 4, 8, 10].

IP has increased the automation of CPM, overcome the limitations of manual observation and collected construction site information methods. Evaluation of progress monitoring can be achieved by comparing as-built and as-planned models [5, 15]. IP is used to collect and analyze the actual status of the construction sites by integrating image processing with CV-based tools and algorithms and with BIM. The automated CPM procedure includes: (a) capturing as-built from construction sites; (b) retrieving information from the collected data and captured images; and (c) estimating progress [5, 9, 16].

Several researchers have made a valuable contribution to automated CPM by adopting IP [4, 8, 16]. For instance, Deng et al., [8] proposed a method that combines IP with CV and BIM for automated progress monitoring of tiles. To train a tile classifier, support vector machines (SVMs) and feature extraction methods (local binary patterns (LBPs)) are used. The enhanced edge identification technique finds the boundaries of constructed tiles in the images provided. The boundary line coordinates are then transformed into a real-world coordinate system using the calibration of the camera. Using information from room profile information from the BIM model and the camera location, the tiled area can be determined automatically. This model has a high accuracy of roughly 91.17 percent for tile categorization [8]. In addition to that, Hamledari et al., [4] used 2D digital images and a CV-based algorithm to find the current state of wall construction automatically. The proposed module recognised wall installation, plaster, and paint states, as well as studs and electrical outlets. Three databases of indoor building site images were used to validate the suggested technique. The model's good accuracy rates and quick performance allow it to be used at indoor sites and to provide information on the current status to future progress monitoring systems. The suggested technique has drawbacks such as its inability to recognise metallic boxes and the present state of partitions with restricted sight, its inability to evaluate a partially painted or plastered state, and its accuracy in poor shot scenes [4].

2.8 Resources Tracking

Tracking construction sites’ resources is essential in many construction applications like productivity recognition, progress monitoring, and safety management. CV-based technologies have been adopted for tracking several resources on construction sites simultaneously using a network of cameras without the need for tags [17]. This feature makes vision tracking a costeffective and time-efficient choice for crowded sites where a large number of resources must be tracked in a small area [17].

Several researchers have made valuable attempts for tracking resources (labours, cranes, dumper, loader, materials, etc.) using IP and CV-based technologies [17, 18]. For instance, Yang et al., [19] presented a comprehensive video camera tracking system for several workers, where machine learning is used in the tracking procedure. It employed kernel analysis to find statistically meaningful joint feature-spatial relationships, which were subsequently utilized to develop a construction worker appearance model. The tracking technique can track many workers in a given video sequence as shown in Fig. 3 [19]. Another study for tracking earthmoving equipment actions using support vector machine classifiers and spatio-temporal features was proposed by Golparvar et al., [20]. The testing findings showed average accuracies of 86.33 percent for excavator action identification and 98.33 percent for truck action recognition, respectively, indicating the potential for the suggested method's use for automated construction activity tracking and analysis [20].

Fig. 3
4 photographs. The above 2 photographs illustrate a construction vehicle moving load. The below photographs highlight a container having some construction material.

Example for tracking construction workers by cameras, proposed by [19]

Full size image

2.9 Material Classification

Automatically monitoring construction sites using site image collections necessitates the recognition of semantic information such as construction materials [21, 22]. A valuable attempt for material classification has been presented by Dimitrov and Golparvar [21], where a vision-based material classification system based on single images captured under unknown site lighting conditions and viewpoint was proposed as shown in Fig. 4 [21]. A database of 20 common building materials with more than 3000 images (about 150 images per category) was constructed and utilized for validation with an accuracy of (97.1%) was recorded [21].

Fig. 4
A scanned photograph. It illustrates 7 different types of construction material. They are wood shingles, asphalt, grass, rough brick, and compact soil.

Example for construction material classification and labelling proposed by [21]

Full size image

Another study by Brilakis et al., [23] decomposed an image into colour, texture, and structure features using a series of content-based filters. Where an image was divided into cluster regions, using databases that were built and utilized to compare the feature signature of each cluster to develop material image classification. Also, Zuh and Brilakis [22] suggested machine learning algorithms for recognizing concrete material regions. To begin, the construction site image was segmented into regions using segmentation. After that, Artificial Neural Network (ANN) and Support Vector Machine (SVM) were used to classify visual characteristics based on texture and color. Experiment results showed that ANN performed better than SVM, with an average precision and recall of about 80%.

The following table 1 represents some valuable research for monitoring and recognizing some activities in construction sites, in addition to the adopted tools and algorithms in each study.

Table 1 Examples for construction activities monitoring using image processing and CV-based technologies
Full size table

3 Challenges and Opportunities for Adopting Image Processing in Construction

3.1 Challenges

Although the great development in capturing devices, high storage databases, and easy access to the Internet support image processing in construction, several challenges reduce the effectiveness and accuracy of IP applications in construction [5, 13, 26]. These challenges can be divided into two major groups: those related to the adopted device and those related to the construction jobsite. The device challenges are mainly related to specifications of the captured device, distance of the device to the object (which change the scale of the tracked objects), angle of capturing (which change the scale of the tracked objects), images overlapping, device calibration, and image resolution [3, 5, 6]. While the challenge of construction job sites is related to the dynamic environment of the construction sites, occlusions, crowded job sites, lighting conditions, and weather [5, 26].

3.2 Opportunities

Despite researchers’ efforts, it has been observed that current studies did not encompass all complex construction sites and were based on a small sample of construction worker activities and machines separately [3, 5, 6]. To maximize the value of IP, future research may focus on monitoring the entire construction site, which would entail several people and machines operating at the same time [3, 5, 6]. It is also vital to develop the applications of IP and CV-based technologies through workshops, training, and conferences to keep current on knowledge and monitor technological trends, particularly in the field of IP and CV technologies. In addition to that, integrating IP, CV, and BIM together will increase the chance for more development of IP applications in the construction industry, and this will have a valuable and great impact on the development of construction and improve the chance of successful project completion [3, 5, 6].

Figure 5 depicts the conceptual framework in which the use of image processing in conjunction with CV and BIM will increase the efficiency of collecting and monitoring various elements of construction sites. This will result in increased construction productivity, improved safety monitoring, and effective decision-making. If these elements are improved in construction, it will lead to fewer errors, lower costs, taking timely corrective actions, and, ultimately, successful construction projects.

Fig. 5
A flowchart of the conceptual framework. It begins with image processing. It is followed by C V based technologies, progress monitoring, material classification, effective construction monitoring. It ends with effective decisions and successfull project completions.

Conceptual framework

Full size image

4 Conclusions

The research focuses on several areas where IP may be applied in construction projects, in addition to challenges that IP adoption faces and opportunities for more improvement in the construction sector. The use of IP in construction raises the automation monitoring for several aspects of construction projects, such as progress monitoring, resource tracking, material classification, productivity analysis, and structural element recognition. which results in reducing and eliminating the limitations of traditional monitoring and collecting methods, enhancing the decision-making process, reducing errors, and improving timely corrective actions. All these items increase the chances of a successful project's completion.

Although the great development in capturing devices, high storage databases, and easy access to the Internet support IP in construction, there are several challenges that reduce the effectiveness and accuracy of image processing applications in construction. These challenges can be divided into two major groups: those related to the adopted device and those related to the construction jobsite. The device (camera) challenges are mainly related to specifications of the captured device, distance of the device to the object, angle of capturing, overlapping, device calibration, and image resolution. While the challenge of construction job sites is related to the dynamic environment of the construction sites, occlusions, crowded job sites, lighting conditions, and weather.