Introduction

A construction site is an engineering phenomenon where many mechanistic workflows interoperate in a predicted manner (Beardsworth et al. 1988). The predicted behavior is the result of the model-based view that engineers use when they design buildings and infrastructure. The model-based engineering approach in Architecture, Engineering, Construction, Operation and Ownership (AECOO) ranges from modeling labor productivity (Thomas et al. 1990), to model-based design and engineering (Rebolj et al. 2008), to information-based modeling (Suermann et al. 2009), and to critical thoughts about the building information modeling (BIM) approach (Turk 2016). Information-based modeling allows for more complex construction projects, therefore the risk of them not progressing on schedule has become a big concern. Monitoring complex transdisciplinary activities on the construction site is a pressing problem that challenges practitioners and researchers to address change management, as it affects people and their customs, and practices and achieve shorter project duration and minimum project costs (Omar and Nehdi 2016).

Unplanned events during the construction phase may lead to undesired consequences, like safety risks (Keng and Razak 2014), and risks of delays and schedule overruns (Arashpour et al. 2015). To minimize the disruptive effects of the events, continuous monitoring of all scheduled activities on the construction site must be ensured. The monitoring process involves manual data collection obtained by direct human observation and/or automated data collection, i.e., measurements (Paolo Rocchi 2016), which result in large datasets.

Many construction projects lose the benefits of carefully designed construction schedules, because they ignore precise monitoring. This is a paradox, because the software tools allow for a complex construction schedule design (time overlap of activities, determination of critical paths, analysis and simulation of the schedule), which, on the other hand, makes it difficult to effectively control while using traditional approaches (direct human observation, reporting) due to the large amount of activities and information. Therefore, construction projects used measuring of the indirect indicators of the construction progress, such as recording workers’ productivity on the construction site with the use of various devices, e.g., access registration, monitoring material delivery to the construction site, cost accounting, monitoring activities with additional staff (interviewers), etc. In the last ten years, improved smart sensors and digital cameras have enabled automated onsite measurements, data acquisition, and image capture (Podbreznik and Rebolj 2005; Kim et al. 2013b).

The constant activity of people, computers, machines, sensors, and any other data-generating devices or agents produce and store an enormous amount of information that needs careful and extensive analysis to become useful. Nowadays, engineering processes, such as building construction, have become sources of the big data (Raguseo 2018). The data are mainly unstructured, such as documents, spreadsheets, digital pictures and videos, acquired signals and measurements, and have to be fed back into BIM systems to steer the project management as required. Actually, BIM can be seen as the main contributing big data technology (Eastman et al. 2011). With the advancement of BIM, data from the construction site can be used for the comparison and validation of the onsite situation with the referential digital model (BIM).

The digital link between the building geometry and the project schedule is formed via a BIM 4D model, which represents a temporal synchronization between the product (building) and a process model (tasks at construction site, (Chau et al. 2004; Rebolj et al. 2008)). This concept has been developed to aid effective construction management (Kim et al. 2013a). Project planning, construction progress monitoring and decision-making are all supported by the BIM 4D model. However, development and the use of BIM 4D models do not solve the problem of manual data gathering and time-consuming schedule updating procedures. Creating a detailed BIM 4D model for a project is a very demanding and time-consuming task (Meža et al. 2014). The same stands for keeping the model up to date, which is also a manual and labor-intensive task (Podbreznik and Rebolj 2005; Kim et al. 2013a). Despite these problems, it is necessary to keep the model up to date to use it for the purpose of project progress tracking and as a basis for decision-making (Navon and Sacks 2007). From the project perspective, accurate construction progress measurement is critical for the success of a building project (Akhavian and Behzadan 2012; Kim et al. 2013b).

As the project progress tracking and oversight are so crucial, there is a clear motivation for the automation of onsite information gathering as well as the automation of BIM 4D model updates. Hence, several technologies have been developed for automated data collection. These technologies include laser scanning of construction sites (Bosché 2010; Akula et al. 2013; Zhang and Arditi 2013; Bosché et al. 2015; Wang et al. 2015), GPS-based location tracking (Jiang et al. 2015), the use of RFID technology, or bar codes (Navon and Sacks 2007), augmented reality (Wang et al. 2013), and the application of video cameras (Podbreznik and Potočnik 2010, 2013; Brilakis et al. 2011; Kim et al. 2013a; Golparvar-Fard et al. 2013; Yang et al. 2015).

When research is focused on accuracy, correctness and timely information delivery, the price of data collection becomes an important issue. The use of video cameras is one of the most cost-efficient approaches of data collection for construction (Brilakis et al. 2011; Rodriguez-Gonzalvez et al. 2014). Effective price performance can also be achieved using hybrid solutions where several heterogeneous data sources are combined into a system to attain a higher quality of collected information. Such systems allow low-cost equipment to be used without losing output quality (Kuipers et al. 2014).

Big visual data have significant potential for change management by comparing the as-built with as-planned by BIM (Han and Golparvar-Fard 2017). Collections of images can come from static fixed-location cameras and dynamic drone-borne, robot-borne, or man-worn photographic devices. Sophisticated computer-vision algorithms have been derived to register any kind of images to BIM-generated views to show the project progress, visualize possible deviations, and follow the scheduled activities. This elevates the project management to a higher-quality level by decreasing the chance of human mistakes due to the complexity and diversity of the problems. The interactivity of man and machine introduces a characteristic cybernetic loop which enhances the technical and technological processes and results considerably, but lacks environmental, social, and global components.

The introduction of the Man–Machine–Environment System Engineering (MMESE) paradigm opened up a new perspective and included environmental feedback regarding the design and management of engineering projects. MMESE primarily focuses on the relationship between man, machine and the environment, and looks for an optimum from safety, high efficiency and economy standpoints: “man” referring to working people as the subject in the workplace, e.g., operators, decision-makers; “machine” being the general name for any object controlled by man, e.g., tools, machinery, computers, systems and technologies; and the “environment” describing the specific working conditions under which man and machine interact, e.g., temperature, noise, vibration, hazardous issues, etc. (Long and Dhillon 2016). Although the proposed approach represents a step further in the area of Sustainability Science (Saito et al. 2017), it only focuses on the local project issues, such as incorporating the potential risks in local environments to the project decision-making process and monitoring those conditions that may, for example, endanger men, destroy equipment, or degrade the environment itself (Xiaoyan and Zhongpeng 2014).

On a global scale, however, the construction industry’s concern has leaned lately against improving the social, economic and environmental indicators of sustainability. Studies of Life Cycle Assessment (LCA) contribute to the optimization of these aspects, from the extraction of raw materials to the final disposal of waste building materials (Ortiz et al. 2009). LCA is based on the long-term monitoring of all activities on construction sites in particular. By analyzing the large datasets collected from various projects in different places and under different conditions, the main factors that jeopardize sustainability can be extracted and, consequently, they can be properly handled over time in subsequent projects elsewhere. This can only be achieved through a knowledge exchange in a participatory transdisciplinary approach (Westberg and Polk 2016).

This paper focuses on a transdisciplinary approach that orchestrates the domains of project management, building information modeling, building construction based on a shared building information model and construction schedule, and knowledge engineering facilitated through the new ICT framework. The framework upgrades the ideas of MMESE and was initially verified within a field experiment on a construction site where the domains’ big data (site images) were automatically collected and processed. The big data collected are limited to images of the externally visible building elements. Our approach includes applied research with science–practice relationships. In Sect. 2, a methodology and a new framework for sustainable management of big visual data is described, Sect. 3 deals with experiments and results, while Sect. 4 discusses the proposed approach, from the sustainability point of view in particular, and concludes the paper with recommendations for sustainable management of construction site big data.

Methodology for sustainable management of construction site big visual data

A quick reflection on today’s computer-aided technologies will show that individual IT approaches involved in the cybernetics man–machine paradigm cannot meet the requirements of sustainability. The MMESE context circumvents this deficiency by deploying a wider spectrum of computer technologies being connected and integrated into continuous development with respect to local environmental parameters. This propels a broad interdisciplinary interaction, even if just the technology and engineering areas are considered. Take, for example, information systems, databases, data mining, which on one hand manage today’s enormous amounts of data, and imaging technologies, pattern recognition, computer vision, computer graphics, computer communications, and self-learning systems, that, on the other hand, process the data and turn them into information.

A single construction site perspective is not enough for the introduction of a sustainable framework. The construction site, where designers, architects, investors, builders, electrical and mechanical engineers, informatics engineers and environmentalists meet, is a component in a much more complex system of living environment, social development and entrepreneurial engagement. They contribute to the framework that implies a sequence from the off-site BIM 4D model to the actual onsite construction works, where monitoring and consequential corrections of the construction works yield big datasets. Over time, a wide range of data is collected from several different construction sites and construction projects, including vital information on the success and effectiveness of the implementation (technical, financial, environmental, sustainability, etc.). This way, an increasingly more efficient knowledge base can grow, which enables the following projects to be launched more optimally and implemented more efficiently. Not only the experience of individual people counts, but also the digital knowledge assets provided by computers can be incorporated into the validation of decisions.

It is important to build on past knowledge, which today’s computer technology can store successfully and use for better decision-making in the future. Specifically, we can talk about growing knowledge bases that contain the knowledge, performance and effects of construction projects, such as BIM models, actual situations and complementary information, which are crucial when aiming at sustainability that can be added over time, not only during construction, but also later during the use and maintenance of facilities.

Taking into account all of the important issues mentioned above, we derived a new sustainable framework based on big data that accompany the life cycles of engineering and construction projects. Experimental setups and protocols will be exemplified in the next section.

The core of the proposed framework performs data acquisition. The most important is the visual information captured in videos with lower resolution and time-lapse photography with higher resolution. Supplementary data are provided by different sensors that measure the local environment parameters on the construction site, such as temperature, humidity, wind speed, and noise. Videos and images need much more computer resources than sensory data when stored, transferred, and processed. It would be difficult to install suitable computer performance on a construction site for several reasons, mostly due to the lack of clean, air-conditioned rooms, protection, security, and difficult maintenance. At the same time, online data transferring with wide bandwidth links, e.g., optical cabling, is a rare commodity on construction sites. The solution we propose is dedicated computer hardware that integrates enough local storage to save acquired data during a required time interval, say one working day. The device is connected to a mobile network to upload data to a powerful network server. Even if the uplink is interrupted due to an inferior quality of communication signals, today’s performances can cover data transfers during less busy periods if a single data (visual and sensory) stream is sent through a single link.

Although low-cost high-resolution cameras, e.g., GoPro (Park et al. 2017) may be used in the place of the imaging device described, we decided differently for the reasons explained in the Discussion section. We designed and constructed a special recording unit (RU) whose software supports the following main properties and tasks (Zazula et al. 2013):

  • A general-purpose portable computer equipped with a large data repository, input facilities for analogue and digital video and image acquisition, smart sensor connections, and an output GSM module; it collects, stores locally, and uploads the data periodically from a construction site;

  • A photographic device adapted for streaming video and acquiring simultaneous time-lapse full-resolution images;

  • Hermetic housing with special cooling systems to protect the electronic equipment and guarantee operation even in aggravated circumstances.

Data from one or several RUs are transferred to and permanently stored on a network server. The server site is divided into three layers: application, application server, and data layer (Fig. 1). The server runs the applications that extract information, help decision-making, and support sustainable improvement of subsequent projects, all based on databases in the data layer.

Fig. 1
figure 1

Technical framework for management of construction site big visual data

Full size image

The RUs periodically push data to the site controller, which stores them in a data repository. A data processor pulls data from the data repository for transformation, validation and data cleansing before writing the data onto an intermediate data repository. A data reasoner leverages implicit knowledge from the intermediate data repository using a reasoner relying on a domain knowledge model.

We began testing the proposed approach with a development of modules based on computer-vision algorithms to properly visualize the scenes, recognize moving subjects and their trajectories, fuse several images of the same scene, reconstruct the as-built objects and compare them with the as-planned information in BIM. The next section introduces two construction sites where we verified the implementation of the framework illustrated by Fig. 1.

Experiments and results

Two field experiments were involved in the initial verification of the methodology proposed in Sect. 2. The first construction site of a lookout tower called Vinarium was remotely monitored throughout its building phase. Vinarium’s BIM model (Tibaut et al. 2015) was prepared in the design phase and a detail of it is outlined in the right side panel of Fig. 2. The BIM model resides in the knowledge base of the proposed framework (see Fig. 1). The corresponding situation on the construction site is depicted in the left side panel of Fig. 2. It is captured by a RU and is stored in the data repository of the proposed framework in Fig. 1. Figure 3 is also an example of an as-built project situation (left panel) and the corresponding as-planned BIM visualization (right panel) showing the Vinarium tower in its final form. The second construction site, the Medical faculty, was used to experiment with the recognition of people, which contributes to the sustainable collection of big visual data (see Fig. 6 below).

Fig. 2
figure 2

Construction works (construction site “Lookout tower Vinarium”, left) monitoring with the as-planned BIM model (right)

Full size image
Fig. 3
figure 3

The as-built lookout tower Vinarium (left) and corresponding as-planned BIM model (right)

Full size image

The Vinarium lookout tower experimental setup

The Vinarium tower was built as a reinforced concrete building with steelwork consisting of an assembly of structural steel columns.

A single RU was initially planned at a fixed distance from the construction site. We expected that manually increasing the elevation of the RU would be done during with the tower’s growth. But, this would have two drawbacks: (a) due to the RU’s viewing angle, the whole building would not be seen during the later construction phases; and (b) by only having one-view images of the construction site available, a 3D reconstruction of the building elements could only be inferred from comparisons with the BIM-induced information. Therefore, we installed two RUs at the same distance of approximately 30 m from the tower (a RU is depicted in Fig. 4). The elevation of the two RUs was different to cover the upper and lower part of the facility independently.

Fig. 4
figure 4

Recording unit as mounted at the construction site

Full size image

Each RU incorporated a Zoltac portable computer with a 1,8 GHz dual-core processor, 3 G bytes of memory, and the 32-bit Lubuntu 12.04 operating system. A Nikon COOLPIX L120 camera was controlled by a special hardware controller we developed to force constant low-resolution (640 × 480 pixels) video streams and simultaneous time-lapse high-resolution (4320 × 3240 pixels) images (Zazula et al. 2013). A GSM modem was connected to the computer to establish an uplink with a powerful data server.

The separate video streams and time-lapse still images were buffered locally on the RUs’ storage. RUs uploaded the recorded images via a GSM link to the site controller. When the line was too busy, or even broken, the data transfer was delayed and completed during more relaxed periods, such as overnight. The images were then stored in the raw image repository (Data Repository in Fig. 1). Low-resolution videos were tentatively processed by our tracking software, while pairs of time-lapse high-resolution still images were being taken in a synchronous way by the two RUs and were merged by a pre-processor on the server first (saved by the data processor to the intermediate data repository in the proposed framework, Fig. 1). The streams of merged images are accessible through a web application (monitoring, application layer) as depicted in Fig. 5. In our experiment, 52,852 pairs of time-lapse still images (170 GB of storage) were taken in regular time intervals of 5 min during the 199 project days. This was the input to our computer-vision processing algorithms as explained in the next subsection.

Fig. 5
figure 5

Web application for managing construction site big visual data

Full size image

Processing of onsite images

As two onsite RUs were used in our experiment, the first processing step was merging pairs of images uploaded to the Data Repository and storing the resulting image in the intermediate data repository, as denoted in Fig. 1. We applied the image stitching pipeline algorithm, which was implemented in the OpenCV (Open Source Computer Vision Library, https://opencv.org) and derives from the well-known algorithm for automatic panoramic image stitching using invariant features (Brown and Lowe 2007). The algorithm takes the two images from the two corresponding RUs at the same time and stitches them into a single resulting image with a given height (1500).

Videos and sequences of time-lapse images were further processed by computer-vision algorithms. They are divided in two groups: (a) the scanning of video streams extracts information on detected objects, their motion and structure; (b) the analysis of full-resolution time-lapse images enhances the 3D reconstruction of the objects. The algorithms belong to the data processor that reads images from the data repository to store the resulting outputs onto the intermediate data repository, as proposed by the framework in Fig. 1.

Video analysis at the Maribor Medical Faculty construction site

The resolution of the images in the video streams is kept fairly low due to a massive transfer that must be supported by the communication lines from construction sites to the data server. In our experiments, the VGA resolution was selected. In the starting phase, our video analysis took advantage of simple approaches to detect motion in images (Kolarič 2013). The problem of moving light sources or reflections may decrease motion detection considerably. Therefore, we pre-processed videos using homomorphic filters first to eliminate illumination changes (a filtered image is depicted in Fig. 6). Image intensity differences in blocks of subsequent images point out the regions of motion. Due to the input noise in images, the images of differences may also be very noisy. However, detection peaks belong to moving objects. We deployed Gaussian thresholding to construct the regions of moving objects, and they took part in the construction of trajectories for moving objects throughout the intervals of visibility in the videos.

Fig. 6
figure 6

An example of recognizing and masking the moving objects on a construction site, some of them being workers

Full size image

An additional task of object detection was the recognition of people. The regulations on public access to personal information forbid spreading video or image material containing humans without their formal consent. It would be very impractical to expect construction-site workers to sign formal consents concerning visual information obtained of them while present on a construction site. The way we solved this problem involves the fact that construction-site workers move a lot of the time. Indeed, there are other moving objects besides humans, but by masking all of them, we are able to hide human forms. An example from the construction site is given in Fig. 6.

Our simple approach to track the moving objects did not entirely fulfill the expectations on reliability and accuracy. The reasons and possible upgrades are discussed in the Discussion section. We compared the algorithm-based detections with human observations statistically. Sensitivity (Se) and precision (Pr) were computed as follows:

$$ {\text{Se}} = \frac{\text{TP}}{{{\text{TP}} + {\text{FN}}}} $$
$$ \Pr = \frac{\text{TP}}{{{\text{TP}} + {\text{FP}}}} $$

where TP stands for true positive, i.e., all correct detections, FN for false negative for all missed moving objects, and FP for false positive for all detections of non-moving objects (erroneous image regions). The overall sensitivity and precision yielded 0.89 and 0.61, respectively.

Time-lapse, high-resolution images of as-built for comparison with as-planned in BIM

The two RUs mounted on the Vinarium construction site covered the same parts of the scene from different angles. Multiple views obtained this way could support 3D scene reconstruction by well-known structure from motion algorithms (Ponce and Forsyth 2012). However, we tested a different approach that combines onsite images and the views on BIM-generated models from different camera poses, i.e.. its position and orientation.

The zoom of the cameras in our RUs are not allowed to change. This way, cameras can be calibrated in the lab environment before they are taken to the construction site. Also, radial distortions were eliminated. A further simplification is possible due to the fact that RUs with cameras are static and remain in the same place on the construction site throughout the building project. Thus, the pose of the cameras in regard to the construction scene does not change and can be determined once the RUs are mounted. Periodic verification of the obtained pose is beneficial anyway. When the cameras’ poses are known, their relationship with the BIM-generated models is based on a coarse-to-fine alignment of the as-built and as-planned views.

The initial coarse position and orientation of the RUs was obtained manually. Then, we placed the initial BIM-generated model, as obtained from the knowledge base of the proposed framework in Fig. 1, according to the established cameras’ positions and virtually projected its components onto the cameras’ planes. The coordinates of all of the visible corners were included in a cloud of model points. Another cloud of points stems from the corresponding real images as obtained from the intermediate data repository of the proposed framework in Fig. 1, characterizing the corner points, too. We detected those by the Harris corner detector. By selecting the corresponding points in both point clouds, a coarse initial alignment related to the real-world images to the BIM-generated models, and vice versa (Lin and Fang 2013). The information is saved in the intermediate data repository of the proposed framework in Fig. 1.

For the given images (Fig. 7), a finer matching with the model was achieved by edge detection and aligning the building components as described in (Čuš-Babič et al. 2018). These are ready in the intermediate data repository for the as-built, real-world images and in the knowledge base for the 4D BIM-planned model as proposed by the framework in Fig. 1. The ratio of overlapping edges of visible components depends on image quality and the success of the detection applied, but it primarily indicates the existence of as-built components in regard to the BIM-planned components (comparison done between the situations synchronized in time by the Data Reasoner of the proposed framework in Fig. 1). The overall overlap of the edges equal 66.7% in our case. By setting the threshold for an edge in as-built images to be recognized as planned at 22%, an overall recognition rate of as-planned components yielded 97%.

Fig. 7
figure 7

Construction site image edge detection overlapped with the transparent wire-frame geometry from the as-planned BIM model: the red edges are detected in as-built images, while the green edges belong to the as-planned BIM-related information, both synchronized in time

Full size image

We discuss how the described approach can be upgraded and how it can contribute to the sustainability in the next section.

Discussion and conclusions

From the broader systems approach perspective, our paper emphasizes the interdependence of different discipline fields embracing AECOO and human activities. We want to associate natural sciences with social, economic and humanistic influences. New innovative approaches in construction related to all the disciplines and fields of computer science, informatics, etc., are the focus of our research. Economic aspects are implicitly related, from the point of view of improving the quality of construction projects with optimized costs and long-term effects on local and global economies. It is less obvious that systems that collect and record all relevant big data from previous projects in a form that enables fast hardware browsing, connectivity and learning provide the basis for sociological studies and social progress for the benefit of an individual. Above all, it is important to emphasize the reduction of adverse environmental impacts and the environmental component of the technological framework system that we proposed. Human-friendly technical solutions, technologies and the environment are more humane and, in fact, direct individuals from a day-to-day struggle for survival to the principles of humanity.

The approach we followed in our research and development is based on big datasets collected by the RUs on various construction sites in various circumstances. As explained, the automated acquisition of images and environmental signals was achieved by the design of our own RUs. The initial experiments focused on imaging only, but the hardware and system software are ready to support a variety of additional sensors connected in construction-site piconets.

An explanation is also necessary concerning the choice of the cameras in RUs. Low-cost quality sports cameras are available, such as the GoPro (Park et al. 2017), which support fast video recording and time-lapse high-resolution imaging as well. Indeed, recorded data can be transferred to another device in close proximity, but the stand-alone operation is rather limited by the amount of data. This means that a more powerful computer must support the camera’s operation. Even if this support could be offered by today’s smartphones, the needed amount of local data storage in case of saturated uplinks exceeds the phones’ capabilities. A decision in favor of a portable computer seemed logical, and since we had a patented setup with a photographic device instead of a video camera, we deployed it in our experiments.

We owe additional clarification of the algorithm for tracking the moving objects. The one we applied is rather simple, which was dictated by our wish to run it in real time. It showed considerably low precision; therefore, a better solution was looked for. Fast and accurate object recognition software has been reported to be on smartphones lately, which has a remarkable recognition rate of various objects and scenes on single still images. The algorithm is based on deep-learning neural networks (Zhu et al. 2017). To train the network properly, a huge amount of annotated referential images of all possible objects must be available. Opposed to the training databases that have been built for everyday human surroundings, including persons, animals, vehicles, etc., no similar database on building-construction components and construction-site elements is available to the best of our knowledge. Nevertheless, our further research will involve the recognition and tracking of objects based on deep-learning approaches. This can lead to more reliable information on material flows, personal engagement and interaction, the utilization of machines, and activities that may cause environmental danger.

Once all project-related data are gathered into the intermediate data repository Fig. 1, the data reasoner compares it to the information obtained from the knowledge database, which includes BIM 4D models too. Not only are the discrepancies between the as-planned and the as-built detected, but subsequent decision-making is also based on the stored conditions for sustainable construction. Any technological and technical solution is critically evaluated according to sustainability. If the sustainability was jeopardized, a change of solution would be suggested.

In addition to that, our future research will adopt the Earned Value Management (EVM) approach (Project Management Institute 2013) that drives the systematic project management process by being based on the comparison of work performed and work planned based on the as-built visual big data and the BIM 4D from our system, respectively. Using the input data, the following EVM metrics (performance indicators) can be calculated at any given time in the project:

  • Budgeted cost for work scheduled (BCWS) is value of the work planned to be accomplished (planned value),

  • budgeted cost for work performed (BCWP) is value of the work accomplished (earned value),

  • actual cost of work performed (ACWP) is cost of the work accomplished (actual cost).

Our approach has impact on the economic, social and environmental sustainability factors for the broader construction site context. Contribution to economic sustainability starts with the consumption of the monitoring results from our system (Fig. 1), which gives automated input for calculation of the EVM status indicators:

  • % schedule = BCWS·100/total budget at completion,

  • % complete = BCWP·100/total budget at completion,

  • % spent = ACWP·100/total budget at completion.

If EVM metrics like cost efficiency (= BCWP/ACWP) and schedule efficiency (= BCWP/BCWS) have favorable positive values, this means that the estimate of total cost at completion meets the total allocated project budget.

Direct benefits of such agile construction project management are reduced management costs, project running costs, maintenance costs and better productivity. Indirect benefits are gained through improved satisfaction of clients (residents, tenants) and better image and reputation of construction industry, which enables repeat business. Faster response to unplanned events and improvement of client satisfaction has an economic impact, but also improve social and environmental performance.

Generally, social impacts are difficult to measure but they usually go along with economic benefits. Filtered images from the big visual dataset as collected with our system impact the social sustainability, because they can be used for public presentation of building progress with the aim to engage local people and to develop public trust. For a construction company, this can minimize trouble and maximize support during construction. Construction project disturbances cannot be always avoided and that is where communication to the surrounding community and businesses complemented with the up-to-date visual information from construction site can minimize negative impact. Use of the video analysis for identification of moving objects on construction site can anonymously identify movements of workers, equipment and supplies. As for the movement of workers, analysis of their movement trajectories can improve health and safety standards on the construction site and optimize workers’ efforts in the workplace. The right working environment is also the basis for good relationships, which maximizes satisfaction and productivity.

The approach and the system we propose also impact some of the environmental sustainability factors related to the environmental performance, which includes:

  • Design-phase considerations related to the embodied energy in materials and components for the building. Environmental evaluation data is obtained from the during the early construction project phases and is particularly accurate for the off-site construction of building component, which are then transported and assembled on the construction site. The information can then be stored in the BIM model (Antón and Díaz 2014). If available, this information can implicitly be accessible with our system through the available BIM model.

  • Air (dust and fumes) and noise pollution measurement on the construction site can be performed with algorithms used for dust detection and additional noise level sensors that can be integrated in RUs (Fig. 4). The data can then be compared with the thresholds for health-harmful pollution levels stored in the knowledge base (Fig. 1) and trigger air quality alert to the site manager.

  • Vehicle movements recorded by the RUs (Fig. 4) can be used to identify vehicles and relate them to the supply chain tasks modeled in the BIM 4D model (e.g., off-site constructed modular building components minimize waste on construction site).

  • Monitoring of waste emerging from the construction site could partly be performed with our system if additional detection algorithms would be developed for identification of waste sources. This upgrade would assist site manager in estimation of the volume of waste leaving the construction site to a certified disposal site.

The importance of properly analyzing big visual data obtained from construction sites has been shown by (Han and Golparvar-Fard 2017). When the deviations of the as-built from the as-planned are at least known on a daily basis, in many cases, practically in real time, the project management can decide promptly and with regard to the sustainability requirements. A variety of situations may appear, some supported by previously known solutions saved in a general knowledge base (according to the proposed framework in Fig. 1), some not yet met and have to be decided on the spot.

All decisions and their semantics upgrade the knowledge base. It becomes a source of valuable information for Sustainability Science after a project is finished. We believe that a lot of important information can be made public from such project databases, although disclosure restrictions are imposed by the participating companies. We are going to develop this idea along with the future construction project based on our RUs and supporting software.

The research will have an impact on the follow-up stages of construction projects coming after new construction, like refurbishment, operation and demolition, because there will be less deviation from the as-planned projects. Better construction project monitoring will allow for the realization of the planned sustainability criteria of the construction projects, like the sustainability assessment method, environmental assessment rating, etc.

As a beneficial side effect of our project, the Vinarium BIM model could also be used for other potentially interested groups—for instance, to create a digital experience for visitors.

Our research outcomes attracted the interest of both construction practitioners (i.e., investors, consultants) and transdisciplinary research domains (i.e., system integration, BIM, computer vision, big data management, knowledge management, construction scheduling, sustainability in building lifecycle). This undoubtedly calls for further applied research, and the participation of all those stakeholders is crucial for developing the idea of managing big visual data as presented here towards the construction projects entirely supported by the sustainability science.