Keywords

1 Introduction

With modern advancements in computational methods, optics, and graphics computing, 3D scanning is rapidly becoming more prevalently adopted in society. High-density 3D scanning can be performed at rates of real time or faster, thus broadening the scope of applications to which these technologies can be applied. A structured light scanning the system projects different light patterns, or structures, and captures the light as it falls onto the scene. It then uses the information about how the patterns appear after being distorted by the scene to eventually recover the 3D geometry. The potential speed of data acquisition, noncontact nature, the availability of necessary hardware, and the high precision of measurement offered by modern 3D structured light scanning technologies are what make them highly adoptable into industries such as medicine, biology, manufacturing, security, communications, remote environment reconstruction, and consumer electronics. As the number of applications in which structured light techniques are employed increases, more interesting and challenging problems arise. It should be noted that there is not one 3D sensing technology that solves each issue and works as a general solution. The handbook by Zhang (1), which discusses many of the major 3D acquisition technologies, can be used to identify advantages and disadvantages of each approach. This article, however, focuses on discussing structured light scanning techniques. Structured light techniques have benefited greatly from recent advancements in digital technology. To achieve real time 3D data acquisition and reconstruction, much computational power is required, yet it can now be matched by today’s modern computers; even some modern tablet computers can be used for these purposes. Given this, it is clear that the barrier to entry for some applications to use these technologies is quite low. However, in the past; if a manufacturing operation, for example, wanted to use these techniques, it may have been quite difficult to obtain the minimally required hardware, let alone to deploy the actual system itself. Modern advancements have ensured that the required hardware is relatively easily available. Software techniques required to reconstruct 3D data have improved greatly as well and these will also be discussed. To be useful for a wide range of applications, a system that can capture and reconstruct 3D information in real time (online), instead of retroactively (offline), is necessary. Real time in this context typically refers to acquiring, processing, and visualizing the captured 3D data at speeds of at least 24 Hz (2). As mentioned earlier, advancements in hardware have made this a possibility, yet it is not a mere trivial task; many problems had to be solved before the technology reached this point.

As mentioned earlier, structured light scanning is only one method of 3D data acquisition. Within the area of structured light scanning, there are many different techniques, as well. One method stands out that has shown many advantages and is derived from laser interferometers: digital fringe projection (DFP). The DFP technique involves varying sinusoidal patterns; using these methods, speeds of up to 120 Hz have been realized (3). However, the DFP approach has limits when it comes to its actual implementation, due to the sinusoidal pattern itself.

Eight-bit grayscale images are required to display the sinusoidal patterns, yet modern projectors can only display 8-bit patterns at certain rates. Due to these speed limitations, among other reasons (such as nonlinearity calibration and correction), other methods have been developed. These other methods allow for increased speed while limiting the increase of system’s complexity. Such improvements to DFP include the squared binary defocusing technique (4). Whereas speed barriers were approached on today’s digital image projection units with 8-bit images, 1-bit structured images can be projected much faster. By displaying 1-bit patterns through a defocused projector, the bits are blended together to reproduce a natural sinusoidal pattern. Another benefit, other than speed, of the binary defocusing technique is its lack of additional complexity due to calibration. As only two intensity values are being projected, nonlinearity calibration can be ignored. The smaller data transfer rate, increased rate of projection, and lack of complexity make this technique very advantageous and 3D shape measurement speeds greater than 120 Hz can be achieved. Using binary defocusing and digital light processing (DLP) platforms, Zhang et al. have been able to successfully develop a system that performed 3D shape measurement at tens of kHz (5). This article aims to provide a review of the different principles used in structured light scanning technologies, 3D data acquisition, and, by extension, a summary of its many different fields of application. By no means does this article position itself as a comprehensive body of work covering all structured light technologies and their finer details; several other surveys have been written regarding structured light techniques that may be useful, as well (6, 7). It should be noted that the content relies on published work, either in journals or in conference proceedings, done by us based on our own experiments or by others in the field. Section 2 will cover different structured pattern encodings. Section 3 will detail the steps required to properly calibrate a structured light scanning system. Section 4 will discuss structured light scanning at real-time speeds including DLP technology, 3D data acquisition, processing, and visualization. Finally, this article concluded the discussion and future directions for 3D scanning, specifically, structured light scanning.

Recently, in rapid growth, vision computer technology has significantly assisted to reconstruct three-dimensional (3D) comminuted objects. Non-contact 3D scanning as one of the cutting technologies in computation methods has become more popular and widely adopted in industrial manufacturing, medicine, and security [1,2,3,4]. Different technologies for 3D model scanning, typically classified into active and passive techniques. Passive scanning methods refer to recording a series of photographs from various angles and finding the corresponding photos using photogrammetric. While active scanning methods reconstruct the surface object by projecting a laser beam or a light pattern to the object's surface. There is not one 3D scanning technology that could be used to solve each issue and works as a general solution. Among high levels of 3D scanning, the structured light scanning system attracted more attention due to the potential speed of data acquisition, non-contact, availability of hardware in the market, and precision of measurements. The main system of structured light scanning works by projecting different light patterns and capturing the light once it falls on the scene. Those capturing pieces of information that contain distortion light patterns are used to reconstruct 3D geometry [5]. The three-dimensional model generated by structured light techniques has certain difficulties regarding the complexity of the objects and applications. The structured-light approach for 3D reconstruction is proposed which is based on 3D triangulation of optical rays generated by a video projector and recorded by a high-resolution digital camera. The system is calibrated in one step by projecting a colored chessboard pattern on top of a targeted planar surface. Commonly, a 3D model is represented by detailed digital surface models, either in point clouds or in triangular meshes format. The rendering process employs to generate texture from real imagery [6]. The structure light projection techniques can be used as powerful methods to reconstruct three-dimensional shape objects. Several commercial structured light systems are available and the others are under research development regarding the improvement of scanning accuracy, fast speed scanning, and effectiveness and optimization of calibration issues. A recent study, reported by Nguyen H., et al. [7] proposed a hybrid structured light technique with the deep convolutional neural networks for a single-shot 3D shape reconstruction. Even though the measurement accuracy of this technique is inferior to the existing 3D imaging methods, it shows potential performance in simple and robust. Wissmann et al. [8] presented a high-speed and low-cost approach for structured light pattern sequence projection as well as assessed the error sources. Ye and Song [9] show a practical approach for the optimization of calibration parameters in a structured light system. The results show that the accuracy of the 3D reconstruction is improved by the proposed approach in comparison with the conventional calibration methods. Liu et al. [10] proposed a polynomial method for measurement fringes projection system with the results of parts geometry accurately constructed after the system calibration. The system is an optically based measurement system that can provide feedback for optimization in the manufacturing process. The objective of this study is to reconstruct the 3D shape object and to determine its deformation.

2 Materials and Methods

To reconstruct 3D shape objects, a structure light projection system is built by using a computer system with CPU i5-9300H/9th Gen RAM 8GB DDR4, a projector with a resolution of IN116xa, WXGA 1280 × 800, aspect ratio 16:9, and a camera with a resolution of SX520 HS 16.0 Megapixel 2592 × 1944 pixels. The distance between a testing object and the imaging system was about 1.3 m. The principles illustration of the structure light projection setup is shown in Fig. 1.

Fig. 1.
figure 1

Illustration of a structured light projection system

Full size image

Experimental tests of 3D scanning began with the installation and setup of the light projection system concerning the triangulation principle of structured light. The scanning parameters under control include the distance from a camera to an object (L, 130 mm), the distance between the projector and a camera (D, 193 mm), and angle (α, 22º), as shown in Fig. 2. To determine scanning accuracy that equipped into the instruments and captured images deviation as well, the calibration test was carried out by comparing the results of the imaging pixels to a standard chessboard pattern.

Fig. 2.
figure 2

Setup position of projector and camera structure light projection system

Full size image

For tests, a square plate (steel, aluminum, and rubber) is mounted in a rectangular frame and penetrated horizontally by a cylindrical punch. The plane deformation is observed in the structure light scanning system. A fringe pattern is projected on the plate surface and recorded by a high-resolution camera. The changing fringe positions on the plate surface during perforation are then processed by a computer to give topography information of plane deformation. For reconstruction of the 3D model shape object, a thin material test was periodically punch-loaded 1 mm in distance for each captured image. The pressure direction of loading was pointed perpendicular to the projection planes. The loading displacement was used as representative for controlling the amount of applied pressure on the surface of the object test. A projector is used to project a grayscale pattern to the surface object. The digital camera is used to capture object deformation in time function. Overall, the scanning system was recorded for any displacements on the object surface during the pressure loading and stored in digital images format using image processing software. Images filtering, enhancement, and thresholding process were addressed to improve the quality of captured images before reconstructing the 3D shape model. Scale Invariant Feature Transform (SIFT) algorithm was employed to match images of a 3D model shape object after deformation. 3D surface models of object testing presented in the form of a Red-Blue-Green (RGB) color histogram.

3 Results and Discussion

A structure light projection system consisting of a projector and a camera was built for the reconstruction of the 3D shape objects. Three kinds of materials sheets were addressed by punch loading for identifying the feasibility of the method, as shown in Fig. 3. The pressure load is absorbed by the thin-plate surface to form a contour displacement. A plate deformation was captured and measured to refer to a profilometric histogram. The average out-of-plane surface displacement was range from 0.5–30 mm for each 30–60 repetitions. The comparison results of the recording and reconstructing 3D plane surface punch load are presented in the graph. For initial conditions before punch loading, the grayscale patterns were projected onto the test object surface and recorded using a camera. The grayscale lines pattern in the display monitor looked like a straight line on the object surface, then creating concave profiles after punch load. The comparison results between displacements of the punch loading movement onto the surface objects (3.4158 mm) to deflection measurement on the surface plane with grayscale patterned (3.4206 mm). Different results between two measurements in the range of 0.5 mm–30 that resulted from the deflection ~2.5 mm, shown in Fig. 4. Typical images of the projected fringe pattern from the camera after the punching process are presented in 3D topography, shown in Figs. 5, 6, and 7. The punch shaping after load displacement was be seen in the softened materials.

Fig. 3.
figure 3

Reconstruction 3D shape of the punch load on the projected plate surface, (a) rubber sheet, (b) steel plate, and (c) aluminum plate. The arrow sign shows the punch loading region on the object testing surface.

Full size image

Based on the loaded reconstruction images from the cameras and the projector projection, the displacement has been measured with small errors. The out-of-plane displacement of the plate as the representative of axial distance during punch loading from the center has been plotted, It can be seen that the proposed system has potential feasibility not only to generate 3D shape objects but also offering the out-of-plane measurement displacement.

Fig. 4.
figure 4

Measurement of experimental punch displacement compared to simulations (polyfit)

Full size image
Fig. 5.
figure 5

3D topographic of shape reconstruction rubber sheet, (a) before (b) after punch loading

Full size image
Fig. 6.
figure 6

3D topographic of shape reconstruction steel plate, (a) before (b) after punch loading

Full size image
Fig. 7.
figure 7

3D topographic of shape model aluminum plate, (a) before (b) after punch loading

Full size image

4 Summary

In this paper, a structure light projection was built based on a camera and projector for the reconstruction of a 3D shape object. The proposed system shows the feasibility as an alternative non-contact measurement to replace conventional methods. The out-of-plane displacement results include some error of ~2.5 mm for displacement range 0.5 mm–30 mm. Future research addressed to improve the scanning system performance refer to calibration accuracy, increasing the processing speed, and reducing the number of projection images.