Keywords

1 Introduction

In recent years, Additive Manufacturing (AM) has been developed in terms of design and functions. 3D printing of objects has increased rapidly and become more open to the public. Additive Manufacturing is simply the appellation for rapid prototyping, also called 3D printing [1]. The principle of AM process is to add material layer by layer. A layer symbolized a cross-section of the material obtained from a Computer-Aided Design (CAD) model. Earlier, AM was used to visualize products as they were being developed. Today, some end-use products for automobiles, aircraft, medical implants, dental restoration, and even fashion products are manufactured using AM techniques [2]. The combination of AM with robotics permits the manufacture of larger scale objects. A 6 DOF robot manipulator offers more flexibility and workspace compared to 3D printers. Robotics additive manufacturing process has the following advantages [3, 4]: (1) Product manufacturing time is significantly reduced; (2) Since parts are not cut from a larger block, material waste is nearly reduced to zero; (3) Complex large parts are now able to be manufactured; (4) Finally, the automatization of the process comes up with less supervision in the working environment.

AM integration in the industrial sector aims to create high-quality complex parts in a large number. However, a new set of problems arises. The geometry, substrate temperature, cooling rate, power, speed, and many other factors are identified to influence the deposition quality [5]. The “teach and playback” robots do not meet the requirements to ensure a stable deposition. The process is usually open-looped with process parameters fixed in advance. Thus, any unexpected change in the deposition environment results in a weld seam quality change and affects the final part accuracy. Therefore, the “teach and playback” method cannot automatically regulate the deposition process, leading to poor weld seam quality [6]. Indeed, this method is still widely used in industrial production. Today's objective is to replace “teach and playback” robots to automatically monitor the surrounding environment and correct real-time disturbances in the deposition process. Therefore, an autonomous system is essential to fill the existing shortages with this open-loop system. 3D printing is MIMO (Multiple-Input Multiple-Output), time-varying, nolinear, with multiple parameters affecting the deposition such as the velocity, wire feeding velocity, shielding gas, power or current, welding direction [7, 8]. A wrong combination or variation of these parameters leads to disturbances, thus induces defects in the produced part [9, 10], thus the motivation to have a sensor-based system [11]. The selected sensor shows a preview of the future deposited part. For instance, a pyrometer can only measure the temperature, whereas a camera provides a larger spectrum of deposition features. In fact, it is crucial to choose a sensor displaying the dynamics of the process and the weld pool characteristics. We believe that a vision-based system offers the requirements to extract weld pool information that can be used later for a real-time feedback correction system of the parameters to achieve better product accuracy.

The paper provides some key advantages of vision-based systems in AM and aims to inspire future research in vision-based monitoring in the AM industry. In Sect. 2, a review of vision-based is presented. A profile extraction is developed, and a methodology is proposed in Sect. 3; Sect. 4 concludes by summarizing this paper's findings.

2 Vision-based Sensing Techniques

In the literature, many works related to visual sensing to improve AM process have been reported. We begin this section by citing some of them before investigating the vision-based systems. The works treated in the literature include sensing features such as layer height, bead geometry, thermal history, etc.

In [9], Mazumder et al. investigated and analyzed independent parameters on the fabricated parts. Logic gates were used to combine the photodetector signals before a feedback system received them to control the deposit height. It is found that increasing the number of photodetectors in the feedback controller eliminated the directional dependence of the height controller on the material deposition, increased the material density slightly, and reduced the surfaces’ roughness by an average of 14–20%. However, the use of multiple sensors does not have major effects on the material properties of the fabricated parts that need to be considered to improve the deposited parts’ accuracy.

In [12], Bi et al. studied the influence of using different process control strategies on the dimensional accuracy, microstructure, and hardness of the final part. An infrared (IR)-temperature generated signal from the melt pool and is connected to a PID controller that regulates the laser power to control the melt pool temperature. Thin walls were deposited and compared under different processing conditions. It is found that with a constant set-value used in the process control, the quality of the deposited parts improves slightly. However, some discontinuities were still visible at the part’s edges due to low cooling time. Therefore, preset values must work into a certain threshold to compensate for melt pool variation.

In [13], Toyserkani et al. developed a feedback control system to enhance the deposition quality. A CCD camera provides pool images. A pattern recognition algorithm is used to extract the height (z) and liquid/solid interface dimensions and angle in real-time. The camera is connected to a knowledge-based PID controller that adjusts the laser pulse energy. The deposition performed using this approach showed an improvement in the accuracy and overcame disturbances around the operating point. However, this approach is limited to simple straight deposition, not to curvature or complex deposition processes.

In [14], Cheng et al. presented a closed-loop control model that measures the final part’s defects. A CCD camera with a Gaussian function is used as a shape descriptor and takes the surface layer’s physical characteristics. The measurements are then fed to different process dynamics models designed in SIMULINK to compute the next layer’s deposition flow rate compensation. The results show some accuracy in the reconstruction and detection of patterned surface shapes. However, the proposed approach reconstructs the 3D surface by using approximation and ignores other variations occurring in the deposition process.

Heralić et al. [15] developed a monitoring system to control the deposition process in real-time, mainly the bead height and width. Two cameras and a projected laser line are combined with an integrated PI controller and a feed-forward compensator for the bead width and the bead height. The control signals were the wire feed rate and laser power. Single-input, single-output (SISO) models represented the relationship between the bead geometry and the control signals. Single bead walls were deposited, and the results show good deposition stability using this approach. However, Heralić method’s is not fully automated as the start and stop sequences are manually performed. Also, automatic feedback control considering other environmental disturbances in the deposition process can enhance the approach’s robustness.

In [16], Doubenskaia et al. investigated the laser impact zone temperature during the deposition process. A bicolor pyrometer is used to measure the temperature and a CCD camera integrated with a PHENIX PM-100 machine to monitor the thermal radiation. Experiments were conducted under variable operational parameters (hatch distance, printing velocity, layer), and the result showed that the recorded signals are sensitive to the variation of the process parameters. However, as AM is complex, a pyrometer cannot interpret laser impact Zone as other environmental factors influence the pool surrounding.

Xiong et al. [17] developed a real-time measurement for the bead geometry (width and height). Two cameras to capture the images, a Gaussian filter to remove noises, a Sobel operator to detect the edge of the bead, and Hough transform algorithms for curve fitting composed the system. The goal was to monitor the nozzle to the top surface distance to eliminate the height errors during the deposition process. The validation tests showed the reliability of the vision-based online measurement. However, the main process parameters’ correlation needs to be considered to develop a robust feedback control system using this approach.

In [18], Ocylok et al. investigated the process parameters’ influence on the melt pool geometry. A camera is used to measure the melt pool’s length, area, and diameter at different process parameters. The values are compared with the cross-sections and the laser spot diameter. An analysis software performs the calculation for each of the images. The experimental results of Ocylok et al.’s work show that the camera provided accurate information of the weld seam and allowed the detection of variations of process parameters influencing the melt pool size. However, it is difficult to observe other process parameters’ real influence except for the power with this approach.

In [19], Grobert et al. developed a methodology to detect discontinuities in the deposition. A high-resolution digital single-lens reflex (DSLR) camera is used to collect multiple images at each build layer. Such linear Support Vector Machines (SVM) technique used a binary classification technique that detects flaws in the images. Two build structures (flaw and normal) were used to classify the process conditions. A computed tomography (CT) scan is used to label each layer. The images generated are permitted to identify fusion, porosity, and inclusions with automated analysis tools or manual inspection. Cross-validation experiments reveal that the camera-based system gives an accuracy greater than 80%. This approach can be more robust if it can detect near edges of parts and improve image resolution or with additional sensors to manufacture complex geometries.

Clijsters et al. [20] studied online monitoring and estimation of the quality of the deposition. The system comprises a high-speed near-infrared (NIR) thermal CMOS camera and a photodiode connected to a field-programmable gate array (FPGA) to transfer the images at high sample rates of at least 10 kHz to the control unit. The measured melt pool data are transferred on a 2D map grid for analysis and interpretation using a mapping algorithm. The experiments show that this model’s images display the melt pool variation, thus defects in real time. However, the detection resolution, speed, and other deposition parameters must be studied for a robust model.

In [21], Xiong et al. developed a vision-based real-time sensing that adjusts the wire feed speed in the next layer. A camera with composite filters is mounted to a robot connected to an external control unit. The images are processed with a Gaussian filter that removes the noise, performs edge detection, calculates the optimal threshold, and a Hough transformation algorithm for edge fitting. A PID controller adjusts the wire feed rate. Deposition of thin-walled allowed to validate the approach’s effectiveness. However, the wire feed rate only, as a controlling signal, is not enough to guarantee deposition stability because the other process parameters, for instance, the power, has a larger influence in the fabrication process.

Summarizing, the selection of a sensor system in 3D printing process has to overcome some of the deposition challenges by offering high-quality information to build a reliable monitoring process.

3 Case Study of a Molten Pool Profile Extraction

Real-time sensing of additive manufacturing processes remains a challenging topic in modern manufacturing technology. The first step is to place ourselves in a welder’s position to imitate and extract weld pool size (i.e., sensing of the printing process). Then, identify or develop a sensing device similar to human sensing systems. The sensor needs to detect some of the deposition process conditions and adapt to environmental variations. The sensing device should acquire the seam dynamics features and gives two-dimensional or three-dimensional information of the weld pool surface. Therefore, machine vision and extraction of weld seam geometric information are techniques that can be employed. A camera is convenient to capture the melt pool images of the process. CCD (Charge Coupled Device) is suitable for quality control of the 3D printing process [22], as it provides sufficient information of the weld seam such as the position of the wire, the shape of the pool as well as the shape of the solidified deposition, etc. Compared with other sensing devices, a camera is contactless and the signal detection does not affect the printing process.

After the images are taken, an image processing algorithm is required [23]. The image processing algorithms need to adapt to some changing conditions. Many existing and efficient algorithms exist, such as curve fitting, integral edge detection, image recovery, neural network edge identification, etc. The algorithms are used to extract some feature information from the weld seam [24]. In the context of LWAM, the shape of the molten pool can be the main feature to be extracted and apply a control rule. In summary, using an adapted camera, the tracked information in the weld pool can be obtained by computer image processing techniques and provides real-time status of the process.

A molten pool frame image is shown in Fig. 1.

Fig. 1
figure 1

A complete frame of a molten pool image

Full size image

3.1 Image Data Acquisition

Real-time monitoring of weld pool dynamics depends on the extracted information used to calculate the weld pool’s geometric features [25, 26]. In the image acquisition, a camera coupled with a robust image processing algorithm is crucial. Usually, the molten pool’s raw image cannot be directly exploited because of the nonlinearity and disturbances in the process. These two factors add difficulties in image acquisition and processing. Image processing aims to get the relevant information by using appropriate image features while suppressing undesired distortions. However, the relative motion between the image acquisition device and the weld pool significantly affects the information acquirement.

A flow chart of a molten pool image processing is shown in Fig. 2.

Fig. 2
figure 2

Flow chart of molten pool image processing during the deposition process

Full size image

The following steps describe an image extraction process of the molten pool. The algorithm was coded in Python and is composed of the following steps:

Image Filtering: First, we removed the noise in the images. During the image acquisition and quantification, interferences are present and cause some signal disturbances that lead to fuzzy images. A Gaussian and median filter [27] are combined to obtain a better image while preserving the weld pool edge. Also, considering the processing speed, a median filter gives the processed outputs faster.

Edge Detection: A Sobel filter [28] is then applied to create a frame for easy recognition of the melt pool profile. Essential features in the image are preserved and non-necessary characteristics removed. We noted a high reduction in the image’s size after this step.

Profile Extraction: Finally, a contour extraction is applied to reconstruct the image obtained from edge detection while keeping the pool’s shape precisely. Various feature extraction techniques, such as Fourier descriptor, extraction of occupancy, endpoints features, Zernike moments, etc., are proven to be efficient in more complex tasks [29].

The results of the image processing are shown in Fig. 3. The image size is 153 X 482 pixels, a standard deviation of 4 for the Gaussian filter, a Kernel size of 7 for the median filter and a contour level of 0.02 are used.

Fig. 3
figure 3

a Original, b Gaussian filter, c Median filter, d Edge detecting, e Profile extracting

Full size image

4 Experimental Set-up Approach

In robotics AM, regulating only the laser power or wire feeding velocity is not enough to obtain a stable deposition. The laser power and the wire feed rate are the most important parameters because the weld seam size increases with these parameters. An experimental set-up composed of a Laser power source, a wire feeder, a camera, a controller, and the robot is proposed in Fig. 4. The profile in Fig. 3. Can be used as a reference shape where any change in the shape profile or the width (w) or height (h) of the melt pool would generate an error. The relationship between the process errors and the molten pool's process parameters must be defined using a control rule, and finally, the control center will monitor the deposition process. The proposed set-up is shown in Fig. 4.

Fig. 4
figure 4

Structure diagram of the proposed experimental set-up

Full size image

5 Conclusion

The use of sensing technology in additive manufacturing processes has made significant progress. Many fields, such as image acquisition and processing methods, weld seam modeling, and process parameters control, are needed to develop online monitoring for AM. With the development of machine learning, researchers and engineers of AM subfields are urged to work together to create a real intelligent seam tracking of the 3D printing process to make the future of 3D printing robots brighter. This paper reviewed the literature on the use of a vision-based approach in the additive manufacturing process. A case study of the profile extraction process was also performed, and a sensing methodology to inspire future research was presented. The image processing results could be used to generate a graphical representation of the processed images and develop a control rule to monitor the deposition paths.