Abstract
Additive Manufacturing (AM) of metallic objects is the process of adding layer-upon-layer of material to produce a hard-compact physical object. During the deposition, many process parameters, such as thermal history, power, material feeding rate, microstructure, etc., are involved and need to be regulated. The soar of the 3D printing process in various sectors has recently added a set of challenges, creating a massive demand for an automated process monitor to ensure a good deposition. In this paper, a literature review of vision-based techniques in additive manufacturing process is presented. A molten pool profile extraction methodology is performed. The image processing results can be used to create a visual monitoring technique and inspire future research in the use of vision sensing in 3D printing process.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
In recent years, Additive Manufacturing (AM) has been developed in terms of design and functions. 3D printing of objects has increased rapidly and become more open to the public. Additive Manufacturing is simply the appellation for rapid prototyping, also called 3D printing [1]. The principle of AM process is to add material layer by layer. A layer symbolized a cross-section of the material obtained from a Computer-Aided Design (CAD) model. Earlier, AM was used to visualize products as they were being developed. Today, some end-use products for automobiles, aircraft, medical implants, dental restoration, and even fashion products are manufactured using AM techniques [2]. The combination of AM with robotics permits the manufacture of larger scale objects. A 6 DOF robot manipulator offers more flexibility and workspace compared to 3D printers. Robotics additive manufacturing process has the following advantages [3, 4]: (1) Product manufacturing time is significantly reduced; (2) Since parts are not cut from a larger block, material waste is nearly reduced to zero; (3) Complex large parts are now able to be manufactured; (4) Finally, the automatization of the process comes up with less supervision in the working environment.
AM integration in the industrial sector aims to create high-quality complex parts in a large number. However, a new set of problems arises. The geometry, substrate temperature, cooling rate, power, speed, and many other factors are identified to influence the deposition quality [5]. The “teach and playback” robots do not meet the requirements to ensure a stable deposition. The process is usually open-looped with process parameters fixed in advance. Thus, any unexpected change in the deposition environment results in a weld seam quality change and affects the final part accuracy. Therefore, the “teach and playback” method cannot automatically regulate the deposition process, leading to poor weld seam quality [6]. Indeed, this method is still widely used in industrial production. Today's objective is to replace “teach and playback” robots to automatically monitor the surrounding environment and correct real-time disturbances in the deposition process. Therefore, an autonomous system is essential to fill the existing shortages with this open-loop system. 3D printing is MIMO (Multiple-Input Multiple-Output), time-varying, nolinear, with multiple parameters affecting the deposition such as the velocity, wire feeding velocity, shielding gas, power or current, welding direction [7, 8]. A wrong combination or variation of these parameters leads to disturbances, thus induces defects in the produced part [9, 10], thus the motivation to have a sensor-based system [11]. The selected sensor shows a preview of the future deposited part. For instance, a pyrometer can only measure the temperature, whereas a camera provides a larger spectrum of deposition features. In fact, it is crucial to choose a sensor displaying the dynamics of the process and the weld pool characteristics. We believe that a vision-based system offers the requirements to extract weld pool information that can be used later for a real-time feedback correction system of the parameters to achieve better product accuracy.
The paper provides some key advantages of vision-based systems in AM and aims to inspire future research in vision-based monitoring in the AM industry. In Sect. 2, a review of vision-based is presented. A profile extraction is developed, and a methodology is proposed in Sect. 3; Sect. 4 concludes by summarizing this paper's findings.
2 Vision-based Sensing Techniques
In the literature, many works related to visual sensing to improve AM process have been reported. We begin this section by citing some of them before investigating the vision-based systems. The works treated in the literature include sensing features such as layer height, bead geometry, thermal history, etc.
In [9], Mazumder et al. investigated and analyzed independent parameters on the fabricated parts. Logic gates were used to combine the photodetector signals before a feedback system received them to control the deposit height. It is found that increasing the number of photodetectors in the feedback controller eliminated the directional dependence of the height controller on the material deposition, increased the material density slightly, and reduced the surfaces’ roughness by an average of 14–20%. However, the use of multiple sensors does not have major effects on the material properties of the fabricated parts that need to be considered to improve the deposited parts’ accuracy.
In [12], Bi et al. studied the influence of using different process control strategies on the dimensional accuracy, microstructure, and hardness of the final part. An infrared (IR)-temperature generated signal from the melt pool and is connected to a PID controller that regulates the laser power to control the melt pool temperature. Thin walls were deposited and compared under different processing conditions. It is found that with a constant set-value used in the process control, the quality of the deposited parts improves slightly. However, some discontinuities were still visible at the part’s edges due to low cooling time. Therefore, preset values must work into a certain threshold to compensate for melt pool variation.
In [13], Toyserkani et al. developed a feedback control system to enhance the deposition quality. A CCD camera provides pool images. A pattern recognition algorithm is used to extract the height (z) and liquid/solid interface dimensions and angle in real-time. The camera is connected to a knowledge-based PID controller that adjusts the laser pulse energy. The deposition performed using this approach showed an improvement in the accuracy and overcame disturbances around the operating point. However, this approach is limited to simple straight deposition, not to curvature or complex deposition processes.
In [14], Cheng et al. presented a closed-loop control model that measures the final part’s defects. A CCD camera with a Gaussian function is used as a shape descriptor and takes the surface layer’s physical characteristics. The measurements are then fed to different process dynamics models designed in SIMULINK to compute the next layer’s deposition flow rate compensation. The results show some accuracy in the reconstruction and detection of patterned surface shapes. However, the proposed approach reconstructs the 3D surface by using approximation and ignores other variations occurring in the deposition process.
Heralić et al. [15] developed a monitoring system to control the deposition process in real-time, mainly the bead height and width. Two cameras and a projected laser line are combined with an integrated PI controller and a feed-forward compensator for the bead width and the bead height. The control signals were the wire feed rate and laser power. Single-input, single-output (SISO) models represented the relationship between the bead geometry and the control signals. Single bead walls were deposited, and the results show good deposition stability using this approach. However, Heralić method’s is not fully automated as the start and stop sequences are manually performed. Also, automatic feedback control considering other environmental disturbances in the deposition process can enhance the approach’s robustness.
In [16], Doubenskaia et al. investigated the laser impact zone temperature during the deposition process. A bicolor pyrometer is used to measure the temperature and a CCD camera integrated with a PHENIX PM-100 machine to monitor the thermal radiation. Experiments were conducted under variable operational parameters (hatch distance, printing velocity, layer), and the result showed that the recorded signals are sensitive to the variation of the process parameters. However, as AM is complex, a pyrometer cannot interpret laser impact Zone as other environmental factors influence the pool surrounding.
Xiong et al. [17] developed a real-time measurement for the bead geometry (width and height). Two cameras to capture the images, a Gaussian filter to remove noises, a Sobel operator to detect the edge of the bead, and Hough transform algorithms for curve fitting composed the system. The goal was to monitor the nozzle to the top surface distance to eliminate the height errors during the deposition process. The validation tests showed the reliability of the vision-based online measurement. However, the main process parameters’ correlation needs to be considered to develop a robust feedback control system using this approach.
In [18], Ocylok et al. investigated the process parameters’ influence on the melt pool geometry. A camera is used to measure the melt pool’s length, area, and diameter at different process parameters. The values are compared with the cross-sections and the laser spot diameter. An analysis software performs the calculation for each of the images. The experimental results of Ocylok et al.’s work show that the camera provided accurate information of the weld seam and allowed the detection of variations of process parameters influencing the melt pool size. However, it is difficult to observe other process parameters’ real influence except for the power with this approach.
In [19], Grobert et al. developed a methodology to detect discontinuities in the deposition. A high-resolution digital single-lens reflex (DSLR) camera is used to collect multiple images at each build layer. Such linear Support Vector Machines (SVM) technique used a binary classification technique that detects flaws in the images. Two build structures (flaw and normal) were used to classify the process conditions. A computed tomography (CT) scan is used to label each layer. The images generated are permitted to identify fusion, porosity, and inclusions with automated analysis tools or manual inspection. Cross-validation experiments reveal that the camera-based system gives an accuracy greater than 80%. This approach can be more robust if it can detect near edges of parts and improve image resolution or with additional sensors to manufacture complex geometries.
Clijsters et al. [20] studied online monitoring and estimation of the quality of the deposition. The system comprises a high-speed near-infrared (NIR) thermal CMOS camera and a photodiode connected to a field-programmable gate array (FPGA) to transfer the images at high sample rates of at least 10 kHz to the control unit. The measured melt pool data are transferred on a 2D map grid for analysis and interpretation using a mapping algorithm. The experiments show that this model’s images display the melt pool variation, thus defects in real time. However, the detection resolution, speed, and other deposition parameters must be studied for a robust model.
In [21], Xiong et al. developed a vision-based real-time sensing that adjusts the wire feed speed in the next layer. A camera with composite filters is mounted to a robot connected to an external control unit. The images are processed with a Gaussian filter that removes the noise, performs edge detection, calculates the optimal threshold, and a Hough transformation algorithm for edge fitting. A PID controller adjusts the wire feed rate. Deposition of thin-walled allowed to validate the approach’s effectiveness. However, the wire feed rate only, as a controlling signal, is not enough to guarantee deposition stability because the other process parameters, for instance, the power, has a larger influence in the fabrication process.
Summarizing, the selection of a sensor system in 3D printing process has to overcome some of the deposition challenges by offering high-quality information to build a reliable monitoring process.
3 Case Study of a Molten Pool Profile Extraction
Real-time sensing of additive manufacturing processes remains a challenging topic in modern manufacturing technology. The first step is to place ourselves in a welder’s position to imitate and extract weld pool size (i.e., sensing of the printing process). Then, identify or develop a sensing device similar to human sensing systems. The sensor needs to detect some of the deposition process conditions and adapt to environmental variations. The sensing device should acquire the seam dynamics features and gives two-dimensional or three-dimensional information of the weld pool surface. Therefore, machine vision and extraction of weld seam geometric information are techniques that can be employed. A camera is convenient to capture the melt pool images of the process. CCD (Charge Coupled Device) is suitable for quality control of the 3D printing process [22], as it provides sufficient information of the weld seam such as the position of the wire, the shape of the pool as well as the shape of the solidified deposition, etc. Compared with other sensing devices, a camera is contactless and the signal detection does not affect the printing process.
After the images are taken, an image processing algorithm is required [23]. The image processing algorithms need to adapt to some changing conditions. Many existing and efficient algorithms exist, such as curve fitting, integral edge detection, image recovery, neural network edge identification, etc. The algorithms are used to extract some feature information from the weld seam [24]. In the context of LWAM, the shape of the molten pool can be the main feature to be extracted and apply a control rule. In summary, using an adapted camera, the tracked information in the weld pool can be obtained by computer image processing techniques and provides real-time status of the process.
A molten pool frame image is shown in Fig. 1.
3.1 Image Data Acquisition
Real-time monitoring of weld pool dynamics depends on the extracted information used to calculate the weld pool’s geometric features [25, 26]. In the image acquisition, a camera coupled with a robust image processing algorithm is crucial. Usually, the molten pool’s raw image cannot be directly exploited because of the nonlinearity and disturbances in the process. These two factors add difficulties in image acquisition and processing. Image processing aims to get the relevant information by using appropriate image features while suppressing undesired distortions. However, the relative motion between the image acquisition device and the weld pool significantly affects the information acquirement.
A flow chart of a molten pool image processing is shown in Fig. 2.
The following steps describe an image extraction process of the molten pool. The algorithm was coded in Python and is composed of the following steps:
Image Filtering: First, we removed the noise in the images. During the image acquisition and quantification, interferences are present and cause some signal disturbances that lead to fuzzy images. A Gaussian and median filter [27] are combined to obtain a better image while preserving the weld pool edge. Also, considering the processing speed, a median filter gives the processed outputs faster.
Edge Detection: A Sobel filter [28] is then applied to create a frame for easy recognition of the melt pool profile. Essential features in the image are preserved and non-necessary characteristics removed. We noted a high reduction in the image’s size after this step.
Profile Extraction: Finally, a contour extraction is applied to reconstruct the image obtained from edge detection while keeping the pool’s shape precisely. Various feature extraction techniques, such as Fourier descriptor, extraction of occupancy, endpoints features, Zernike moments, etc., are proven to be efficient in more complex tasks [29].
The results of the image processing are shown in Fig. 3. The image size is 153 X 482 pixels, a standard deviation of 4 for the Gaussian filter, a Kernel size of 7 for the median filter and a contour level of 0.02 are used.
4 Experimental Set-up Approach
In robotics AM, regulating only the laser power or wire feeding velocity is not enough to obtain a stable deposition. The laser power and the wire feed rate are the most important parameters because the weld seam size increases with these parameters. An experimental set-up composed of a Laser power source, a wire feeder, a camera, a controller, and the robot is proposed in Fig. 4. The profile in Fig. 3. Can be used as a reference shape where any change in the shape profile or the width (w) or height (h) of the melt pool would generate an error. The relationship between the process errors and the molten pool's process parameters must be defined using a control rule, and finally, the control center will monitor the deposition process. The proposed set-up is shown in Fig. 4.
5 Conclusion
The use of sensing technology in additive manufacturing processes has made significant progress. Many fields, such as image acquisition and processing methods, weld seam modeling, and process parameters control, are needed to develop online monitoring for AM. With the development of machine learning, researchers and engineers of AM subfields are urged to work together to create a real intelligent seam tracking of the 3D printing process to make the future of 3D printing robots brighter. This paper reviewed the literature on the use of a vision-based approach in the additive manufacturing process. A case study of the profile extraction process was also performed, and a sensing methodology to inspire future research was presented. The image processing results could be used to generate a graphical representation of the processed images and develop a control rule to monitor the deposition paths.
References
Gibson I, Rosen D, Stucker B (2010) Additive manufacturing technologies, vol 238. Springer
Yakout M, Elbestawi M, Veldhuis SC (2018) A review of metal additive manufacturing technologie. Solid State Phenom 278:1–14
Evjemo LD, Moe S, Gravdahl JT, Dubonnet OR, Gellein LT, Br⊘tan V (2017) Additive manufacturing by robot manipulator: an overview of the state-of-the-art and proof-of-concept results. In: 22nd IEEE ETFA
Fernando R (1998) 3D printing with metals. Comput Control Eng
Khorasani AM, Gibson I, Goldberg M, Littlefair G (2016) A survey on mechanisms and critical parameters on solidification of selective laser melting during fabrication of Ti-6Al-4V prosthetic acetabular cup. Mater Des 103(5):348–355
Chen SB, Wu J Real-time control of weld pool dynamics during robotic GTAW. IntellIzed Methodol Arc Weld Dyn Process 221–273
Regan P, Prickett P, Setchi R, Hankins G, Jones N (2016) Metal based additive layer manufacturing: variations, correlations and process control. Procedia Comput Sci 96:216–224
Müller J, Grabowski M, Müller C, Hensel, Unglaub J, Thiele K, Kloft H, Dilger K (2019) Design and parameter identification of wire and arc additively manufactured (WAAM) steel bars for use in construction. June 2019 Metals—Open Access Metall J 9(7):725
Mazumder J, Dutta D, Kikuchi N, Ghosh A (2000) Closed loop direct metal deposition: art to part. Opt Laser Eng 34:397–414
Marrey M, Malekipour E, El-Mounayri H, Faierson EJ (2019) A Framework for optimizing process parameters in Powder Bed Fusion (PBF) process using Artificial Neural Network (ANN). Procedia Manuf 34:505–515
Khosravani MR, Reinicke T (2020) 3D-printed sensors: current progress and future challenges. Sens Actuators A: Phys 305:111916, 15 April 2020
Bi G, Gasser A, Wissenbach K, Drenker A, Poprawe R (2006) Characterization of the process control for the direct laser metallic powder deposition. Surf Coat Technol 201(6):2676–2683
Toyserkani E, Khajepour A (2006) A mechatronics approach to laser powder deposition process. Mechatronics 16(10):631–641
Cheng Y, Jafari M (2008) Vision-based online process control in manufacturing applications. IEEE Trans Autom Sci Eng 5(1):140–153
Heralić A, Christiansson AK, Ottosson M, Lennartson B (2010) Increased stability in laser metal wire deposition through feedback from optical measurements. Opt Lasers Eng 48(4):478–485
Doubenskaia M, Pavlov M, Grigoriev S, Tikhonova E, Smurov I (2012) Comprehensive optical monitoring of selective laser melting. Publ 2012 Phys J Laser Micro Nanoeng
Xiong J, Zhang G (2013) Online measurement of bead geometry in GMAW-based additive manufacturing using passive vision. 2013 Meas Sci Technol 24:115103
Ocylok S, Alexeev E, Mann S, Weisheit A, Wissenbach K, Kelbassa I (2014) Correlations of melt pool geometry and process parameters during laser metal deposition by coaxial process monitoring. Phys Procedia 56:228–238
Gobert C, Reutzel EW, Petrich J, Nassar AR, Phoha S (2018) Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. Addit Manuf 21:517–528
Clijsters S, Craeghs T, Buls S, Kempen K, Kruth JP (2014) In situ quality control of the selective laser melting process using a high-speed, real-time melt pool monitoring system. Publ 2014 Int J Adv Manuf Technol Vol
Xiong J, Liu G, Pi Y (2019) Increasing stability in robotic GTA-based additive manufacturing through optical measurement and feedback control. Robot Comput-Integr Manuf 59:385–393
Pan Y https://www.universal-robots.com/blog/simplify-robot-programming-with-g-code. [Online accessed: 27–08–20]
Nandi C, Caspi A, Grossman D, Tatlock Z (2017) Programming language tools and techniques for 3D printing. SNAPL 2017
Yana X, Gu P (1996) A review of rapid prototyping technologies and systems. Comput Aided Des 28(4):307–318
Chen SB, Wu J (2008)Intelligentized methodology for arc welding dynamical processes visual information acquiring, knowledge modeling and intelligent control. Springer, p. 72
Zhang W, Liu Y, Wang YX, Zhang Y (2012) Characterization of three-dimensional weld pool surface in gas tungsten arc welding. July 2012 Weld J 91(7)
Kumar A, Sodhi SS (2020) Comparative analysis of gaussian filter, median filter and denoise autoenocoder. In: Published in: 2020 7th international conference on computing for sustainable global development
Ganesan P, Sajiv G (2017) A comprehensive study of edge detection for image processing applications. In: Published in: 2017 international conference on innovations in information, embedded and communication systems (ICIIECS)
Kumar G, Bhatia PK (2014) A detailed review of feature extraction in image processing systems. In: February 2014 IEEE fourth international conference on advanced computing & communication technologies
Acknowledgments
This work was supported by the Interrreg V-A Grande Région “Fabrication Additive par Dépôt de Fil” (Fafil) project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Mbodj, N.G., Plapper, P. (2023). Review and Methodology on Vision-based Sensing Approach in Metal Additive Manufacturing Process. In: Conte, G., Sename, O. (eds) Proceedings of 10th International Conference on Mechatronics and Control Engineering . ICMCE 2021. Lecture Notes in Mechanical Engineering. Springer, Singapore. https://doi.org/10.1007/978-981-19-1540-6_3
Download citation
DOI: https://doi.org/10.1007/978-981-19-1540-6_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-1539-0
Online ISBN: 978-981-19-1540-6
eBook Packages: EngineeringEngineering (R0)