1 Introduction

Additive manufacturing (AM) can produce a product directly from a three-dimensional (3D) model data with advantages on the capability of producing sophisticated and customizable components, reducing production time, and flexible use for various repairs and freeform fabrication [1, 2]. Wohlers Report 2019 stated that the income in AM field is expected to reach 15.8 billion dollars in 2019 [3]. As an important field, metal AM has been used widely in many areas, such as in aerospace, transportation, medical, and remanufacturing. The revenue from metal AM increased by approximately 41.9% and continued a 5-year streak with more than 40% growth annually [3]. Therefore, the society will pay increasing attention to metal AM for a long time.

AM has seven process categories, namely, binder jetting, directed energy deposition (DED), material extrusion, material jetting, powder bed fusion (PBD), sheet lamination, and vat photopolymerization [4]. DED and PBD are the two main methods used for processing metal materials. DED is an AM process that uses thermal energy to fuse materials by melting while being deposited [4]. PBD is an AM process, in which thermal energy fuses regions of a powder bed selectively [4]. DED technology is the main object reviewed in this paper. The energy sources of DED mainly include laser, electron beam, and arc plasma. Figure 1 shows the advantages and disadvantages of three DED methods [5]. This paper will describe the three methods as laser-based directed energy deposition (L-DED), electron-based directed energy deposition (E-DED), and arc-based directed energy deposition (A-DED). Two types of feedstocks, powders and wires, are mainly used in metal AM. Metal powders are usually applied in L-DED. Metal wires can be used in all types of DED that include laser beam, electron beam, gas metal arc, gas tungsten arc, and plasma arc as energy sources [6].

Fig. 1
figure 1

Advantages and disadvantages of DED with different energy sources [5]

Full size image

DED is an advanced technology used in 3D deposition forming, surface cladding, and part remanufacturing. The schematic is seen in Fig. 2. It has the advantages of multifunctional homogeneous or heterogeneous structures because of the layered manufacturing technology and the multiple-powder handling capability. Moreover, DED can produce metal parts with superior quality and strength because of its inherent rapid heating and cooling feature [10]. However, despite all these advantages, many technical challenges, including low repeatability and quality, difficulty in achieving the adaptive control, problematic and expensive post-process inspection, and accumulative error, continue to hamper the widespread adoption of DED and achieving its full potential [11].

Fig. 2
figure 2

Schematic of a L-DED [7], b E-DED [8], and c A-DED [9]

Full size image

To overcome the aforementioned challenges, IMAC technology has attracted increasing attention from researchers in this field. IMAC technology acquires various sensing signals by sensing equipment in situ. The sensing signals can be generated from the DED process itself and can also be produced from other devices, such as an X-ray radiation source and a structured light laser diode. Subsequently, these signals are processed, identified, and transformed into process signatures, which can indicate product qualities. Lastly, the process qualities are be controlled adaptively online or layer-to-layer based on the relationships among process parameters, process signatures, and product qualities. Online IMAC technology exhibits the monitoring and control behavior in the same layer. Layer-to-layer IMAC technology implements the control behavior in the other layers after exhibiting the monitoring behavior. The diagrammatic sketch of IMAC technology is shown in Fig. 3.

Fig. 3
figure 3

Relationships of the different parts in IMAC

Full size image

IMAC technology is beneficial to DED. First, some process signatures, including molten pool geometric characteristics and feedstock, temperature, and spectrum process signatures, are obtained through the studies of IMAC to indicate product qualities, such as geometric accuracy, defects, microstructure, and property. Therefore, if the process signatures can be monitored and controlled in real time, then the quality and repeatability of the products can be guaranteed. Second, the nondestructive test after the DED process cannot discover the problems in time during the processing. This phenomenon wastes time and material resources when defects or deviations emerge. Thus, process signature is an opportunity for the computer or the operating crew to correct the technological parameters or cancel long builds when the defects or deviations emerge. Third, some areas or structures, such as net-like ones, which are difficult to be post-process inspected, can be examined by IM technology during the processing. Finally, the monitoring data will be used in revealing the processing mechanisms and verifying theory and simulation models.

Recently, several reviews are devoted to IMAC for DED. Tapia and Elwany [12], Everton et al. [13], and Chua et al. [14] reviewed the IM of DED and summarized the sensing equipment and monitored objects that have been realized. Yan et al. [15] reviewed the thermal characteristics and parameter–thermal behavior–quality relationships during PDF and DED processes. Bandyopadhyay and Traxel [16] explained the existing monitoring methods that could be applied in modeling verifications. He et al. [17] mainly described the optical monitoring techniques and an emerging full-field deformation in laser-based DED. Aiming at the adaptive control of DED, Tapia and Elwany [12] and Shamsaei et al. [18] reviewed the studies that focused on control methods and objects of DED. Some beneficial reviews [19,20,21,22] for PBF that could be considered in DED were also available. Compared with the previous reviews of IMAC in DED, this paper refers to them suitably, combines the studies in recent years, and focuses on the three parts of IM technology, namely, sensing signals, process signatures, and monitorable product qualities. Therefore, this work mainly describes existing sensing signals and relevant devices, illustrates process signatures that can be acquired by these signals, and reviews ongoing studies about product qualities monitored by process signatures. Lastly, existing problems and future research will also be specified.

2 Commercial sensing, monitoring, and control systems

Many companies that produce AM devices have integrated IM modules or toolkits in DED devices. In addition, some companies focus on building a system or providing a solution that will improve product qualities through IMAC. The toolkit name, developer, monitored process signatures or objects, possible parameter that can be altered, and sensing equipment are shown in Table 1. An open-source toolkit was also mentioned in the literature [23].

Table 1 Commercial IMAC toolkits for DED
Full size table

3 Sensing signals and equipment

The interaction between the high-energy beam and metal material is a complex coupled process with physical metallurgy changes that lead to the processing region, such as molten pool in a heat, convection, and mass transfer state. Meanwhile, the process signatures during the DED process are directly correlated with product qualities. However, these process signatures are difficult to monitor and identify accurately because of a complex process environment. In this section, we provide an overview of the acquired sensing signals and sensing equipment that are used to obtain the signals. The sensing signals can be categorized into image signals with a visible spectrum, image signals with an infrared spectrum, special spectral intensity signals of a limited small region, acoustical signals, X-ray signals, and other signals in terms of the types of monitored signals. Sensing signals, sensing equipment, process signatures, and research literature in DED can be seen in Table 2. Studies that focused on the IM of E-DED are limited and are mostly conducted by the National Aeronautics and Space Administration (NASA) [115, 116].

Table 2 Sensing signals, sensing equipment, process signatures, and research literature in DED
Full size table

3.1 Image signals with a visible spectrum and sensing equipment

Image signals usually refer to pictures of the molten pool, depositing layers, and feedstocks, which can be commonly monitored by visible light signal sensors, such as charge-coupled device (CCD) and complementary metal-oxide-semiconductor (CMOS) cameras. The main problems in the acquisition of image signals with a visible spectrum include heat source interference, spatter interference, and difficulty in tracing the small dynamic molten pool. Therefore, researchers and engineers consider several key points, including technical parameters and installation methods of cameras, illumination devices, optical filters, and structured light, when using these cameras.

Technical parameters of cameras

Except for common technical parameters, such as pixels and frame rate, dynamic range is an important parameter when using a visible light camera in a high-energy beam process. The use of high dynamic range cameras is considered effective in avoiding heat source and scatter interferences. However, low image quality is the barrier to spread commercial high dynamic range cameras. Although the frame rate of this camera is low, it basically meets the sensing requirement of an adaptive control because of the image processing time.

Installation methods of cameras

Image signal acquisition methods are classified as coaxial [24, 111] and paraxial monitoring [25] in terms of the installation position of the cameras. The schematic is shown in Fig. 4. The coaxial monitoring method is applicable to the automation because of its ability to trace the small dynamic molten pool. However, this method can be only used in L-DED and mainly obtains 2D images of the molten pool with a small visual spectrum range. Thus, it is not suitable for E-DED and A-DED [187].

Fig. 4
figure 4

Schematic of a coaxial monitoring and b paraxial monitoring [27] in L-DED

Full size image

Illumination devices

Auxiliary illumination devices, such as light-emitting diode, laser diode, UV array, and vertical-cavity surface-emitting laser [188, 189], are beneficial for extracting accurate visual information. Laser light sources are preferred when gathering the visible lighting signatures of a molten pool. Illuminants can also be installed coaxially or paraxially. Compared with paraxial installation, passive lighting, which is based on the lighting of process areas, and coaxial illuminants are more attractive from the perspective of practical application. Meanwhile, although the illuminant can improve accuracy and clarity, it may disturb some spectrum signals.

Optical filters

Optical filters are universally used in the acquisition of image signals to filter the optical spectrum of a high-energy beam, powder scattering, and arc scattering. In addition, they are also part of a 3D camera, which can obtain the profile of DED products.

Structured light

A structured light with a wavelength that is in the range of 405–520 nm is suitable for eliminating the error in the line position estimation caused by an identical wavelength of glowing surface and laser [190]. Meanwhile, a rough surface that can enhance a diffuse reflection and absorb noises can be considered to improve measurement accuracy [190].

Currently, image signals with a visible spectrum have been used to monitor the geometric characteristics of molten pools, molten pool temperature, feedstocks, plumes, geometric characteristics of deposited layers, etc. as shown in Table 2.

3.2 Image signals with an infrared spectrum and sensing equipment

Temperature process signatures are significantly relevant to product quality because DED uses high-energy heat sources to fuse metal powder or wire. In contrast to the optical signals with a visible spectrum, the optical signals with an infrared spectrum are more reliable in calculating temperature data. The main reason is that the visible spectrum will be disturbed by other process phenomena or experiments, such as different colors of deposited layers or substrates. Therefore, image signals with an infrared spectrum are important in determining the temperature distribution of molten pools and deposited layers.

CMOS/CCD camera, which can gather the infrared signals of short wavelength, and thermal imagers with different infrared wavelength ranges are the main sensing equipment used to obtain image signals with an infrared spectrum. Both types of sensing equipment can obtain temperature data after a blackbody furnace calibration. The relevant references are listed in Table 2. In addition, images with an infrared spectrum can also eliminate noises and disturbances, as shown in Fig. 5. Compared with Fig. 5(b). the image of Fig. 5(a) can be used to extracted the binary image(as shown in Fig. 5(c)) more easily. Because of atmospheric absorption, three infrared wavelength ranges, three infrared wavelength ranges, 0.9–1.7 μm (short wavelength), 3–5 μm (medium wavelength), and 8–14 μm (long wavelength) are usually used in infrared cameras.

Fig. 5
figure 5

Images of a molten pool: a original infrared image, b ordinary image, and c binary image showing the boundary of melt pool [51]

Full size image

Some problems need to be addressed when the infrared cameras are applied in DED. First, an optical filter that eliminates the wavelength range of a laser beam should be used because the wavelength ranges of some laser sources and infrared cameras with short wavelength are partially coincident. Second, infrared cameras with short and medium wavelengths have better monitoring abilities on high-temperature monitoring than those with long wavelengths. The reason is based on the Wien displacement law [191], which suggests that the wavelength of radiation spectrum decreases with the increase in temperature. Third, emissivity of metals with different states need to be ensured, and measured thermal radiation quantities need to be calibrated by a blackbody furnace because of sharp variations of emissivity with temperature and surface thermochemistry for improving accuracy [73]. Lastly, Table 3 shows the advantages and disadvantages of different devices to serve as a relevant selection basis for researchers and engineers.

Table 3 Comparison of different equipments applied in image signals with infrared spectrum
Full size table

Currently, image signals with infrared spectrum have been used in the temperature monitoring of molten pools, feedstocks, and deposited layers, as shown in Table 2.

In addition to the visible and infrared spectrum, ultraviolet spectrum is also tested to obtain the molten pool images for avoiding the interference of strong visible light and infrared radiation [189].

3.3 Special spectrum signals and sensing equipment

Spectrum signal acquisition methods can be classified as coaxial and paraxial monitoring in terms of the installation position of sensors. Coaxial monitoring is applied in L-DED. However, the measured spectrum range is limited because of the complex optical system in a laser head. Paraxial monitoring is adaptive to every type of DED. Two types of data exist based on the acquired spectrum data. One type is a larger range of spectrum wavelength obtained by photodiodes, spectrometers, and hyperspectral cameras. A significant problem about this type is plasma plume over the molten pool. Although a lot of useful information can be obtained from the spectrum of the plasma plume, the existence of plasma plume needs to be evaluated because the energy density of DED cannot always produce plasma plume [138]. The other type is a confirmed single, double, and multispectrum wavelengths obtained by a single-color, dual-color, or multicolor pyrometers. The key point of this type is the emissivity set for an accurate temperature. The emissivity problem is the same as the image signals with an infrared camera above. Furthermore, the colorimetric temperature measurement based on double and multispectrum wavelengths is an effective method to avoid unknown emissivity and particle interference.

Currently, the sensing equipment involves pyrometers, photodiodes, spectrometers, and hyperspectral cameras. The process signatures that can be obtained by special spectrum signals mainly include the molten pool characteristics, feedstock characteristics, plasma plume over the molten pool, etc. These process signatures are shown in Table 2.

3.4 Acoustic emission signals and sensing equipment

Compared with the three aforementioned signals, acoustic emission signals generated by stress waves can provide abundant information about powder transmission, pores, or cracks [165,166,167]. These signals are also studied in PBF and fused deposition modeling, which can be a reference in DED [192,193,194,195]. Some problems, such as the analysis of high-frequency acoustic signals, overheating substrates, and the noise from the powder that strikes the substrate, impede the application of acoustic emission signals. In addition, when the stress wave source is positioned differently relative to the propagation path to the detector, the same acoustic emission mechanism may lead to different detected signals [196].

3.5 X-ray signals and sensing equipment

In recent years, X-ray signals are gradually applied in high-energy heat source processes, including DED. The sensing equipment mainly consists of an X-ray radiation source, an undulator, a slit, a scintillator, and a high-speed camera, as shown in Fig. 6. The evolution mechanisms of molten pools and feedstocks during the DED process are the primary subjects investigated. By contrast, X-ray signals are more widely used in welding, melting, and PBD [172, 197,198,199,200,201,202,203,204,205,206].

Fig. 6
figure 6

Schematic of X-ray sensing equipment [171]

Full size image

3.6 Other signals and sensing equipment

Except for the aforementioned signals, those that involve the electromotive force signals of thermocouples, visible lighting signals of displacement sensors, and current signals of the Hall effect current sensor are also used in DED. The relevant references are shown in Table 2.

Electromotive force signals of thermocouples

Thermocouples are usually applied to measure the temperature of substrates. However, as contact measurement devices, when thermocouples are applied in the DED monitoring, some problems need to be paid more attention, including the high temperature above the melting point of metals, temperature rising of the sensors resulted from irradiation [174], and low response speed. Currently, thermocouples are mainly applied to verify the accuracy of simulation models. The use of thermocouples is more accurate than infrared temperature measurement, which is greatly affected by emissivity.

Visible lighting signals of displacement sensors

The principle of displacement sensors used in DED is triangular surveying. The visible lighting signal usually comes from the light spot generated by a laser diode, which is a part of a displacement sensor. The wavelength of the laser diode in most displacement sensors is about 650 nm. Thus, the high-temperature metal can possibly interfere with the measurement accuracy of the displacement sensors.

Current signals of the Hall effect current sensor

Current signals are mainly used in wire DED. Resistances of wire and a molten pool can be calculated by current and voltage signals.

4 Process signatures obtained from sensing signals

Process signatures are obtained or calculated by sensing signals. Some sensing signals may contribute to the same process signatures. For example, temperature process signatures can be obtained by an infrared camera, pyrometer, hyperspectral camera, thermocouple, and visible light camera. This section will review the process signatures that have been acquired by using the sensing signals of L-DED and A-DED. These DED types will be described aiming at some common process signatures. However, many methods can be applied in all the three DED processes.

4.1 Molten pool, deposited layer, and substrate geometric characteristics

Molten pool and deposited layer geometric characteristics are usually obtained through image signals with a visible or an infrared spectrum. Since the image signals can indicate geometric characteristics directly and are widely applied with the development of image processing technology, a vast majority of works has been performed to determine the geometric characteristics. Width, length, height, outline, and area of a molten pool; width and height of deposited layers; and deformations of the substrate and deposited layers are some interesting geometric characteristics in previous studies.

4.1.1 Two-dimensional geometric characteristics of a molten pool

Geometric characteristics can be mainly acquired by image signals. Moreover, these characteristics can be acquired when the relationships between the other signals and the geometric characteristics of a molten pool are determined.

Image signals with a visible spectrum

Coaxial monitoring is an important advantage of L-DED, especially on the monitoring of 2D molten pool geometric characteristics because it can determine the 2D geometric characteristics of a molten pool directly without performing a coordinate conversion. Currently, many companies that produce laser heads can provide coaxial visual monitoring modules. The width, length, outline, and area of molten pools have been exacted through the coaxial monitoring method and image processing [24, 28,29,30,31,32,33, 36, 49, 50, 111, 114]. The main difference of molten pool images between coaxial monitoring and paraxial monitoring is the optical filter system. The optical filter system of coaxial monitoring is integrated into the laser head with limited wavelength. For example, the wavelength that can pass PRECITEC YC52 is only 520–720 nm [37]. However, the image signals of coaxial and paraxial monitoring can be processed by using similar image processing methods. The images are shown in Fig. 7.

Fig. 7
figure 7

Images of a molten pool: a infrared image, b video image, and c the contour of the molten pool on infrared image [36]

Full size image

In L-DED, the molten pool characteristics are commonly exacted by using a threshold segmentation method with a user-defined or calculational threshold value [33]. Song et al. presented a phase congruency melt pool edge extraction approach, except for the traditional threshold segmentation method. This method has a better robustness but a similar execution time compared with fix and Otsu threshold methods [27]. Deng et al. [110] used a grayscale gradient algorithm to obtain the boundary of a molten pool and calculate the molten pool area in real time. The Otsu threshold method was also applied in A-DED [52]. In addition, Xiong et al. [53,54,55] obtained the edge points of the deposited layer width through the Laplacian operator, and then a Hough transform was used to determine the width.

Image signals with an infrared spectrum

When image signals with an infrared spectrum are used to eliminate noises and disturbances, the data process methods are similar to image signals with a visible spectrum [36, 51, 110, 111]. In addition, the geometric characteristics of a molten pool can also be obtained on the basis of the melting point [70, 112, 115, 116, 121, 122], as seen in Fig. 8.

Fig. 8
figure 8

Two-dimensional geometric characteristics of a molten pool obtained by image signals with an infrared spectrum [121]

Full size image

Special spectrum signals

Except for image signals with a visible or an infrared spectrum, special spectrum signals can also indicate the geometric characteristics of a molten pool. Miyagi et al. [30] investigated the correction between molten pool widths and spectrum signal acquired by photodiodes. This study indicated that the molten pool widths could be obtained by spectrum signals. Moreover, the temperature distribution of a molten pool was determined by the spectrum signals using a spectrometer or hyperspectral cameras; the molten pool boundary could be ensured by the melting point [112, 138, 150, 152, 153].

4.1.2 Height of a molten pool or deposited layers

The 2D horizontal geometric characteristics of the aforementioned molten pool are usually attained and calibrated by one camera. On the same principle, the third-dimension height of deposited layers or a molten pool can be calculated by one camera that is perpendicular to the laser beam and the deposition direction [40,41,42,43,44,45, 48, 53, 58, 66, 67, 207, 208]. Furthermore, a binocular vision system or a laser-based 3D optical scanning system is applied in height monitoring [49, 50, 96,97,98,99,100,101,102,103,104,105,106,107,108].

Except for optical filters, the height measurements of different DEDs are similar. In L-DED, Mazumder et al., Song et al., and Asselin et al. measured the geometric characteristics that involved the layer heights online by using a trinocular CCD-based binocular vision system. In the trinocular CCD-based binocular vision system, two cameras can calculate the height data, and the remaining one is used to improve monitoring accuracy [10, 39, 46, 47, 209]. Heralić et al. [49, 50, 101] measured the geometric characteristics of deposited layers layer-to-layer by using a noncontact laser-based 3D optical scanning system that consisted of paraxial CCD/CMOS camera and a green structured light laser diode at 512 nm wavelength and 5 mW power. In addition, some coaxial monitoring methods of molten pool or deposited layer height have been presented. Donadello et al. [60, 61] obtained a coaxial scattered probe light with a coaxial probe laser beam. Subsequently, they calculated the height based on the scattered light, but when a molten pool existed on process multilayers, the scattered light might be disturbed. In A-DED, similar researches have also been conducted in [102,103,104,105,106,107,108]. The use of displacement sensor is another way to acquire deposited layer heights [163].

4.1.3 Deformation of the substrate and deposited layers

A displacement sensor and a digital image correlation system have been applied to reach the deformation state in DED. The substrate was usually clamped by a fixture when studying substrate deformation, as seen in Fig. 9 [126]. The displacement sensor could obtain the deformation values in real time. The method was also used in [179, 184]. With regard to deposited layers, digital image correlation systems could obtain the deformation layer to layer or online [59, 62, 63]. A digital image correlation system with two cameras could gather 3D data [59, 62], and the system with one camera could obtain 2D data [63, 64]. In the same way, the digital image correlation system with one camera has also been applied in the monitoring of substrate deformation [68].

Fig. 9
figure 9

Experimental setup for distortion measurements by a displacement sensor (LDS) [126]

Full size image

4.2 Feedstock process signatures

DED is a coupling interaction among substrates, heat sources, and feedstocks. The process signatures about feedstocks are beneficial for improving the understanding and stability of DED process. The feedstocks of DED have two types, powders and wires. The focused points of these two types are different. With regard to powders, three procedures should be conducted before powders enter a molten pool, leave powder pots, transfer in the powder delivering pipe, and transmit between nozzles and the molten pool. The two latter are the main focused points in powder DED. In wire DED, the transfer state has elicited the researchers’ attention.

4.2.1 Powder DED

On the one hand, the delivering state and mass flow rate are concerned targets when powders are being transferred in the powder delivering pipe. Now, acoustic emission and spectrum signals were both used to inspect the delivering state and flow rate. Ding et al. [36] used an optoelectronic sensor that consisted of a diode laser, a photodiode, a small rectangular glass chamber, and asset of lenses, to inspect the changes in powder density based on the diffusion, absorption, and reflection of the powder stream that passed through the glass chamber. Whiting et al. [167] also performed the powder mass flow rate monitoring and realized the function by an acoustic emission sensor.

On the other hand, when powders are transmitted between the nozzles and the molten pool, the powder flow is the research object. The powder flow between nozzles and a molten pool or a substrate is commonly observed through image signals with a visible or infrared spectrum. The image signals can obtain the concentration distribution of the powder stream (as shown in Fig. 10) [70, 74,75,76], the flow velocity of powders [70,71,72,73,74,75], and the temperature distribution [69, 70]. First, Liu et al. [70, 74] obtained the side powder concentration at different distances from the nozzle by a green laser and a CCD camera when powder feeding was paraxial by focusing on concentration distributions. Balu et al. [75] determined the powder concentrations of NT-20, WC, and premixed powders with different carrier–gas flow rates. Balu et al. also acquired the coaxial powder concentrations at different stand-off distances by using a 45° mirror. Second, with regard to the flow velocity of powders, Liu et al. [70, 74] captured the particle tracks to calculate the flow velocities of powders by using a CCD camera with 5000 fps. The averaged photodiode velocities were obtained when the carrier–gas rate or powder feeding rate was changed. Similar works that include the effects of various process parameters on flow velocities of powders are shown in [71,72,73, 75]. Lastly, the temperature distributions of the powder flow could be monitored by a color CCD based on the colorimetric temperature measurement or the image signals with an infrared spectrum based on thermal radiations [69, 70].

Fig. 10
figure 10

Images of different concentration distributions of powder stream [75]

Full size image

4.2.2 Wire DED

The transfer state of a wire during the DED process is a main concern that needs to be addressed. Abe et al. [77] confirmed the heat input range based on the observed dripping of the molten metal. Zhan et al. [78] presented a vision-based measuring method to detect the deviations of the wire-feeding position to avoid deposition defects and dimensional errors. Furthermore, the resistances of a molten pool and wire were measured to identify the transfer state, as shown in Fig. 11 [185, 186].

Fig. 11
figure 11

Wire transfer modes [185, 186]

Full size image

4.3 Temperature process signatures of the molten pool, deposited layer, and substrate

Considering that DED uses the high-energy heat source as the focused thermal energy source to fuse metal powder or wire, the temperature process signatures, which can be gathered online by temperature sensors, is significantly relevant with product qualities. The methods used for gathering the temperature signal are divided into contact measurements, such as thermocouples, and noncontact measurements, such as pyrometers and thermal cameras. In the last section, the temperature process signatures of feedstocks are shown. This section will focus on the temperature process signatures of the molten pool, deposited layer, and substrate. In terms of temperature process signatures, the monitoring methods in L-DED, EB-DED, and AP-DED are similar.

4.3.1 L-DED

The temperature process signatures are classified into a single-point temperature of molten pool, the temperature distribution of molten pool, temperature distribution around the molten pool, and temperature distribution of the substrate. The single-point temperature of molten pool was monitored by single-color [70, 72, 122, 135,136,137, 159, 160, 210,211,212], dual-color [7, 24, 117, 130, 154,155,156,157,158, 161,162,163], or a multicolor pyrometer [71, 73, 117, 119, 120]. CCD camera [25, 80, 81], hyperspectral (or spectrometers) [112, 138, 150,151,152,153], and infrared cameras [70,71,72, 79, 112, 113, 117,118,119, 121, 122, 125, 127,128,129,130, 132, 133] were used for the measurement of the temperature distribution of a molten pool based on spectral and blackbody calibration. The temperature of a solid–liquid interface can also be used as a calibration [121]. The effective monitoring devices for the temperature distribution of the substrate or deposited layers were thermocouples [34, 48, 59, 68, 70, 122, 173,174,175,176,177,178,179] and infrared cameras [117, 120, 131, 213]. In addition, temperature measurement by the images with two distinct wavelengths has also been attempted to obtain the molten pool temperature characteristics coaxially [32]. Fabbro et al. [88, 214] used an infrared CMOS camera, which was calibrated by a spectrometer real time, to acquire the temperature distribution of a molten pool. After gathering the temperature distribution, the date of cooling rate, temperature gradient, and other parameters related to temperature can be calculated. These data can be monitored to indicate the product qualities or verify the simulation models.

Except for the ways to measure the temperature data quantificationally, Akbari et al. [53] indicated that the temperature state of molten pools with their contour profile and the lengths of molten pools with varied cooling velocities were obviously diverse.

4.3.2 A-DED

The temperature distribution of the substrate or deposited layers was mainly monitored by thermocouples [85, 124, 180,181,182,183] and infrared cameras [77, 123, 124, 134]. In addition, Wu et al. [85, 164] used a pyrometer to record the temperature change in every deposited layer. The temperature distribution measurement of the molten pool mainly relied on infrared camera [77, 104, 105].

4.3.3 E-DED

The temperature process signature monitoring of E-DED was mainly completed by NASA. They used a CCD/CMOS camera, which filters the visible spectrum, to acquire the temperature distribution of a molten pool and a deposited layer. The camera was calibrated by a blackbody radiation source [115, 116].

4.4 Spectrum process signatures of molten pool and deposited layer

Spectrum process signatures of molten pools and plasma plume over the molten pool have great potential to be used in the temperature process signatures and feedstock process signatures. Photodiodes [24, 26, 30], spectrometers [52, 83, 138,139,140,141,142,143,144,145,146,147,148,149], and hyperspectral cameras [112, 150,151,152,153] are the common devices used for gathering the spectrum signatures from molten pools or deposited layers. The spectrum process signatures from a spectrometer are shown in Fig. 12. In the spectrum process signatures, the spectral characteristic analysis is the main work. Now, an electron or plasma temperature [144, 146, 149], a spectral line intensity ratio [83, 139,140,141,142, 144,145,146, 148], and a spectral intensity [52, 138, 139, 143, 147] have been obtained.

Fig. 12
figure 12

Emission lines from the DED experiment [139]

Full size image

4.5 Acoustic emission process signatures

Acoustic emission process signatures are mainly acquired by acoustic emission sensors. Deposited layer and powder mass flow were the monitored objects. Gaja and Liou [166] analyzed acoustic emission signals through an unsupervised pattern recognition analysis (K-means clustering) in conjunction with a principal component analysis. Whiting et al. [167] obtained the acoustic emission of powder transmission and acquired the real-time powder mass flow rate by a calibration.

4.6 Other process signatures

Interaction mechanisms of a laser beam, feedstock, and substrate are also important points that can be investigated on the basis of the sensing signals of a high-speed camera. Haley et al. [86] monitored the L-DED process paraxially by using a high-speed camera with up to 200,000 frames per second and 3.6 μm pixel resolution. The velocity distributions, impact, and floating phenomenon of powder particles were revealed when a powder particle touched a molten pool [86]. Abe et al. [87] observed the melting process of metal powders and substrate and presented four stages, namely, powder gathering, solidifying in a globular shape, hump forming, and hump remelting, during powder L-DED. Gharbi et al. [88] analyzed melt pool fluctuations with different duty cycles when the laser was set as a pulsed laser regime. Gharbi et al. [38] also found two situations when Ti–6Al–4V alloy powders impact the molten pool.

Focusing on A-DED, Guo et al. [94] used a high-speed camera to observe the physical generating process of arc and droplets during a compulsively constricted A-DED process. Xiong et al. [58] obtained visual images of inclusion disturbance in the molten pool, at the molten pool boundary, and on the deposited layer, as shown in Fig. 13(a), (b), and (c), respectively. Xu et al. [91] also observed the molten pool behavior, such as wetting and spreading, on a smooth and rough surface condition, respectively.

Fig. 13
figure 13

Disturbance of inclusions in the image during the A-DED process. a Inclusion in the molten pool. b Inclusions at the molten pool boundary. c Inclusions at room temperature [58]

Full size image

5 Development of the IM of product quality in DED

The relationships among process parameter, process signatures, and product qualities are the key to the studies of IMAC technology. This section will show the product qualities or monitored objects that have been monitored by these process signatures in Section 4. The mapping of the literature on monitored objects with respect to process signatures is shown in Table 4.

Table 4 Mapping of the literature on monitored objects with respect to process signatures
Full size table

5.1 Geometric accuracy

5.1.1 Forming appearance

Forming appearance is a product quality that can be observed directly without further inspection. It is usually considered a primary monitored object and provides a global assessment of the DED process. Any process signature that exists during an entire DED process can indicate surface quality. When the process signature changes unexpectedly, the surface quality may decrease. The characteristics of molten pools, such as geometric characteristics, temperature, and spectrum, are the common process signatures that indicate surface quality.

Molten pool, deposited layer, and substrate geometric characteristics

In powder DED, numerous studies proved that the molten pool area was correlated with forming appearance. Hu et al., Ding et al., and Stockton et al. [36, 111, 113, 115] found that the stable molten pool areas could acquire a uniform wall thickness when processing multilayer products. Deng et al. [110] proved the relationship between the areas of molten pool and cross-section areas of deposited layers. Except for area characteristics, the correlations between other geometric characteristics and forming appearances are also significant. Tang et al. [37] investigated the evolution behavior of forming appearances based on molten pool geometric characteristics. The results showed that the molten pool geometric characteristics could indicate all the respective key forming appearances in the four typical DED processes (e.g., single-track single-layer cladding, single-track multilayer accumulation, multitrack single-layer overlap, and multitrack single-layer overlap). For example, in multitrack single-layer overlap, the unsmooth surface was observed by the molten pool characteristics resulted by different overlap rates, as shown in Fig. 14; in the single-track multilayer overhang accumulation, the process fails because of inadequate and unstable flow velocities of a molten pool, as shown in Fig. 15. In wire DED, Motta et al. [90] used a high-speed camera to monitor the DED process of a metal wire and observed the appearance of dripping and stubbing defects. Radel et al. [65] presented a deposited radius to indicate the forming appearance. Xiong et al. [93] cleared the phenomenon of instability and overflow of a molten pool, and when this phenomenon occurred, the deposited layer might collapse.

Fig. 14
figure 14

a–c Molten pool characteristics of different overlap rates [37]

Full size image
Fig. 15
figure 15

a–c Failure in single-track multilayer overhang accumulation because of inadequate and unstable flow velocities of a molten pool [37]

Full size image

Temperature process signatures of molten pool and deposited layer

In L-DED, Bi et al. [26, 135,136,137] emphasized that the temperature signals gathered by a Ge photodiode significantly affected the forming appearance. This phenomenon was proven by the forming failure and edge defects of a thin-wall accumulation and surface qualities of cylinder cladding. In A-DED, Abe et al. [77] researched the influence of the interlayer temperature on the forming appearance, and the result showed that the lower interlayer temperature could indicate a stable process state.

Spectrum and plume process signatures

Focusing on spectrum process signatures, Liu et al. [149] obtained spectral signals from plasma plume during the A-DED process and calculated the electron temperature. The electron temperature, including the mean values and standard deviations, could be considered indicators of the instability of the forming appearance [149]. Aiming at plume process signatures, Nassar et al. [84] studied the relationships of process parameters, plume process signatures, and formation appearance. The plume process signatures contained plume geometry and intensity. The result showed plume process signatures could reflect the DED state [84].

Other process signatures

In wire DED, the resistance of the molten pool and the wire was affected by the transfer state [185, 186]. Therefore, Hagqvist et al. [185, 186] attempted to use resistance to indicate the process stability and found that an ideal resistance could maintain a remarkable forming quality. In addition to single process signature, multisensor information fusion was also performed in the monitoring of geometric accuracy. Zhao et al. [52] applied a cooperative awareness method of spectrum, vision, and electrical parameter, which had a higher recognition rate than single-source monitoring, to identify the changes in process parameters and formation appearances.

5.1.2 Dimensional accuracy

The width and height of deposited layers and the deformation of a substrate are important objects in the monitoring of dimensional accuracy. Geometric characteristics acquired by some cameras with different wavelengths are effective ways to monitor dimensional accuracy. In current studies, the width and height of molten pool are considered the width and height of deposited layers. Therefore, the monitoring methods focused on the molten pool geometric characteristics are the same as the geometric characteristics of deposited layers.

Width

The widths of molten pools refer to the distance between the left and right solid–liquid interfaces of a molten pool along the scanning direction. Solid–liquid interfaces can be monitored on the basis of the grayscale changes in the visual images or the position with a fusion temperature of metal material. Cameras with different wavelengths can all observe the grayscale changes. The positions with a fusion temperature need to be acquired by infrared cameras or calibrated spectral sensing equipment. The relevant contents and references are shown in Section 4.1.1 and Table 2, respectively. In addition to measuring the widths of deposited layers based on the molten pool widths, the widths of deposited layers can also be monitored by one camera perpendicular to the upper surface of a deposited layer, binocular vision system, or 3D camera system directly. These methods have been reviewed in Section 4.1.1 and Table 2.

Height

The height monitoring of deposited layers is similar to width monitoring. The molten pool heights have been determined by a camera perpendicular to the scanning direction and laser beam and binocular vision system. The 3D camera system is not suitable for monitoring a molten pool because the linear structured light of a 3D camera is obstructed by the molten pool with high brightness. By contrast, deposited layer heights can be monitored by the three devices mentioned above, as seen in Table 4. Except for visual methods, displacement sensors are also used in DED to record the changes in every layer. The contents have been reviewed in Section 4.1.1. In addition, Tan et al. and Bennett et al. [131, 156] studied the relationship between the temperature characteristics and the deposited layer height and realized the height monitoring based on temperature process signatures.

Deformation

The substrate or deposited layer deformations can also be considered the height or width variations. Therefore, if necessary, the deformations can be described in detail. In addition, displacement sensors and a camera that captured 2D data have been used in monitoring substrate deformations [68, 126, 179, 184]. The 2D data from one camera and the 3D data from two cameras could be applied in monitoring the deposited layer deformation [59, 62,63,64]. After determining the relationship among process parameters, temperature process signatures, and deformation data, Yan et al. [126] predicted the deformation angle by heating areas during DED.

5.2 Microstructure and property

5.2.1 Microstructure, phase transformation, and mechanical property

Temperature process signatures greatly affect the microstructures of deposited layers. Therefore, in the current studies, the main process signatures used in monitoring microstructures include the temperature process signatures of the molten pool, deposited layer, and substrate. Farshidianfar et al. [127, 128] proved that real-time cooling rates could define the size of the solidification structure and mode of solidification of 316L stainless steel. Huang et al. [216] combined the temperature monitoring data of Farshidianfar et al. [127] with temperature simulation, and used thermal gradient (G) and solidification rate (R) to correlate product qualities. The results indicated that the dendrite arm spacing was more sensitive to scanning speed, and the cooling rate (G × R) and G/R value had sufficient abilities to predict the microstructure when depositing 316L stainless steel and Inconel 625 [216]. Bennett et al. [132] presented a longer cooling and solidification time result in coarse microstructures in processing Inconel 718, and the ultimate tensile strengths would decrease. In their work, the temperature data were characterized as the weighted cooling and solidification time, which was the weighted normalized solidification rate times plus the normalized cooling rate [132]. Wu et al. [164] examined the effect of heat accumulation on oxidation, microstructure, and mechanical properties. Muvvala et al. [159] used a pyrometer to record the molten pool lifetime and cooling rates when depositing Inconel 718 with 30 wt% WC as a ceramic phase. They found that an optimum molten pool lifetime could provide the highest wear resistance of the coating. Applying similar methods, Muvvala et al. [160, 210,211,212] also finished many excellent works that focus on different materials in references. Compared with the evident influences of temperature process signatures, Wolff et al. [129] researched the variation of microhardness by relying on simulation and temperature monitoring. They found that the composition mixture between the powder and the substrate decided the microhardness rather than the cooling behavior of a molten pool in DED Inconel 718 on AISI 1045 carbon steel.

In addition to temperature process signatures, molten pool geometric characteristics and spectrum process signatures are also applied in DED to monitor the microstructure, phase transformation, and mechanical properties. Concerning molten pool geometric characteristics, Akbari and Kovacevic [51] built an empirical relationship between the cooling rate and the molten pool area. They found that a short interlayer time in thin-wall accumulation would decrease the cooling rate and result in coarse grain size and low ultimate tensile strength [51]. Based on spectrum process signatures, Song et al. [140] presented a method to predict phase transformation. They found the odd points of the line intensity ratios that exist when a phase transformation change happened, and they had potential to monitor the phase transformation [140].

5.2.2 Elemental composition

Elemental composition is mainly monitored by spectrum process signatures. Bartkowiak [143] proved the feasibility to monitor the elemental composition by spectrum process signatures. Song et al. [139] utilized laser-induced plasma to monitor the composition of Ti–Al binary powders during DED with a support vector regression method. Song et al. [141] also realized the Cr composition monitoring using spectrum process signatures by the spectral line intensity ratio when depositing pure chromium on H13 tool steel, as seen in Fig. 16. Meanwhile, Shin et al. [146] also used the laser-induced plasma method to monitor the Ni composition during depositing Inconel 718. They found that the regression line of the Ni-I (352.45 nm)/Cr-I (399.11 nm) line ratio had the most accurate monitoring [146]. Lednev et al. [142, 145, 148] applied laser-induced breakdown spectroscopy to analyze the key components (e.g., carbon and tungsten) quantificationally after nickel alloy reinforced with tungsten carbide grains were deposited, as shown in Fig. 17.

Fig. 16
figure 16

Schematic of the detection of laser-induced plasma [139]

Full size image
Fig. 17
figure 17

a Schematic and b photograph of the laser-induced breakdown spectroscopy probe for in situ elemental composition analysis [145]

Full size image

5.2.3 Metallurgical bonding

In addition to microstructural and elemental composition, the metallurgical bonding state is also an important performance index. Metallurgical bonding state can be considered the dilution depth. In fact, dilution depths can hardly be monitored directly. Therefore, some process signatures related to dilution depths are used to predict or monitor dilution depths. Abe et al. [87] used a high-speed camera to observe the metallurgical bonding state and cleared the phenomenon when the poor or overfusion state appeared. Bi et al. [24, 136] applied the temperature process signatures to acquire the correction between dilution state and IR-temperature signals gathered by a 1400–1600-nm Ge photodiode. Farshidianfar et al. and Wolff et al. [127, 129] also used temperature process signatures to study the relationship between cooling rate and dilution depth and found their direct and obvious relation. With regard to spectrum signals, Ya et al. [144] utilized the IR signal and electron temperature as the spectrum process signatures to monitor the metallic bonding state. The result showed that the IR signal could indicate the onset of metallic bonding, and the electron temperature could indicate the extent of dilution [144].

5.3 Metallurgical defect

Internal metallurgical defects are a critical point for DED to impact its quality and repeatability. Realizing the IM of defects provides a probability to stop this process to avoid waste of time and materials or eliminate these defects in situ. Stutzman et al. [83] utilized plume images and spectral signals to monitor the internal metallurgical defects compared with the post-build CT scans. The results showed that the two signals were consistent with each other, that is, they face the same experimental phenomenon, and both had the potential to indicate internal metallurgical defects [83].

Pores and cracks are two main objects in the monitoring of metallurgical defects. Barua et al. [80] observed the image changes in red-hot deposited layers and obtained the images when cracks and pores occurred. Hammell et al. [117] used an infrared camera to determine the temperature distribution when cracks appeared. Gaja and Liou [166] utilized an acoustic emission sensor and released the monitoring of pores and cracks by combining an unsupervised pattern recognition analysis and a principal component analysis. The acoustic emission process signatures were also used by Wang et al. to monitor crack generation and expansion. They found that the number of cracks would increase with deposited areas, layer thickness, and cooling rate [165].

Furthermore, the porosity, which is similar to pores, is also monitored in DED. Zhang et al. [89] used a coaxial visual monitoring camera to realize the porosity monitoring based on a deep learning-based method with a classification accuracy of 91.2%. Khanzadeh et al. [130] obtained the temperature characteristics of a molten pool and built the relationship between temperature characteristics and porosities based on the self-organizing map method.

6 Existing problems and future research

With the expedite development of IMAC technology in DED, there are still some existing problems and deficiencies in aforementioned parts, including sensing signals, process signatures, product qualities, and commercial equipment/software, which have been reviewed in Sections 2, 3, 4, and 5. Many works can be performed to meet the requirements of IMAC.

First, more advanced sensing devices, such as in situ X-ray diffraction devices, an electromagnetic acoustic transducer, and some radiation sensors, should be applied to acquire sensing signals. Especially for E-DED, the photons, electrons, X-rays, neutral particles, ions, and backscattered electrons can provide rich signals to understand and quantify the DED process [116]. In addition, some sensing signals, such as eddy currents, coherence scanning interferometry [217], and backscatter electron detection, which have been used in PBF, can also be utilized in DED [218]. Meanwhile, a deposition head, especially the laser head with a coaxial monitoring ability, needs to integrate as many sensing signals as possible to adapt to the requirements of high flexibility and adaptability.

Second, process signatures are a bridge between process parameters and product qualities. Therefore, characterization and quantization methods need to be stable, reliable, and speedy, especially for real-time monitoring. Moreover, a larger number of process signatures from sensing signals are necessary. However, when expanding the scope of monitorable objects or improving inspecting accuracy, the data size will be larger simultaneously. Thus, maintaining the processing speed is a considerable problem. Meanwhile, because of the complex DED mechanisms, the applied situations of process signatures must be restricted based on DED equipment, range of process parameters, materials, process environment, and other possible factors. Even a calibration that aims at different situations is important. Furthermore, compared with PBF, signal processing, identification, and dimensionality reduction methods based on artificial intelligence can be paid more attention.

Third, in product qualities of DED, geometric accuracies are the main researched objects now. However, because of the near-net shaping of DED, the processed parts usually need to be subtractive machined again. Therefore, a very accurate dimension is not obligatory in DED. When ensuring the successful forming about the geometric accuracy, more methods need to focus on the monitoring of microstructures and metallurgical defects, especially for the hardly direct monitored objects, such as dilution depth, thermal stress, phase transformation, metallurgical bonding, etc. Moreover, multisensor information fusion methods and artificial intelligence algorithms, which have been used in the monitoring of welding defects [219], applied in DED are minor.

Lastly, the commercial equipment and software are restricted strongly because of the limitations of the DED equipment, the range of process parameters, materials, process environment, etc. Therefore, the IMAC equipment and software companies currently provide the sensing modules that gather sensing signals and can supply process signatures that have been processed. Fewer companies present an IMAC system that includes product qualities.

In summary, the authors believe that these points can be paid more attention. First, more sensing signals and equipment can be tried in DED, especially for monitoring methods that have been used in welding or PBF. Second, some methods need to be presented to obtain accurate emissivity during DED. Third, the characteristics of a molten pool, which can reveal the process of melting, flowing, and solidification, need to be analyzed systematically based on the visual features. Lastly, we think that multisensor information fusion and intelligent learning algorithms can play an important role in the analysis and identification of the process signatures and product qualities obtained.

7 Conclusion

Recently, a consensus that AM plays an important role in the industrial field has been reported. DED technology, as an important type of AM, has the advantages of 3D deposition forming, surface cladding, and parts repairing. However, the reliability and repeatability of DED still cannot meet the stringent requirement of high-end and high-value equipment. These problems represent major challenges toward fully exploiting the potentiality offered by DED.

In this review, we proved an overview of research efforts conducted in the area of IMAC, which is considered an effective method to overcome the aforementioned challenges. The research on IMAC is reorganized as sensing signals, sensing equipment, process signatures obtained from sensing signals, and product qualities that can be monitored. So far, image signals with different wavelengths are the main signals in DED used for monitoring the geometric accuracy and temperature distribution. More advance signals can be used in the IMAC of DED. Some objects that are difficult to be monitored, such as dilution depths, pores, and phase transformation, have been observed in terms of temperature, spectrum, and acoustic emission process signatures successfully. However, in terms of geometric accuracy, the methods or abilities for monitoring the other product qualities of DED still need to be improved. Multisensor information fusion and artificial intelligence that focuses on processing or identifying process signatures and product qualities are the research emphases.