1 Introduction

Welding processes can be classified into penetrative processes and filling processes based on their primary purposes [1]. Gas tungsten arc welding (GTAW) and gas metal arc welding (GMAW) serve as the respective benchmark processes [1]. GMAW is widely used in welding applications [1] and has been the initial process employed for wire arc additive manufacturing (WAAM) [2, 3], also known as welding-based rapid prototyping [2]. In a filling process, the first desirable property is the controllability of the heat proportion applied to both the wire and the workpiece [1]. Unfortunately, achieving such desired controllability is not feasible with the benchmark GMAW process due to its underlying principle. The principle, which involuntarily applies only half of the power at the anode to melt the wire compared to the power at the cathode/workpiece [1], is the cause of this limitation.

The welding community has been continuously striving to improve existing processes and pioneer innovative methods to achieve greater desirability. Examples of these efforts include laser welding, A-TIG [4, 5], K-TIG [6, 7], and double-sided arc welding [8, 9] for penetrative processes, as well as controlled short-circuiting GMAW [10] and AC GMAW [11] for filling processes. In a typical GMAW application, the wire functions as the anode, facilitating the transfer of the molten wire to the workpiece through electromagnetic forces [12]. However, in this polarity, the voltage on the workpiece is approximately twice that of the anode voltage [13]. By alternating the polarity of the wire, AC GMAW can reduce the heat applied to the workpiece while increasing the heat on the wire, effectively detaching droplets of the molten wire [11]. For short-circuiting GMAW, which utilizes resistance heat during the short-circuiting period to heat the wire only, precise control of metal transfer without spattering is essential. A specific technique that retracts the wire during the short-circuiting period has resulted in a precisely managed metal transfer process, leading to the successful commercialization of cold metal transfer (CMT) [14]. CMT has found extensive application in WAAM [15, 16]. Furthermore, the polarity of CMT has been alternated to combine the benefits of AC GMAW and CMT [17].

Fundamentally understanding GMAW as the benchmark filling process is challenging without a concept like “effective heat”, i.e., the heat imposed on the wire. Decoupled control of mass and heat stands as the ultimate goal of all process innovations aiming for an ideal filling process. The extent of this decoupling can be quantified by comparing the effective heat to the heat applied to the workpiece. In an open arc situation (without short circuiting), the ratio is \({r\approx IV}_{wire}/(I{V}_{wire}+{IV}_{work}){=V}_{wire}/({V}_{wire}+{V}_{work})\), with the resistance heat and the heat in the arc column omitted. When the wire functions as the anode, as is typical in many applications, \(r\approx 1/3\), and the ratio remains constant. Conversely, if the wire functions as the cathode, \(r\approx 2/3\), which serves as the stringent upper limit for AC GMAW, although the achievable ratio is likely much lower, ensuring a minimally necessary electrode-positive period for successful metal transfer. In the context of short-circuiting transfer processes, the ratio during the open arc period also equals 1/3, meaning that effective heat is only predominantly increased during the short-circuiting phase. In all scenarios, the upper limit of the achievable ratio is confined by the fundamental structure of the welding system, where the arc forms between the wire and workpiece, resulting in the passage of the same current through both elements.

The DE-GMAW process, invented by the corresponding author at the University of Kentucky, involves the addition of a second or bypass electrode to establish a secondary arc between the wire and the bypass electrode [18]. Consequently, the current bifurcates into two branches after passing through the wire, resulting in \(I={I}_{wire}={I}_{bypass}\)+ \({I}_{work}\), and a ratio of \(r{\approx {I}_{wire}V}_{wire}/({I}_{wire}{V}_{wire}+{{I}_{work}V}_{work}).\) Given that \({I}_{work}\) can be reduced to zero, DE-GMAW can reach an \(r\) value approximating 1, surpassing the fixed 1/3 ratio. The process has been effectively controlled and expanded, transitioning from a non-consumable to a consumable bypass electrode [19]. Furthermore, the introduction of the second current branch has enhanced metal transfer [20].

One challenge associated with DE-GMAW is the requirement for placement of the bypass electrode in close proximity to the wire tip to maintain the bypass arc. Additionally, the electrode must be positioned outside the central region of the arc column to minimize its impact on arc behavior. In a controlled laboratory setting, these conditions can be achieved by appropriately securing the bypass electrode in advance. However, if the position of the wire tip varies, it becomes necessary to either readjust the placement of the bypass electrode or, at the very least, monitor the process to ensure that DE-GMAW operates as intended.

As a result, the operation of the bypass electrode must be automated using a follower robot capable of dynamically adjusting the bypass electrode’s position in real-time. To achieve this objective, our initial investigation focuses on the arcing phenomenon, which we categorize into three distinct modes: parallel arc, serial arc, and single arc. The preferred mode is the parallel arc, characterized by the immediate branching of current into two paths after following from the wire. Subsequent analysis has revealed that the transition from the parallel to serial arc mode might not be sharply defined, leading to mixed mode. Consequently, we further refine the classification to incorporate a mixed arc/transition mode. This comprehensive classification framework enables us to closely monitor the DE-GMAW process, ensuring its consistent operation within the desired mode, which is targeted by our first step to robotize DE-GMAW.

2 Process background

In a typical GMAW process, an arc is created through the flow of current across the ionized gaseous gap from the wire to the workpiece. The heat generated by the arc melts the wire, which subsequently deposits into the workpiece. The productivity is determined by the achieved wire melting speed.

As in the reference [21], if the metal transfer is in spray mode (melting current great than 250 A for steel wire of 1.2-mm diameter), the melting speed \(\dot{m}\) of the mass, arc current \(I\), wire extension \(L\), and cross-section area of the wire \(S\) can be formulated as:

$$\dot{m}=5.1\times \frac{{10}^{-13}{I}^{2}L}{S}+2.2\times {10}^{-6}I$$
(1)

where the first term corresponds to the resistance heat and the second is due to the arc. In a first order approximation, \(\dot{m}\propto I\). Both Eq. (1) and its approximation mean that increasing the melting speed of the mass is realized by increasing the arc current. However, the current going through the wire equals to the current going through the workpiece:

$$I={I}_{work.}$$
(2)

The heat imposed by the arc on the workpiece \(({{I}_{work}V}_{work}\Delta t\)) during the arc application time \((\Delta t)\) thus increases involuntarily and proportionally if the current is increased to increase \(\dot{m}\). Increasing the current to increase \(\dot{m},\) hence, enlarges the weld pool, resulting in workpiece distortion and the accumulation of residual stress.

To address this challenge, the double-electrode GMAW process has been proposed as a solution to reduce the heat input towards the workpiece during the welding process [22], as shown in Fig. 1, by adding a bypass electrode along the side of the GMAW torch. This double-electrode process breaks the melting current (\(I\)) going through the wire into the workpiece current (\({I}_{work}\) or base metal current \({I}_{bm}\)) and bypass current (\({I}_{bypass}\) or \({I}_{bp}\)):

Fig. 1
figure 1

DE-GMAW system (a) and arc phenomena in DE-GMAW (b)

Full size image
$$I={I}_{bm}+{I}_{bp}$$
(3)

As a result, in this bypass current setup, the melting current will not be equal to the base metal current. The current passing through the workpiece can be controlled by adjusting the bypass current, while the total current flowing through the wire will remain unaffected. This allows for the maintenance of the wire melting speed while controlling the heat generated on the workpiece to a desired level.

It is obvious that the fundamental characteristic of the DE-GMAW process (Fig. 1(a)) is the presence of the bypass arc (Fig. 1(b)). The stability of this bypass arc is crucial for the proper functioning of DE-GMAW. In previous studies, this stability was achieved by pre-determining/fixing the position of the bypass electrode in relation to the GMAW torch. However, when the wire extension, wire feed speed, arc voltage, or other welding parameters change, the position of the bypass electrode would need to be adjusted to sustain this stability optimally. To address this problem, an automatic robot welding system is needed to manipulate the bypass electrode. The control algorithm to command the robot can be learned and generalized from those demonstrated by skilled human welders.

3 Proposed approaches

The arc image, as shown in Fig. 1 (b), is apparently the most direct raw information to derive the arc mode. However, processing the arc image to identify the wire, bypass electrode, and arc paths appears to be challenging. Further studies are also needed to derive the mode from the identified wire, bypass electrode, and arc paths wire. As the arc image contains sufficient raw information, which is crucial for deep learning [23], to determine the mode, an ideal solution is to train a deep-learning model with the image as the input and the mode as the output. Unfortunately, although humans can decide the mode through careful observation, labeling the needed large number of images is not practical. This is particularly the case when involving the mixed arc mode. To find automatically obtained alternative labels, we propose to study the physical process which leads us to identify the voltage across the bypass arc gap as such an automatically obtainable label. The needed large quality of labels can thus become available.

One may wonder if the bypass voltage may replace the arc image to monitor the mode of the DE-GMAW process. We note that the arc volage is fluctuating and is not as direct as the arc image. The arc voltage may also change with other factors/parameters such as the material of the wire, the material of the bypass electrode, the shield gas, the used power source etc., while the arc image only changes with the arc mode. As such, in this study, all these factors/parameters are kept the same, but the learned model to process the image is transformative and generalizable beyond the experimental conditions used in this study.

With the automatically obtained bypass voltage, thresholds are still needed to classify into different modes. We propose to first study the experimental data to determine thresholds. However, after mixed mode is added, the thresholds become fuzzy. As such, we will then propose using the k-means method to automatically decide how many modes/classes make the best sense and determine the thresholds.

We note that the DE-GMAW process has not been robotized, and this work is the first effort toward its robotization. As such, we do not have a freely controllable robotized process and do not know how to control it yet. We propose to learn from human welders to address this challenge. Hence, we have established an experimental system where a tractor moves the GMAW torch, and a human welder observes the arc to adjust the bypass electrode. As the human welder has never operated such a process, his/her control of the bypass electrode cannot be ideal. This creates various possible scenarios we can learn from for how and why human welders succeeded and failed for us to develop the control algorithm, to adaptively adjust the bypass electrode by the follower robot, through learning from human welders. Hence, these scenarios, that simulate manufacturing scenarios in laboratory conditions, can allow more reasonably study how the modes may change during manufacturing.

4 Experiments

The experimental system is shown in Fig. 2. The electrical system to operate the process is still to be same as given by Fig. 1. During experiments, a human welder operates a GTAW torch that holds the bypass electrode while collaborating with the moving tractor that carries a GMAW torch. The tractor moves at a fixed speed, and the human welder adjusts the bypass electrode based on the observation, including the arc and relative position of the wire tip and bypass electrode. The movement of GTAW torch that holds the bypass electrode is tracked by an IMU (inertial measurement unit) sensor attached to it. A Point Grey camera FL3-FW-03S1C is attached to the tractor to image/record what the welder observes. Examples of the recorded images are shown in Fig. 3. The GMAW power source used (Power Source 1 or Main Power Source in Fig. 1, not shown in the figure) is Miller Auto-Continuum 350a operating at constant voltage (CV) mode, and the bypass power source (Power Source 2 or Bypass Power Source in Fig. 1, not shown in the figure) is Miller Maxstar 210 operating at the constant current (CC) mode. The current for Power Source 2 (\({I}_{2}\)) was set at \(100 \mathrm{A}\), while voltage for Power Source 1 (\({V}_{1}\)) was set at \(33 \mathrm{V}\) and wire feed speed was \(270\mathrm{ IPM }(6.9 \mathrm{m}/\mathrm{min})\). During the process, the arc image, voltage, and current are sensed synchronously.

Fig. 2
figure 2

Experimental setup. The power sources are not shown

Full size image
Fig. 3
figure 3

Examples of observed arc phenomena

Full size image

In particular, we will measure the voltage across the wire and bypass electrode which are connected to the positive and negative terminal of Power Source 2, denoted as \({V}_{2}\). We will also use the current supplied by Power Source 2 which has been denoted as \({I}_{2}\). The current supplied by Power Source 1 has been denoted as \({I}_{1}\). The voltage across the wire and workpiece is \({V}_{1}\). The definitions of these voltages and currents can be seen in Fig. 1. As will be analyzed, \({I}_{1}={I}_{bm}\) and \({I}_{2}={I}_{bp}\) are true only when the process operates in the desired mode of the DE-GMAW. Also, \({V}_{2}={V}_{bypass\_arc}\) is true only when the process operates in the desired mode.

5 Experimental results and analysis toward automatic labeling

Observation of recorded videos shows that skilled human welders distinguished among different modes to decide their subsequent control actions in adjusting the bypass electrode, without the knowledge of the voltage associated. Consequently, to generalize the expertise from human welders, raw image representing their observation should be an ideal input information source and be used as the input of their model. However, while manually labeling the image for their observed mode as output to train the model is feasible, such labeling will be very time consuming and is not practically feasible. Therefore, an automatically generated label, which can also reflect the arc mode, should be discovered. To this end, we analyze the underlying process.

Examining the images uncovered that there are three distinct modes as shown in Fig. 4: open arc (Fig. 4(a)), parallel arc (Fig. 4(b)), and serial arc (Fig. 4(c)).

  1. a.

    Open/Single arc mode: In Fig. 4 (a), there is no bypass arc established between the wire and the bypass electrode. Therefore, no current passes through to the bypass electrode. That is, \({I}_{bp}=0.\) In this case, \({V}_{2}\) is the open-circuit voltage \({V}_{oc}\) which is the highest voltage provided by the power source used to power the bypass loop. This mode is featured by \(\left[{I}_{2},{V}_{2}\right]=\left[0,{V}_{oc}\right]\), as shown in Fig. 5 (a). This mode can be further illustrated in Fig. 6 (a).

  2. b.

    Parallel arc mode: In Fig. 4 (b), in addition to the main arc (MA or Arc1) between the wire and workpiece, there is also an arc (BA or Arc2), as evidenced by the bright passage between the wire and bypass electrode. In this case, \({V}_{2}\) is the voltage of the bypass arc established between the wire and bypass electrode, i.e., \({V}_{bp}\), and \({I}_{2}={I}_{2}^{*}\) where \({I}_{2}^{*}\) is the current setting for Power Source 2 (100 A in this study, although it can be freely adjusted). As such, \({V}_{bp}\) will be determined by the bypass arc governed by well-known arc physics. This mode is featured by \(\left[{I}_{2},{V}_{2}\right]=\left[{I}_{1}^{*},{V}_{bp}\right],\) as shown in Fig. 5 (b), and this mode can be further illustrated in Fig. 6 (b). As can be seen from Fig. 5 (b), \({V}_{bp}\) has been reduced from \({V}_{oc}\) to the range \([20\mathrm{V}, 40\mathrm{V}]\).

  3. c.

    Serial arc mode: In Fig. 4 (c), there is also another arc (Arc2) in addition to the main arc. However, this Arc2 is established between the bypass electrode and workpiece, rather than the wire. This Arc2 is thus not the bypass arc we desire from the process. In this case, the bypass loop changes from “power source-wire-electrode-power source” to “power source-wire-workpiece-electrode-power source.” We observed an increase in \({V}_{2}\) while \({I}_{1}={I}_{bp}^{*}\) still holds. We now have \(\left[{I}_{2},{V}_{2}\right]=\left[{I}_{1}^{*},{V}_{1}+{V}_{wb}\right]\) where \({V}_{wb}\) is the voltage between the workpiece and bypass electrode. \({V}_{2}\) is thus higher exceeding the range associated with the parallel arc mode. This mode results in Fig. 5 (c) and can be further demonstrated in Fig. 6 (c).

Fig. 4
figure 4

DE-GMAW modes. (a) Open arc; (b) parallel arc; (c) serial arc. WT, wire tip; BE, bypass electrode; MA, main arc; WP, workpiece; and BWA, arc between bypass-electrode and workpiece. Bypass voltage \({V}_{bp}\) is measured between the wire and bypass electrode for the voltage between WT and BE

Full size image
Fig. 5
figure 5

Measured \({I}_{2}\) and \({V}_{2}\) in different modes. a Open arc mode; b parallel arc mode; c serial arc mode

Full size image
Fig. 6
figure 6

Electrical principles of different operation modes

Full size image

We now analyze these three modes and the observed phenomena. We are particularly interested in why \({V}_{2}\) differs and how \({I}_{bm}\) changes in different modes:

  1. a.

    Open arc mode: As shown in Fig. 6 (a), there is no passage between \({V}_{2}(+)\) (wire) and \({V}_{2}(-)\) (bypass electrode). As such, it is well understood that \(\left[{I}_{2},{V}_{2}\right]=\left[0,{V}_{oc}\right]\). In this case, the current to melt the wire is completely provided by Power Source 1, i.e., \({I}_{1}=I(\dot{m})\) which is considered a fixed amperage for the given \(\dot{m}\) (approximately proportional to the wire feed speed). In this case, \({I}_{bm}={I}_{1}=I(\dot{m})\). This is the benchmark GMAW process.

  2. b.

    Parallel arc mode: As shown in Fig. 6 (b), there is a direct passage between \({V}_{2}(+)\) (wire) and \({V}_{2}(-)\) (bypass electrode). As such, it is well understood that \(\left[{I}_{2},{V}_{2}\right]=\left[{I}_{2}^{*},{V}_{bp}\right]\) where \({V}_{bp}\) is the voltage of the bypass arc:

    $${V}_{bp}={V}_{anode}+{V}_{column}+{V}_{catode}$$
    (4)

    where \({V}_{anode}\), \({V}_{column}\), and \({V}_{catode}\) are the voltage fall on the arc anode (wire), voltage of the arc column (bright passage observed), and the voltage fall on the arc cathode (bypass electrode). Per the arc physics, \({V}_{anode}\) and \({V}_{catode}\) are approximately constants determined by the materials of the electrodes (wire and bypass electrode for the bypass arc), while \({V}_{column}\) increases with the arc length (distance between the wire tip and the bypass electrode). In summary, \({V}_{bp}\) is from one arc!

    In this mode, \(I(\dot{m})\) is provided by \({I}_{1}\) and \({I}_{2}\). As \({I}_{2}\) directly flows from the wire to the bypass electrode, \({I}_{bm}={I}_{1}=I\left(\dot{m}\right)-{I}_{2}<I\left(\dot{m}\right)\). The current imposed on the workpiece is thus reduced from \(I\left(\dot{m}\right),\) and the reduction increases as \({I}_{2}\) increases.

  3. c.

    Serial arc mode: As shown in Fig. 6 (c), there is no direct passage from the wire to the bypass electrode. As such, \({I}_{2}\) also flows from the wire to the workpiece so that \({I}_{bm}={I}_{1}+{I}_{2}\)= \(I\left(\dot{m}\right).\) That is, \(I\left(\dot{m}\right)\) is fully imposed on the workpiece. This makes the arc power on the workpiece to become the same as in the benchmark GMAW. However, \({I}_{2}\) returns from the workpiece to the bypass electrode (\({V}_{2}(-)\)). Additional arc power \({{I}_{2}V}_{anode}\) is thus imposed on the workpiece. The heat input in the workpiece is actually increased from that in the benchmark GMAW. This is the worst case among the three modes. Hence, DE-GMAW must be operated in the desired Parallel Arc mode and supervised or controlled.

    This mode is featured by

    $$\begin{array}{c}{V}_{2}=\left({V}_{1}\left(+\right)-{V}_{1}\left(-\right)\right)+\left({V}_{1}\left(-\right)-{V}_{2}\left(-\right)\right)=\\ \left({V}_{anode1}+{V}_{column1}+{V}_{cathode1 }\right)+{(V}_{anode2}+{V}_{column2}+{V}_{cathode2 })\end{array}$$
    (5)

    where the first parenthesis is the voltage from the wire to the workpiece and the second is that from the workpiece to the bypass electrode. Although \({{V}_{anode1}\ne V}_{anode2},\) \({V}_{cathode1 }\ne {V}_{cathode2}\) and\({V}_{column1}\ne {V}_{column2}\), we can still see\({V}_{2}\approx 2({V}_{anode}+{V}_{column}+{V}_{cathode })\). As such, \({V}_{2}\) in this mode should be higher than \({V}_{2}\) in the Parallel mode. Voltage \({V}_{2}\) thus has the physics foundation to be used to classify the operation modes.

As illustrated in Fig. 7, the voltage during a DE-GMAW process can be segmented corresponding to (0) State 0 for the desired Parallel Arc mode; (1) State 1 for the undesired Serial Arc mode, and (2) State 2 for the undesired benchmark GMAW mode. This segmentation helps distinguish the different arc states.

With the assistance of a skilled human welder, 20 weld trials were conducted, and a total of 10,062 data pairs \([{I}_{k}, {S}_{k}]\) have been collected, where \(k\in [1, 10062]\), where \({I}_{k}\) is the kth arc image and \({S}_{k}\) is the corresponding segmentation (label) from \({V}_{2}\).

Fig. 7
figure 7

Voltage-based segmentation

Full size image

6 Network and training

The employed convolutional neural network (CNN) architecture, depicted in Fig. 8, predicts a label \(S\) from input image \(I\). The input image, sized \(256\times 256\), is passed through the CNN via typical convolution layers followed by pooling layers. This process is repeated five times with the convolution layer parameters as (1, 16, 5, 1, and 2), (16, 32, 5, 1, and 2), (32, 64, 3, 2, and 1), (64, 128, 3, 2, and 1), (128, 256, 3, 2, and 1), and pooling layers of (2 and 2). ReLU activation and batch normalization are applied between each convolution and pooling process. After processing the input image into a \(256\times 1\) feature vector, it is passed through two continuous fully connected layers, reducing its dimension from 256 to 128 and finally to 3. Softmax process is then performed on the resulting \(3\times 1\) feature vector to obtain the real class of the input image.

Fig. 8
figure 8

CNN for classification

Full size image

The training and validation process has been conducted on an NVIDIA GTX 2080 graphic card. With the dataset collected containing 10,062 data pairs, a total of 9592 data pairs were used for the training process, while the remaining 470 data pairs were utilized for the validation process. The model was trained iteratively 200 times with SGD optimizer and cross-entropy loss under Python environment with Pytorch library. During the training and validation process, the dataset has been shuffled to ensure that the data has been drawn randomly.

The loss results during the training and validation processes are displayed in Fig. 9. The validation loss stabilizes after epoch 33, while the training loss continues to decrease until the end. The model with the minimum validation loss at epoch 67 is selected, and the test results are presented in Fig. 10.

Fig. 9
figure 9

Training and validation curve

Full size image
Fig. 10
figure 10

Validation result

Full size image

For a more intuitive view of the results, Fig. 11 displays the voltage, two reference lines, and the errors obtained from the \(Label-Predict\) result. Figure 12 displays the confusion matrix of the result. Clearly, in the first segment of the data, where the voltage exceeds \(70 \mathrm{V}\) (indicating only GMAW arc with open-loop voltage), the model’s accuracy is high with no errors observed. Similarly, during the second segment, where the voltage fluctuates around \(30 \mathrm{V}\), representing a successful DE-GMAW process, the model also performs accurately. However, in the third segment, which marks the transition between the successful bypass arc and the undesired serial arc, the model makes several mistakes. Using voltage to calculate mean square loss for the wrong predictions gives 0.039.

Fig. 11
figure 11

Analysis of the prediction error: where errors occur

Full size image
Fig. 12
figure 12

Confusion matrix of the result

Full size image

7 K-means classification

K-means clustering [24] is a method to group data points into clusters. It minimizes the sum of squared distances between points and their cluster centroids. The process involves iteratively assigning points to the nearest centroid and updating centroids. The goal is to find centroids that minimize the total distance within each cluster. For each data point \({x}_{i}\), find the nearest centroid \({c}_{j}\) and assign \({x}_{i}\) to cluster \(j\):

$${min}_{j}{\Vert {x}_{i}-{c}_{j}\Vert }^{2}$$
(6)

For each cluster \(j\), update its centroid \({c}_{j}\) to the mean of all data points assigned to it:

$${c}_{j}=\frac{1}{\left|{s}_{j}\right|}{\Sigma }_{{x}_{i}\in {s}_{j}}{x}_{i}$$
(7)

In this process, \({s}_{j}\) represents the set of data points assigned to cluster \(j\). By iterating between these two steps, a state of convergence is achieved as the assignments and centroids reach stability. The algorithm effectively partitions the data into \(K\) clusters for the predetermined value of \(K\). The value of \(K\) is a hyperparameter pre-defined and will significantly influence the quality of clustering. To determine the appropriate number for \(K\), the gap statistic method [25] is employed by:

$$Gap\left(K\right)=Avg\left[\mathrm{log}\left({J}_{null}\right)\right]-\mathrm{log}\left({J}_{actual}\right)$$
(8)

where \({J}_{null}\) is the sum of squared distances for the randomly generated data, \({J}_{actual}\) is the sum of squared distances for the actual data clusters. \(Avg\left[\mathrm{log}\left({J}_{null}\right)\right]\) represents the average of the logarithms of the sum of squared distances for the null data clusters. The Gap Statistic compares the difference between the average logarithm of the sum of squared distances for the null data clusters and the logarithm of the sum of squared distances for the actual data clusters. Determining the optimal number of \(K\) that provides a good balance between capturing meaningful patterns in the data and avoiding overfitting or underfitting. With the voltage data collected during the experiment. The result was shown in Fig. 13. Clearly 3 clusters should be optimal number in the dataset. This verifies our classification clusters proposed based on physical analysis of the process. However, K-means algorithm avoids manually assigning the thresholds.

Fig. 13
figure 13

Gap statistics result for an optimal \(K.\)

Full size image

We performed the K-means algorithm on the dataset with \(K=3\). The result is illustrated in Fig. 14.

Fig. 14
figure 14

K-means clustering result

Full size image

Using K-means clustering, the 3 Cluster Centroids were determined as 4.5, 7.5, and 2.9. Notably, 2.9 corresponds to our desired cluster dataset. To further investigate, the identical model structure was trained using the same dataset, with the labels substituted from the K-means clustering results. The training curve is illustrated in Fig. 15.

Fig. 15
figure 15

Training and validation curve

Full size image

The validation error plot, following the format of Fig. 11, is presented in Fig. 16.

Fig. 16
figure 16

Analysis of the prediction error in classification using K-means clustering based thresholds: where errors occur

Full size image

Similar with the model training with the manually assigned threshold label, K-means based labeling performs well with the first two segments of the data, and the error only occurs when the voltage was fluctuating significantly. This demonstrates that regardless of equipment variations, an accurate classification model can be trained without the necessity of prior knowledge about the welding power source, i.e., obtaining a reasonable label group does not require knowing the voltage/current of the welding system.

8 Fine classification of desired mode

Figure 17 is the histogram of \({V}_{2}\) which has been used to label the mode. As can be seen, while the Open Arc mode is completely separable, there is not a clear boundary to separate between the desired Parallel Arc and undesired Serial Arc. Analysis of arc images suggests that there is a mixed mode where a portion of \({I}_{2}\) directly flows from the wire to the bypass electrode while another portion flows to the workpiece and then from the workpiece to the bypass electrode, as shown in Fig. 18.

Fig. 17
figure 17

Histogram of \({V}_{2}\) measured across the wire and bypass electrode

Full size image
Fig. 18
figure 18

Examples of the mixed mode in DE-GMAW

Full size image

As the Serial Arc mode is the worst, it is beneficial to prevent a mixed mode from being accepted during supervisory monitoring. To this end, for the image that has been classified to and accepted as the Parallel Arc mode, we propose to pass the image also into a fine classification network. We note there are cases where we have the perfect Parallel Arc mode, but the bypass electrode appears to be too close to the wire to cause a possible collision. As such, we propose to further classify the Parallel Arc mode, based \({V}_{2},\) into three sub-clusters: Too Close, Most Desired, and Low Confidence.

Within the \({V}_{2}\) interval [\(min{V}_{2}^{*},max{V}_{2}^{*}\)] for the Parallel Arc, there are no clear boundaries for further separation. This is understandable as they are indeed in the same mode of operation. Our proposed fine classification is to provide an additional safeguard. As such, we propose to divide this voltage interval into three subintervals of equal length, \(\Delta {V}_{2}=(max{V}_{2}^{*}-min{V}_{2}^{*})/3\). A new but smaller dataset was formed, for the data with \({V}_{2}\in\) [\(min{V}_{2}^{*},max{V}_{2}^{*}\)], and is classified using two thresholds \(min{V}_{2}^{*}+\Delta {V}_{2}\) and \(min{V}_{2}^{*}+2\Delta {V}_{2}\). We trained the same model with the new dataset. The result is shown in Fig. 19. The prediction accuracy is approximately 80%.

Fig. 19
figure 19

Analysis of the prediction error in fine classification within desired mode: where errors occur

Full size image

Although the accuracy is lower than the previous model, it is important to recognize that this fine classification represents an advanced attempt to distinguish the arc state within a very narrow segment. The input images were highly similar, and the voltage exhibited only small differences while the voltage also fluctuated. This fine classification is to determine whether the state could be considered the most optimal, with the other two states also being acceptable. This suggests that the deep learning model is capable of identifying arc states even with very small differences. With an increase in data, the model’s performance for the fine classification should also be expected to improve.

9 Conclusion and future work

The DE-GMAW provides an effective approach to separately control mass and heat input [26]. To ensure that DE-GMAW delivers the intended benefits, we have developed a deep learning model capable of processing captured images to classify the operational states/modes of the process. This forms the foundation for robotizing the DE-GMAW process, even in the face of potential variations that might deviate it from the desired mode.

The solution presented in this work is the culmination of a series of novel concepts and innovations. Given the complexity of the process, which has not been previously automated, we propose learning from human welders. To facilitate this, an experimental system has been established, allowing for demonstrations by human welders. Overcoming the challenge of acquiring a substantial number of labels required for deep learning, we analyze the process and suggest utilizing the voltage across the wire and bypass electrode. Furthermore, we advocate for automatic classification using the K-means approach, supplemented by fine classification within the accepted mode range to ensure optimal operation.

This work marks the pioneering step towards automating DE-GMAW, with a specific focus on supervisory monitoring. Subsequent efforts will prioritize learning from human welders for real-time adjustments, leading to a fully adaptive, automated DE-GMAW system capable of functioning amidst continuous variations in manufacturing conditions.