Weld penetration in situ prediction from keyhole dynamic behavior under time-varying VPPAW pools via the OS-ELM model

Abstract

In situ monitoring and accurate detecting of welding quality have been one of the common challenges of automatic welding process. This paper contributes an intelligent decision-making framework for the weld penetration prediction from the keyhole dynamic behavior under time-varying VPPAW pools. Initially, a series of dynamic experiments under different welding conditions were conducted to acquire the backside images of keyhole and corresponding backside bead width. Then, the geometry appearance of keyhole was described by the supervised descent method (SDM)–based image processing algorithm. Subsequently, the internal correlation between the keyhole characteristics and the backside width was further derived to help understand the nonlinear and time-varying VPPAW process. Finally, a novel dynamic model based on an online sequential extreme learning machine (OS-ELM) was designed to predict the weld penetration as measured by the backside bead width in real time. Extensive experiment results further verify and validate that the proposed dynamic OS-ELM model is significantly better than other state-of-the-art algorithms in terms of predicting accuracy, efficiency, and robustness.

1 Introduction

As “one of the most prospective advanced welding technologies in the twenty-first century,” the keyhole variable polarity plasma arc welding (VPPAW) has many advantages of high energy density, strong penetrability, and free porosity [1,2,3,4]. It has seen widespread use in high-end aerospace manufacturing area such as the space shuttle external tanks and rocket propellant tanks. Due to the unique physical attributes of plasma arc, it has brought a series of issues, e.g., the stability of keyhole molten pool, the complexity of welding process, and the dependability of welding quality [5]. Especially for constant parameters, open-loop VPPAW process, the keyhole collapse, or burn-through may easily occur in the thin-walled plates if disturbances such as abrupt change of heat dissipation, butt-joint gap, or misalignment exist. Therefore, how to guarantee the dynamic presence of the keyhole molten pool for obtaining a desired weld remains one of the greatest challenges in the automatic VPPAW process. In view of the keyhole state is a core element for determining the weld penetration and joint quality, it is quite necessary to obtain the sensing information in situ for reflecting the keyhole dynamic behavior. Therefore, using the “keyhole effect” as an entry point, most welding researchers have made extensive efforts in investigating the relationship between the keyhole signatures and weld joint penetration. The earlier researches mainly focus on some indirect in situ sensing methods including plasma cloud charge [6], efflux plasma voltage [7], and audible sound signals [8, 9] to monitor and control the keyhole plasma arc welding process. Nevertheless, the above methods mainly concern with variation of individual physical phenomena to reflect the keyhole status, not directly provide the shape and dimension information of keyhole.

To date, with the rapid development of machine vision technology, most welding researchers have paid more attention to visual sensing of keyhole molten pool, which could provide intuitive information related to the weld quality [10,11,12,13,14,15,16]. Hence, Zheng et al. [10] acquired the front-side image of the keyhole puddle and extracted the keyhole geometrical sizes to further realize the feedback control for weld formation during the VPPAW process. However, it is hard to capture a complete and clear keyhole image from the front-side workpiece due to the large-diameter plasma torch and limited standoff distance. To better observe the keyhole appearance, Liu et al. [13] developed a vision-based sensor to acquire the keyhole images from the backside of the workpiece. Based on the mechanism analysis of keyhole formation, the authors qualitatively investigated the effect of the keyhole features on the weld quality. Additionally, Wu et al. [14, 15] described the keyhole variation mechanism based on a thermal-force model and then proposed a novel intelligent model to relate the welding parameters to the backside keyhole characteristics. Nevertheless, detailed studies should be done to further verify that the keyhole behavior does contain sufficient information on the weld penetration. The establishment of the correlation between keyhole behavior and weld penetration demands more accurate in situ inspections and a suitable welding dynamic model.

For a full-penetrated weld, a key indicator of determining the weld quality is the weld penetration status which can be quantitatively evaluated by the backside bead width [16,17,18]. However, it is very difficult to achieve the in situ measurement for the backside width because of the strong plasma arc disturbance during VPPAW process. Hence, how to derive the complex correlation between the keyhole behavior and the backside width is of utmost importance towards a better control of weld joint penetration. To our knowledge, the artificial neural networks (ANN) are capable of dealing with complex database with nonlinear and dynamic relationships, which have been widely used to perform tasks such as nonlinear approximation, prediction, and classification in a multivariable and complicated industrial process, especially in the area of welding process [19,20,21,22]. In this regard, Cook et al. [23] applied the artificial neural networks (ANN) to predict weld bead geometry depending on the process parameters and weld pool characteristics of gas tungsten arc welding (GTAW) process. Chen et al. [24, 25] further developed a size and shape neural network model (SSNNM) for a dynamic GTAW process to relate the welding parameters to the backside weld width. However, traditional ANN algorithms have a good capacity of nonlinear identification, but have the disadvantages of slow training speed, low generalization capability, etc. Moreover, the fluctuating weld conditions or external interferences challenge the application of ANN model for penetration prediction because an off-line trained model is hardly adapt to the constantly changing weld pool.

To develop the capability for predicting the weld penetration despite large variations in keyhole geometry, a novel dynamic OS-ELM model will be introduced to accomplish this objective in this study. It is an easy-to-use and incremental fast learning algorithm for single–hidden layer feedforward neural networks (SLFNs) [26]. The OS-ELM could provide the online learning property that guarantees the trained model matches the real complicated and changing environment perfectly [27, 28]. Therefore, recent years have received significant attentions on time-series prediction, image processing, and large-scale data processing area. Yang et al. [29] proposed a novel OS-ELM model–based manufacture execution system for the position in situ identification under the dynamic changing environment, which can overcome the problem of mismatch between the model and the process for a RFID-positioning system. Frances-Villora et al. [30] applied the OS-ELM algorithm as a good candidate for the hardware implementation of real-time online learning applications, due to its combination of high training speeds and a tight use of resources. And also, Yu et al. [31] put forward a novel ensemble OS-ELM based on combination weight (CWEOS-ELM) for fast-changing data stream classification. The results demonstrate that the proposed algorithm not only has a better generalization performance but also provides a faster learning procedure.

Nevertheless, the advantages of evolving OS-ELM approach have not been penetrated into welding process domain up to now. Enlightened by a certain similarity between the complex welding process and the above industrial process, our study will further explore the feasibility of the OS-ELM model for weld penetration in situ prediction under varying VPPAW pools. Therefore, the structure of this paper was organized as follows. In Section 2, we briefly described the VPPAW experiment platform. Section 3 detailed an adaptive image processing algorithm for characterizing the keyhole behavior. Section 4 introduced a novel OS-ELM-based welding penetration in situ prediction model and illustrated the performance results comparing with some classical algorithms. Section 5 highlighted our conclusions.

2 VPPAW experimentation overview

2.1 Experiment apparatus

The VPPAW experimental platform is mainly composed of a welding power supply, a moveable manipulator, a low-cost flexible vision sensing system, and an industrial computer-based process controller. The basic controller can automatically adjust the welding parameters through its analog output interface to the welding power source system. As seen in Fig. 1, a charge-coupled device (CCD) camera was equipped with a proposed narrow band-pass filter (central wavelength 660 nm, bandwidth 35 nm). It was located behind the backside of workpiece to capture the keyhole images when the workpiece is penetrated to form a stable keyhole. The plasma torch and camera were attached to a moveable manipulator controlling by the computer. The keyhole image (200 × 200 8-bit digital image matrices) can be acquired at the sampling rate of 45 Hz. To better obtain the clear keyhole image, the welding angle and distance between the camera and torch center were set to 15° and 260 mm respectively in our research.

Fig. 1

The architecture of VPPAW experimental apparatus

2.2 Experiment conditions

Generally, the practical welding condition such as workpiece contamination, thermal-induced deformation, or varied clearance/misalignment may cause an abrupt change on the keyhole pool. Indeed, these interface factors have a direct influence on the welding thermal distribution on the welded workpiece. Therefore, to investigate the effects of keyhole behavior on the weld joint penetration when the welding interferences exist, the weld parameters were varied randomly to simulate the time-varying welding process. As shown in Table 1, a series of vertical-up butt-welding experiments were conducted on 2219 aluminum alloys with uniform dimensions 300 × 50 × 5.5 mm. The welding process parameters (welding current and plasma gas flow rate) determining weld quality were randomly adjusted according to the process controller, as shown in Fig. 2.

Table 1 Detailed VPPAW process parameters

Fig. 2

The variation of VPPAW current and plasma gas flow

With the rapid changing of weld heat input, the keyhole behavior continues to change along with the variation of backside bead width (see Fig. 3). As demonstrated, the weld penetration status is of importance to weld formation and joint quality, and the backside weld width (W_b) is proposed to evaluate the weld penetration quantitatively. In this study, the backside welds were captured off-line by CCD camera after welding, and then, the actual weld widths were computed based on the checkerboard calibration method (according to Fig. 8a). To establish the relationship between the backside width and welding penetration, it is necessary to develop an appropriate evaluation criterion for the weld penetration (see Table 2).

Fig. 3

Keyhole image acquisition and the backside weld appearance

Table 2 A detailed evaluation criterion for VPPAW penetration status (NP none penetration, PP partial penetration, FP full penetration, OP over penetration)

Figure 3 further shows that the keyhole behavior can roughly reflect the variation trend of backside width under the random changing of welding condition. With a low heat input, the workpiece is not fully penetrated and the back keyhole has not appeared; after that, with the increase of heat input, the keyhole sizes increase obviously and the backside width also increases accordingly. Due to the inherent nonlinearity and dynamics of welding process under varying welding conditions, there exists a complicated relationship between the keyhole features and the backside weld width. Therefore, in order to achieve accurate predictions of the weld penetration, nonlinear models, rather than linear models, are used to correlate the keyhole features with the weld width.

Moreover, the backside weld width correlates with the keyhole features nonlinearly and dynamically when the VPPAW pool varies, and the keyhole cannot maintain a relatively stable state. Therefore, it is essential to develop an effective and robust image processing algorithm to characterize the keyhole dynamic behavior. In this regard, most researchers have applied some traditional edge detection algorithms to plasma arc welding in the flat position [13]; however, it is difficult to handle the keyhole images in case of partial occlusion during vertical upward VPPA welding. Indeed, the dynamic balance inside the keyhole is more susceptible to the strong interferences from spatter (slag) and unsteady plasma jet; the keyhole behavior cannot always maintain a stable status (see Fig. 4) under varying welding parameters.

Fig. 4

a–d The keyhole images at different moments

3 Keyhole image processing using SDM

3.1 Brief review of SDM approach

To overcome the aforementioned drawbacks, we developed an accurate and robust algorithm for extracting the keyhole periphery based on supervised descent method (SDM). Comparing with some traditional edge extraction algorithms, SDM learns in a supervised manner generic descent directions and can resolve many shortages of second-order optimization schemes [32]. The SDM has been widely applied in facial feature detection and tracking fields in recent years [33, 34].

Indeed, the keyhole edge extraction has a certain similarity to the face alignment/detection process; hence, our study will investigate the feasibility of the SDM algorithm on keyhole image processing. In this section, an overview of proposed SDM approach for addressing the keyhole images was briefly reviewed which mainly contains training and testing sessions. Some processing stages were shared by the training and testing sessions (indicated by the dotted box in Fig. 5).

Fig. 5

a A manually labeled keyhole image (200 × 200 pixels) with 8 landmarks. b Blue outline represents a specified keyhole detector on the training images which provides the mean landmarks

3.1.1 Training session

• Step 1: Given a keyhole image I ∈ ℜ^l × 1 of l pixels and the coordinate vector of keyhole edge landmarks (p is the number of landmarks in each keyhole image), I(x) ∈ ℜ^2p × 1 is feature extraction function. Then, we detect the specific keyhole edge landmarks and further capture the SIFT (scale invariant feature transform) features to achieve a robust representation for keyhole edge information. During training, we will assume that the correct p landmarks (in our case 8) are known, and we will refer to them as x_∗ (see Fig. 5a). To reproduce the testing scenario, we ran the face detector on the training images to provide an initial configuration of the landmarks (x₀), which corresponds to an average shape (see Fig. 5b). Therefore, the objective function for keyhole alignment is:

$$ f\left({x}_0+\varDelta x\right)={\left\Vert h\left(I\left({x}_0+\varDelta x\right)\right)-{\phi}_{\ast}\right\Vert}_2^2 $$

(1)

where ϕ_∗ = h(d(x_∗)) refers to the feature (SIFT) of the actual shape. The main idea behind SDM algorithm is to calculate a series of descent directions to produce a sequence of updates (x_k + 1 = x_k + Δx_k) starting from that converges to during training.

• Step 2: The first shape increment Δx₁ can be written as a generic combination of SIFT feature vectors ϕ₀ plus a bias term b₀ in Eq. 2:

$$ \varDelta {x}_1={\boldsymbol{R}}_0{\phi}_0+{\boldsymbol{b}}_0 $$

(2)

where R₀ indicates a descent direction, which represents matrix of regression coefficients. Therefore, SDM will calculate a sequence of generic decent directions and bias terms in Eq. 3:

$$ {x}_k={x}_{k-1}+{\boldsymbol{R}}_{k-1}{\phi}_{k-1}+{\boldsymbol{b}}_{k-1} $$

(3)

• Step 3: For each k^th training iteration, R_k and b_k are learned from a distinct linear regression, minimizing the difference between the true shape and predicted shape as expressed in Eq. 4:

$$ \arg \underset{\boldsymbol{R},\boldsymbol{b}}{\min }{\left\Vert \varDelta {x}_{\ast}^i-{\boldsymbol{R}}_k{\phi}_k-{\boldsymbol{b}}_k\right\Vert}_2^2 $$

(4)

where $\varDelta {x}_{\ast}^i={x}_{\ast}^i-{x}_k^i$ is the desired increment of shape ${x}_k^i$ from the current iteration to the true shape in the training set after 5 steps for all training keyhole images.

3.1.2 Testing session

Most of the steps in testing session are the same with the training session. Firstly, we obtain a testing image and find the specific landmarks using SIFT descriptor around each landmark. Then, with the help of the well-trained SVM regression model, we can acquire the predicted keyhole landmarks, with a root mean square error (RMSE) comparing with true edge landmarks. According to Eq. 5, the root mean square error (RMSE) is calculated on the differences between the SDM-based detection (yellow points) and true shape (red points) on keyhole images. It is used to measure the alignment accuracy of the keyhole edges.

$$ RMSE=\left(\sqrt{\frac{1}{N}\sum \limits_{i=1}^N{\left({x}_t^i-{x}_e^i\right)}^2}+\sqrt{\frac{1}{N}\sum \limits_{i=1}^N{\left({y}_t^i-{y}_e^i\right)}^2}\right)/2 $$

(5)

where (${x}_t^i$, ${y}_t^i$) is the coordinate values of the ith SDM-based detection point, and (${x}_e^i$, ${y}_e^i$) is the coordinate values of the corresponding true point. Figure 6 shows the complete flow diagram of SDM algorithm for the keyhole image processing process.

Fig. 6

Block diagram of SDM algorithm for keyhole image processing (red points refer to manual landmarks and yellow points refer to initialized/predicted landmarks)

3.2 Result validation using SDM algorithm

In this section, we will perform some experiments with the keyhole images to verify the proposed SDM approach. We acquired a total of 215 keyhole images (each image has a size of 200 × 200 pixels) corresponding to different penetration statuses and conducted a model training (38%) and testing (62%). To better represent the whole keyhole shapes, we manually selected several edge points in sequence during the training phase. The performance was evaluated using RMSE indicator, which can be calculated on the difference between the predicted keyhole landmarks and the true keyhole landmarks.

In addition, Fig. 7 shows an example of the keyhole edge estimation taken for a test sequence. With regard to each keyhole image, the RMS errors were almost within 10 and the average RMS error was nearly 7.004. To make sense of the numerical results, several location results on different keyhole images were presented to validate the effectiveness of the SDM algorithm. As depicted in Fig. 7, we cannot observe obvious differences between the predicted and true shapes. Note that in the 49 and 74 frames of keyhole images, the SDM algorithm could accurately track the keyhole edge even in cases of partial occlusion. Moreover, the algorithm was implemented in OpenCV platform, and the computation time per image needs 80~120 ms, which is suitable for real-time image processing during VPPAW process.

Fig. 7

The RMSE results between the SDM-based detection (yellow points) and true shape (red points) on keyhole images

3.3 Keyhole behavior characterization

After detecting all 8 landmarks from the keyhole images using the SDM-based approach, it is essential to extract the keyhole complete boundary to further analyze the keyhole behavior in detail. Indeed, the keyhole shape is almost oblate according to Fig. 4; we will apply a common ellipse-fitting algorithm to extract the keyhole edges. In this regard, we respectively defined the keyhole length (K_l), width (K_w), length-to-width ratio (K_r), and keyhole area (K_a) as the keyhole feature parameters (see Fig. 8), which further to holistically descript the keyhole dynamic behavior. The length-to-width ratio (K_r) is used to describe the oblateness of the keyhole [35].

Fig. 8

The keyhole behavior characterization using ellipse-fitting algorithm (yellow points refer to the acquired landmarks)

To obtain the actual keyhole geometric characteristics, we will transform the pixel numbers into the real dimensions both containing x-axis (along the welding direction) and y-axis (perpendicular to the welding direction) based on a 6 mm × 6 mm checkerboard calibration method. The four points A_i (i = 1~4) have the coordinates (X_i, Y_i), and the distance of adjacent two points along the same direction is 6 mm. The calibration coefficients CX and CY can be determined as Eq. 6. Thus, with the acquired keyhole images and the calibrated correlation, we can calculate the actual sizes of keyhole and further represent the morphology and size of the keyhole in the actual welding process.

$$ {\displaystyle \begin{array}{l} CX=6/\sqrt{{\left({x}_2-{x}_1\right)}^2+{\left({y}_2-{y}_1\right)}^2}=0.05128\mathrm{mm}/\mathrm{pixl}\\ {} CY=6/\sqrt{{\left({x}_3-{x}_1\right)}^2+{\left({y}_3-{y}_1\right)}^2}=0.05042\mathrm{mm}/\mathrm{pixl}\end{array}} $$

(6)

4 OS-ELM-based dynamic process modeling

As depicted above, the measured keyhole pool was characterized by its width, length, length-to-width ratio, and keyhole area which could identify the weld penetration. To our knowledge, the substantial thermal inertia effect exists in VPPA welding of aluminum alloy, which indicates the backside weld width at any moment may not be determined not only by the keyhole pool at the same moment, but also by the adjacent keyhole pool [15]. This implies that the correlation between the keyhole pool and weld penetration is highly nonlinear and time-varying which can be demonstrated in Fig. 9. Hence, a dynamic OS-ELM model was considered to perform in situ prediction of the backside width using the keyhole characteristic parameters.

Fig. 9

a Measured data for keyhole characteristic parameters. b Backside bead width and weld appearance formation.

4.1 Review of OS-ELM algorithm

As an extension of basic ELM, the online sequential extreme learning machine (OS-ELM) is an incremental fast learning algorithm for single–hidden layer feedforward neural networks (SLFNs) [29]. For a given dataset of learning samples N = {(x_i, w_bi)| x_i ∈ Rⁿ, w_bi ∈ R^m, i = 1, …, N}, where x_i refers to the feature vector of the keyhole that contains m visual features that are acquired by the SDM approach, w_bi is the measured backside weld width, and N is the sample number. In OS-ELM, the learning process is divided into two phases, i.e., an initialization phase and a sequential phase, which was described as follows [33]:

• Step 1: Initialization phase

A small chunk of training data is used to initialize learning ${N}_0={\left\{\left({x}_i,{w}_{bi}\right)\right\}}_{i=0}^{N_0}$ from the given training set, and set N₀ ≥ L.

• (1) Randomly assign the input parameters. For the number L of hidden RBF nodes: G(a_i, b_i, x) = g(b_i‖x − a_i‖), where a_i and b_i are the center and biases of the ith RBF node

• (2) Calculate the initial hidden layer output matrix H₀:

$$ {H}_0={\left[\begin{array}{l}G\left({a}_1,{b}_1,{x}_1\right)\kern1em \cdots \kern1em G\left({a}_L,{b}_L,{x}_1\right)\\ {}\kern2.12em \vdots \kern3.24em \vdots \kern3em \vdots \\ {}G\left({a}_1,{b}_1,{x}_{N_0}\right)\kern0.44em \cdots \kern1em G\left({a}_L,{b}_L,{x}_{N_0}\right)\end{array}\right]}_{N_0\times L} $$

(7)

• (3) Estimate the initial output weight β⁽⁰⁾

For the target vector ${T}_0={\left[{t}_1,\dots, {t}_{N_0}\right]}^T$, the problem is to minimize H₀β = T₀. In OS-ELM, H^† is given by

$$ {H}^{\dagger }={\left({H}^TH\right)}^{-1}{H}^T $$

(8)

which is called the left pseudoinverse of H. Hence, the solution to minimize H₀β = T₀ is given by β⁽⁰⁾ = P₀H₀T₀, where ${P}_0={\left({H}_0^T{H}_0\right)}^{-1}$.

• (4) Set k = N₀, and then go to step 2 to learn other samples one by one

• Step 2: Sequential learning phase

• (5) Present the (k + 1)^th chunk of new observations ${N}_{k+1}={\left\{\left({x}_i,{t}_i\right)\right\}}_{i=\left(\sum \limits_{j=0}^k{N}_j\right)}^{\sum \limits_{j=0}^{k+1}{N}_j}$; N_{k + 1} denotes the sample number in the (k + 1)^th chunk.

• (6) Compute the partial hidden layer output matrix H_{k + 1}:

$$ {H}_{k+1}={\left[\begin{array}{l}G\left({a}_1,{b}_1,{x}_{\left(\sum \limits_{j=0}^k{N}_j\right)+1}\right)\kern2.12em \cdots \kern3em G\left({a}_L,{b}_L,{x}_{\left(\sum \limits_{j=0}^k{N}_j\right)+1}\right)\\ {}\kern3em \vdots \kern6.12em \vdots \kern5.12em \vdots \\ {}G\left({a}_1,{b}_1,{x}_{\left(\sum \limits_{j=0}^{k+1}{N}_j\right)+1}\right)\kern2.12em \cdots \kern2em G\left({a}_L,{b}_L,{x}_{\left(\sum \limits_{j=0}^k{N}_j\right)+1}\right)\end{array}\right]}_{N_{k+1}\times L}\kern1em $$

(9)

• (7) Set ${T}_{k+1}=\left[{t}_{\left(\sum \limits_{j=0}^k{N}_j\right)+1},\cdots, {t}_{\left(\sum \limits_{j=0}^{k+1}{N}_j\right)+1}\right]$,

$$ {\displaystyle \begin{array}{l}{K}_{k+1}={K}_k+{H}_{k+1}^T{H}_{k+1}\\ {}{\beta}^{k+1}={\beta}^k+{K}_{k+1}^{-1}{H}_{k+1}^T\left({T}_{k+1}-{H}_{k+1}{\beta}^{(k)}\right)\end{array}} $$

(10)

From Eq. 10, it is found that ${K}_{k+1}^{-1}$ is used to compute β^(k + 1). To avoid calculating the inverse in a recursive process, the Woodbury formula is applied to derive the equations:

$$ {K}_{k+1}^{-1}={K}_k^{-1}-{K}_k^{-1}{H}_{k+1}^T{\left(I+{H}_{k+1}{K}_k^{-1}{H}_{k+1}^T\right)}^{-1}{H}_{k+1}{K}_k^{-1} $$

(11)

Set ${P}_{k+1}={K}_{k+1}^{-1}$; then, the equation for updating ${P}_{k+1}={K}_{k+1}^{-1}$ can be written as:

$$ {\displaystyle \begin{array}{l}{P}_{k+1}={P}_k-{P}_k{H}_{k+1}^T{\left(I+{H}_{k+1}{P}_k{H}_{k+1}^T\right)}^{-1}{H}_{k+1}{P}_k\\ {}{\beta}^{\left(k+1\right)}={\beta}^{(k)}+{P}_{k+1}{H}_{k+1}^T\left({T}_{k+1}-{H}_{k+1}{\beta}^{(k)}\right)\end{array}} $$

(12)

• (8) Set k = k + 1; go back to (7) to train the next sample. After training all the samples, the OS-ELM algorithm can predict the backside width w_bi based on the keyhole feature vector x_i and RBF parameters (a_i and b_i), which as presented [36]:

$$ {w}_{bi}=f\left({x}_i\right)={\sum}_{i=1}^L{\beta}_iG\left({a}_i,{b}_i,{x}_i\right) $$

(13)

4.2 OS-ELM-based penetration prediction system

In this section, Fig. 10 shows a flowchart of OS-ELM-based in situ penetration prediction system for VPPAW process. It mainly includes four parts, namely, (1) a real-time visual-based monitoring system; (2) a SDM-based keyhole characteristic parameter acquisition module; (3) an OS-ELM-based regressive prediction model construction; (4) an in situ welding penetration prediction output model.

Fig. 10

The flow diagram of the OS-ELM-based penetration estimation system

4.2.1 Data acquisition

The real-time visual-based monitoring system used the CCD camera to capture the backside keyhole images to reflect the VPPAW penetration status as presented in Section 3. Then, we obtained the keyhole features based on SDM approach and combined with backside widths to compose a keyhole dataset D_i = {X_i,W_bi}, where X_i = {K_li, K_wi, K_ri, K_ai} is the keyhole features of the ith sample, and W_bi refers to the corresponding backside width (off-line measured using a structure-light vision system after welding). Further, the total dataset was randomly split into two sub-datasets which contain a training dataset D_Train and a testing dataset D_test in a ratio of 7:3. Then, D_Train was applied to train the OS-ELM-based regressive prediction model through its initialization and sequential learning phases, and D_test was employed to predict the weld backside width.

4.2.2 Data pre-processing

Before establishing the OS-ELM-based model, to ensure all of the keyhole features vector and backside width have the same contribution for training and testing datasets, all values from the VPPAW process should be linearly normalized between [− 1, 1] by the following formula:

$$ y=\frac{2\ast \left(x-{x}_{\mathrm{min}}\right)}{x_{\mathrm{max}}-{x}_{\mathrm{min}}}-1 $$

(14)

where x_min and x_max denote the maximum and minimum values of the original dataset, and y refers to the new dataset after normalizing.

4.3 Performance evaluation and result discussion

4.3.1 Experiment scenario

In this section, a nonlinear OS-ELM model was used to develop an in situ welding penetration monitoring system for predicting the welding backside width. To avoid overly complicated process and consider the impact of historical values on current unknown values, the input variables of dynamic OS-ELM model were adopted as K_l (t − i), K_w (t − i), K_r (t − i), K_a (t − i), and W_b (t − j), and the output variable was backside width W_b (t), where t denotes the current moment, i = 0, 1, 2, j = 1, 2. Its prediction accuracy/generalization performance can be evaluated using root mean square error (RMSE) as follows:

$$ RMSE=\sqrt{\sum \limits_{i=1}^N{\left({\hat{w}}_b(i)-{w}_b(i)\right)}^2/N} $$

(15)

where ${\hat{w}}_b(i)$ is the predicted backside width at instant i, w_b (i) is the measured backside width at instant k, and N is the number of data points.

4.3.2 Model parameter selection

For the proposed OS-ELM-based prediction model, the number of hidden layer nodes L is a unique parameter to be manually selected; hence, we will vary the value of L from 5 to 50 with an interval of 5. The optimal parameter was thus chosen based on the lowest RMSE value. Additionally, the number of initial training data N₀ was empirically set to N₀ = L + 50 according to [33]. For the OS-ELM with the Gaussian RBF activation function G(a, b, x) = exp(−x − a²/b), the centers and the impact width were randomly selected from the range [− 1, 1] and [0.1, 3.5] respectively. In our experimental simulations as mentioned before, the 1178 dataset samples were randomly divided into a training dataset D_Train with 824 samples and a testing dataset D_test with 354 samples. The performance comparison of different hidden layer nodes L for training and testing datasets was presented in Fig. 11. When the hidden layer nodes L exceeded 25, the performance of OS-ELM on the training and testing datasets remained stable. Thus, we selected the hidden layer nodes L as 30 for the following experimental simulations.

Fig. 11

The RMSE values for different numbers of hidden nodes

4.3.3 The performance comparison of different algorithms

To demonstrate the superiority of OS-ELM model in predicting the backside width, the performance of OS-ELM-based penetration identification system was compared with other popular algorithms which include BP neural networks (BPNN), support vector regression (SVR), and extreme learning machine (ELM). To make this a fair comparison, we first selected their corresponding best performance parameters for each algorithm by computing RMSE values, which were averaged over 30 trials. In addition, the average training/testing time and RMSE for predicting the backside weld width were summarized in Table 3. It can be seen that the training and testing time taken by OS-ELM is much less than that by ELM, SVR, and BPNN. However, out of all learning algorithms, OS-ELM can obtain the lowest testing RMSE for predicting the weld width. The comparison results demonstrate that the proposed OS-ELM is significantly superior to the other state-of-the-art algorithms in terms of predicting accuracy and calculating time. This is mainly because OS-ELM is trained with the initial samples, and then, newly arrived samples could be directly input to model without the previous trained data. Moreover, during the sequential learning phase, the newly samples can real-time update the previous trained network parameters (include H and β) for the next stage training, which will greatly strengthen the generalization capacity of OS-ELM algorithm.

Table 3 Performance comparison between OS-ELM and other algorithms

4.3.4 Online OS-ELM model validation for VPPAW process

To further evaluate the performance of the dynamic OS-ELM model, the developed SDM-based algorithm for keyhole image processing has been applied to construct an in situ penetration monitoring system for predicting the backside bead width. As depicted in Fig. 12, the welding current and gas flow rates were firstly randomly varied to achieve different joint penetrations, and the keyhole features were obtained by using SDM algorithm. Secondly, we will use the optimal parameter (L = 30, N₀ = 80) of the OS-ELM model to predict the backside weld width which was shown in Fig. 13. Finally, the weld penetration status is estimated by predicting the weld width according to the evaluation criterion (see Table 2). The calculation of the proposed online OS-ELM model validation for VPPAW process is operated in less than 35 ms on a 2.8-GHz Intel Core i7 processor. Therefore, the monitoring and estimation of the weld penetration status can be well implemented online.

Fig. 12

The schematic of in situ VPPAW penetration monitoring system

Fig. 13

Online OS-ELM predicting model verification experiment. a The acquired keyhole parameters based on SDM algorithm. b Online estimated backside width and the measured backside width

The future research will investigate the effects of keyhole behaviors on the welding penetration in different thickness (3–8 mm), and prove the feasibility and robustness of the OS-ELM model for predicting the welding penetration under different dimensions and types of the workpiece.

5 Conclusion

This paper developed an innovative in situ weld penetration prediction framework based on the keyhole dynamic behaviors and OS-ELM model to monitor the nonlinear and time-varying VPPAW process. The conclusions can be drawn as follows:

• (1) The SDM-based image processing algorithm can accurately characterize the keyhole behavior in describing the visual geometrical appearance, which can give a good description of the variation trend of resultant weld penetration.

• (2) The correlation between the keyhole features (including keyhole length, width, area, and length-to-width ratio) and the weld penetration under different welding conditions revealed inherent nonlinear and time-varying characteristics of VPPAW process.

• (3) The proposed OS-ELM model can precisely track the dynamics of the backside bead width based on the keyhole behaviors, thereby realizing in situ prediction of the weld penetration despite the varying VPPAW pools. The experimental results further showed that the proposed OS-ELM approach improved performance and generalization over the other state-of-the-art approaches such as ELM, BPNN, and SVR in terms of accuracy, efficiency, and robustness.