Abstract
Laser vision-sensing technologies are the most widely used to detect weld seam profiles during the intelligentized robotic welding process (IRWP) with thick steel plates, in which the weld seam profile extraction technology plays a crucial role for guiding the welding torch in real time. This paper presents an effective method to extract the weld seam profile from the intense arc background. To emphasize the weld seam profile in images and produce saliency maps at the initial stage, a top-down visual attention model is proposed using the target-driven characteristics of the weld seam profile and splashes. Due to the interference data surviving in the saliency map, a visual attention–based strategy is suggested to gradually discern the larger segments of the weld seam profile through local competition of dynamic saliency based on clustering results. For ineffective weld seam profile extraction resulting from empirical parameters used in the weld seam profile extraction process, the exponentially weighted moving average (EWMA) control chart is employed to implement fault detection and diagnosis (FDD) by monitoring irregular changes of slopes of the extracted weld seam profile. In the final stage, a novel step is arranged to retrieve the possible loss of the weld seam profile. Using the proposed method, validations are carried out using the welding experiments with T-joints and butt joints. Experimental results show that the ratio of successful extraction is over 97% and more stable welding processes with better welds are obtained. This method lays a good foundation for the general weld seam profile extraction process and shows a potential industrial application to the IRWP.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Intelligentized robotic welding systems are one of the hot research branches in the welding field [1]. To realize the real automatic welding process, a great number of operations must be implemented such as weld quality monitoring, seam tracking, and decision making of welding process parameters [2, 3]. Many types of signals produced in the intelligentized robotic welding process (IRWP) can be used for implementing the above operations. Of these signals, visual information is the most widely used [4, 5], and structured light-based laser vision-sensing technologies are commonly used for robotic thick plate welding [6,7,8] to detect the weld seam profile. With the identified feature points of the extracted weld seam profile, the welding torch can be positioned for each pass and the real-time deviation of the welding torch can be eliminated with control algorithms. Therefore, weld seam profile extraction is a crucial step for subsequent operations. However, there are various adverse factors that influence image acquisition and result in the interference data surrounding the weld seam profile such as fumes, arc light, and splashes [9]. In addition, it is true that the larger depth of the groove of the workpiece the more inhomogeneous weld seam profile in terms of intensity, and the weld seam profile that crosses the groove is relatively weak. These are obstacles to integrally extracting the entire weld seam profile for each sampling time. Effective algorithms that can detect more details of the weld seam profile with higher ratios of effective extraction are continuously sought by researchers.
Extensive research concentrates on algorithms or schemes of weld seam profile extraction. These methods can be summarized in three types: traditional, visual attention–based, and target tracking methods. The related studies are reviewed as follows.
Traditional methods mean that some classic algorithms such as various filtering algorithms for denoising are used in the extraction process. In addition, searching algorithms for thresholding also belong to this category. For the denoising on the FPGA-based processing result, Pritschow et al. [10] presented a 2D high-pass filter to enhance edges and discontinuities in pixel intensity, and designed four special filter masks for detecting the weld seam profile. Moon et al. [11] eliminated noise caused by open arc and surroundings and used a pattern matching algorithm to detect the center position of the weld seam profile. To implement the multipass welding process based on visual sensing, a robust algorithm of weld seam profile extraction is proposed by Gu et al. [12], which consists of median filtering, thresholding, denoising, thinning, and curve fitting. Since the laser beam is strongly reflected from smooth metal surfaces, an intensity maximum searching algorithm is proposed to extract the weld seam profile in the region of interest by Nguyen and Lee [13]. A multiple peak detection algorithm similar to intensity-based searching methods for weld seam profile extraction is proposed by Chen et al. [14] to overcome arc interference in the high strength reflection area. An adaptive convolution procedure by Yin et al. [15] is presented, in which the weld seam profile center extraction method based on the geometry information and correlation is implemented, and followed with piecewise fitting to acquire seam profile centerlines. Traditional methods for weld seam profile extraction are relatively easy to be implemented and work effectively in most cases when arc light is kept out or the depth of the groove is relatively small.
Since the regular arc region contributes to positioning the welding torch in real time [16], it is useful except a barrier to weld seam profile extraction. Recently, visual attention models developed from visual attention mechanism are proposed for rapid scene analysis and image classification using saliency, which can also extract objects of interest from the complex background. Zhou et al. [17] proposed saliency detection to locate the weld pool region for weld seam extraction in the pipeline welding process. Gharsallah and Braiek [18] exploited an off-center saliency map as a feature to represent image pixels and realize weld defect inspection using radiography images. Li et al. [19] presented a visual saliency model based on local contrast to emphasize weld seam candidates for weld seam detection. Visual saliency evoked from this mechanism is also presented to emphasize the weld seam profile from the arc background [20, 21]. More complicated visual attention models investigated in a bottom-up (scene-dependent) manner are proposed to augment the saliency of the weld seam profile against the arc region [22, 23]. Because of the weld pool, back weld pool, and the seam in the same image, a visual attention–based method is also adopted to detect the small areas related to the weld pool by Yan et al. [24]. To eliminate heavy noise, Gong et al. [25] developed a multilayer hierarchy vision processing architecture integrated with bottom-up and top-down inference algorithms, in which the bottom-up process acts as a basic information collector while the top-down process performs acts as a missing data hunter. Visual attention–based methods use saliency to distinguish the weld seam profile from the arc background, and one defect of these methods is that interference with significant orientation characteristics (e.g., splashes) is simultaneously strengthened.
Currently, target tracking algorithms derived from visual attention are developed to track targets in images or videos. Classic target tracking algorithms use the correlation between adjacent images to determine the optical flow pattern. To extract the weld seam profile, a seam tracking system based on the morphological image processing method and a continuous convolution operator tracker (CCOT) algorithm are presented by [26]. These methods are also used to directly track an appointed feature point of the weld seam profile [27]. However, these algorithms cannot accurately position the entire weld seam profile because the profile possesses large spans in images. These methods are improper to track the feature point of the weld seam profile because the necessary tracking point changes with the position of the pass.
The weld seam image using active vision can be divided into two categories: the first category is that only the weld seam profile is in the image whereas the arc region is kept out, and the second one contains both. There are no algorithms that are fit for all applications to weld seam profile extraction, and the existing methods do not arrange a remedy for ineffective extraction. Note that empirical parameters account for ineffective extraction in most cases.
The study in this paper concentrates on two aims. The first one is striving to develop a general method to extract the weld seam profile for thick plate welding with the typical T-joints and butt joints, and the second one is attempting to build a fault detection and diagnosis (FDD) mechanism for weld seam profile extraction with a higher probability of successful extraction. To achieve the two targets, a method using top-down (task-dependent) visual attention model is first suggested, which fully uses the visual imaging characteristics of the weld seam profile and splashes to augment the saliency of the former. A dynamic saliency-based strategy is then presented to gradually discern the larger pieces of the weld seam profile in size through local competition at the initial stage. In the second stage, the exponentially weighted moving average (EWMA) control chart is investigated to overcome the disadvantage of the proposed method resulting from some empirical parameters used in this process and implement FDD.
2 Analysis of imaging characteristics of weld seam profiles and splashes
The spectrum generated in the metal active gas (MAG) arc welding process shows that the intensity of arc light is lower when its wavelength is from 630 to 660 nm. To maximize the intensity of the laser stripe (weld seam profile) as well as weaken arc light, the structured light-based laser whose central wavelength is about 660 nm is usually adopted. The weld seam profile is relatively weak when the entire arc region is not kept out (see Fig. 1a–c). On the contrary, using appropriate dimmer glass and optical filters, the weld seam profile is captured with higher intensity when the arc region is concealed (see Fig. 1d). The welding system and the visual sensing system refer to [23] and the imaging characteristic analysis of the weld seam profile consult [28].
There are four characteristics of weld seam profiles and splashes that are used as target-driven factors to enhance the saliency of the former. The first characteristic is that the weld seam profile usually covers larger space than splashes do in the horizontal direction. The second one is that the thickness of the weld seam profile in the vertical direction is nearly a constant (about 5 pixels on average) whereas the thickness of the splash is not uniform. The third is that splashes usually cross the weld seam profile rather than are parallel to it with higher probabilities. In addition, the intensity of the weld seam profile in images with the arc region is weaker than that of the weld seam profile in images without the arc region because a dimmer glass must be used to acquire the more regular shape of the arc region (more regular shape can more accurately position the welding torch).
3 The proposed top-down visual attention model
Classic computational models of human visual search have embraced the idea of a saliency map to accomplish preattentive selection [29], which are mainly based on bottom-up visual attention and mostly implemented for scene analysis with a generated saliency map. For accurate weld seam profile extraction, these models are not competent to such task, and the test results using three classic models are given in this section. Meanwhile, since our previously presented models are mainly based on data-driven, they are sensitive to splashes. However, top-down visual attention models can strengthen the weld seam profile along with weakening interference as possible. In this paper, a top-down visual attention model is developed with these characteristics described in Section 2, as shown in Fig. 2.
3.1 Orientation feature detection
Compared with the intensity of the weld seam profile, its orientation characteristics are more salient than the arc region’s. Therefore, orientation feature detection is mainly considered in the proposed model. Gabor filters are used to produce orientation features (Offline tests determine the appropriate filtering angles). Note that two sets of filtering angles are used for different thickness of the workpiece, namely {5°, −5°} for less than 30 mm and {±5°, 80°} for over 30 mm. Orientation feature detection results with different filtering angles are linearly integrated into the final orientation feature map.
3.2 Denoising
Because of intense electromagnetic disturbance occurring during the welding process, the background of the captured image is full of different noise, which always survives after thresholding is carried out. In this study, it is removed as [12] first.
3.3 Saliency map
Since the weld seam profile is usually divided into several pieces after thresholding, the nearest neighbor clustering is used to gather the data points as clusters. There are three metric methods that use imaging characteristics of weld seam profiles and splashes to further strengthen the saliency of the weld seam profile. The data points belonging to the same cluster have the same saliency. The first metric is called length metric, which is described as
where Li is the maximum Euclidean distance between the two points of the ith cluster, n is the number of the cluster, and \( \overset{\sim }{L_i} \) is length saliency. This metric effectively enlarges the saliency of the pieces of the weld seam profile with larger lengths whereas fully lowers the saliency of the interference data points. Width metric is the second metric method, which intentionally uses the thickness (this statistical parameter is referred to as Tn) of the weld seam profile to further strengthen the weld seam profile as well as weaken the interference data points:
where Ti represents the average thickness of the ith cluster. Moreover, width uniformity metric is developed to measure the saliency of each cluster via variance \( \frac{1}{m_i}\sum \limits_{j=1}^{m_i}{\left({T}_{i,j}-\overline{T_i}\right)}^2 \):
where mi is the column number of the ith cluster, Ti, j is the thickness of the jth column of the ith cluster, and \( \overline{T_i} \) is the average thickness of all clusters. The three saliency maps are normalized and summed into the final global saliency map S like [30]:
where Ν(⋅) is a normalization operator, with which \( \overset{\sim }{L_i} \), \( \overset{\sim }{T_i} \), and \( \overset{\sim }{V_i} \) are normalized to 0–255, respectively. To show the effectiveness of the proposed method in this paper, three classic bottom-up based models are tested as shown in Fig. 3.
Figure 3 shows these classic models are not fit to accurately enhance the saliency of the long and narrow weld seam profile, whereas the top-down–based method proposed in this paper can intentionally highlight the weld seam profile against the interference background.
4 Strategy of weld seam profile extraction via local saliency competition
It is unrealistic to extract the weld seam profile at once because the orientation saliency of the weld seam profile is different in various regions. Piecewise extraction is a solution to this problem. Using the generated saliency map and combing the visual attention process by our eyes, the pieces of the weld seam profile is extracted gradually. The related strategy is presented through local saliency competition containing two steps: discerning the main parts of the weld seam profile and determining its details (see Fig. 4).
It is crucial to correctly discriminate the first piece of the weld seam profile. The rule of determining the first piece is selecting the first cluster (from left to right) that is strengthened with all three metric methods.
4.1 Candidate clusters
Since only one cluster is judged as part of the weld seam profile from candidate clusters in each discrimination loop, the rule of determining candidate clusters needs to be instituted. Candidate clusters are from the clusters whose data points are on the right of the last discerned piece in each loop. If the number of these clusters that satisfy the above requirement in each loop is Tempnum, the number of candidate clusters NCC is set as follows: NCC = 3 if Tempnum ≥ 3, NCC = 2 if Tempnum = 2, and NCC = 1 if Tempnum = 1.
4.2 Competition using local saliency
In each loop, saliency is still used to implement segment extraction of the weld seam profile. The saliency of each candidate cluster is dynamic, which is described as:
where j(j = 1, 2, ⋯, NCC) is the number of candidate clusters in the tth loop, Dt, j is the smallest Euclidean distance between the last discerned cluster and the jth candidate cluster, Lt, j is the maximum distance between two points of the jth candidate cluster, and St, j is the global saliency of the jth candidate cluster produced with Eq. (4). The candidate cluster with the maximum dynamic saliency \( {S}_{t,j}^{\hbox{'}} \) is picked in each loop.
4.3 End rule of discerning weld seam profile
The end rule of discerning the weld seam profile is Cp = Cl (see Fig. 4 and p is the subscript used in the loop). The rule does not work when larger splashes lie on the right of the entire laser stripe. The FDD method proposed in this paper accounts for this case. The saliency map with bigger V-groove in Fig. 3 is used to show the extraction process (Fig. 5).
To show the effectiveness of the proposed method in this paper, our two published methods are applied as shown in Fig. 3a. The extraction results in Fig. 6 indicate that the proposed one here is closer to a general solution to different weld seam profile extraction than the previous methods and can extract more details of the weld seam profile.
5 EWMA-based FDD for weld seam profile extraction process
Due to process parameters used in the proposed method, it is necessary to respond to their ineffectiveness during the welding process. The goal of FDD in this paper is to ensure higher probabilities of weld seam profile extraction.
The slopes of the effectively extracted weld seam profiles of different passes with T-joints and butt joints are statistically analyzed, and their three characteristics are revealed. Firstly, the slopes of the initial part of the weld seam profile are nearly a constant 0 (butt joints) or a negative constant (T-joints). Secondly, for the butt joint, the slopes change only once from positive to negative or from negative to positive with relatively big variations except in the case of the backing weld. In addition, the slopes of the final part of the weld seam profile are also nearly a constant 0 (butt joints) or a positive constant (T-joints). These statistical characteristics are shown in Fig. 7, in which the slopes are calculated using Eq. (6).
where j represents the data point and y(⋅) and x(⋅) are two coordinates of data points in images, respectively. Each slope is determined with 8 adjacent data points. Note that the points used to calculate slopes are from linear weld seam profiles like the images in Fig. 7. Figure 7 shows that slopes of the weld seam profiles of different passes gradually change, and the changes of the monotonous intervals of slopes are generally small apart from the backing weld. The FDD method proposed in this paper is based on these characteristics, and the structure of the proposed FDD method is shown as Fig. 8.
5.1 Performance indexes and fault detection rules
In this paper, three performance indexes are defined to monitor whether the extracted weld seam profile is effective for T-joints and butt joints. The first performance index is the mean of the first 50 slopes using the data points of the first cluster \( \overset{\sim }{C_1} \) whose number of data points is over 50, which is referred to as AVSF. The second index is defined with the mean of the last 50 slopes using the data points of the last cluster \( \overset{\sim }{C_l} \) and referred to as ASLF. The third one is the biggest length of the monotonous intervals of slopes called BLMI, in which the first slope and the last one must be contrary sign and they both deviate from 0 to at least 0.3. Table 1 gives these definitions.
For butt joints, the first and the last identified clusters contain the horizontal parts of the weld seam profile. Therefore, the benchmarks of performance indexes I and II are both set to 0 to monitor ineffective extraction. Performance index III is used to monitor ineffective extraction with the interference data for the middle extraction process and set to 2.0. For T-joints, the benchmarks of performance indexes I, II, and III are 0.4, 0.7, and 2.0, respectively.
The general diagnosis rule for Tn is that it should be narrowed down when performance index I and/or II deviate(s) from the corresponding benchmark(s). The rule for NCC is that it should be increased if performance III deviates from its benchmark. The rules are set for both butt joints and T-joints. Tables 2 and 3 list the detailed rules, in which “↑” and “↓” indicate the changes of the performance index compared with its benchmark. The sign “―” indicates no change. Moreover, “∨” is the logic OR operation.
It must be noted that the fault occurring in the middle of the extraction process is directly detected through large changes of slopes rather than EWMA because the change in this case is so much.
5.2 Fault detection using EWMA
The EWMA method was originally presented by Roberts [33] for detecting small shifts described as Eq. (7). Thomson et al. [34] introduced EWMA control charts for the early detection of fouling of the heat recovery system. Zhao et al. [35] used the EWMA control chart to detect faults in a statistical way to improve the ratios of correctly detected points for centrifugal chillers. In this study, the EWMA control chart is used to detect the deviations of slopes of the designated part of the weld seam profile from the benchmarks.
where \( \lambda \left(0<\lambda \le 1\right) \) is a constant weighting factor for determining the importance of the current group mean \( \overline{X_i} \) (the mean of ith sample group which has constant size n) and Zi is the ith EWMA value. Z0 = μ0 is the expected value (namely the corresponding benchmark). The control limits used in the EWMA control chart are calculated using Eqs. (8) and (9):
where UCL and LCL are the upper and lower control limit respectively; L is the coefficient of the control limit; σ is the standard deviation of samples. n is set to 50. L and λ are influenced with average run lengths (ARL) when monitoring targets are in control and out of control. Statistical results suggest that λ should be small, which contributes to detecting tiny shifts with better ARL. L and λ depend on σ and the desired UCL/LCL.
6 Validation of the proposed FDD method
For butt joints, UCL is set to 0.23 for AVSF and ASLF using the characteristics of slopes of splashes. L is assigned to 75 when λ is 0.5 and σ is 0.04. To validate the proposed FDD method, 186 images are tested offline and the results show that the ratio of effective weld seam profile extraction increases from 92.4 to 98.6% (see example in Fig. 9).
For T-joints the test of the third pass is conducted, in which L is set to 71 and UCL is set to 0.8 when λ is 0.5 and σ is 0.025. In this case, faults usually occur in the middle and end stages of the weld seam profile extraction process. With the proposed FDD method, the ratio of successful extraction rises from 91.8 to 97.3%. Another welding test is done to show the influence of the FDD process on welds. Figure 10 illustrates two typical diagnosis processes and gives two welding results, which shows that the welding process with the proposed FDD in this paper is more stable than the process without this FDD process.
7 Discussion
In this paper, we investigate a FDD method based on EWMA for improving the success probability of weld seam profile extraction during the traditional MAG welding process with thick steel plates. Our results show that the slopes of the weld seam profile seem to be an effective factor to determine whether the extracted weld seam profile is effective and the EWMA control chart is competent for implementing FDD by monitoring irregular changes of these slopes.
To our knowledge, this is the first FDD method for weld seam profile extraction with slopes in the robotic arc welding field with thick steel plates. This finding is significant for smoothing the welding process and improving welding quality, which offers useful references when laser vision sensing is used and empirical parameters are set in visual information extraction technologies.
To date, various visual sensing systems have been reported and many algorithms have been proposed for weld seam profile extraction. These algorithms [4,5,6] are used to cope with the case in which the thickness of the steel plate is less than 30 mm and the intensity of the laser stripe is high and uniform even in the groove region. It is relatively easy to extract the weld seam profile in this case. However, when the thickness increases to more than 50 mm, the intensity of the laser stripe becomes nonuniform, which brings about the issue described in this paper. Therefore, the segmental extraction process like this paper is an effective solution to weld seam profile extraction. Differing from [21,22,23], the top-down visual attention in this paper is implemented with the binary result of the final global saliency map whereas the previous visual attention methods are designed with the law image. The metric methods in this paper are also different from the previous ones.
Empirical parameters are usually used in various algorithms [12, 15, 21,22,23] for weld seam profile extraction. Different noise and unknown interference may hit the defects of these empirical parameters because the welding process is full of randomness. The FDD process to effectively adjust these parameters to adapt to unknown noise and interference is necessary. Currently, there is a little research that proposes methods to detect the faults because of empirical parameters for weld seam profile extraction during the welding process. We previously used the phenomenon of over proximity of the identified feature points of the extracted weld profile to trigger the FDD process [36]. Using the abnormal change of slopes in this study saves time cost in contrast to the previous one because the step of identifying the feature points of the extracted weld seam profile (using slopes) is not involved.
The method in this paper exposes its defects because of thresholds still used for UCL/LCL, λ, etc. These settings account for the rest percentage of effective weld seam profile extraction. Developing more effective methods and investigating more effective monitoring signals (e.g., the derivative of the slope) are our next research interest. Meanwhile, the corresponding strategy of adjusting the empirical parameters is studied in the future.
8 Conclusions
In conclusion, the proposed target-driven visual attention model combining the EWMA-based FDD method is used to extract weld seam profiles for the robotic MAG welding process with thick steel plates. Conclusions are as follows:
-
1.
The abnormal changes of slopes of the extracted weld profile are a relatively effective monitoring object to determine whether the weld seam profile extraction process is effective.
-
2.
With the proposed FDD method in this paper, the percentage of successful weld seam profile extraction is over 97% for the typical butt joints and T-joints.
-
3.
Using this proposed method, a more stable welding process with better welds is acquired.
-
4.
The proposed method in this work is nearly a general solution to weld seam profile extraction using the traditional welding methods to join thick steel plates with laser vision-sensing technologies. The method here shows a potential industrial application to the field of intelligentized robotic arc welding.
References
Chen SB, Lv N (2014) Research evolution on intelligentized technologies for arc welding process. J Manuf Process 16(1):109–122
Teimouri R, Baseri H (2015) Forward and backward predictions of the friction stir welding parameters using fuzzy-artificial bee colony-imperialist competitive algorithm systems. J Intell Manuf 26(2):307–319
Pashazadeh H, Gheisari Y, Hamedi M (2016) Statistical modeling and optimization of resistance spot welding process parameters using neural networks and multi-objective genetic algorithm. J Intell Manuf 27(3):549–559
Xu Y, Fang G, Chen S, Zou JJ, Ye Z (2014) Real-time image processing for vision-based weld seam tracking in robotic GMAW. Int J Adv Manuf Technol 73(9–12):1413–1425
Muhammad J, Altun H, Abo-Serie E (2017) Welding seam profiling techniques based on active vision sensing for intelligent robotic welding. Int J Adv Manuf Technol 88(1–4):127–145
He L, Wu S, Wu C (2017) Robust laser stripe extraction for three-dimensional reconstruction based on a cross-structured light sensor. Appl Opt 56(4):823–832
Zeng J, Chang B, Du D, Wang L, Chang S, Peng G, Wang W (2018) A weld position recognition method based on directional and structured light information fusion in multi-layer/multi-pass welding. Sensors 18(1):129
Kiddee P, Fang Z, Tan M (2016) An automated weld seam tracking system for thick plate using cross mark structured light. Int J Adv Manuf Technol 87(9–12):3589–3603
Yamazaki K, Suzuki R, Shimizu H, Koshiishi F (2012) Spatter and fume reduction in Co2 gas- shielded arc welding by regulated globular transfer. Weld World 56:12–19
Pritschow G, Mueller S, Horber H (2002) Fast and robust image processing for laser stripe-sensors in arc welding automation in industrial electronics, 2002. ISIE 2002. Proceedings of the 2002 IEEE International Symposium on 2
Moon HS, Kim YB, Beattie RJ (2006) Multi sensor data fusion for improving performance and reliability of fully automatic welding system. Int J Adv Manuf Technol 28(3–4):286–293
Gu WP, Xiong ZY, Wan W (2013) Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor. Int J Adv Manuf Technol 69(1–4):451–460
Nguyen HC, Lee BR (2014) Laser-vision-based quality inspection system for small-bead laser welding. Int J Precis Eng Manuf 15(3):415–423
Chen H, Liu W, Huang L, Xing G, Wang M, Sun H (2015) The decoupling visual feature extraction of dynamic three-dimensional V-type seam for gantry welding robot. Int J Adv Manuf Technol 80(9–12):1741–1749
Yin XQ, Tao W, Feng YY, Gao Q, He QZ, Zhao H (2017) Laser stripe extraction method in industrial environments utilizing self-adaptive convolution technique. Appl Opt 56(10):2653–2660
Ye Z, Fang G, Chen S, Zou JJ (2013) Passive vision based seam tracking system for pulse-MAG welding. Int J Adv Manuf Technol 67(9–12):1987–1996
Zhou PY, Li J, Shen NM, Li F (2014) An improved weld seam extraction method using saliency detection for pipe-line welding based on GMAW and passive light. Appl Mech Mater 598:160–163
Gharsallah MB, Braiek EB (2015) Weld inspection based on radiography image segmentation with level set active contour guided off-center saliency map. Adv Mater Sci Eng 2015:1–10
Li N, Wang Z, Xu H, Sun L, Chen G (2016) Weld seam detection based on visual saliency for autonomous welding robots. In: 2016 IEEE workshop on advanced robotics and its social impacts (ARSO). IEEE
He YS, Chen YX, Wu D, Huang YM, Chen SB, Han Y (2015) A detection framework for weld seam profiles based on visual saliency. In: Tarn TJ, Chen SB, Chen XQ (eds) Robotic welding, intelligence and automation. RWIA 2014. Advances in intelligent systems and computing, vol 363. Springer, Cham
He Y, Chen H, Huang Y, Wu D, Chen S (2016) Parameter self-optimizing clustering for autonomous extraction of the weld seam based on orientation saliency in robotic MAG welding. J Intell Robot Syst 83(2):219–237
He Y, Chen Y, Xu Y, Huang Y, Chen S (2016) Autonomous detection of weld seam profiles via a model of saliency-based visual attention for robotic arc welding. J Intell Robot Syst 81(3–4):395–406
He Y, Xu Y, Chen Y, Chen H, Chen S (2016) Weld seam profile detection and feature point extraction for multi-pass route planning based on visual attention model. Robot Comput Integr Manuf 37:251–261
Yan Z, Na L, Huang Y, Chen S (2014) Feature characters extraction with visual attention method based on three-light-path weld pool images. Trans CHN Weld Inst 35(8):53–56
Gong Y, Dai X, Li X (2010) Structured-light based joint recognition using bottom-up and top-down combined visual processing in image analysis and signal processing (IASP) in 2010 International Conference on Image Analysis & Signal Processing, IEEE
Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: learning continuous convolution operators for visual tracking in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)
Zou Y, Chen T (2018) Laser vision seam tracking system based on image processing and continuous convolution operator tracker. Opt Lasers Eng 105:141–149
Li X, Li X, Ge S, Khyam MO, Luo C (2017) Automatic welding seam tracking and identification. IEEE Trans Ind Electron 64:7261–7271
Ramirezmoreno DF, Schwartz O, Ramirezvillegas JF (2013) A saliency-based bottom-up visual attention model for dynamic scenes analysis. Biol Cybern 107(2):141–160
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal 20:1254–1259
Itti L (2005) Models of bottom-up attention and saliency. Neurobiol Attention:576–582
Bernhard S, John P, Thomas H (2007) Graph-based visual saliency. In advances in neural information processing systems 19:Proceedings of the 2006 ConferenceMIT Press 545-552
Roberts SW (1959) Control chart tests based on exponentially weighted moving average. Technometrics 1:239–250
Thomson M, Twigg PM, Majeed BA, Ruck N (2000) Statistical process control based fault detection of CHP units. Control Eng Pract 8:13–20
Zhao Y, Wang S, Xiao F (2013) A statistical fault detection and diagnosis method for centrifugal chillers based on exponentially-weighted moving average control charts and support vector regression. Appl Therm Eng 51:560–572
He Y, Yu Z, Li J, Ma G, Xu Y (2019) Fault correction of algorithm implementation for intelligentized robotic multipass welding process based on finite state machines. Robot Comput Integr Manuf 59:28–35
Funding
This work is financially supported by the National Natural Science Foundation of China under the grant no. 51575349, 51665037, and 51575348, and is partly supported by the State Key Laboratory of Smart Manufacturing for Special Vehicles and Transmission System (GZ2016KF002).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
He, Y., Yu, Z., Li, J. et al. Weld seam profile extraction using top-down visual attention and fault detection and diagnosis via EWMA for the stable robotic welding process. Int J Adv Manuf Technol 104, 3883–3897 (2019). https://doi.org/10.1007/s00170-019-04119-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00170-019-04119-w