Introduction

The packaging is the necessary procedure of LED process components. The purpose of packaging is to protect chip, internal circuit and weld line, isolate steam and dust, and prevent radiation and provide external connection. Using appropriate substrate material for packaging gives the LED component necessary mechanical protection. The heat sinking structure is packaged appropriately to enhance the component reliability and operating life. The optical design of lead frame reflector can lead the light to the required direction effectively to increase the luminous efficiency. As the SMD-LED has small size, wide viewing angle and light weight, it becomes the main packaging mode (PIDA 2011). The defect in the LED package component surface and the weld line form are the key factors influencing the component quality. Therefore, this study discusses the LED package component surface defect detection.

We reviewed the previous literature on automatic optical inspection. Most product defects occur in the packaging stage (Chen et al. 2012; Perng et al. 2014; Hsu and Chen 2016), and automated production is essential for competing in the intensely competitive LED market. Therefore, methods of automating production technology in the packaging stage are critical for LED production. In order to promote LED products quality, Lin and Chiu (2011) applied grey relational analysis to the frequency components in block discrete cosine transform domain (DCT), and attenuate the large-magnitude frequency components. The tiny flaws from uneven illumination backgrounds could be inspected. Lin and Chiu (2013) proposed a blemish detection system for curved LED lenses. A spatial domain image with equal sized blocks was converted to DCT domain and the suspected blemish blocks could be determined by the multivariate statistical method. Perng et al. (2012) employed a machine vision system combining an automatic system-generated inspection regions method to inspect SMD-LED. The operations of normalization and segmentation were applied and could detect the defects include inverse polarity, mouse bites, missing gold wires, and surface stains. Kuo et al. (2015) used an automatic inspection system for LED chip using two stages back-propagation neural network. Such defects include fragment chips, scratch marks and remained gold on the pad area, scratch marks on the luminous zone, and missing luminous zone. Chang (2012) developed an auto optical inspection system for SMD-LED. The defects include mixed-material, resin-tearing and tombstone. The count of gradient operator is used to recognize LED chip, the correlation coefficient between regions of interest and flawless image is selected to recognize, and the geometric relationship was established to calculate erected angle.

According to the aforesaid studies of LED package component detection, there has not been detailed detection of SMD-LED component. Figure 1 shows the SMD-LED structure discussed in this study, size: 3 mm width, 6.5 mm length, including 2 chips, 2 reflectors, 4 wires and phosphor region. In the LED package component detection, the weld line is likely to be influenced by light, resulting in low contrast and difficult identification, and the lead-frame for packaging is produced by stamping, electroplating and injection, there will be dimensional tolerance between LED components in the special domain, increasing the detection difficulty. Figure 2 shows the image without defect and the defect images resulted from four production processes, including missing component, no chip, wire shift and foreign material.

Fig. 1
figure 1

Illustration of the structure of SMD-LED

Full size image
Fig. 2
figure 2

One non-defective and four defective samples: a non-defective sample; b missing component sample; c no chip sample; d wire shift sample; e foreign material

Full size image

In order to overcome the low contrast of weld line and difficult identification, as shown in Fig. 3, this study discusses image enhancement. Celik and Tjahjadi (2011) proposed an algorithm that enhanced the contrast of an input image using interpixel contextual information. The algorithm applied a two-dimensional histogram of the input image constructed using a mutual relationship between each pixel and its neighboring pixels. There will be artifact for histogram value, and it is not objective enough to set the parameters according to human vision result. Celik (2012) presented a two-dimensional histogram equalization algorithm which utilized contextual information around each pixel to enhance the contrast of an input image. The algorithm was based on the observation that the contrast in an image can be improved by increasing the grey-level differences between each pixel and its neighboring pixels. But it must set the size of adjacent domain first, but there is no criterion of the setting, the enhancement effect is subject to man-made influence. Jang et al. (2016) used the lighting condition and minimizing the structural distortion to image contrast enhancement that improved the perceptual visual quality for adaptive contrast enhancement. Kim et al. (2016) presented an entropy-based contrast enhancement method in the wavelet domain. The proposed approach can reveal the details and color information of low-light images without any post-processing. In order to increase the computing efficiency, the concept of the global contrast enhancement mentioned in the above two literatures can’t be applied.

Li and Ren (2012) proposed MLC image enhancement for rail inspection. The definition of MLC was used for contrast conversion and normalization of the pixels with gray level lower than the mean in local area to implement enhancement. The rail defect and the weld line in this study have the same feature, which are lower gray level in local area of image, but the enhancement effect is poor when the background gray level changes largely. Therefore, this study refers to MLC image enhancement and improves it, so as to implement enhancement with greater background gray level changes. The innovative method is to screen the pixels which can represent local area by the mean and standard deviation in local area and the weight is added in the contrast conversion equation to enhance the low contrast weld line.

Fig. 3
figure 3

Low contrast weld line

Full size image

As the dimensional tolerance in lead frame results in difficult detection, this study does not use spatial domain for detection, but uses the concept of spectral domain texture anomaly detection for global foreign material defect detection of SMD-LED component. The spectral domain texture analysis techniques include wavelet transform, Gabor transform and Fourier transform. Tolba (2011) introduced an approach for defect detection in homogeneous flat surface products. The neural network was used probabilistic for defect classification based on the maximum posterior probability of the Log-Gabor based statistical features.

Tsai and Huang (2003) presented a global approach for the automatic inspection of defects in textured surfaces. The proposed method was based on a global image reconstruction scheme using the Fourier transform (FT). By finding an adequate radius in the spectrum space, and setting the frequency components outside the selected circle to zero, the periodic, repetitive patterns of statistical textures could be removed by using the inverse FT.

Aiger and Talbot (2010) presented the phase only transform for unsupervised surface defect detection. The method corresponded to the discrete Fourier transform (DFT), normalized by the magnitude. The PHOT could remove the regularities, at arbitrary scales, from the image while preserving only irregular patterns considered to represent defects. Aiger and Talbot’s method is unsupervised and directly use POFT to each single test image to find the defect. However, it cannot detect defects structured as a scratch or line, which may be seen as regular patterns in a single test image (Bai et al. 2014; Aldea and Le Hégarat-Mascle 2015; Tsang et al. 2016). Bürger and Pauli (2013) proposed an unsupervised segmentation of anomalies in sequential data, images and volumetric data using MPHOT. It was based on the fusion of the PHOT in scale space using only one adaptive sensitivity parameter. The PHOT used the discrete Fourier transform (DFT) to remove the regularities while it detected small defects and pattern boundaries. The proposed multiscale extension allowed the precise segmentation of large anomalies as well.

Referring to the multiscale phase conversion proposed by Bürger and Pauli (2013), the multiscale extension and image fusion techniques are imported into the multiscale adaptive Fourier analysis proposed in this study, the innovative texture anomaly detection method is used for foreign material detection, replacing the present visual inspection operation.

This paper is organized as follows: section one is introduction, section two is approach for SMD-LED defects inspection, section three is experimentation and discussion, and section four is conclusions.

Fig. 4
figure 4

SMD-LED SMD-LED image capture system

Full size image
Fig. 5
figure 5

SMD-LED defect detection process

Full size image

Approach for SMD-LED defects inspection

The overall process of automated optical inspection (AOI) comprises the image capture of hardware part and image processing and analysis of software part, which are introduced in this chapter.

Image capture system and program development

The image capture system of this study comprises camera, lens and light source configuration and PC. The system setup is shown in Fig. 4. The light source is configured by annular blue ray and coaxial blue ray. The line scan camera is combined with 1X lens, and the blue filter is located between the camera and lens. The computer with operating system 32-bit Windows7 operation interface developed by Microsoft and processor Intel (R) Core (TM) i5-3.10GHz as the program development environment, the image processing and analysis programs and user operation interface are composed by MATLABR2008a.

Fig. 6
figure 6

Histogram of image with component and component missing image: a image with component; b histogram of (a); c component missing image; d histogram of (c)

Full size image

SMD-LED defect detection process

The SMD-LED defect detection process is designed according to the features of defects, as shown in Fig. 5, divided into missing component ‘no chip’ wire shift and foreign material parts. The missing component defect is identified according to the histogram characteristics, and then two Normalized Cross Correlations are implemented for template matching to locate the component and solder joint, and the maximum correlation coefficient value during solder joint location is used for identifying no chip. Afterwards, the weld line and foreign material defect detection is implemented, the SMD-LED appearance defect detection is completed. The detection processes are described in this chapter.

Missing component defect inspection

The histograms of image with component and component missing image are calculated respectively, as shown in Fig. 6. The two histograms have significant differences in specific histogram range, especially in gray levels 100–255. Therefore, this study uses the total number of pixels \(\alpha \) of this specific histogram range as rapid sieving analysis index of component and missing component defect recognition. The total number of pixels of specific histogram range is defined as Eq. (1), where \(f_{i}\) is the number of pixels with gray level of i in the test image. If the total number of pixels \(\alpha \) in the specific histogram range of test image is smaller than \(\mathbf{T}_{\mathbf{1}}\), it is identified as missing component defect, otherwise there is component, the detection must be carried on.

$$\begin{aligned} \alpha =\sum _{i=100}^{255} {f_i } \end{aligned}$$
(1)

No chip defect inspection

When the image is confirmed with test component, the Normalized Cross Correlation is used for template matching to locate the SMD-LED component of test image and capture the component region of interest (ROI). The normalized cross-correlation is expressed as Eq. (2), where r(u, v) is the correlation coefficient value, f(x, y) is the test image, \(\bar{f}_{uv} \) is the average value of f(x, y), and the template image T(x, y) is flawless SMD-LED component \(\bar{T}\) is the average of T(x, y). The correlation coefficient of each pixel is calculated. In order to avoid long computing time resulted from huge amount of calculation, this study uses the Fast normalized cross-correlation (FNCC) method proposed by Lewis (1995) to locate component and capture component ROI. Afterwards, the coordinate relative displacement is used, as shown in Fig. 7, to frame the 8 solder joint small area ROIs automatically, the relative coordinates (\(d_{x},d_{y})\) between the prerecorded probable small area ROI (\(x_{sr},y_{sr})\) of solder joint in standard component and the upper left pixel (\(x_{s},y_{s})\) of component are used, as shown in Fig. 7a, and the maximum correlation coefficient coordinates (\(x_{m},y_{m})\) during template matching are used to frame the coordinates (\(x{^\prime }_{sr},y{^\prime }_{sr})\) of solder joint small area ROI automatically, as shown in Fig. 7b. The fast normalized cross-correlation is used for solder joint location in solder joint small area ROI, so as to capture 4 groups of weld line ROI. The maximum correlation coefficient value (\(\beta _{1}\), \(\beta _{2})\) of 4 solder joints’ ROI location in Fig. 8 is used to identify no chip defect. If the maximum correlation coefficient (\(\beta _{1}, \beta _{2})\) of test image is smaller than the threshold (\(\mathbf{T}_{\mathbf{2}},\mathbf{T}_{\mathbf{3}})\), it is identified as no chip defect, otherwise the detection must continue.

$$\begin{aligned} r(u,v)=\frac{\sum \nolimits _{x,y} {[f(x,y)-\bar{{f}}_{uv} ]} [T(x-u,y-v)-\bar{{T}}]}{\sqrt{\sum \nolimits _{x,y} {[f(x,y)-\bar{{f}}_{uv} ]^{2}} [T(x-u,y-v)-\bar{{T}}]^{2}}} \end{aligned}$$
(2)
Fig. 7
figure 7

Schematic diagram of coordinate relative displacement: a standard component; b test component

Full size image
Fig. 8
figure 8

4 solder joint ROIs of recognized no chip defect

Full size image

Wire defect inspection

In order to reduce the amount of computation and image complexity, after the solder joint location of probable small area ROI of 8 solder joints, the ROI images of 4 groups of weld line are captured, the 4 weld line ROI images are processed, including noise filtering, contrast enhancement, entropy of histogram and morphology. Afterwards, the appropriate threshold is selected to segment the weld line. Finally, the weld line defect is judged by the region parallel with solder joints at both ends and at distance of solder joint radius. If the weld line exceeds this region, it is identified as weld line defect, otherwise the detection continues.

Fig. 9
figure 9

Image processing result of weld line ROI: a original weld line ROI image; b noise filtering result; c contrast enhancement result; d binarization result; e upper left solder joint removed; f weld line defect recognition

Full size image

As the SMD-LED manufacturing process needs fluorescent agent, the image forms noise-like structure which influences the definition of weld line in image, as shown in Fig. 9a, influencing the threshold-based weld line segmentation. Therefore, in the ROI image processing procedure of 4 weld lines, the Gaussian low-pass filter and similarity between blocks are used for 3D block matching noise filtering (Dabov et al. 2006, 2007), the result is shown in Fig. 9b. Due to the three-dimensional structure of weld line, partial weld line segments in the image have low contrast or the gray level is close to even higher than other background regions, resulting in difficult identification, as shown in Fig. 3. The weld line cannot be segmented by using global binarization, but the weld line has low gray level in local area of image. The MLC intensification proposed by Li and Ren is an intensity conversion method for the pixels with this feature, but the gray level of background in weld line ROI local area changes significantly, partial background pixels will be intensified as a result. In order to overcome this problem, the local area mask W size is designed according to the direction of change in the weld line ROI image background after noise filtering, as shown in Fig. 10, and the improved Michelson-like contrast is proposed, the average gray level \(\mu _W \) enough to represent local mask corresponding area is calculated for intensity conversion, expressed as Eq. (3). The weight concept is put in intensity conversion to intensify the low contrast weld line, expressed as Eqs. (5) and (6). The improved MLC intensification result is shown in Fig. 9c.

$$\begin{aligned} \mu _{W} =\frac{\sum \limits _{i=1}^{w} \sum \limits _{j=1}^{h} {\alpha _{(x+i-1,y+j-1)}}}{num(\alpha \ne 0)} \end{aligned}$$
(3)

where \(\alpha \) is the matrix of pixel gray level in local area within mean \(\pm 1\) standard deviation, expressed as Eq. (4). \(w\times h\) is the local area mask size, \(num(\alpha \ne 0)\) represents the nonzero pixel number in matrix \(\alpha \), and I is original gray level in local area.

$$\begin{aligned}&\alpha =\left\{ {{\begin{array}{ll} \alpha _{(x+i-1,y+j-1)} \\ =I_{(x+i-1,y+j-1)} &{}\quad if\; \mu _W -\sigma _W<I_{(x+i-1,y+j-1)} \\ &{} \quad < \mu _W +\sigma _W \\ 0&{}\quad {otherwise.}\\ \end{array} }} \right. \nonumber \\&\quad i=1, 2, 3, \ldots ,w \quad ' \quad j=1, 2, 3, \ldots ,h \end{aligned}$$
(4)
$$\begin{aligned}&MLC_{(x,y)} =\left\{ {{\begin{array}{ll} {\frac{I_{(x,y)} -\mu _W }{I_{(x,y)} +\mu _W }\times weight,} &{}\quad {if {I_{(x,y)} <\mu _W } } \\ {0,} &{}\quad { otherwise}. \\ \end{array} }} \right. \end{aligned}$$
(5)
$$\begin{aligned}&{ weight}=\frac{\mu _W -(I_{(x,y)} -\mu _W )}{\mu _W } \nonumber \\&\quad \quad \quad \quad =2-\frac{I_{(x,y)} }{\mu _W } \end{aligned}$$
(6)

Afterwards, the intensified weld line ROI image is segmented by automatically selected threshold, the entropy of histogram E(t) is calculated by Eqs. (7) and (8), as shown in Fig. 11a. According to the distribution characteristic of entropy of histogram, the entropy of histogram is smoothed 100 times by Eq. (9), as shown in Fig. 11b, the maximum value in the local minima set is found out as the optimum segmentation threshold \(t^{{*}}\) for image segmentation and morphology. The upper left block of No. 3 weld line ROI is removed, the weld line segmentation results are shown in Fig. 9d and e. The local minima set is defined as \(E(t)<E(t-1)\) and \(E(t)<E(t+1)\).

$$\begin{aligned} P(t)= & {} \sum _{i=0}^t {p_i } \end{aligned}$$
(7)
$$\begin{aligned} E(t)= & {} -\sum _{i=0}^t {\left( {\frac{p_i }{P(t)}\cdot \ln \frac{p_i }{P(t)}} \right) } \;\;\;\;t\in [0,255] \end{aligned}$$
(8)
$$\begin{aligned} E(t)= & {} \frac{E(t-1)+E(t)+E(t+1)}{3}\;\;\;\;t\in [0,255] \end{aligned}$$
(9)

Finally, the connection between the coordinates of two solder joints of weld line is taken as center line, the distance to the center line is defined as 10 pixels, the weld line defect is judged according to this range. If the segmented weld line exceeds this range, meaning there is weld line defect, as shown in Fig. 9e. The area between the green parallel lines is the weld line defect defining range, and the red pixels are the pixels outside the range.

Fig. 10
figure 10

Schematic diagram of local area mask size

Full size image
Fig. 11
figure 11

Automatic selection of segmentation threshold: a entropy of histogram; b entropy of histogram smoothed 100 times and optimal threshold value \(t^{*}\)

Full size image

Foreign material defect inspection

In foreign material detection, as the lead frame process for packaging results in white shell between lead frames and slight difference in internal electrode, and the weld line position and form are unfixed, this study does not use spatial domain processing for foreign material defect detection of SMD-LED component.

This study uses the concept of detecting texture anomaly for global foreign material detection of SMD-LED component, the foreign material can be regarded as the irregularity of texture. The MAFA proposed in this study is used to eliminate the regularity of texture in SMD-LED component, the abnormal texture is highlighted by threshold segmentation and morphology. The texture anomaly is defined as foreign material defect candidate block in this study, and the foreign material defect block is screened by image masking and the feature that the foreign material defect has gradient value on the edge. Finally, the screened foreign material defect area is calculated to judge foreign material defect component. The overall foreign material defect detection process is shown in Fig. 12. The steps of process are discussed and the detection result is analyzed in this section.

Fig. 12
figure 12

Overall foreign material defect detection process

Full size image

The aforesaid template matching captured component ROI is used as the input image of foreign material defect detection, the MAFA is implemented to eliminate the regularity of SMD-LED component texture. The MAFA combines the multiscale extension and image fusion techniques proposed by Bürger and Pauli with the adaptive Fourier analysis proposed in this study. The MAFA process is shown in Fig. 13.

Fig. 13
figure 13

Multiscale adaptive Fourier analysis process

Full size image

Overusing Laplacian Pyramid Decomposition (LPD) results in image blurring and defect distortion. In order to avoid severe shape and area distortion of defect, the LPD is expressed as Eq. (10), where l is the number of layers of Gaussian Pyramid, w(m, n) is Gaussian filter mask. The first-order and second-order LPD are implemented for ROI image I(x, y) of input component to obtain \(G_1 \) and \(G_2 \), the LPD result is shown in Fig. 14.

$$\begin{aligned} \left\{ {{\begin{array}{ll} {G_0 (x,y)=I(x,y)} &{}\quad if \quad l=0 \\ G_l (x,y)=\sum _m \sum _n w(m,n)G_{l-1}\\ (2x+m,2y+n) &{}\quad {{ otherwise}.} \\ \end{array} }} \right. \end{aligned}$$
(10)

where m and n are image size.

Fig. 14
figure 14

Laplacian Pyramid Decomposition of test image: a\(G_0\); b\(G_1\); c\(G_2\)

Full size image

Afterwards, images \(G_0 \), \(G_1 \) and \(G_2 \) of different scales and their adaptive Fourier masks \(G_0 (u,v)\), \(G_1 (u,v)\) and \(G_2 (u,v)\) are created. Take the adaptive Fourier mask of \(G_0 \) as an example, the Discrete Fourier Transform F(u, v) is implemented first, and the Fourier spectrum \(\left| {F(u,v)} \right| \) is taken, expressed as Eq. (11), the spectrum image of \(G_0 \) is shown in Fig. 15a.

$$\begin{aligned} \left| {F(u,v)} \right| =\left[ {R^{2}(u,v)+I^{2}(u,v)} \right] ^{1/2} \end{aligned}$$
(11)

where (u, v) is the spectral domain coordinate variable, R is the real part of F(u, v), I is the imaginary part of F(u, v).

The spectrum image shows the energy distribution of image in spectral domain, where closer to the spectrum image center represents the low frequency part of image, otherwise high frequency part. This study uses this spectrum characteristic to design Adaptive Duality Fourier Mask, the low frequency energy of image is weakened to eliminate the regularity of texture in the image, so as to increase the defect contrast.

In order to enable the Fourier mask to highlight spectral energy distribution, the logarithmic transformation is implemented by Eq. (12) for the spectrum image. The \(G_0 \) converted spectral distribution is shown in Fig. 15b. Afterwards, the converted spectrum value \(F_{\log } (u,v)\) is normalized by Eq. (13) to 0\(\sim \)1.

$$\begin{aligned} F_{\log } (u,v)= & {} \log (1+\left| {F(u,v)} \right| ) \end{aligned}$$
(12)
$$\begin{aligned} {F}'(u,v)= & {} \frac{F_{\log } (u,v)-\min (F_{\log } (u,v))}{\max (F_{\log } (u,v))-\min (F_{\log } (u,v))} \end{aligned}$$
(13)

In order to attenuate the high spectral energy (low frequency) region and maintain low spectral energy (high frequency) region, the pair conversion is implemented by Eq. (14) for the normalized spectrum value \(F{\prime }(u,v)\). Fig. 15c, d show the Adaptive Fourier mask and screening factor distribution respectively.

$$\begin{aligned} G(u,v)=(1-{F}'(u,v))^{k} \end{aligned}$$
(14)

where k is the conversion factor \(k \quad>\)1, \(k=2.5\) is selected in this study.

Fig. 15
figure 15

Adaptive Fourier mask and coefficient distribution: a Fourier spectrum image of \(G_0 \); b spectral distribution after logarithmic transformation; c, d screening factor distribution

Full size image

Images \(G_0 \), \(G_1 \) and \(G_2 \) of different scales and their Adaptive Fourier masks \(G_0 (u,v)\), \(G_1 (u,v)\) and \(G_2 (u,v)\) are created by repeating Eqs. (11)–(14). The Adaptive Fourier masks and the corresponding Fourier spectrum image dot product are used for spectral domain filtering, the Inverse Fourier Transform is implemented for the filtered image to obtain the filtering results \(M_0 (u,v)\), \(M_1 (u,v)\) and \(M_2 (u,v)\), expressed as Eqs. (15) and (16). The texture regularity is removed by Adaptive Fourier filter proposed in this study, as shown in Fig. 16.

$$\begin{aligned} M(u,v)= & {} G(u,v)\left| {F(u,v)} \right| \end{aligned}$$
(15)
$$\begin{aligned} g(x,y)= & {} \frac{1}{MN}\sum _{u=0}^{M-1} {\sum _{v=0}^{N-1} {M(u,v)e^{j2\pi (ux/M+vy/N)}} } \end{aligned}$$
(16)
Fig. 16
figure 16

Result of texture regularity removed: a\(M_0 (u,v)\); b\(M_1 (u,v)\); c\(M_2 (u,v)\)

Full size image
Fig. 17
figure 17

Foreign material defect detection result

Full size image
Fig. 18
figure 18

GUI of SMD LED inspection system

Full size image

The Mahalanobis distance transform is implemented by Eq. (17) for the filtering results \(M_0 (u,v)\), \(M_1 (u,v)\) and \(M_2 (u,v)\) to highlight the defect for subsequent threshold segmentation, where \(x_1 \), \(\bar{{x}}_1 \) and \(\sigma _1 \) are the gray level average and standard deviation of filtering results respectively, and the MD image is reduced by Laplacian Pyramid reduction to the original image size. Finally, the images \(M{D}'\) after pyramid reduction are given different weights for image fusion to complete MAFA, expressed as Eq. (18). The image fusion result is shown in Fig. 17a. An appropriate segmentation threshold \(T_0 \) and compensation coefficients \(s_{l}^{\gamma } \) shall be selected for image fusion, so as to obtain the scale adaptive threshold \(T(s_l )\), then the fusion index \(\alpha \) is applied to highlight the flaws. The pixel greater than 1 after image fusion is the defect pixel. The optimum segmentation threshold \(T_0 \) is 9 in this study, the segmentation result is shown in Fig. 17b. The threshold selection will be introduced in Sect. 3.1.

$$\begin{aligned} MD= & {} \sqrt{\left( {\frac{(x_1 -\bar{x}_1 )}{\sigma _1 }} \right) ^{2}} \end{aligned}$$
(17)
$$\begin{aligned} O\__{{ fusion}}= & {} \sum _{i=1}^m {\left( {\frac{M{D^{\prime }_{l}} }{T(s_l )}} \right) ^{\alpha }},\;T(s_l )=T_0 \cdot s_{l}^{\gamma } \end{aligned}$$
(18)

After morphological closed operation of segmentation result, the image masking is run to filter the reflective cup and the weld line pixels of the image segmented in Sect. 2.2.3. The image masking of Fig. 14a is shown in Fig. 17c. The white region in the mask is the filtered part, the black region is the reserve part, and the reflective cup position is calculated by coordinate relative displacement, the dark defect in the reflective cup region is detected, merged with image masking filtering result, but the block which is not foreign material still may be segmented. Therefore, the gradient average of edge pixels of each defect is calculated according to the feature that the foreign material defect has obvious change of gradient on the edge, the non-foreign material block is removed, the defect area \(\delta \) is calculated, the block smaller than 35 pixels is removed, the foreign material defect segmentation result is obtained, as shown in Fig. 17d.

Experimentation and discussion

The selection criteria of parameters \(\mathbf{T}_{\mathbf{1}}, \mathbf{T}_{\mathbf{2}},\mathbf{T}_{\mathbf{3}}\) and \(T_{\mathbf{0}}\) for previous experiment will be described in Sect. 3.1. The experimental results are discussed in Sect. 3.2. This study uses MATLAB R2008 as program development software for algorithm and creating graphical user interface. The GUI of SMD LED inspection system is shown in Fig. 18, including the real-time defect detection results and reports of various samples, the user can pick up and view various sample defect images, so as to know the defect condition and adjust process parameters.

Analysis of parameter settings

The SMD LED inspection algorithm proposed in this study contains four parameters which influence the detection result, which are the total pixel number threshold \(\mathbf{T}_{\mathbf{1}}\) of specific gray level range (100–255) for missing component defect recognition as fast sieving index, the template matching correlation coefficient thresholds \(\mathbf{T}_{\mathbf{2}}\) and \(\mathbf{T}_{\mathbf{3}}\) for no chip defect recognition, initial segmentation threshold \(T_0\) of estimated scale adaptive threshold in image fusion of MAFA algorithm. The parameter selection criteria are described in this section.

Determining the threshold values \(T_{1},T_{2}\) and \(T_{3}\)

There are 176 images with component and 52 component missing images for training the fast sieving indicator (total number of pixels in specific gray level range), the fast sieving index distribution is shown in Fig. 19. This study uses statistical three-standard deviation to select the threshold, the average of rapid sieving analysis indicator average with component minus triple standard deviation \(\mu _1 -3\sigma _1 \) and rapid sieving analysis index average of missing component defect plus triple standard deviation \(\mu _2 +3\sigma _2 \) is used as the threshold \(\mathbf{T}_{\mathbf{1}}\) for recognizing missing component defect. The threshold is calculated as Eq. (19).

$$\begin{aligned} \hbox {T}_1 =\frac{\left( {(\mu _1 -3\sigma _1 )+(\mu _2 +3\sigma _2 )} \right) }{2} \end{aligned}$$
(19)
Fig. 19
figure 19

Fast sieving index distribution

Full size image

In the selection of template matching correlation coefficient thresholds \(\mathbf{T}_{\mathbf{2}}\) and \(\mathbf{T}_{\mathbf{3}}\) for no chip defect recognition, there are 39 images without chip and 136 images with chip for threshold training. The maximum correlation coefficient distribution is shown in Figs. 20 and 21, the maximum correlation coefficient average of chip component minus triple standard deviation and maximum correlation coefficient average of no chip component plus triple standard deviation are substituted in Eq. (19) to calculate the threshold, and the black dotted lines are drawn in Figs. 20 and 21.

Fig. 20
figure 20

Maximum correlation coefficient distribution of images with and without chip in the upper two ROIs of Fig. 8

Full size image
Fig. 21
figure 21

Maximum correlation coefficient distribution of images with and without chip in the lower two ROIs of Fig. 8

Full size image
Fig. 22
figure 22

The simulated cross-region dark defect and bright defect are added in flawless sample respectively: a dark defect, b bright defect

Full size image
Fig. 23
figure 23

Segmentation threshold—segmentation area precision curve

Full size image
Fig. 24
figure 24

Detection result comparison: a test image, b PHOT, c MPHOT, d this study

Full size image

Determining the threshold values \(T_{0}\)

In terms of selection of initial segmentation threshold \(T_0 \) for estimating scale adaptive threshold in image fusion of MAFA algorithm, according to the definition of foreign material defect, the drawing software is used to simulate defects in the image without defect, which are smaller than the average gray level of dark background (electrode) by 40 and greater than the average gray level of bright background (fluorescent glue filled region) by 40, as shown in Fig. 22a, b. The precision of error matrix is calculated according to the segmentation result and actual area of simulated defect as the evaluation index of segmentation threshold selection.

The initial segmentation threshold of this study is selected according to the curve formed from the segmentation threshold and precision of defect segmentation area, as shown in Fig. 23. When the threshold is 9, the dark defect or bright defect has the maximum repeatability (99.99, 99.98%). Therefore, this study selects T0 = 9 as segmentation threshold for image fusion

Method verification and comparison

The non-supervisory texture anomaly detection algorithm proposed in literatures and the detection method proposed in this study are used for comparative verification. The PHOT proposed by Aiger and Talbot (2010), MPHOT proposed by Bürger and Pauli (2013) and MAFA proposed in this study are used to eliminate the image texture regularity, and the optimal threshold value is set according to the result of texture regularity elimination and the same image masking is run. The detection results are shown in Fig. 24b–d. It is observed that the PHOT cannot segment large defect completely, and there are some fine speckles, so that the detection is incorrect. The MPHOT is combined with the concept of Multiscale Gaussian Pyramid to improve the incomplete segmentation of PHOT when the defect area is large. The detection result shows the large defect can be segmented, but the defect shape and area are distorted. The defect segmentation results of the MAFA proposed in this study are closest to the actual defect shape and area, and there is no non-defect pixel segmented, meaning the proposed MAFA can segment defect effectively and correctly.

Table 1 The performance of the proposed approach compares with PHOT, MPHOT
Full size table

The performance of the proposed approach compares with other approaches in the literatures are presented. A defect area of 332 pixels in the SMD-LED is chosen for the segmentation of PHOT (Aiger and Talbot 2010), MPHOT (Bürger and Pauli 2013) and this proposed MAFA methods. The accuracy is 99.90, 99.92 and 99.98%, respectively as shown in Table 1. It is calculated from Eq. (20), meaning the MAFA proposed in this study has the best segmentation effect.

$$\begin{aligned} \text { Accuracy rate }= \frac{\textit{Correctly identified pixels}}{\textit{The total number of pixels}} \end{aligned}$$
(20)

The SMD-LED automatic defect detection system developed in this study uses 457 SMD-LED images, including 427 components with defect and 30 components without defect. This study aims at five common and important defects, which are missing component, no chip, wire shift, wire broken and foreign material defects. The forms and features of different defects are analyzed, the overall defect type detection process and image processing technology are designed for detection. The aforesaid trained threshold is used for test, the 449 SMD-LED components are identified correctly, 3 are over judged and 5 are missed. The overall recognition rate of this system is 98.25%, meaning the detection system proposed in this study has good recognition accuracy rate.

Conclusions

The proposed method in this study is used for automatic defect detection. It detects the important defects in SMD-LED package components, including missing component, no chip, wire shift and foreign material. In order to overcome the difficult identification as the weld line is subject to light rays, the improved MLC enhancement is proposed, and the segmentation threshold is selected by entropy information to segment the weld line successfully. In order to overcome the effect of the tolerance of component size and internal electrode and unfixed weld line position resulted from lead frame process on foreign material detection result, the MAFA is proposed in the concept of texture anomaly detection for foreign material defect detection. The result proves that the proposed method can segment the defect effectively and correctly compared with the PHOT and MPHOT. The overall recognition rate of this system is 98.25%, contributing to the large market demand and high component quality of LED industry. The labor cost and misrecognition of manual inspection can be reduced effectively, the product yield and quality are improved. The MAFA proposed is applicable to other texture anomaly detection fields.