1 Introduction

In the milling process, the tool wear has important effects on the processing quality. Excessive tool wear would result in a significant reduction in the quality of the machined surface [1]. The tool damage is one of the main reasons leading to unexpected downtime [2]. The cost of tools and tool replacements accounted for 25% of the total cost [3]. Tool failures accounted for 35% of the total downtime of the milling machine [4] and downtime caused by tool replacement accounted for 20% of the total downtime [5]. In order to reduce the adverse effects of tool wear on processing, the tools need to be replaced before it is damaged. In traditional tool condition monitoring (TCM), the production personnel observe the vibration, noise, and machining time during the machining process based on their own experience to judge the wear status of the tool, which is strongly affected by subjective judgment. If the tool is replaced when the tool wear does not reach the wear standard, the tool is not fully utilized, which will increase the manufacturing cost. In actual processing, only 50–80% of the effective life of the milling cutter is used when changing the tool [6, 7]. If the tool is continued to use when it is severe wear, the accuracy and surface quality of the machined parts would be reduced [8]. In severe cases, safety accidents may even occur. Therefore, exploring a method of automatically monitoring tool wear will reduce processing costs and improve processing quality, which is of great significance.

At present, according to the principle, tool wear monitoring technology can be divided into two categories: indirect method and direct method [9]. The indirect method uses the correlation between related parameters like acoustic emission signals, the cutting force during the machining process, and the tool wear status to indirectly judge the tool wear status. Zhang et al. [10] proposed a method for tool life prediction and wear state evaluation. A new wear model based on the change of wear rate under different milling conditions was established and verified. The results showed that the model has good flexibility. Gomes et al. [11] acquired acoustic emission signals and vibration signals, thus establishing a tool wear monitoring model based on support vector machine to monitor the wear of micro milling cutter. The results showed that this method could effectively classify tool wear. Móricz et al. [12] used spindle power signals and artificial neural network to realize off-line and on-line monitoring of micro milling cutter in machining process of ceramic. The relationship between wear stage and measurement parameters was determined. Zhou et al. [13] developed a wireless rotary vibration measuring tool holder system that could simultaneously measure the vibration signals of the three axes. The superior effectiveness and sensitivity of the system were verified by carrying out milling experiments on Ti6Al4V. Yang et al. [14] proposed a new method to judge tool wear state based on milling force. The milling force was predicted by deducing a theoretical formula based on undeformed chip thickness. The support vector machine with different cores was used to classify tool wear state. The feasibility of the proposed method was verified by the cutting experiment of Ti6Al4V. Zhou et al. [15] proposed a milling tool monitoring method based on two-layer corner core extreme learning machine and binary differential evolution. The method included multiple signals, while it did not require preset function or optimization of hyperparameters. Two milling experiments were carried out and the results showed that the inventive method had low error. By summarizing previous studies, it turns out that the indirect method has the advantage of on-line tool wear monitoring, since signals can be acquired in real time during machining, which could save production time while monitoring tool wear and be able to check sudden damages of the tool. Nevertheless, the indirect method is usually based on more expensive special experimental equipment and the monitoring is easily affected by external factors and experimental conditions. Generally, it is difficult to accurately obtain the value of tool wear, since the tool wear is reflected by signals. Therefore, compared with the direct method, the accuracy of the indirect method is hard to guarantee [16].

With the continuous development of machine vision technology and image processing algorithms, monitoring tool wear through a vision system has the advantages of high detection accuracy, low cost, and less susceptibility to external factors. The direct method of monitoring tool wear has been widely studied and applied. Yu et al. [17] proposed a wear area edge detection method based on morphological component analysis, which reduced the influence of image noise on the extraction of wear edges. Subsequently, based on this method, Zhu and Yu [18] applied a region growing algorithm to extract the wear area of the target image. García-Ordás et al. [19] presented a new approach to categorize the wear of cutting tools based on local binary patterns description over the local texture of tool images. The texture of different wear patches was evaluated by computer vision and machine learning to judge the wear state of the tool. Fong et al. [20] developed a tool wear image measurement system based on cross-correlation analysis. The sensitivity and accuracy of TCM technology were improved by cross-covariance analysis of worn and unworn tool images. The high efficiency of the developed measuring system is proved by analyzing the cutting tools such as the drill bit, end mill, taper tap, and carbide insert. Qin et al. [21] proposed an automatic end-milling cutter monitoring method based on dynamic image sequence, which could obtain the dynamic image sequence of the milling cutter when the spindle rotated. You et al. [22] designed an image-based tool wear monitoring method, which located the wear area by combining homomorphic filtering and histogram comparison, thus segmenting the wear area by GrabCut model. The experimental results showed that this method could accurately judge the degree of wear. Mamledesai et al. [23] realized tool wear monitoring based on computer vision, convolutional neural network, and transfer learning. This method effectively solved the problem of insufficient data in TCM system. Pagani et al. [24] proposed a deep learning method based on image processing, which judged the wear degree of cutting tools with chip color as the feature. However, this method was only suitable for fixed machining processes because of the great influence of workpiece materials and cutting process parameters on chip color. Compared with the indirect method, the direct method can reflect the changes of tool wear more intuitively. It is not susceptible to the influence of factors such as machining parameters and vibration during the machining process. Therefore, it has better robustness and higher detection accuracy. Nevertheless, the direct method usually requires the acquisition of information related to tool wear during machining intervals, which will occupy the production time. In addition, it is hard to realize on-line monitoring because of the insensitivity to sudden damages.

Based on previous research, in order to further improve the automation level of tool wear monitoring, this paper designs an automatic tool wear monitoring system based on the direct method. This system is intended to be the basis of the further TCM system including the direct and indirect methods simultaneously. The methods of automatic acquisition of wear images and automatic extraction of wear areas are explored. In order to verify the effectiveness of the monitoring system in this paper under actual processing conditions, milling experiments of the nickel-based superalloy Inconel 718 were carried out under internal cooling conditions. In order to reflect the tool wear status more realistically, the average wear area of flank is used as a technical index and its reliability to reflect the tool wear is discussed. The relationship between surface texture and tool wear is explored by analyzing the characteristic parameters of the processed surface texture of workpiece.

2 Tool wear analysis

2.1 Tool wear monitoring system design

2.1.1 Tool wear evaluation index

During milling, the tool is in constant contact and friction with the workpiece as well as chips. The resulting high temperature and high pressure lead to the tool coating to peel off during the machining process, thus causing the tool wear. The impact is greater for some difficult-to-machine materials. Kasim et al. [25] found that when milling nickel-based superalloys, the high temperature load attached to the processing area caused severe bonding wear of the tool, which greatly reduced the quality of the processed surface. Wang et al. [26] carried out a nickel-based superalloy milling experiment. The results showed that the temperature of the contact area between the tool and the workpiece was greater than 1000 K. When the tool is worn, the material strength of the processing area is greatly reduced, which affects the processing quality. Therefore, it is very important to select an appropriate standard to evaluate tool wear.

Figure 1 shows the tool wear form in the milling process [9]. Due to the greater rigidity of the workpiece, the flank face is in contact with the workpiece and the wear is severe, so the width of the flank face wear zone VB is usually used as the index for tool wear [27]. When machining nickel-based superalloys, the tool wears quickly. According to the actual situation, the tool wear evaluation standard in this paper is set as follows: when the flank face of tool is regularly worn, the average wear width of the flank VBave shall not exceed 0.25 mm. When the flank face of tool is irregularly worn, the maximum wear width of the flank VBmax shall not exceed 0.5 mm. In addition, since the tool wear area is in a complex mechanical and thermal environment and has irregularities, it is necessary to analyze the characteristics of the flank face wear area to reflect the wear status of the tool more reliably [18]. Therefore, this paper determines that the width of the frank face VBmax and VBave are used as conventional indexes of the degree of tool wear, while the average wear area of flank face AVB is used as an additional index to reflect the tool wear status more truly and comprehensively. In addition, compared with wear width and area, this paper will also explore surface texture of workpiece caused by tool wear.

Fig. 1
figure 1

Wear form of milling tool [9]

Full size image

2.1.2 Tool wear monitoring platform

The tool wear monitoring system is composed of hardware and software. The hardware part realizes the function of effectively acquiring images, while the software part realizes the function of analyzing the acquired wear image and extracting the wear value. The hardware part includes In-Sight 5403 CCD industrial camera, P/N 119-2043R LED ring light source and PD2-5024 light source controller produced by Cognex from USA, CF25HA-1 industrial lens produced by Fujinon from Japan. The software part is Cognex In-Sight explorer and the designed tool wear monitoring software.

This paper establishes the tool wear monitoring platform shown in Fig. 2 to achieve the acquisition and processing of tool wear images under complex working conditions. The working principle of the system is as follows: the camera fixture is set on the worktable of the machine tool. After determining the best shooting angle, keep the relative position of the camera and the tool unchanged. When measuring the tool wear during processing is needed, the tool returns to the original set position. At this time, the spindle is kept rotating at a constant speed of 120 r/min and the camera is triggered to acquire a full frame image. The acquired images are transmitted to the computer through the Ethernet interface for processing and the current tool wear value would be extracted. In order to explore the relationship between surface texture of workpiece and the tool wear, pictures of the surface texture are taken for analysis after the measurement.

Fig. 2
figure 2

Schematic diagram of tool wear monitoring platform

Full size image

2.2 Tool wear image analysis

In order to realize the automatic extraction of the tool flank wear value, it is necessary to process and analyze the tool wear images acquired by the CCD camera. The tool wear image analysis process is shown in Fig. 3.

Fig. 3
figure 3

Tool wear image analysis process

Full size image

2.2.1 Image preprocessing

In order to effectively analyze the wear value, the image for further processing must contain clear and complete tool flank. Due to the uncertainty of the spindle rotation angle at the end of milling processing, it is difficult to realize automatic detection. In order to reduce the detection error caused by the different shooting angles of the tools, this paper proposes to use the structural similarity index (SSIM) to filter the acquired images of the tools, considering the brightness, contrast, and structure factors. The image with the highest similarity to the target image is further processed. The formula for calculating structural similarity is as follows [28]:

$$ l\left(x,y\right)=\frac{2{\mu}_x{\mu}_y+{C}_1}{\mu_x^2+{\mu}_y^2+{C}_1} $$
(1)
$$ c\left(x,y\right)=\frac{2{\sigma}_x{\sigma}_y+{C}_2}{\sigma_x^2+{\sigma}_y^2+{C}_2} $$
(2)
$$ s\left(x,y\right)=\frac{\sigma_{xy}+{C}_3}{\sigma_x{\sigma}_y+{C}_3} $$
(3)
$$ \mathrm{SSIM}\left(x,y\right)=\frac{\left(2{\mu}_x{\mu}_y+{C}_1\right)\left(2{\sigma}_{xy}+{C}_2\right)}{\left({\mu}_x^2+{\mu}_y^2+{C}_1\right)\left({\sigma}_x^2+{\sigma}_y^2+{C}_2\right)} $$
(4)

where l(x,y) is the brightness similarity, c(x,y) is the contrast similarity, s(x,y) is the structure similarity, and μx, μy, σx, σy, and σxy are the mean, standard deviation, and covariance between the two images x and y. C1, C2, and C3 have small values to avoid the situation that the denominator approaches zero when describing the low brightness and contrast area, which leads to unstable measurement results. The value range of SSIM is [−1,1]. If the similarity of the compared images is higher, the value is closer to 1. The image with the highest SSIM value is selected as the image for subsequent processing, as shown in Fig. 5b.

The unprocessed original image contains a lot of information. Since the tool flank wear area is the region of interest (ROI) for tool wear monitoring, the extraction of the wear area is a prerequisite for further processing. The corner points usually reflect important changes in the image, which can reduce the amount of information while retaining important features. In this paper, Harris corner detection is used to detect the corners of the original image. All the detected corners are sorted in the vertical direction. Since the highest corner point of the image is the highest point in the vertical direction of the tool wear area, its neighborhood is taken as the ROI area. The principle of Harris corner detection is as follows: a specific window is moved on the image. When the gray scales in the window changes greatly, the corner point is included in the window. The function for calculating the gray change value of the window, Harris matrix as well as the simplified function after Taylor expansion and ignoring the high-order remainder are as follows:

$$ E\left(u,v\right)=\sum \limits_{x,y}w\left(x,y\right){\left[I\left(x+u,y+v\right)-I\left(x,y\right)\right]}^2 $$
(5)
$$ E\left(u,v\right)\cong \left[u,v\right]M\left[\begin{array}{l}u\\ {}v\end{array}\right] $$
(6)
$$ M=\sum \limits_{x,y}w\left(x,y\right)\left[\begin{array}{cc}{I}_x^2& {I}_x{I}_y\\ {}{I}_x{I}_y& {I}_y^2\end{array}\right] $$
(7)

where w(x,y) is the window function, I(x,y) is the original gray scale, I(x + u,y + v) is the gray scale after window translation of (u,v), and M is the Harris matrix.

This function is a quadratic elliptic function. The size of the ellipse is determined by the eigenvalues of the Harris matrix (λ1,λ2) and the direction is determined by its eigenvectors. The image features are classified according to the eigenvalues of matrix, thus judging the corner point. The corresponding function of the corner point for classification is as follows:

$$ R={\lambda}_1{\lambda}_2-k{\left({\lambda}_1+{\lambda}_2\right)}^2 $$
(8)

where R is the corresponding function of the corner point and k is the empirical function which usually equals to 0.04~0.06.

When R is a large positive number, it is judged that the corner point is detected. By sorting the corner points and extracting the neighborhood of the highest corner point in the vertical direction, the automatic cropping of the ROI is realized, as shown in Fig. 4c.

Fig. 4
figure 4

Image cropping: a standard image, b image with highest SSIM value, c region of interest recognition

Full size image

Due to the interference of the external environment and other factors, the images acquired by the CCD camera often contain a lot of noises [29], as shown in Fig. 5a. These noises will have a great impact on the image quality. In order to improve the quality of images, the acquired images must be denoised. Compared with other denoising algorithms, median filtering has an excellent noise reduction effect. The principle of median filtering is as follows: any pixel (i,j) in the image is taken as the center point. The gray scales of the pixel in its neighborhood are sorted and then the median value is taken as the new gray scale of the point. The median filter with a 7 × 7 mask is selected as the denoising algorithm in this paper. Figure 5 shows the gray scale distribution before and after the median filtering. A superior denoising effect is achieved and the edges of the image are kept intact, so the image would not be too smooth to keep information.

Fig. 5
figure 5

Image gray scale distribution. a Original image. b Image after median filtering

Full size image

In order to highlight the wear area on the image and effectively distinguish it from the irrelevant area, it is necessary to enhance the image. Piecewise linear transformation is an algorithm based on gray scale transformation, which can effectively suppress irrelevant areas and is insensitive to noises, so that wear area could be easily extracted. Three-stage linear gray scale transformation is used and the equation is as follows:

$$ g\left(x,y\right)=\left\{\begin{array}{c}\frac{c}{a}f\left(x,y\right),0\le f\left(x,y\right)<a\\ {}\frac{d-c}{b-a}\left[f\left(x,y\right)-a\right]+c,a\le f\left(x,y\right)<b\\ {}\frac{e-d}{g-b}\left[f\left(x,y\right)-b\right]+d,b\le f\left(x,y\right)<g\end{array}\right. $$
(9)

where f(x,y) is the original gray scale of any pixel, g(x,y) is the gray scale of the pixel after linear transformation, and a, b, c, d, e, and g are constants, which determine the slope of the linear transformation and the stretching range of gray scale transformation. In order to obtain a clear wear edge, the linear transformation parameters are debugged, and the final parameters are taken as follows: a = 100, b = 140, c = 30, d = 100, e = 256, and g = 270.

Figure 6 shows the histograms of grayscale value before and after linear transformation. It can be seen that after linear transformation, the distribution of pixel points is more concentrated, thus effectively highlighting the wear area. Figure 7 shows the effect of median filtering and linear transformation. It can be seen from the processed image that the wear area is clearly distinguished from the irrelevant background, which is benefit for obtaining tool wear value.

Fig. 6
figure 6

The histograms of grayscale value. a Before linear transformation. b After linear transformation

Full size image
Fig. 7
figure 7

Tool wear image. a Original image. b Median filtering. c Linear transformation

Full size image

2.2.2 Threshold segmentation

In order to extract the wear contour boundary of the tool, it is necessary to segment the wear area and background on the image. The choice of threshold determines the effect of image segmentation. The maximum between-class variance method (Otsu) [30] is a threshold segmentation method that calculates the between-class variance of the image and takes the threshold corresponding to the maximum between-class variance as the best threshold. Since the wear area and the background gray scale characteristics are different, the smaller the variance between the classes, the more blurred the information is mixed by the threshold, which is not conducive to extracting the wear area. When the variance between the classes is greater, it indicates that the two are more distinct. The image after the threshold segmentation process is shown in Fig. 9a. It could be seen that the Otsu method could effectively segment the wear area and the background. Nevertheless, it still contains irrelevant areas and requires further processing.

In order to obtain more reliable wear area, the binary morphology is used to process images after using Otsu method. Binary morphology is a method of processing image collections by constructing structural elements, including image erosion and expansion. The image erosion of the image X using the structural element S is denoted as X ⊖ S. If S is still in the set X after being translated by a distance x, the position of the point is recorded and the new set (S)x of points obtained is the result of image erosion. The image expansion of the image X using the structural element S is denoted as X ⊕ S. If the intersection of the new set \( {\left(\hat{S}\right)}_x \) obtained after S is shifted by x and the set X is not an empty set, the position of the point is recorded and the new set of points \( {\left(\hat{S}\right)}_x \) obtained is the result of image expansion. The equations of erosion and expansion are as follows:

$$ X\ominus S=\left\{x|(S)x\subseteq X\right\} $$
(10)
$$ X\oplus S=\left\{x|\left[\left(\hat{S}\right)x\cap X\right]\ne \varnothing \right\} $$
(11)

The opening operation means that the image is first eroded and then expanded to separate the seemingly connected pixels. The closed operation means that the image is first expanded and then eroded to connect the intermittent pixels. The flexible use of opening and closed operations could effectively optimize the wear area and facilitate the extraction of its edges. In addition, in order to eliminate the influence caused by irrelevant areas with a large difference from the wear area, the areas of a small number of pixels accumulated are removed.

Figure 8 shows the changes of the wear image before and after the morphological processing. The white area in the lower right corner of Fig. 8b has been removed after processing, as shown in Fig. 8c. The results show that the morphological processing can effectively eliminate the burrs generated by the Otsu threshold segmentation and make the edges smoother. In addition, the influence of irrelevant regions on edge extraction is also eliminated.

Fig. 8
figure 8

Threshold segmentation. a Otsu method. b Before morphological processing. c After morphological processing

Full size image

2.2.3 Edge detection

After the threshold segmentation process, the pixel grayscale value changes sharply at the tool wear boundary and the wear contour boundary could be extracted. Compared with traditional operators, Canny operator has excellent edge detection ability and good robustness [9].

Canny edge detection steps are as follows: first, the image is convolved with the Gaussian template to smooth the image. Secondly, the gradient magnitude and direction of the tool wear image are calculated. The non-maximum values of the image edge are suppressed and only the gradient maximum points are retained as the candidate edge. Then, the upper and lower bounds of the threshold are set. Keep the non-isolated edges between the bounds and the edges larger than the upper bound of the threshold. Finally, the closed operation is used to smooth the extracted edges. Figure 9 shows the result of edge detection. It can be seen that the wear edge obtained by using Canny operator for edge detection is clear and complete.

Fig. 9
figure 9

Edge detection. a Canny edge detection. b Tool wear edge

Full size image

2.2.4 Wear value extraction

In order to extract the wear value from the edge of the tool wear contour, it is necessary to calibrate the camera pixel equivalent. Firstly, fix the camera’s shooting angle, focal length, and other factors. Then, use the ruler as a reference standard and calculate the ratio of the number of pixels N occupied by the grayscale value on the ruler to the actual scale value D as the pixel equivalent of the camera. The equation of pixel equivalent is as follows:

$$ K=D/N $$
(12)

The row scan and column scan on the tool wear edge are performed. The maximum wear width of the flank VBmax is the maximum number of pixels between the smallest column and the largest column in each row. The average wear width of the flank VBave is the ratio of the total number of pixels in the wear area to the number of rows. In addition, the total number of pixels in the wear area is used to calculate the area of the flank wear AVB. Therefore, the tool wear value could be extracted.

2.3 Surface texture of workpiece analysis

Texture analysis has always been a very challenging direction in machine vision and pattern recognition, so it has been widely studied [19, 31, 32]. The texture refers to a group of basic elements or spatial organization composed of primitive elements, the basic microstructure in natural images and the atoms of human forward-looking visual perception [33]. The texture would obey some statistical properties and the appearance of different texture would be different. Tool wear is directly reflected in the surface topography of the processed workpiece. As shown in Fig. 10, machining by a new tool produces a smooth surface, so that the incident light is regularly reflected, making the texture in the image appear consistent. When the tool is worn, the grooves and ridges of the tool are unevenly distributed as well as the surface of the workpiece becomes rough. The diffuse reflection of incident light is more intense, which means that a rough surface is brighter than a smooth surface. Therefore, this paper will discuss the relationship between machined surface texture and tool wear in milling by using the gray-level co-occurrence matrix method.

Fig. 10
figure 10

Surface texture of workpiece. a Machined by a new tool. b Machined by a worn tool

Full size image

The gray-level co-occurrence matrix method is a method based on statistics. First, for any point (x, y) and another point (x + Δx, y + Δy) next to the former one, set the grayscale value of the pair of points as (f1, f2). Then, the point (x, y) is traversed through the entire image, so different pairs of points (f1, f2) are obtained. Each pair of points is recorded and arranged into a square matrix. Finally, the number of occurrences of (f1, f2) is recorded and normalized to the probability p(f1, f2) and the gray-level co-occurrence matrix is obtained. While the deviation value (Δx, Δy) is different, the gray-level co-occurrence matrix would also be different. When the deviation value (Δx, Δy) of the image is smaller, the grayscale values of two points are generally similar. This phenomenon is reflected in the gray-level co-occurrence matrix as the diagonal of the matrix and the nearby values are larger. For images with rapid texture changes, the value distribution of each element of the matrix is relatively uniform. To be specific, the following parameters are usually used to describe texture features.

2.3.1 Angular second moment (ASM)

This feature is an index to describe the texture thickness of an image. The smaller the ASM value is, the less smooth the region is and the more irregular texture is. On the contrary, when the ASM value is greater, the probability that the pixels have the same gray-level is greater, which means that the image is smoother and the texture is more regular. The equation of ASM is as follows:

$$ ASM=\sum \limits_{f_1}\sum \limits_{f_2}{\left[p\left({f}_1,{f}_2\right)\right]}^2 $$
(13)

2.3.2 Contrast (CON)

Contrast represents the grayscale difference between different grayscale areas of the image. There is a significant difference in contrast between bright and dark areas. For the machined surface image, the surface scratches processed by the new knife are narrow and thin. When the tool is worn or blunt, the scratches would become thicker and wider. Generally speaking, the scratches have a strong ability to refract light, resulting in the low contrast of texture images processed by new tools and the high contrast of texture surfaces processed by worn tools. The equation of CON is as follows:

$$ CON=\sum \limits_{f_1-{f}_2}{\left({f}_1-{f}_2\right)}^2\left[\sum \limits_{f_1}\sum \limits_{f_2}p\left({f}_1,{f}_2\right)\right] $$
(14)

2.3.3 Correlation (COR)

The correlation is embodied in the local grayscale correlation of the texture. If the grayscale value in neighborhood is the same, the processed surface of the workpiece is smoother and the correlation is greater. On the contrary, the correlation is smaller. The equation of COR is as follows:

$$ COR=\frac{\sum \limits_{f_1}\sum \limits_{f_2}{f}_1{f}_2p\left({f}_1,{f}_2\right)-{\mu}_x{\mu}_y}{\sigma_x{\sigma}_y} $$
(15)

where:

$$ \left\{\begin{array}{c}{\mu}_x=\sum \limits_{f_1}{f}_1\sum \limits_{f_2}p\left({f}_1,{f}_2\right)\\ {}{\mu}_y=\sum \limits_{f_2}{f}_2\sum \limits_{f_1}p\left({f}_1,{f}_2\right)\end{array}\right. $$
(16)
$$ \left\{\begin{array}{c}{\sigma}_x^2=\sum \limits_{f_1}{\left({f}_1-{\mu}_x\right)}^2\sum \limits_{f_1}\sum \limits_{f_2}p\left({f}_1,{f}_2\right)\\ {}{\sigma}_y^2=\sum \limits_{f_2}{\left({f}_2-\mu y\right)}^2\sum \limits_{f_1}\sum \limits_{f_2}p\left({f}_1,{f}_2\right)\end{array}\right. $$
(17)

2.3.4 Entropy (ENT)

This indicator is used to measure the amount of information in the image. It reflects the disorder or complexity of the texture in the image. Larger entropy value means that the texture is more complex. On the contrary, lower entropy value means that the texture is more regular. The equation of ENT is as follows:

$$ ENT=-\sum \limits_{f_1}\sum \limits_{f_2}p\left({f}_1,{f}_2\right){\log}_2p\left({f}_1,{f}_2\right) $$
(18)

By calculating the four indicators of ASM, CON, COR, and ENT, tool wear can be explored from the aspect of surface texture of workpiece.

3 Experiment

3.1 Experiment conditions

In order to verify the reliability of the monitoring system as well as explore the relationship between surface texture of workpiece and tool wear, the experimental platform was built as shown in Fig. 11. The designed experiment was carried out on the MVC650 CNC milling machine. The workpiece is a nickel-based superalloy Inconel 718 with a size of 120 mm × 60 mm × 35 mm. As nickel-based superalloys are difficult-to-machine materials and internal cooling technology can effectively dissipate the milling heat [34], the milling experiment was carried out under directional internal cooling condition. The internal cooling cutter is a Sandvik CoroMill R390 020A20-11M internal cooling cutter from Sweden and the type of insert is R390-11T3 04E-PL S30T, the TiAlN coated carbide insert.

Fig. 11
figure 11

Experiment site

Full size image

The visual monitoring platform is composed of CCD camera, camera fixture, industrial lens, ring light source, compute, etc. In order to reduce the impact of the vibration the machine tool on the position of camera, the CCD camera was fixed on a worktable with a certain distance from the machine tool spindle through the camera fixing bracket. In order to reduce the impact of cutting fluid sputtering on imaging under directional internal cooling condition, a lens protective cover was installed in front of the lens to guarantee the quality of images. A handheld digital microscope model AM4113ZT Dino-Lite Premier from China was used to measure the true wear value of the tool. The image acquisition under internal cooling condition is shown in Fig. 12.

Fig. 12
figure 12

Image acquisition under internal cooling condition

Full size image

3.2 Experiment design

In order to verify the reliability of the monitoring system and to explore tool wear and the change of the surface texture, the experiments consisted of two parts. The first part was an accuracy detection experiment of monitoring system based on the tool wear image. In order to avoid the contingency of the shape of tool wear zone, nine sets of inserts are used to explore the change of tool wear with different processing parameters by the design of orthogonal experiments. The second part was a texture analysis experiment. The gray-level co-occurrence matrix method was used to explore the relationship between surface texture of workpiece and tool wear. Two experiments are as follows:

3.2.1 Experiment 1

Taking cutting speed v, feed per tooth fz, and cutting depth ap as factors, a three-factor three-level orthogonal experiment was designed. The milling parameters are shown in Table 1. There were nine sets of experiments. The milling method was down milling, the milling width ae was 15 mm, the milling length L was 450 mm, and each set of experiment milled 20 times. After a set of milling experiment was completed, the tool returned to the set image acquisition point and the CCD camera acquired the tool wear image automatically. Then the worn tool was disassembled and the wear value was measured with a handheld microscope, which was compared with the result obtained by the monitoring system.

Table 1 Milling parameters of experimental 1
Full size table

3.2.2 Experiment 2

As shown in Table 2, a set of processing parameters was selected to carry out the full-life milling experiment. In order to explore the relationship between the surface texture of workpiece and tool wear value, the insert was removed and the wear value was measured every interval milling time T. At the same time, the texture image of the machined surface was acquired and the characteristics were extracted.

Table 2 Milling parameters of experimental 2
Full size table

3.3 Application prospect of integration with indirect method

Only using machine vision for tool wear monitoring has some disadvantages, as tool images could only be acquired during processing intervals, which may occupy a lot of production time and is difficult to check sudden damages. However, the indirect method could obtain the information related to tool wear in real time, while the accuracy is inferior to that of the direct method. Therefore, the proposed method will be combined with the indirect method. The complete tool wear monitoring system keeps the advantages of the machine vision method while acquiring the acoustic emission signals during processing in real time, as shown in Fig. 13. Machine learning algorithms would be proposed to control the frequency of image acquisition according to the emission signal system. When the acoustic emission signal system shows the tool wear is slight, the tool image would not be acquired to save production time. With the deepening of tool wear, the frequency of image acquisition is gradually increased to obtain the accurate value of tool wear. When sudden damages occur, this system could also control the camera to acquire images. Under the premise of ensuring a lot of production time will not be occupied, the real and reliable value of tool wear can be obtained. The complete system has the advantages of the direct and indirect methods, which is a potential work and also has positive application prospects. The details and the application prospect of the system are still being developed.

Fig. 13
figure 13

Tool wear monitoring system with indirect and direct methods

Full size image

4 Results and discussion

4.1 Tool wear image

Figure 14 shows the tool wear calculated by the monitoring system and measured by microscope. Figure 15 shows the comparison between the real tool wear values measured using the microscope and the wear values calculated by the monitoring system. The results show that the minimum error rate of the average wear width of the flank face VBave measured by the monitoring system and the real value is 0.88%, the maximum error rate is 5.73%, and the average error rate is 2.88%. The minimum error rate of the maximum wear width of the flank face VBmax is 1.36%, the maximum error rate is 5.17%, and the average error rate is 2.66%. Therefore, the monitoring system can effectively monitor the wear of the tool and the accuracy meets the industrial requirements.

Fig. 14
figure 14

Tool wear value. a Calculated by monitoring system. b Measured by microscope

Full size image
Fig. 15
figure 15

The result of accuracy detection experiment. a Average wear width VBave. b Maximum wear width VBmax

Full size image

Figure 16 shows the comparison between the wear area calculated by the monitoring system and the wear value. It can be seen that the wear area of the flank face AVB has the same changing trend as the wear width of the flank face under different processing parameters. Therefore, the wear area of the flank face AVB can be used as one of the effective indexes for the tool wear status.

Fig. 16
figure 16

The comparison between the wear area and the wear value

Full size image

4.2 Surface texture of workpiece

In order to facilitate the calculation, the texture images were compressed from 256 gray-levels to 16 gray-levels. The gray-level co-occurrence matrix when the image angles are 0°, 45°, 90°, and 135° were calculated by GLCM. The values of ASM, CON, COR, and ENT were extracted to describe the surface texture image of the workpiece. The averages of the 4 directional features were taken as the final results. Figure 17 shows the changes of the four features extracted from the gray-level co-occurrence matrix with the increase of milling time. It can be seen that when the machining time reached 28 min, the tool had entered a stage of severe wear.

Fig. 17
figure 17

Effect of milling time on texture-related parameters. a ASM. b CON. c COR. d ENT

Full size image

As shown in Fig. 17a, for the surface texture of workpiece image, the ASM value decreases with the increase of the processing time, showing a negative correlation. In the initial processing, the tool wear is slight. The texture after machining is relatively smooth and changes regularly, so the ASM value is larger. As the machining progresses, the texture changes irregularly due to tool wear and the ASM value becomes smaller. From Fig. 17b, before 28 min, the CON value tends to increase slowly because of the continuous tool wear. After 28 min, the surface quality of the workpiece deteriorates, greatly enhancing the ability of the workpiece surface to reflect light. Therefore, the CON value is greatly increased. It can be seen from Fig. 17c that the COR value decreases with the increase of processing time. The deterioration of the workpiece surface quality caused by the tool wear leads to a difference in the grayscale value in the neighborhood of any point. Therefore, the COR value shows a downward trend. Figure 17d shows that the ENT value of the image of surface texture increases with the increase of the processing time and it has a greater increase after 28 min. The tool enters a period of severe wear, which means that the geometry of the tool changes and the surface texture of workpiece changes become complicated. Therefore, rich information is contained in the image and the ENT value is increased. To sum up, using the surface texture of workpiece can analyze the tool wear from another aspect and using GLCM can clearly see the change trend of related parameters, which is of great significance to the judgment of tool wear.

5 Conclusions

In order to solve the problems of low tool utilization and high cost caused by tool wear in the milling process, this paper proposes an automatic tool wear monitoring system. The tool wear was characterized from different aspects and the accuracy of the system was verified by milling experiments. The specific conclusions are as follows:

  1. 1.

    A tool wear monitoring platform based on machine vision was established. Except the width of the flank face, the wear area of flank face was used as a new tool wear evaluation index. The tool wear monitoring system and detection software were designed. Structural similarity algorithm and Harris corner detection algorithm were used to solve the problem of automatic shooting tool angle. The automation of the monitoring was achieved. The use of binary morphology method improved the image quality, resulting in smooth and complete tool wear edges.

  2. 2.

    A superalloy milling experiment was carried out to verify the reliability of the monitoring system. The results showed that the error rate between the wear value calculated by the monitoring system and the real wear value measured using the microscope was within 5.73%. The accuracy met the industrial requirements. In addition, the wear area and the wear width were consistent in trend under different processing parameters, which means that the average wear area of flank could be used as one of the indexes for judging tool wear.

  3. 3.

    The gray-level co-occurrence matrix method was used to explore the relationship between the surface texture of workpiece and the tool wear. Four characteristic parameters of ASM, CON, COR, and ENT are used to characterize the texture. The results showed that the ASM and COR values decreased while the CON and ENT values increased with the increase of tool wear. Therefore, GLCM-based surface texture analysis could be used as a new aspect to measure tool wear.

Our future work will focus on the integration of machine vision, acoustic emission signals, and machine learning algorithms, thus developing the on-line tool wear monitoring system based on the combination of direct and indirect methods.