1 Introduction

Like most manufacturing processes, machining is optimized to achieve high economic productivity and machining quality [1]. Economic productivity is characterized by indicators like material removal rate (MRR) and energy consumption, while the machining quality is described by tool life and surface finish of the components. Therefore, machining optimization is applied to find the best cutting conditions, such as process parameters and cutting fluids, which improve machining performance indicators, such as energy consumption, MRR, surface roughness, and tool life [2]. However, research has shown that monitoring or controlling tool wear progression during machining is a key to optimizing the performance indicators, as presented in [3,4,5], among others. Tool wear progression can be monitored or controlled by applying an indirect method of sensors or a direct method of machine vision system—a process called TCM [6]. On the other hand, the growing trend of industry 4.0 computing networks led to new designs of cyber-physical systems that transmit data between interconnected devices, paving the way for real-time intelligent monitoring and control of tool wear progression by decision-making algorithms [7]. Therefore, the success of TCM depends on the system’s capability to select feature attributes that correlate with tool wear using data processing techniques and artificial intelligence (AI). In this case, artificial intelligence is considered a powerful tool to decipher and explore the complex underlying physics of tool wear evolution, including flank wear measurement, failure modes detection, and tool performance optimization.

The indirect TCM methods are applied to monitor tool wear progression using changes in force, torque, vibration, acoustic emission (AE), power, motor current, and sound signals extracted from various sensors, as presented in [6, 8,9,10,11], among others. These sensors can be mounted on different locations of the machining environment to acquire various tool wear signals (Fig. 1), making indirect TCM more convenient in detecting tool wear progression for various machining operations. Although one type of sensor can be used in TCM, the sensor fusion technique uses a synergy of feature attributes for a more accurate tool wear representation during prediction [12]. For example, Segreto et al. [13] applied wavelet packet transform (WPT) to select features extracted from acoustic emission, cutting force, and vibration signals to predict flank wear using a classical artificial neural network (ANN) model during the turning of Inconel 718. The ML model yielded a mean absolute percentage error (MAPE) of 5.17%. Although it is useful to extract time–frequency features for a more accurate tool wear representation, the shift-variance characteristic of WPT varies the energy [14], making it inflexible to extrapolate the time-invariant flank wear evolution under unprecedented failure modes, especially when milling superalloys like Inconel 718. In addition, applying a synergy of sensors’ signals involves complex feature extraction techniques and a high initial investment cost [3].

Fig. 1
figure 1

The sensor fusion technique during the turning operation

Full size image

Among the indirect TCM methods, force signals from a dynamometer are more reliable in predicting tool wear progression during machining, even though the technique is expensive compared to low-cost vibration or acoustic emission sensors. Moreover, cutting force is directly associated with the mechanics of machining [15], making it a direct indicator of tool performance for most machining operations. For example, a counter-propagation network yielded a high accuracy in predicting tool wear levels of the boring process using a 3-D map of force signals during TCM [16]. Furthermore, Gouarir et al. [17] applied features extracted from force signals to classify tool wear levels using a convolutional neural network (CNN), minimizing the complexity of features selection and noise filtering while applying multifeature or sensor-fusion techniques. However, CNN is a variant of deep learning (DL), vulnerable to overfitting issues under small datasets [18]. Although Gouarir’s model showed the ability to capture the intrinsic features of force signals, the model achieved a mean accuracy of 90%, which was not satisfactory due to a rather small dataset. Even though sensors are frequently applied for most TCM applications, they are not efficient in detecting the dominant wear mechanisms and failure modes of the tool’s cutting edge. For example, the vibration signals can detect a change in frequency amplitude due to tool wear evolution but not the dominant failure modes, such as chipping, notching, flaking, and BUE, which diagnostically indicate the extent of wear severity on the tool’s cutting edge [19]. Moreover, sensors are less accurate when predicting the value of flank wear width (VB) to determine the actual failure criterion and time for tool replacement.

On the other hand, machine vision technology is used to detect the failure modes and measure the actual progressive VB. It applies the conventional methods of digital microscopes to detect tool wear progression by extracting micrographs of the wear regions to diagnostically analyze dominant failure modes and wear mechanisms on the tool’s cutting edge [20, 21]. Although these methods are accurate, they involve machining interruption, human observation, and the high cost of tool wear inspection over a large sample size during in-process TCM application. Meanwhile, due to recent advances in computer vision and imaging technology, the direct method of using high-speed industrial cameras to monitor tool wear progression is receiving more attention due to its cost-effectiveness in detecting a large sample size while reducing machining disruptions, human observation, and manual handling of tools [22]. This can be achieved by mounting industrial cameras (CCD or CMOS) on the machining environment to acquire and transmit tool wear data to a computer vision software, which carries out a variety of tool wear detection—a process called MV-TCM (Fig. 2) [23]. It was reported that MV-TCM systems are quick and efficient in detecting tool wear at an accuracy above 95% [24]. For example, Wu et al. [25] applied CNN auto-encoders to extract feature attributes from various wear regions of the tool’s cutting edge to classify the tool wear of CVD and PVD-coated inserts during CNC milling of Inconel 718. The model showed an average precision recognition rate of 96.20% and MAPE of 4.76%, indicating the effectiveness of CNN features in predicting tool wear progression when milling Inconel 718. Furthermore, MV-TCM played a significant role in determining the failure criteria of the tools by real-time prediction of progressive VB during machining [26]. For example, Mikolajczyk et al. [27] applied a single category-based classifier to predict VB progression using image features extracted from the tool’s cutting edge. The model achieved an absolute mean relative error of 6.7% for tools’ entire life change. However, such techniques, which rely on segmentation are vulnerable to low prediction accuracy under the light variance, multimodal characteristics of the images, and changes in failure patterns on the wear region.

Fig. 2
figure 2

A CNC milling machine with a cyber-physical MV-TCM setup

Full size image

The greatest challenge with most MV-TCM systems is that industrial cameras are difficult to mount in some machining environments, such as CNC milling centers, where high-pressure flood coolant is applied to reduce heat and friction when cutting hard-to-machine metals [28]. In addition, most CCD/CMOS cameras are affected by smearing due to coolant droplets, light diffusion, and swarf, producing low-resolution tool wear images during the MV-TCM data acquisition process [29]. These images may not show adequate features to predict failure modes, primarily because of the poor pixel distribution and luminosity, which distract various failure patterns of the wear region. However, if AI is applied to enhance the resolution of these images and learn from their feature attributes to identify the failure patterns, it can be integrated with the MV-TCM system for a more accurate tool wear detection and optimization. In that case, not only will the in-process MV-TCM predict tool wear progression but also diagnostically detect dominant failure modes to show the extent of wear severity on the tool’s cutting edge. In doing so, the machinists can be notified of the unprecedented failure modes at the early cutting stage to adjust the cutting conditions before catastrophic failure. Therefore, this paper shows the advances and practical application of machine vision systems in tool wear monitoring and optimization. It further highlights the advances in artificial intelligence to enhance MV-TCM capability for in-process failure modes detection and classification.

2 Overview of MV-TCM

The in-process MV-TCM system consists of image acquisition, feature extraction, and tool wear detection [25]. Image acquisition is achieved by mounting the CCD or CMOS cameras to capture tool wear images in the machining environment (Fig. 2). More literature on MV-TCM image acquisition is summarized in Table 1. It was reported that mounting image acquisition cameras during MV-TCM was quite challenging due to splashing coolant and swarf, limiting the camera’s locations in the machining environment [30]. As a result, the images had uneven blurriness and poor contrast, affecting the clarity of the features used to distinguish failure patterns on the wear region [29]. Hence, most researchers applied MV-TCM in dry cutting conditions or minimum quantity lubrication to reduce the effect of image noise due to smearing to enhance the model’s predictive capability, as shown in Table 1. However, the cutting fluid is essential in minimizing the heat, friction, and thermally induced tool wear mechanisms on the tool-workpiece contact zone, reducing rapid tool deterioration during machining [5]. Therefore, efficient MV-TCM systems should also be developed to detect failure modes and wear mechanisms when machining using flood cooling conditions in addition to dry cutting or minimum quantity lubrication.

Table 1 Summary of recent MV-TCM systems
Full size table

Feature extraction is a key to dimensionality reduction and selection of reliable feature attributes that correlate with tool wear progression when applying MV-TCM systems. It involves image processing techniques, which remove the noise from tool wear images, segment the wear region, and extract geometric and textural properties to represent tool wear progression during machining. The most notable techniques used for feature extraction are summarized in Table 2. Duan et al. [31] combined the level set and histogram contour segmentation techniques to remove excess noise due to smearing and extract features for tool wear representation. In addition, other methods like morphological reconstruction and canny edge detection were reliable in extracting geometric properties, such as area, average width, and perimeter of the wear region to represent flank wear progression [32]. Xiong et al. [33] applied a variation level-set segmentation method to analyze the wear area of tool inserts during the milling process. This process concentrated on the wear area as an alternative method to flank wear width (VB) in determining the failure criteria of tools during machining. Furthermore, Prasad and Ramamoorthy [34] used the triangulation stereo MV-TCM system to acquire a 3D map of the wear region to predict flank and crater wear depth. However, the technique could not measure the tool wear progression of grooved inserts and could not reduce noise due to smearing. Devillez et al. [35] also performed a similar analysis of predicting tool wear by varying crater depth, and they experienced a similar challenge of the model’s difficulty to measure tool wear on grooved inserts. In this case, the interpolation and search techniques were used to enhance image focus at different depths, improving the classification accuracy of flank wear, crater wear, and breakage of the tools during machining [36]. It was observed that image processing techniques like fractal analysis, line interpolation, histogram transformation, contour detection, and deep learning were not fully utilized during in-process MV-TCM to extract tool wear features up to date.

Table 2 Summary of image processing techniques applied in MV-TCM
Full size table

In the meantime, features are analyzed by machine learning techniques, such as ANN, support vector machine, fuzzy logic, and regression models to detect tool wear progression. However, the accuracy of tool wear detection depends on the reliability of features extracted by image processing algorithms. For example, Moldovan et al. [37] used the geometric features, wear area, and Euler number, extracted from the tool’s cutting edge, to distinguish worn from unworn tools using one hidden layer neural network. However, classical ANNs like single-layer or multilayer perceptron architectures rely on numerical inputs to predict the system’s output (Fig. 3). These techniques are more reliable in the prognostic analysis of tool wear measurement during MV-TCM, such as predicting flank wear progression to determine the failure criterion of the tools during machining [38].

Fig. 3
figure 3

Tool wear prediction using classical neural networks

Full size image

As observed in Tables 1 and 2, most MV-TCM methods still apply image processing techniques, such as automatic segmentation and edge detection, to extract geometric and textural features that quantify the wear region for tool wear measurement. However, such techniques cannot extract high-level features that distinguish failure modes or wear mechanisms on the tool’s cutting edge, a notable TPO characteristic that was barely discovered by most researchers in MV-TCM application. This simply indicates that most MV-TCM systems still lack the essential capability to enhance the practical application of failure modes detection for a more effective TPO utilization. In this case, a general understanding of TPO techniques is required to briefly highlight their attributes, which can enable them to be used for in-process tool performance optimization.

3 Overview of TPO

While TCM is aimed at detecting the main causes of rapid wear evolution, such as failure modes and wear mechanisms, the TPO techniques are used to find optimum cutting conditions, such as cutting parameters, tool geometry, cutting fluids, and cooling techniques, which minimize tool wear progression during machining. Practically, the two methods must complement each other to provide reliable cutting solutions for in-process tool performance optimization during machining. For example, Halim et al. [58] applied the cryogenic CO2 cooling technique to optimize wear mechanisms and failure modes in high-speed milling of hardened Inconel 718, where abrasion, chipping, and BUE were observed during SEM analysis. However, it was observed that tools experienced less BUE formation due to sufficient heat dissipation of the cryogenic cooling condition. Meanwhile, cutting parameters are considered in-process variables when predicting tool wear progression at constant tool geometry and cooling techniques. For example, the BUE adhering to the depth of cut line increased with cutting temperature due to the high cutting speed and feed rate [59]. In addition, the cutting speed and depth of cut drastically affected flank wear progression during rough turning of Inconel 625, where abrasion and notching were dominant failure modes identified in the process [60]. Therefore, TPO techniques are tailored at finding the significant levels of process parameters that improve tool performance during machining. These techniques are summarized in Fig. 4.

Fig. 4
figure 4

Summary of techniques applied in tool performance optimization

Full size image

3.1 Conventional TPO methods

The commonly used conventional methods are the Taguchi technique, response surface methodology, and mathematical search techniques. The Taguchi technique investigates the effect of different machining parameters on the mean and variance of the performance characteristics, defining how best the whole machining process can be optimized. It applies orthogonal arrays to arrange deviations of the machining parameters with their respective levels [61,62,63], testing the effects of factor interactions on tool performance. In addition, it minimizes the number of runs to reduce the cost of materials for a full factorial experimental design. It is applied when DOE has the intermediate number of parameters whose interactions significantly contribute to the machining performance based on the predefined underlying physics of material science. The steps used in the Taguchi method to optimize tool performance are highlighted in [64]. However, it was noted that this method produces results that do not clearly show which parameters affect which tool performance indicator, in the case of multi-objective machining operations.

On the other hand, response surface methodology (RSM) is a statistical approach for optimizing tool performance by varying the cutting parameters of a machining process [65]. It applies techniques, such as central composite design (CCD) and central composite rotatable design (CCRD), to organize practical interaction terms of process parameters in an experimental design [66]. The CCD uses factorial (cube points), axial (star points), and center points to investigate the effects of the interaction terms during the experiment. The factorial points with significant contribution to the approximation of these terms are coded as − 1 and + 1. Axial points are considered during experimental design to estimate the quadratic terms whose interactions are considered less significant, i.e. (+ α, 0), (−α, 0), (0, + α), (0, − α), where α is the axial distance based on the desirable area called the region of interest for RSM. If α is rotatable, then, the design is called CCRD, which provides equidistant points from the design center with desirable predicted variance to enhance performance quality [67]. Finally, the center points estimate internal error and quadratic terms [66].

RSM was used to select suitable points from the DOE to fit the second-order polynomial function derived from regression equivalence, formulating the mathematical model to predict tool wear at various parameter combinations. For instance, a second-order mathematical model was developed to correlate the variation of cutting force and surface roughness with cutting parameters, mainly speed and feed rate, using RSM during high-speed drilling of Al–Si [68]. ANOVA analysis on the output of experiments designed through CCD was used to prove the significance of feed rate on tool wear progression during machining [69]. Besides, Davoodi and Eskandari [70] analyzed the effect of cutting speed and feed rate on tool wear progression that was again estimated by applying the second-order regression model of RSM designed by the CCRD technique during the turning of N155 iron–nickel-based superalloy. It was noted that the main advantage of RSM is the utilization of significant interaction terms to formulate an objective function, reducing the material cost incurred during a full factorial experiment. However, not all machining data can fit a second-order regression model, limiting RSM to highly correlated and structured data for linear regression analysis.

The mathematical search techniques (MSTs) are linear, nonlinear, and dynamic formulations used to optimize tool performance based on functional constraints. Such techniques are applied when the objective function is differentiable, and the constraints are explicitly defined to achieve an exact solution to a machining operation. In linear programming (LP), both the objective and constraint functions are linear in nature, making them suitable for solving linear problems of tool performance optimization. For instance, an integer LP (ILP) variant was applied to select the best depth of cut that minimized the machining cost of a multi-pass turning operation [71]. Besides, nonlinear programming (NLP) is more efficient than most LPs in formulating multivariable and multi-objective functions. However, it was observed that they are still inefficient in providing an optimal global solution for highly complex nonlinear or discreet problems like the 4-stage machining operations [72]. In this case, the dynamic programming (DP) techniques are considered the most efficient models in solving discrepancies faced by LP and NLP in finding optimal global solutions to highly complex and discreet machining operations. However, research has shown that these MSTs are not utilized in predicting tool wear progression during the machining process because of many interdependent variables and their respective relationships, which are not cost-effective during TPO [64]. In addition, as mentioned earlier, these techniques are limited to differentiable problems, which may not apply to some real-life machining operations.

3.2 Evolution techniques

Several evolution techniques (ET) are applied to optimize tool wear progression during machining. These techniques include genetic algorithm (GA), particle swarm optimization (PSO), and Artificial Bee Colony (ABC). GA is the commonly used ET method, which provides a population of optimum parameters by applying reproduction, crossover, and mutation [73]. It produces sets of parameters called new generations from the initial solutions or population created from various combinations of cutting parameters, as shown in Fig. 5. For example, Prasad et al. [74] applied GA to find optimum cutting speed, feed rate, and depth of cut during the turning process, where the near-optimal solution of tool performance was achieved at the 33rd generation of the iteration. Gao et al. [75] applied GA to optimize tool wear and cutting force using speed, feed rate, and axial depth of cut during the milling process, where GA was reported efficient in optimizing tool wear progression for single or multi-pass cutting operations, thereby avoiding local optimal solutions [76]. In another research, Sen et al. [77] compared the performance of artificial intelligence meta-modeling tools in predicting flank wear of TiAlN coated carbide inserts during the MQL-assisted milling operation of Inconel 690. The results showed that a variant of GA called gene expression programming (GEP) had a high performance in terms of MAPE and correlation coefficient (R2) than ANN by 3.135% and 1.3%. However, like any other GA, GEP requires a preceding condition of a fitting approximation function, which provides a specific estimation capability to evaluate all kinds of linear and nonlinear objective functions [78].

Fig. 5
figure 5

Summary of NSGA II technique

Full size image

To overcome some challenges faced by a single objective GA (SoGA), multi-objective evolutionary algorithms (MOEAs) play a significant role in finding multiple Pareto solutions using a single run to minimize computational time [76, 79]. Bouzakis et al. [80] applied MOEA to find the cutting parameters that optimized various objective functions of the milling operation. It was observed that MOEA could also be applied to optimize the performance indicators of multi-objective turning operations [81]. To further enhance the MOEA capability, a nondominant sorting genetic algorithm (NSGA) was introduced to achieve a true Pareto-optimal region, overcoming the general weakness of MOEAs in computing various solutions in a single run [82]. However, due to high computational time and low accuracy in finding the optimal solutions, NSGA was involved in NSGA II to provide more accurate solutions than NSGA and other contemporary MOEAs like Pareto-Archived Evolution Strategy (PAES) [83] and Strategy Pareto Evolution Algorithm (SPEA) [84]. The general methodology of NSGA II is presented in Fig. 6 [76]. Yusoff et al. [85] compared NSGA II, SoGA, and MOEA in optimizing MRR and surface roughness using pulse time, pulse current, flushing pressure, and electrode rotation of die sink EDM. It was observed that SoGA converged faster, followed by MOEA and NSGA II during optimization. However, NSGA II provided more accurate solutions than SoGA and MOEA. In the meantime, NSGA II capability is not fully utilized to optimize the tool performance of the machining operation.

Fig. 6
figure 6

Summary of NSGA II technique

Full size image

To improve the accuracy of GA techniques in TPO, some hybrid systems are used to utilize attributes of various TPO techniques in achieving a high-level optimization capability. For example, ANN was integrated with NSGA II to optimize tool wear, surface roughness, and MRR during EDM machining operation [86]. In addition, the hybrid system of RSM-NSGA II showed a higher tool optimization performance than the ANN-PSO method during the turning of AISI 52,100 steel with CBN tools [87]. However, the main challenge with all MOEAs and NSGA II is computational complexity [88], especially when evaluating the fitness function. To resolve this issue, the fitness approximation technique is applied during TPO [89]. In this case, the parent fitness function is used to estimate the values of the offspring, or the representative is selected by clustering the solutions. However, fitting the approximate function does not guarantee a global optimal solution because it limits the significance of other solutions that can be used to determine an optimal machining performance.

Particle swarm optimization (PSO) is another evolutionary technique, which applies particles with stochastic velocities to form a swarm, representing feasible optimum cutting parameters that best fit the objective function (tool wear or tool life). By moving these particles in a space of different solutions, the PSO algorithm converges at an optimal point. The methodology of PSO is presented in [90]. Such methods are reliable in solving continuous optimization problems [91]. For example, it was used to optimize tool life, dimensional accuracy, and MRR using feed rate, flow velocity, and voltage of an electrochemical machining (ECM) process [92]. It was also used to maximize tool life and minimize surface roughness, cutting power, and force using spindle speed and feed of the milling operation [93]. To enhance its capability to optimize highly nonlinear machining operations, Huang et al. [94] integrated it with a wavelet neural network (WNN) to detect the cutting power by decomposing the energy and the square error signals using Daubechies-wavelet transform. After training the PSO-WNN with power signals, the model showed a faster convergence and more accurate tool wear prediction than conventional WNN or hybrid GA-WNN. Due to its high speed, simple mathematical operation, and less memory requirement, PSO can be used to optimize the tool performance in-process. However, it is limited to solving simple linear problems of the machining operation as the model’s complexity is difficult to control with highly nonlinear and discreet objective functions.

Apart from PSO, Artificial Bee Colony (ABC) can also be used in TPO by applying the behavior of a bee colony, which consists of employed, unlooked, and scouts bees to optimize the tool performance by maximizing the quantity of nectar (objective function) unloaded at the hive [95]. The employed bees use greedy selection to find, memorize, and evaluate the amount of nectar and share the quality of food sources on a wangle dance. Onlookers choose profitable sources from the wangle dance and later modify them based on probabilistic selection. The food source that is not good for exploitation is abandoned, and their respective employer bees are labeled scouts. The Deb rule introduced by Akay and Karaboga [95] evaluated the feasibility or cost of infeasibility by allowing the lowest cost and most feasible process parameters to be selected during ABC analysis. The general steps involved in ABC are highlighted in [96]. Although ABC is used to optimize single objective functions, it was also applied to solve multi-objective machining problems. Samanta and Chakraborty [97] applied the ABC technique to select the best process parameters, which optimized the performance of EDM and electrochemical micro-machining (ECMM) processes. It was observed that ABS was more suitable for finding voltage, electrolyte concentration, and interelectrode gap, which optimized the radial overcut (ROC) and heat-affected zone (HAZ). However, just like GA techniques, the greatest limitation of ABC is slow convergence, which is caused by selecting the initial population of the machining parameters.

3.3 Soft computing techniques

The problem of high complexity and computational cost, which most conventional and evolutional techniques encounter, can be resolved by applying AI techniques during TPO [64]. These methods include but are not limited to regression models, ANNs, and fuzzy logic. Regression analysis is a simple approach to modeling mathematical equations (linear, nonlinear, logistic, multivariant, etc.), which predict the tool wear progression using input features like sensor signal, cutting parameters, and image features. The regression model was reported to be quicker and more accurate than ANN in predicting tool wear progression during machining [98]. However, it is constrained to accurately predict tool wear within the vicinity of large training datasets. Nevertheless, due to their simplicity, efficiency, and integrability, regression models play an essential role in modeling the objective functions when optimizing tool performance using other TPO techniques, such as RSM [70]. Thangavel et al. [99] developed a regression model using RSM to predict flank wear progression using cutting speed, feed, and depth of cut of a turning operation. The model showed a high performance with a standard estimated error of 0.012 mm, indicating a high correlation between actual and predicted flank wear values. Gupta  [100] further reported that ANN was better than RSM regression and support vector machine (SVM) in predicting tool wear progression during the turning operation. Berkani et al. [101] also developed a regression model using RSM to predict flank wear of CVD-coated carbide inserts using the cutting speed, feed, and cutting time during AISI 304 stainless steel turning operation. The model showed a high correlation of 96.73% and a MAPE of 14.31%.

In general, previous research applied the regression model during RSM by assuming tool wear data can fit a quadratic polynomial function, as presented by [99,100,101], among others. However, such modeling works better when the absolute curvature of the response surface is already known or established by experts from the fundamental physical behavior of tool wear evolution for a particular tool and machining operation [100]. It was observed that such complexity gets even worse when machining superalloys, such as Inconel 718, where various wear mechanisms accelerate and mutilate flank wear curvature during machining [102]. Therefore, these models are unreliable for in-process VB extrapolation because the key descriptors are not easily defined under highly complex machining operations.

The ANN on the other hand uses artificial neurons to learn the knowledge trend of the tool wear dataset without being guided by any mathematical model, making it more robust and efficient to solve linear, nonlinear, and discreet machining operations [103]. It eliminates the complexity of formulating objective functions for intricate tool wear behavior by applying backpropagation to optimize the neurons’ weights during learning, strongly adjusting feature attributes to correlate with tool wear progression [104]. Figure 7 shows the typical ANN model for tool wear monitoring or prediction. Such ANN characteristic was utilized to optimize creep feed during grinding of superalloys [105] and predict the MRR and surface roughness of the abrasive flow machining process [106]. In addition, it was noted that ANNs are built with flexible architectures that can easily integrate with conventional methods to optimize the tool performance during machining [107]. For example, Taguchi-ANN and Taguchi-regression hybrid models were used to select the best cutting speed, feed, and depth of cut, which optimized tool wear during the machining of composites [108]. In addition, it was observed that the hybrid systems predicted the tool performance faster while providing more accurate solutions than regression models. However, the over or under-fitting issues of ANN with insufficient datasets still affect the performance of hybrid models unless the training progress is terminated before converging at an optimal point, making it pessimistic about achieving globally optimal solutions for the machining operation [109].

Fig. 7
figure 7

ANN tool wear optimization technique

Full size image

Apart from regression and ANN models, fuzzy logic is also a soft computing technique for TPO, which generates rules or analytical computation called fuzzification to relate inputs and outputs during modeling. It is used when the relationship between features and outputs is known from fundamental principles, allowing experts to set up objective functions and key parameters during modeling [110]. It can be used to predict tool wear progression of complex machining operations because of its high computation capability, which is enhanced by the fuzzification process (logical rule generation of inputs and outputs based on membership functions) [111]. Ip [112] developed a rule-based fuzzy logic model to optimize tool life and cutting efficiency by controlling the feed of mild steel surface milling operation. Although fuzzy logic accurately predicts tool performance, it involves complex computation, in the case of multivariable and multi-objective machining problems, when formulating logical rules with several objective functions and input parameters [113]. However, it can be integrated with ANN architectures to form hybrid systems, thereby utilizing the superior characteristics of both methods to minimize the computational complexity (ANN) while improving prediction accuracy (fuzzy logic) during TPO [110]. For instance, the artificial neuro-fuzzy inference system (ANFIS) was modeled using features extracted from sound and force signals to predict the flank wear progression during the milling operation [114]. It was noted that the ANFIS could predict tool wear quickly and precisely during milling of 7075-T6 aluminum alloy.

3.4 In-process tool performance optimization

The greatest challenge researchers always face is trying to incorporate these techniques for a more practical real-time application. The idea behind real-time TCM and MPO application is to acquire in-process updates of tool wear evolution and use the information to adaptively control the cutting conditions to improve the performance indicators during machining. In this case, apart from high accuracy, TCM and TPO techniques must be enhanced with high-speed capacity for real-time tool wear detection and performance evaluation. Table 3 summarizes TPO techniques, highlighting their attributes and limitations for in-process tool performance prediction and optimization. It was noted that the conventional and evolution techniques are time intensive and involve high computation complexity, making them not suitable for in-process tool performance optimization. On the other hand, soft computing methods like ANN and regression models are fast and efficient in predicting and optimizing tool performance subject to large datasets, making them more reliable in predicting tool wear evolution than conventional and evolution techniques [114]. It was further noted that most techniques computed optimum process parameters based on the final performance output, like the longest tool life or minimum wear rate. However, this approach does not consider the in-process evolution of failure modes and tool wear mechanisms to reveal the extent of tool deterioration rate at various levels of wear progression.

Table 3 General summary of TPO techniques
Full size table

Typically, tool wear progression is divided into three levels (initial, uniform, and rapid failure stages) [115], as shown in Fig. 8. The uniform wear stage experiences a constant deterioration rate, making it a determinant of economic tool life under optimum cutting conditions. At this stage, tools yield the highest tool performance, including the longest tool life and minimum wear rate [58]. Hence, adjusting the cutting conditions at this stage improves overall tool performance during machining, making it an important region to apply in-process TPO and MPO. Besides, the uniform wear stage is critically used to evaluate significant microscale failure modes and wear mechanisms before they propagate to rapid macroscale or catastrophic failure. On the other hand, initial and rapid failure stages experience different failure modes and wear mechanisms that accelerate tool deterioration rate due to severe wear mechanisms that cause break-in (early rapid wear) and macro-chipping (rapid failure stage). During these stages, wear mechanisms cannot be controlled by adjusting the cutting conditions to prevent tools from failure [116]. Therefore, if the dominant failure modes and wear mechanisms are detected at uniform flank wear level, cutting solutions, such as adjusting cutting parameters and cooling techniques, can be applied to minimize the intensity of wear mechanisms, preventing rapid tool deterioration while tools are still new.

Fig. 8
figure 8

Stages of flank wear progression

Full size image

Therefore, an overall tool performance can be achieved by integrating the two principal components of in-process tool wear analysis: flank wear measurement to prevent premature failure by determining the failure criteria for tools replacement [38] and failure modes detection to determine cutting solutions that minimize rapid wear deterioration rate [117]. Research has shown that more investigations were conducted on in-process flank wear prediction using analytical formulation [118, 119], finite element simulation [120, 121], and machine learning models [122,123,124], among others. Only a few studies were conducted on real-time failure modes detection using in-process MV-TCM systems. However, machine vision systems play a significant role in detecting failure modes and tool wear mechanisms, complementing the TPO techniques in selecting the best cutting conditions that optimize tool performance during machining [117]. To understand the real-time application of the MV-TCM systems, this research first presents the conventional methods and how they are used to complement in-process TPO techniques in optimizing tool performance during machining.

4 Applying machine vision in TPO

4.1 Applying conventional methods of machine vision in TPO

It was observed that most machining industries recommend the best cutting parameters based on the minimum failure modes experienced by tools during machining, confirming the significance of machine vision systems in selecting tools and parameters for various material processing. In this case, digital microscopes are used to analyze failure modes during machining, as shown in Table 4. To apply such methods, tools are detached from the cutter and later analyzed by SEM/EDS to extract micrographs and observe tool wear mechanisms and failure modes [102]. It was reported that abrasion, adhesion, diffusion, attrition, and oxidation were observed using the SEM/EDS analysis [125]. Halim et al. [58] observed that wear mechanisms are the dominant causes of rapid tool failure in most machining operations. In addition, it was also reported that continuous rubbing action in dry cutting operation generates high cutting temperature, which facilitates wear mechanisms, such as abrasion, adhesion, diffusion, and chemical attack, accelerating tool wear by thermal and mechanical cracks to cause rapid wear progression during machining, as presented in [5, 58, 126], among others. As a result, tools in a temperature-controlled machining operation tend to experience less thermally induced wear mechanisms and longer tool life, making cooling techniques and cutting fluids another prominent area of research for in-process TPO, especially when machining superalloys like Inconel 718 [127]. Besides, the wear mechanisms contribute to th rapid occurrence of unprecedented failure modes, such as nonuniform flank wear, chipping, flaking, BUE, thermal cracks, and coating delamination, which can also be detected under SEM and optical microscopes [117].

Table 4 Summary of techniques used to inspect failure modes and tool wear mechanisms on various tools
Full size table

Furthermore, toolmakers and optical microscopes are used to measure the actual values of some failure modes like flank wear width, chipping width, and crater depth, determining the failure criteria or maximum tool life during machining [38]. Due to their accuracy in measuring tool wear progression and observing wear mechanisms and dominant failure modes, conventional methods are used with most TPO techniques to select the cutting parameters that optimize tool performance during machining. For example, SEM was used to observe tool wear mechanisms and the surface profile of machined Aluminium composites, complementing RSM in selecting the best cutting parameters that improved tool life, surface roughness, and MRR [128]. In addition, the literature in Table 5 attributed some cutting solutions as countermeasures to minimizing the dominant failure modes and wear mechanisms during tool performance optimization of various machining operations.

Table 5 Summary of proposed solutions for in-process TPO using various failure modes [38, 142]
Full size table

However, conventional methods cannot be used for real-time tool wear detection because of their sophisticated hardware and software configuration, which cannot be mounted directly on the machining environment to observe the in-process tool wear evolution. In addition, they involve manual handling of tools, which eradicate some evidence of known failure modes, such as BUE. As a result, such methods can be used to analyze a few samples to reduce machining time and the cost of wear inspection, which does not provide substantial information of in-process wear evolution during machining. Furthermore, they rely on human observation, limiting their application to skilled personnel in tool wear characterization. However, with advances in computer vision techniques, high-speed industrial cameras, and artificial intelligence, the cyber-physical MV-TCM systems can overcome the challenges faced while applying these conventional methods, as presented by [29, 41, 49, 50, 129,130,131,132], among others. Furthermore, an AI can learn from feature attributes to utilize its human evaluation capability during in-process MV-TCM, simplifying a decision-making algorithm for tool wear characterization for non-skilled personnel during machining.

4.2 Applying in-process MV-TCM during TPO

In the meantime, the in-process MV-TCM system has also received more attention because of its cost-effectiveness in predicting tool wear progression online. It is used to extract image features that are combined with cutting parameters to train machine learning models, which predict in-process tool performance during machining. For example, a WNN applied the multi-scale wavelet transform and ANN classifier to predict flank wear and surface roughness using speed, feed, depth of cut, and image features extracted from the wear region during the end milling operation [41]. It was observed that WNN had the highest accuracy in selecting the best cutting parameters that optimize tool wear progression compared to deep multilayer neural network (DMLNN), Radial Basis Function Neural Network (RBFNN), and RSM. However, they recommended that controlling the illumination during image acquisition can improve the feature extraction process from tool wear images during the MV-TCM application. Chethan et al. [143] developed a relationship between tool wear signals obtained from the AE sensor and image features extracted by automatic segmentation to assess the effect of machining parameters on flank wear progression. Their approach used the AERSM, AECOUNT, wear area, and perimeter as descriptor variables of flank wear progression. The Taguchi method was used during the analysis to find optimum cutting speed, feed, and depth of cut, which minimized flank wear progression during machining.

Furthermore, researchers noted that the online MV-TCM can provide a more accurate and efficient prediction of tool wear progression than conventional methods. For example, Zhang and Zhang [51] observed that the detection of wear edge points in a ball-end milling process using sub-pixels extracted from the tool’s cutting edge was more accurate than the conventional method of SEM measurement. In another research, Ponce et al. [144] applied an MV-TCM system to detect the evolution of adhesive particles by automatic segmentation to predict changes in the area, thickness, and volume of the BUE on the cutting edge. They observed that the BUE thickness reduced with increasing cutting speed for all values of ADOC. However, there was no predictable trend of progressive wear area and BUE volume with cutting speed and feed rate. Hence, the best cutting speed was selected based on the BUE thickness, making it one of the significant features to be considered for prognostic tool wear analysis when determining the failure criteria of the tools during machining. Although this method successfully selected the cutting speed to optimize tool performance, their study only focused on measuring progressive features of one failure mode, BUE, making it more prognostic than diagnostic in nature. However, a synergistic effect of unprecedented failure modes that affect tool performance during machining can diagnostically be detected to determine the corresponding wear mechanisms, which accelerate tool failure [116].

As highlighted in Tables 4 and 5, the dominant failure modes reveal the nonoptimal tool performance attributed to a drastic change in machining conditions, such as tool grades, cutting parameters, and tool wear mechanisms like abrasion, adhesion, diffusion, and oxidation [102]. Such information is necessary to machinists when selecting the best in-process solutions for minimizing rapid tool wear progression to achieve maximum tool performance. However, these failure modes are barely detected online due to low spatial resolution images during in-process MV-TCM application. Poor resolution distracts the pixel distribution and luminosity of the wear region, drastically mutilating the feature patterns that can be utilized to detect failure modes online [29]. Therefore, if MV-TCM can be enhanced to track the failure modes on low-quality images, it can efficiently complement in-process TPO techniques, achieving better performance than known conventional methods of SEM/EDS and optical microscopes. In this case, Artificial Intelligence (AI) can significantly improve the resolution of tool wear images to extract and learn from their deep feature attributes to predict failure modes. Therefore, MV-TCM enhancement is divided into features engineering and deep learning in this research. Feature engineering focuses on advanced techniques of noise filtering, dimensionality reduction and feature extraction for pattern recognition and tool wear analysis. Deep Learning (DL) focuses on the extraction and adaptive learning of high-level features for more accurate failure modes detection.

4.3 Enhancing MV-TCM with features engineering

The automatic segmentation techniques applied in most MV-TCM applications to extract the image features rely on the bi-modal characteristic of the image, which assumes an image has two distinct colors (wear region and background) [145]. However, online tool wear images contain more than two colors, especially during dry cutting of superalloys, where heat and failure modes like chipping, notching, BUE, catastrophic failure, and flaking distract the wear region’s pixel distribution and feature patterns, as presented in [146, 147], among others. In the meantime, such complexity is avoided by applying textural, fractal, and Fourier transform analysis, which provide the spatial variation of the pixel distribution regardless of the multi-modal characteristic of tool wear images. Kerr et al. [55] used histogram, gray level co-occurrence method (GLCM) features, fractal features, and Fourier spectral analysis to analyze and measure flank wear with texture descriptors on end mills. In addition, Li and An [48] applied the GLCM, watershed transform, and Markov random field segmentation techniques to extract the entropy and average values of grey-level pixels on small wear regions to evaluate tool wear progression during machining. The model demonstrated how multi-feature information can improve tool wear representation more than a single feature. However, the consistency of GLCM features can be affected by the extent of severity in crater depth, flaking, notching, and localized chipping, generating complex textural properties that misrepresent the pixel distribution on the wear region. Hence, research has shown the significance of varying the wear depths when extracting textural features for tool wear analysis during MV-TCM [35]. For instance, the interpolation and search techniques were applied to capture textural information at various wear depths, improving the features representation under inconsistent pixel distribution and complex wear geometry [36].

However, it was noted that such techniques were not fully utilized for in-process MV-TCM due to high computational cost, which is time-intensive for real-time detection of tool wear progression. In that case, online MV-TCM can utilize other pattern recognition and feature extraction techniques like Principal Component Analysis (PCA), Singular Value Decomposition (SVD), and tensor decomposition to avoid the complexity of textural analysis and automatic segmentation on multi-modal tool wear images. PCA is a feature mining technique for dimensionality reduction, which was successfully used for pattern recognition, including face detection and image compression [148, 149], with low sensitivity to noise, less capacity, and low memory usage during online deployment [150]. Furthermore, PCA used an unsupervised learning technique [151, 152], which can eliminate human involvement in selecting reliable feature attributes when training machine learning models for more efficient application in MV-TCM.

Apart from PCA, Singular Value Decomposition (SVD) was also applied for pattern recognition by considering an image as a 2-D matrix with pixels denoting the spatial relationship between adjacent features located within it. It factorizes the matrix into a left singular matrix, a right singular matrix, and singular values [153] – a property well known for noise reduction, as shown in Fig. 9a. Since the basic noise filtering during feature extraction assumes the noise can be isolated from the dominant features of an image, the SVD technique localizes the noise component orthogonal to the data signal subspace [154]. It applies large singular values to reconstruct the dominant features of an image while removing the noise sub-space using the smallest singular values [155, 156]. Such a capability controls the two characteristics of an image, dominant features and noise component [153], which can be utilized to enhance the resolution of tool wear images and extract reliable feature attributes for tool wear prediction during MV-TCM.

Fig. 9
figure 9

Image decomposition using a SVD and b HOSVD (Tucker) to extract patterns of tool wear images

Full size image

Moreover, a series of tool wear images during in-process MV-TCM form a 3-D tensor to represent time-series image data of tool wear progression. In this case, tensor decomposition can be used as higher-order SVD (HOSVD) to accelerate the feature extraction process on time-series image datasets during MV-TCM [157]. It applies the same principles of SVD, only that this time, the singular values are presented in the form of a 3-D core tensor, as shown in Fig. 9b. It can also be used to denoise and watermark tool wear images, thereby filtering the noise to enhance the resolution of the wear region. Therefore, PCA, SVD, and tensor decomposition can be utilized to extract statistical features from tool wear images to train machine learning models for more accurate flank wear prediction with limited datasets during MV-TCM. However, such techniques were not utilized for tool wear analysis during in-process MV-TCM up to date.

4.4 Enhancing MV-TCM with deep learning

Deep Learning (DL) techniques were explicitly introduced to handle complex data [158], liable for extracting complex properties to recognize dominant feature patterns of tool wear images. DL forms a crucial part of computer vision, offering the best solutions for different applications, including face recognition, object detection, and image super-resolution enhancement. It has a robust architecture capable of extracting high-level features of various failure modes during in-process MV-TCM applications. Furthermore, DNNs can directly use images as inputs with features extracted by built-in convolutional layers [158], avoiding the complexity of using separate algorithms for feature extraction when applying classical ANN during tool wear prediction, as shown in Fig. 3. Serin et al. [159] explored the possibility of using such techniques during TCM. They discovered that CNN, DMLNN, recurrent neural network (RNN), and deep reinforcement learning (DRL) are the most promising architectures for online TCM applications with big data from sensors. However, the MV-TCM techniques discussed in this research were not explicitly emphasized in [159]. The overview of existing DNN variants, their computer vision tasks, and notable MV-TCM applications are summarized in Table 6. It was observed that CNN-based architectures are the most applicable models for computer vision tasks, more especially object detection and classification [160]. Although these models can extract deep features from images, it is still challenging to achieve enough datasets to train them, limiting their application to simple classification problems during MV-TCM like classifying worn from unworn tools and region-based tool failure, such as flank wear and crater wear. Researchers tried to apply transfer learning, utilizing pretrained CNN models to extract deep features for training small ML architectures like multi-class and SoftMax regression models for tool wear classification.

Table 6 Summary of DNNs applied in computer vision and the proposed MV-TCM. Annotations are found on the highlighted literature
Full size table

In our previous research, pre-trained AlexNet and VggNet-16 were used to extract features from tool wear images to classify dominant failure modes like chipping, notching, thermal wear, and BUE [161]. It was discovered that the transfer learning technique was successfully used to classify types of failure modes on the small dataset using multi-class SVM and SoftMax regression models (by fine-tuning fully connected layers of the stated pretrained CNN models) with an accuracy above 95%. However, the greatest challenge with such models is the utilization of features like color and shape, causing various false positives and misclassification of multiple failure modes occurring on the same wear region. In addition, a convolutional auto-encoder model developed in Caffe deep learning framework was trained using backpropagation and stochastic gradient descent to classify different types of tool wear using features extracted from a custom tool wear dataset [25]. Again, it was discovered that CNN-based models successfully extracted high-level features from tool wear images to classify different types of regional-based wear. Although the model showed a lower precision of 96.2% than VggNet-16, it was fast and reliable for real-time MV-TCM application. However, this model again considered regional-based failure modes, such as flank and crater wear, where shape becomes the dominant feature for classification.

Nevertheless, research has shown that the same wear region can experience multiple failure modes during machining. For example, when milling Inconel 718, prevalent failure modes that affect tool performance occur on the flank face of the wear region, making it dominant when machining materials with such metallurgical properties [162]. On the other hand, when milling titanium, the crater wear region experiences the failure modes that affect tool performance, making it dominant when cutting such materials [163]. In this case, failure modes cannot easily be identified by shape, or color, but complex features like localization in object detection to distinguish the locations and orientations of the failure patterns on the same wear region. Therefore, advanced DNN models can detect multiple failure modes on the same wear region, an area which is still dormant to date, mainly due to notable challenges in utilizing such powerful DNN architectures during MV-TCM. Some of these challenges are presented as follows.

The first challenge is achieving enough dataset of tool wear images that contain various failure modes to train DNN models. To achieve such large datasets, several experiments must be conducted by varying cutting tools, workpiece materials, cutting fluids, cooling techniques, and machining processes. However, this iterative approach is time intensive and incurs a high cost of materials during the experiment. In this case, techniques like generative adversarial network (GAN) can offer the best alternative to generating more plausible and super-resolution images of the existing tool wear datasets [164], saving time and cost of materials when conducting several offline experiments. GANs apply unsupervised learning techniques to automatically extract feature attributes and learn from their patterns to generate more similar examples [164]. They consist of a series of CNN models called the generators that produce examples and the discriminators that classify the generated images as real or fake by comparing with the original dataset [165], as shown in Fig. 10. The generator’s performance is updated continuously to eliminate the differences between the real and fake images. At the same time, the discriminator improves its skills to distinguish the real from generated images, thereby creating a stiff competition between CNN models called an adversarial game. By doing so, the GAN can produce high-resolution tool wear images that replicate the original dataset [165], making it a reliable solution to improve the size of datasets and enhance the contrast of the wear region during MV-TCM.

Fig. 10
figure 10

The architecture of GAN, which can generate more samples from the original tool wear dataset

Full size image

The second problem faced by online MV-TCM is real-time failure modes detection on the same wear region, described by three expressions: localization, classification and human evaluation capability [179, 195, 196]. It is challenging to detect failure modes because of variations in viewpoints, occlusions, lighting conditions, and changes in feature patterns and orientations on the wear region. The idea behind tool wear detection is to locate a failure mode in an image (localization), categorize it based on dominant feature patterns (classification), and label it based on the highest precision (human evaluation capability). In this case, DL applies high-level features extracted from regions of interests (ROIs) to detect specific objects in an image using regional-based CNN (R-CNN) [174]. The capability and high-speed efficiency of R-CNN to learn complex feature patterns from an ROI without designing the features manually [158] makes it a reliable DNN variant for real-time failure modes detection and classification. The commonly used R-CNNs are fast R-CNN [174], faster R-CNN [194], and you only look once (YOLO) techniques [197]. By creating a bounding box on each failure mode in an image, an R-CNN can extract latent features and coordinates on each ROI localized by the specific bounding box and later assigns scores to the feature maps using multi-class SVM or SoftMax regression models [179]. The non-maxima suppression is then applied to eliminate the labels with low scores while maintaining the classes with the highest precision values [198]. Therefore, the accuracy of R-CNN depends on the distribution of class labels in the dataset, and the robustness of the R-CNN architecture. As a result, it cannot effectively predict unprecedented failure modes outside the training datasets.

To overcome the challenge R-CNN models face, a robust intelligent automation architecture called reinforcement learning (RL) can be used to predict unprecedented failure modes during online MV-TCM application. Deep RL (DRL) applies CNN architectures to create the initial feature states that best describe the attributes of the existing objects (failure modes) in an image [199]. In this way, the CNN model can extract high-level features from different failure modes to train RL before deploying it for real-time MV-TCM applications. DRL approach combines deep learning perception and uncertain decision-making of intelligent automation in reinforcement learning to create an AI, which resembles a human ability to learn, predict and make decisions in a very complex environment [200]. It can be used to detect the exact location of objects in an image (in this case, the exact location of failure modes on the wear region) by training an agent, which refines and resizes the dimension of the bounding box in ROI using feature maps predefined by a CNN model [201]. Besides, it was applied to improve the real-time automatic object segmentation (in this case, wear region segmentation) by controlling an agent to track the contour around the object in an image [202]. Although the task seems challenging, the capability of DRL can be utilized by training an agent to automate the cyber-physical MV-TCM system for more accurate performance, including real-time wear region segmentation and failure modes detection.

The final challenge is inductive reasoning for deciphering underlying relationships between prevalent failure modes and wear mechanisms. As previously stated in Table 5, the occurrence of dominant failure modes reveals the presence of some wear mechanisms that accelerate tool failure. In that case, MV-TCM can be utilized for inductive reasoning during in-process tool performance optimization. Such an intelligent task can be implemented in MV-TCM using DNN variants called graphical neural network (GNN) architectures. GNNs are state of the art in computer vision applications because they use the prevailing relationships between various objects (failure modes in this case) in an image to predict the possible existence of significant information (tool wear mechanisms or other causes of rapid tool failure). GNNs apply graph domains to predict objects on a small dataset [203], making it suitable for failure modes detection by considering that each researcher builds their own dataset based on a specific material, tools, cooling technique, and machining process. The relationships can be modeled in a knowledge graph that uses a graphical convolution network (GCN) or point GNN during the encoding process.

Recently, GCN was used to construct a knowledge graph of a framework position and speed-dependent dynamics of tool-tip based on the vibration signals [193]. First, the stationary tool-tip Frequency response functions (FRFs) of 27 selected positions were obtained by impact tests. Next, a series of spindle speed ramp-up (SSR) tests were conducted to obtain the cutter’s chatter frequencies and axial depths. Then, the inverse stability solution was applied to identify tool-tip dynamics under various cutting conditions to obtain the labeled dataset (dominant natural frequency and damping ratio). With the kernel neural network (KNN) rule, the labeled and unlabeled data were converted into a knowledge graph. The multilayer perceptron (MLP) was introduced to learn the adjacent matrix of the knowledge graph. Using the features and graph networks, a semi-supervised GCN was trained to propagate information from labeled to unlabeled samples, predicting the tool-tip dynamics by integrating with the MLP. Although this research did not focus on MV-TCM, it demonstrated the future of expediting state-of-the-art GNN models for real-time automation of tool performance optimization.

In computer vision technology, objects in an image can be selected as nodes linked by existing relationships to create a knowledge graph [204]. This principle was enhanced by defining the super-subordinate, positive, and negative relations to transmit the confidence between related objects [205]. It was used to detect features in an ROI [206], which were connected based on predicted class networks in R-CNN [207]. For instance, Baldassarre et al. [194] applied Faster R-CNN to extract different features from the nodes and aggregate them to create a graphical network for object detection and classification. Such an approach can be utilized for developing a cyber-physical MV-TCM system during intelligent manufacturing. For instance, failure modes can also be presented in graphical form as nodes connected by their respective attributes, such as localization, as shown in Fig. 11. Then, techniques like point-GNN can be utilized to create the knowledge graphs, mapping feature vectors from each node using classical ANN like multilayer perceptron (MLP) to aggregate them for failure modes detection.

Fig. 11
figure 11

Example of GNN used to detect predicates depicted on the wear region [NMS: non-maxima suppression]

Full size image

However, when constructing the knowledge graphs using relationships generated from various failure modes, factors like varying the workpiece material and cutting tools must be considered. For example, different tools, such as PVD, CVD, and ceramic coated inserts, have various geometric shapes (square, round, etc.) and colors. However, shape and color are also significant image features extracted in the first layers of a CNN [179], making them prominent factors to consider when developing a dataset to train a DNN architecture for MV-TCM application. In this case, the robustness of the predictive model depends on the diverse labels of each failure mode (different scenarios of the same failure mode) in the training dataset, including failure modes from various types of cutting tools with different shapes and colors. On the other hand, the metallurgical properties of workpiece materials substantially influence the type of failure modes experienced by the tools during machining. For example, when machining ductile materials like aluminum and low alloy steel, most tools experience failure modes like BUE and uniform flank wear, which can be observed from the flank or rake face [24, 27]. Therefore, such failure modes can easily be identified and classified on the tool’s cutting edge to create the training dataset for most DNN architectures.

Furthermore, most tools experience unprecedented failure modes, such as progressive chipping, flaking, notching, coating delamination, and galling when machining superalloys like Inconel 718, titanium, and stainless steel due to their complex metallurgical properties [140]. When cutting such materials, it is difficult to identify and classify multiple failure modes on ROIs to create the training dataset because of their dynamic change in location on the tool’s cutting edge. This affects feature localization during object detection, especially when a few failure mode scenarios are included in the training dataset. However, with more scenarios, the DNN architecture can adaptively learn enough feature attributes to enhance its predictive capability. In addition, it requires experts with deep knowledge in tool wear characterization to identify and classify various failure modes on the tool’s cutting edge, creating a more diverse and well-balanced dataset for training DNN models. Therefore, due to the complexity of detecting the unprecedented failure modes when machining superalloys, applying a DNN-based MV-TCM system is now becoming an exciting field of research in the intelligent machining of these difficult-to-cut metals [141].

5 Conclusion

This research shows that by applying artificial intelligence, an ML-based MV-TCM system has the potential to perform more efficiently than conventional or evolutionary techniques in tool performance optimization, providing the cost-effective in-process tool wear prediction during machining. As a result, the cyber-physical MV-TCM system can be improved by DNN techniques, such as DRL, by interconnecting with the CNC machine controllers for a real-time adjustment and control of cutting parameters during in-process TPO without relying upon human input. The following points summarize the findings of this research:

  • Conventionally, digital microscopes are commonly used to observe the failure modes, wear mechanisms, and measure progressive flank wear width during machining. They complement TPO techniques in selecting the best cutting conditions that optimize the tool performance of the machining operation.

  • The soft computing TPO methods like classical ANN, fuzzy logic, and regression models can be used to predict flank wear progression, whereas DNN can detect and classify the dominant failure modes during in-process MV-TCM application.

  • The pattern recognition techniques of SVD, PCA, or tensor decomposition can be utilized for dimensionality reduction and extraction of reliable feature attributes for flank wear prediction during in-process MV-TCM.

  • Pretrained CNN, such as AlexNet, Resnet-50, and VGG-16, can be applied in transfer learning for simple classification problems, such as detecting different types of regional-based wear or classifying worn and unworn wear inserts.

  • GAN can generate super-resolution image data from the prevailing examples of tool wear images to increase the training datasets of deep learning models for real-time failure modes detection and classification during MV-TCM.

  • The real-time detection of failure modes can be implemented by applying R-CNN, DRL, and GNN. The R-CNN can be used to localize dominant failure modes and classify them based on feature attributes extracted from their respective regions of interests (ROIs).

  • DRL can be used for automatic segmentation by training the agent to track the contour around the wear region. It can also be used to detect the unprecedented failure modes not included in the training dataset.

  • Furthermore, GNNs can be applied to model knowledge graphs, relating the failure modes and wear mechanisms through induction reasoning. In this case, R-CNN can integrate with GNNs to detect failure modes using knowledge graphs and apply the results to decipher the underlying wear mechanisms associated with them.

Although these models promise a prominent future for cyber-physical MV-TCM systems in intelligent manufacturing, more research is required to explore how PCA, SVD, tensor decomposition, CNN, R-CNN, DRL, GNN, and GAN can be applied for real-time tool wear detection and machining control. Finally, further research (e.g., time complexity analysis) is also required to demonstrate the industrial applicability, reliability, and scalability of such models for smart monitoring of tool wear progression during in-line production systems.