1 Introduction

In the machining process, chatter refers to the self-excited vibration between the tool and the workpiece, which will lead to significant machining errors, poor surface quality, and poor machining quality of the machine tool [1]. In the actual machining process, it is usually used to select conservative cutting parameters to avoid chatter, which greatly reduces production efficiency. Although the analytical method can predict the chatter in advance, the prediction may not be accurate due to the complex dynamic system involved in the machining process [2], and the signals collected in the milling process can accurately reflect the milling state. Therefore, using signal processing technology for chatter detection has been widely studied.

Chatter detection includes signal preprocessing, feature extraction, and state recognition. Because the signals collected in the machining process are nonlinear, advanced signal processing methods are needed to eliminate the irrelevant information to the machining state in the original signal, restore the essential components of the signal, and further extract the chatter feature. The signal processing methods include time domain analysis, frequency domain analysis, and time–frequency domain analysis. In the cutting process, the original time series signal obtained by the sensor reflects the change of signal amplitude with time, and some statistical indexes can be directly calculated to judge the machining state. Lamraoui et al. [3] took root mean square, variance, peak value, and gap coefficient directly calculated from the time domain acceleration signal as chatter feature. Li et al. [4] calculated the real-time variance of milling force signal in the time domain for chatter detection. However, the time domain analysis method cannot provide frequency domain information. When the cutting parameters change, the time domain method may not be able to detect the occurrence of chatter [5].

As a frequency domain analysis method, fast Fourier transform (FFT) is widely used in chatter detection because of its advantages in capturing frequency information. Gupta and Singh [6] used the local mean decomposition technique to demodulate the collected audio signal, then performed FFT and extracted the chatter feature. Aslan and Altintas [7] used FFT to decompose the current signal provided from the CNC and detect the frequency of chatter from the spectrum. However, the traditional spectrum analysis based on FFT is usually only applicable to the analysis of stationary signals. The signal in the processing process is nonlinear and unstable, so its robustness is poor.

The time–frequency analysis method of simultaneous time and frequency positioning of signals can effectively analyze nonlinear signals and is widely used in feature extraction [8]. Wavelet transform (WT) overcomes the shortcoming that the window size of short-time Fourier transform (STFT) does not change with frequency. Cabrera et al. [9] proposed a chatter identification method based on wavelet analysis of cutting force signals, which has been proven to be effective in four different experimental configurations. Gao et al. [10] proposed the method of cmor continuous wavelet transform (CMWT) based on acoustic signals for detecting the milling chatter of thin-walled parts and analyzed the results of chatter detection and the acquisition of stable region of thin-walled parts in the milling process. However, because the resolution of high-frequency coefficients decomposed by WT is low, the application of high-frequency signals is greatly limited when the chatter frequency is high. Empirical mode decomposition (EMD) is an adaptive method for analyzing non-stationary and nonlinear signals. EMD does not need to set any parameters in advance and has a high signal-to-noise ratio. Shrivastava et al. [11] collected the signals in the turning process and further processed the chatter signal after wavelet denoising (WD) pretreatment by using EMD and performed FFT on the obtained internal model function to identify the chatter frequency. Chen et al. [12] used the method of ensemble empirical mode decomposition (EEMD) to process the signal, which overcame the shortcoming of EMD mode aliasing and used the Fisher discriminant ratio (FDR) to sort the features extracted from the sub-components and selected the features with higher FDR. However, there are some problems in the decomposition of EEMD, such as insufficient envelope overshoot and negative frequency. VMD is a completely non-recursive and adaptive method for signal processing and modal variation, which is more robust than EEMD and EMD. Mou et al. [13] proposed a VMD-ED method for chatter detection of thin-walled structures during milling. The experimental results show that the identification method has a good identification effect. However, when using VMD, the penalty factor \(\alpha\) and the decomposition level K need to be set in advance, and the selection of parameters has a great impact on the decomposition effect. Liu et al. [14] used kurtosis to optimize the parameters of VMD and extracted energy entropy as a chatter feature. However, kurtosis as a fitness function cannot effectively identify the transient shock sequence with periodic distribution, so it is very important to establish a reasonable fitness function to improve the decomposition effect of VMD.

In the machining process, an appropriate chatter index is key to detecting the occurrence of chatter. Entropy is a dimensionless index, which reflects the complexity and randomness of the sequence. The entropy will change significantly when the machining state changes. Zhu et al. [15] used the sample entropy (SE) and energy entropy to identify the chatter in the peripheral milling of the thin-walled workpiece. Yang et al. [16] used approximate entropy and SE as chatter features to detect the occurrence of chatter according to its entropy change. The result of the experiment shows that this method is suitable for detecting both continuous and intermittent chatter. However, the single-scale entropy omits part of the machining state information, which is not enough to detect the accurate time point when the machining state changes [17]. Liu et al. [18] and Chang and Wu [19] respectively applied multi-scale permutation entropy (MPE) and multi-scale entropy (MSE) to chatter detection, which was verified by experiments. However, SE has low calculational efficiency and is greatly affected by sudden change signals. Although the calculation efficiency of permutation entropy (PE) is high, it does not consider the magnitude relationship between amplitudes. To overcome the above shortcomings, Rostaghi et al. [20] proposed a new irregularity measurement index dispersion entropy (DE). This method considers the magnitude relationship between amplitudes, has a fast calculation speed, and is less affected by sudden change signals. The multi-scale process of MDE is the same as that of MSE and MPE, leading to the loss of statistical information during isometric segmentation. Therefore, Azami et al. [21] proposed the RCMDE method on this basis and compared it with several other multi-scale methods, reflecting the advantages of RCMDE in feature extraction.

Considering that the collected machining signal has a pair of interference components, it may lead to the misjudgment of chatter detection; it is necessary to filter the signal to separate the sub-signals with rich chatter information. Cao et al. [22] used the comb filter to preprocess the vibration signal to eliminate the interference of cutting frequency. Lamraoui et al. [23] used a Wiener filter to remove the noise signal related to the free rotation of the spindle. Although these filtering methods can filter the noise signals related to the free rotation of the spindle speed and the fixed colored noise component, they cannot filter the random interference components in the machining process. Therefore, before extracting the chatter feature, it is necessary to select the frequency band containing abundant chatter information [24].

To solve the problem that it is difficult to detect the chatter of thin-walled parts in the milling process, a milling chatter feature extraction method based on OVMD and RCMDE is proposed. Firstly, SE-SCK is used as the fitness function of GA. Secondly, the optimal parameters are used for the VMD decomposition of the signal, and the signal is reconstructed according to the energy ratio. Then, the influence law of scale factor on DE is analyzed, and RCMDE under the scale factor with the largest distinction to the machining state is selected as the chatter detection index. RCMDE can reduce the interference of random signals to detect chatter more accurately. Finally, it is verified by the experiment of side milling variable cutting depth of titanium alloy thin-walled parts.

The rest of this article is organized as follows. Section 2 briefly introduces VMD and chatter detection index RCMDE. Section 3 introduces the chatter feature extraction method based on GA-SE-SCK-VMD and RCMDE. Section 4 introduces the experimental setup and verifies the algorithm. Finally, the conclusions are laid out in sect. 5.

2 Principle of milling chatter detection model

Signal preprocessing and sensitive feature extraction is the key to the early detection of chatter. The signal collected by the sensor is a nonlinear and non-stationary signal in the milling process. VMD decomposes signals based on sequential iterative sifting and has good denoising performance. It is suitable for the processing of nonlinear signals, and non-stationary RCMDE contains more information about the machining state and can effectively analyze nonlinear signals. Therefore, using RCMDE to detect the occurrence of chatter has important advantages.

2.1 Principle of VMD

VMD is an adaptive and completely non-recursive method of modal variation and signal processing. VMD has a more solid mathematical theoretical foundation. The core idea of VMD is to construct and solve variational problems [25].

The variational model is

$$\left\{ \begin{gathered} \mathop {\min }\limits_{{\left\{ {u_{k} } \right\}\left\{ {\omega_{k} } \right\}}} \left\{ {\sum\limits_{k} {\left\| {\partial_{t} \left[ {\left( {\delta \left( t \right) + \frac{j}{\pi t}} \right)*u_{k} \left( t \right)} \right]e^{{ - jw_{k} t}} } \right\|_{2}^{2} } } \right\} \hfill \\ s.t.\sum\limits_{k} {u_{k} = u} \hfill \\ \end{gathered} \right.$$
(1)

where \(u_{k} \left( t \right)\) is the kth modal component, \(\partial_{t}\) is the Tikhonov matrix, \(*\) is the convolution operator, \(\omega_{k}\) is the center frequency of the kth modal component, \(\delta \left( t \right)\) is the pulse function, and u is the original signal.

To transform the constrained variational problem into an unconstrained variational problem, the Lagrange multiplier \(\lambda\) and the quadratic penalty term \(\alpha\) are introduced:

$$\begin{gathered} L\left( {\left\{ {u_{k} } \right\},\left\{ {\omega_{k} } \right\},\lambda } \right) = \alpha \sum\limits_{k} {\left\| {\partial_{t} \left[ {\left( {\delta \left( t \right) + \frac{j}{\pi t}} \right)*u_{k} \left( t \right)} \right]e^{{ - jw_{k} t}} } \right\|}_{2}^{2} + \hfill \\ \left\| {f\left( t \right) - \sum\limits_{k} {u_{k} \left( t \right)} } \right\|_{2}^{2} + \left\langle {\lambda \left( t \right),f\left( t \right) - \sum\limits_{k} {u_{k} \left( t \right)} } \right\rangle \hfill \\ \end{gathered}$$
(2)

The alternating direction method of multipliers is used to solve the variational problem in Eq. (2), and the extended Lagrangian “saddle point” is obtained through iterative updating \(u_{k}^{n + 1}\), \(\omega_{k}^{n + 1}\), and \(\lambda^{n + 1}\), which is the final solution of Eq. (1).

To facilitate the calculation of \(u_{k}^{n + 1}\) and \(\omega_{k}^{n + 1}\) in the iteration process, these two terms are converted to the frequency domain, and the updated expressions of \(u_{k}^{n + 1}\) and \(\omega_{k}^{n + 1}\) are determined as follows:

$$\widehat{u}_{k}^{n + 1} \left( \omega \right) = \frac{{\widehat{f}\left( \omega \right) - \sum\limits_{i \ne k} {\widehat{u}_{i} \left( \omega \right)} + \frac{{\widehat{\lambda }\left( \omega \right)}}{2}}}{{1 + 2\alpha \left( {\omega - \omega_{k} } \right)^{2} }}$$
(3)
$$\omega_{k}^{n + 1} = \frac{{\int_{0}^{\infty } {\omega \left| {\widehat{u}_{k} \left( \omega \right)} \right|}^{2} d\omega }}{{\int_{0}^{\infty } {\left| {\widehat{u}_{k} \left( \omega \right)} \right|}^{2} d\omega }}$$
(4)

The flow chart of the VMD algorithm is shown in Fig. 1.

Fig. 1
figure 1

VMD algorithm flow chart

2.2 RCMDE

The RCMDE algorithm, time series with different scales \(\tau\), \(\tau\) corresponds to different starting points of the coarse-grained process. The RCMDE value is defined as the average value of the dispersion entropy of the coarse-grained sequence.

The kth coarse-grained sequence of the signal \(u = \left\{ {u_{1} ,u_{2} , \cdots } \right\}\) is

$$\begin{gathered} x_{k}^{\left( \tau \right)} = \left\{ {x_{k,1}^{\left( \tau \right)} ,x_{k,2}^{\left( \tau \right)} , \cdots } \right\} \hfill \\ x_{k,j}^{\left( \tau \right)} = \frac{1}{\tau }\sum\limits_{{b = k + \tau \left( {j - 1} \right)}}^{k + \tau j - 1} {u_{b} } \, 1 \le j \le N,1 \le k \le \tau \hfill \\ \end{gathered}$$
(5)

The RCMDE value under scale \(\tau\) is calculated as follows:

$$E_{ECMD} \left( {x,m,c,d,\tau } \right) = - \sum\limits_{r = 1}^{{c^{m} }} {\overline{p} } \left( {r_{{v_{0} v_{1} \cdots v_{m - 1} }} } \right)\ln \overline{p} \left( {r_{{v_{0} v_{1} \cdots v_{m - 1} }} } \right)$$
(6)

where \(\overline{p} \left( {r_{{v_{0} v_{1} \cdots v_{m - 1} }} } \right)\) is the average value of the probability of the dispersion mode corresponding to the coarse-grained sequence, as shown in Eq. (7).

$$\overline{p} \left( {r_{{v_{0} v_{1} \cdots v_{m - 1} }} } \right) = \frac{1}{\tau }\sum\limits_{k = 1}^{\tau } {p_{k}^{\left( \tau \right)} }$$
(7)

where \(p_{k}^{\left( \tau \right)}\) is the probability of the dispersion model corresponding to the kth coarsening sequence under scale \(\tau\).

3 Milling chatter detection method based on GA-SE-SCK-VMD and RCMDE

In the actual machining process, the milling signals collected by the sensor will have many signals unrelated to the machining state, which is very unfavorable for the subsequent feature extraction and the machining state recognition. Therefore, it is necessary to process the collected signal, filter out the noise in the signal, eliminate the interference signals, and extract the machining state representation with high robustness reliability and sensitivity to identify chatter [26].

3.1 Adaptive method of VMD parameters based on SE-SCK

The penalty factor \(\alpha\) and the decomposition level K in the VMD algorithm are preset parameters, and the selection of their values will have a great impact on the signal decomposition effect. How to select appropriate parameters is the premise and the key to accurate signal decomposition in VMD. In this paper, GA is used to optimize the decomposition level K and the penalty factor \(\alpha\) in the VMD algorithm. GA algorithm has a fast global search ability [27] and has been applied to optimize various parameters. When using GA for optimization, it is necessary to determine a fitness function, and GA judges the optimal solution according to the size of the fitness function. In the actual machining process, the cutting system will be subject to aperiodic accidental impact. When the time domain index is used as the fitness function, the accidental impact will affect the time domain index, so the time domain index is not the best parameter optimization index. Therefore, the frequency domain index squared envelope spectral correlated kurtosis (SE-SCK) is introduced as the index of VMD parameters optimization.

Inspired by kurtosis and signal autocorrelation, SE-SCK is a new index [28] proposed by redefining correlation kurtosis, which can accurately identify cyclic transient shocks. As a frequency domain index, Squared envelope spectral correlated kurtosis is not disturbed by accidental shocks. The larger the value of the square envelope spectral correlation kurtosis, the higher the signal-to-noise ratio of the signal and the more obvious the periodic pulse component.

For the zero mean filtered signal \(\widetilde{X}\) with length N, its square envelope signal can be expressed as

$$SE\left(\widetilde{X}\right)={\left|X+j\cdot {\text{Hilbert}}\left(X\right)\right|}^{2}$$
(8)

The variance of the square envelope signal is expressed as

$$D\left[ {SE\left( {\widetilde{X}} \right)} \right] = E\left[ {SE\left( {\widetilde{X}} \right)} \right]^{2} - \left\{ {E\left[ {SE\left( {\widetilde{X}} \right)} \right]} \right\}^{2}$$
(9)

According to the kurtosis [29] formula, the above formula is substituted and sorted out as follows:

$${\text{kurtosis}}\left( {\widetilde{X}} \right) = D\left[ {SE\left( {\widetilde{X}} \right)} \right] + 1$$
(10)

According to the definition of the autocorrelation function, kurtosis is re-expressed as

$${\text{kurtosis}}\left( {\widetilde{X}} \right) = \frac{{r_{SE} \left( 0 \right)}}{{\left[ {r_{{\widetilde{X}}} \left( 0 \right)} \right]^{2} }}$$
(11)

where \(r_{{\widetilde{X}}} \left( 0 \right)\) is the value of the autocorrelation function when the delay coefficient is 0, and \(r_{SE} \left( 0 \right)\) is the value of the autocorrelation function of the square envelope signal \(SE\left( {\widetilde{X}} \right)\) when the delay coefficient is 0.

In combination with Eq. (11), the relevant kurtosis is redefined as

$$RECK\left(\widetilde{X},T\right)=\frac{{r}_{SE}\left(T\right)}{{\left[{r}_{\widetilde{X}}\left(0\right)\right]}^{2}}=N\cdot \frac{\sum_{n=1}^{N}{\left[\left|\widetilde{x}\left(n\right)\right|\cdot \left|\widetilde{x}\left(n\right)+T\right|\right]}^{2}}{\sum_{n=1}^{N}{\left[{\left|\widetilde{x}\left(n\right)\right|}^{2}\right]}^{2}}$$
(12)

Perform discrete Fourier transform on the square envelope signal to obtain the square envelope spectrum. In combination with Eq. (12), SE-SCK can be expressed as

$$SE-SCK\left({f}_{r}\right)=\frac{{f}_{s}}{2}\cdot \frac{\sum_{f=0}^{\frac{{f}_{s}}{2}}{\left[SES\left(f\right)\cdot SES\left(f+{f}_{r}\right)\right]}^{2}}{{\left[\sum_{n=1}^{N}SES{\left(f\right)}^{2}\right]}^{2}}$$
(13)

where \(f_{r}\) is the spindle rotation frequency, \(SES\left( f \right)\) is the square envelope spectrum, and \(f_{s}\) is the sampling frequency.

Using SE-SCK as the fitness function of GA, \(\left[ {k,\alpha } \right]\) at the maximum fitness function value is obtained, which is the optimal parameter of VMD. In this paper, a GA-VMD optimization method with adaptive parameter range search is proposed. The detailed optimization steps are as follows.

  1. 1.

    Coding and group initialization. The encoding method adopts real number encoding. The mapping error after discretization of continuous function in binary encoding has a significant impact on the results. When the chromosome is relatively short-term, the accuracy is not high. The parameter range of the VMD algorithm is preset to \(K \in \left[ {3,10} \right]\), \(\alpha \in \left[ {1000,10000} \right]\), and the population size is set to 4, and the number of iterations is 15.

  2. 2.

    Calculate the fitness function value. Calculate the fitness value corresponding to each chromosome according to Eq. (13), and record the chromosome with the largest fitness value.

  3. 3.

    Crossover and mutation. The parent chromosomes are used to cross to generate offspring, and the offspring chromosomes are mutated. In this paper, single-point crossover and single-point mutation methods are selected.

  4. 4.

    Determines whether to terminate the iteration. If the preset requirements are met, terminate and output the results. Otherwise, return to step 2 until the optimal solution is generated.

3.2 Chatter feature extraction method of the reconstructed signal based on RCMDE

In milling chatter detection, after the signal processing technology is used to process the collected signals, one or more features reflecting the milling state are usually extracted as indicators to identify the occurrence of chatter. Therefore, a chatter feature extraction method based on RCMDE of the reconstructed signal is proposed in this paper. When chatter occurs, the energy of the cutting system is transferred from the spindle rotation frequency and its multiple frequencies to the chatter frequency. Therefore, calculate the energy radio of each IMFs component of the signal after VMD decomposition, select the IMFs component with a high energy ratio to reconstruct the signal, and obtain sub-components with abundant chatter information. The IMFs after VMD decomposition of the signal are represented by \(u_{i} \left( t \right)\), and the calculation formula of the energy of each component is shown in formula (14).

$$E_{i} = \int_{ - \infty }^{ + \infty } {\left| {u_{i} \left( t \right)} \right|}^{2} dt \, i = 1,2,3,...,N$$
(14)

When the chatter occurs, the frequency components gradually gather at the chatter frequency, which results in the change of DE. However, the dynamic system involved in the actual machining process is very complex, so analyzing signals from a multi-scale perspective will better explain the machining state. Due to the equidistant segmentation and averaging of the data in the multi-scale process, the relationship between the segmented data is not considered, which results in the loss of statistical information. And the initial signal points selected under different scale factors are the same, and the result has a certain deviation. RCMDE method is further refined based on the MDE method, which can effectively reduce the lack of some statistical information in the coarse-grained process of MDE and can effectively solve the impact of the initial point position on the calculation results by averaging the positions of multiple initial points, so as to reduce the calculation deviation. Therefore, the RCMDE characteristics of reconstructed signals are more conducive to the early detection of chatter.

The flow chart of the proposed milling chatter detection scheme is shown in Fig. 2. One is the adaptation of VMD parameters. GA algorithm is used to optimize the penalty factor \(\alpha\) and the decomposition level K of VMD, and SE-SCK is used as the fitness function of GA. The second is chatter feature extraction. The proposed OVMD algorithm is used to decompose the signal to obtain a series of IMFs. Selecting the sub-components with a high energy ratio for signal reconstruction. RCMDE is extracted from the reconstructed signal, and the machining state is judged through the changes in features (Table 1).

Fig. 2
figure 2

Flow chart of milling chatter feature extraction

Table 1 Cutting parameters

4 Experimental verification

4.1 Experimental configuration

To better reveal the changing process of the machining state, the variable cutting depth milling experiment was carried out on a CNC machine tool to verify the effectiveness of this method. The experimental site is shown in Fig. 3. The tool used in the experiment is cemented carbide end mill, and the specific parameters are shown in Table 2.

Fig. 3
figure 3

Experimental scene

Table 2 Tool parameters

The milling force signal is collected by a dynamometer (Kistler 9171) and a charge amplifier for chatter detection. The workpiece material is TC4, and its parameters are shown in Table 3. The sampling frequency is set to 5 kHz. The initial workpiece size is 200 mm in length, 100 mm in height, and 5 mm in thickness. The variable cutting depth experiment shows a linear change of axial cutting depth from 3 to 15 mm. All experiments were dry milling.

Table 3 Material properties of Ti6Al4V

4.2 Preliminary division of the machining state

To verify the correctness of the algorithm, several groups of milling experiments are carried out. The machining parameters are shown in Table 1, and the third representative group of parameters is selected. In the milling process, the stiffness of thin-walled parts in the y-direction is small, so the milling force signal in the y-direction is selected for analysis. With the increase of the cutting axial depth, the milling force signal increases slowly. However, due to the chatter of the self-excited vibration between the workpiece and the tool, the milling force signal will fluctuate, obviously. Therefore, the machining state can be preliminarily divided according to the change of the original signal, as shown in Fig. 4a. Through the analysis of the milling force signal, it can be found that the machining state is stable at about 0–6.43 s, slight chatter at 6.43–31.87 s, and severe chatter at 31.87–40 s.

Fig. 4
figure 4

Milling force signal analysis: a time domain signal, b spectrum of the signal at state A, c spectrum of the signal at state B, and d spectrum of the signal at state C

Chatter is a kind of self-excited vibration that reflects the change of frequency and energy distribution in the machining process. Therefore, the chatter can be identified by analyzing the change of frequency. When chatter occurs in the machining process, the main frequency of the signal spectrum will gradually shift from the cutting frequency to the natural frequency of the cutting system, and the cutting frequency includes spindle rotation frequency and its multiple frequencies. FFT can sample data from continuous input and convert the sampled signal to the frequency domain. Therefore, FFT analysis can be used to judge the machining state preliminarily.

Three 0.4 s sub-signals are randomly selected from the above three signals, as shown in Fig. 4a. The selected sub-signals are analyzed by FFT using MATLAB software. The sampling frequency is set to 5 kHz, and the number of sampling points is 2048. The results are shown in Fig. 4b–d, and f is the spindle rotation frequency; the black triangle is the cutting frequency, and the inverted purple-red triangle is the chatter frequency. In Fig. 4b, the signal spectrum is mainly composed of the cutting frequency, which belongs to the stable machining state. In the frequency spectrum of the signal in Fig. 4c, in addition to the cutting frequency, there is also a chatter frequency of 1542 Hz, which has a small amplitude. The cutting frequency still occupies an absolutely dominant position, and the machining state belongs to the slight chatter. In Fig. 4d, the amplitude of the chatter frequency of 1542 Hz increases significantly. Although it does not occupy a dominant position, its proportion in the frequency spectrum increases significantly, indicating that the chatter has broken out and the machining state belongs to the severe chatter.

4.3 Signal preprocessing based on GA-SE-SCK-VMD

The machining state can be judged through FFT analysis, but FFT lacks the function of time to frequency positioning, and FFT is the overall transformation from the time domain to the frequency domain, which cannot separate each frequency band and effectively distinguish the frequency components. To better understand the components of the signal, the GA-SE-SCK-VMD method proposed in this paper is used to decompose the milling force signal. The parameter settings in the algorithm have been given in Sect. 3.1. The GA-SE-SCK-VMD decomposition is performed for the above three groups of signals. The optimization iteration curve is shown in Fig. 5, and the optimal parameters are shown in Table 4.

Fig. 5
figure 5

Change curve of SE-SCK with the number of iterations

Table 4 Optimal parameters and optimal fitness value

Considering that in the subsequent chatter suppression stage, it is necessary to start chatter suppression when slight chatter is detected, so after obtaining the optimal parameters, the VMD algorithm is used to decompose the signal of slight chatter. The obtained sub-components and their spectra are shown in Fig. 6. It can be seen from the figure that the slight chatter signal is decomposed into 6 IMFs; the boundary between the frequency bands of different sub-components is clear, and there is no mode aliasing. At the same time, the chatter frequency is successfully decomposed into the fifth frequency band, and the chatter frequency is about 1542 Hz. The proposed GA-SE-SCK-VMD algorithm can avoid the problem of mode aliasing and successfully separate the chatter frequency band, which proves the effectiveness of the algorithm.

Fig. 6
figure 6

GA-SE-SCK-VMD decomposition sub-component and its spectrum

4.4 EMD decomposition of signals

EMD is a signal processing method based on the shifting process. Based on the local characteristics of the signal, the concept of IMF is proposed, which makes the instantaneous frequency meaningful. IMF has a symmetric envelope defined by local maximum and minimum, respectively [30].

Figure 7 shows the results of the decomposition of the slight chatter signal in Fig. 4 by EMD. After EMD processing, the signal is decomposed into 8 IMFs and arranged in the order of high frequency to low frequency. The frequency bands with cutting frequency and chatter frequency are mainly distributed in the first five sub-components, of which the first sub-component contains most of the signal frequencies, so it is impossible to effectively distinguish the components of these signal frequencies. In the “Signal preprocessing based on GA-SE-SCK-VMD” section, the OVMD algorithm proposed is used to decompose the signal, and the chatter frequency is separated into the fifth frequency band, which avoids the disadvantage of mode aliasing. The signal decomposed by OVMD suffers less interference, so the chatter features extracted are more accurate and more suitable for chatter detection.

Fig. 7
figure 7

EMD component and its decomposition

4.5 Chatter feature extraction of reconstructed signals based on RCMDE

Chatter is characterized by changes in frequency and energy distribution. In the stable state, the cutting energy mainly concentrates on the cutting frequency. When the chatter occurs, the cutting energy is gradually absorbed into the chatter frequency. Therefore, to improve the recognition accuracy, this paper selects the first three IMFs for signal reconstruction according to the criterion of the energy ratio of IMFs components from high by referring to the signal reconstruction method in reference [9]. Although these IMFs are still disturbed by the noise signal, and the cutting frequency is also included in the IMFs with a high energy ratio, the energy is mainly concentrated on the chatter frequency, thus increasing the signal-to-noise ratio.

Although the chatter frequency band can be separated by VMD decomposition, it is not practical to detect chatter from the amplitude spectrum in actual production. So, it is necessary to use quantitative indicators to detect chatter. In this paper, the quantitative indicator RCMDE is used for chatter detection. When calculating RCMDE, embedding dimension m, delay \(\iota\), and scale factor \(\lambda\) should be selected. Rostaghi and Azami [20] suggest that the value of embedding dimension m should not be too large or too small and m is usually taken as 2 or 3. In this paper, we choose the delay \(\iota\) as 1, and the embedding dimension m as 3. However, at present, there is no exact calculation formula to select the scale factor. To determine the best scale of RCMDE, multiple groups of signals are randomly selected from the three segments of signals divided in Fig. 4a. The algorithm proposed is used to decompose and reconstruct the signals, and the change of RCMDE when the scale factor is from 1 to 16 is analyzed from the reconstructed signals. The calculation results of each group are similar, and one group of calculation results is selected for analysis, as shown in Fig. 8. When the scale factor is 1, that is, for the single-scale DE, the entropy values of the severe chatter and the slight chatter are very close, so it is difficult to distinguish between the two machining states. Moreover, the entropy decreases with the increase of the chatter level at the same scale. But the entropy value of the stable state of the single-scale DE is the lowest, which does not meet the requirements. Therefore, single-scale DE cannot effectively judge the milling machining state. When the scale factor is 4 to 16, the entropy decreases with the increase of chatter level. Especially when the scale factor is 4, the distinction between the three machining states is the largest. Therefore, the best index for chatter detection is RCMDE with a scale factor of 4.

Fig. 8
figure 8

Changes of RCMDE with the scale factor of signal decomposed and reconstructed by GA-SE-SCK-VMD

To verify that RCMDE performs better than MDE in chatter detection, the reconstructed signals of three machining states are selected, as shown in Fig. 8, and the change of MDE when the scale factor is from 1 to 16 is analyzed. The results are shown in Fig. 9, and the general trend is the same as that of RCMDE. Because RCMDE selects the initial point for multi-scaling according to the scale factor, the values of RCMDE and MDE are the same in the case of the single scale. It can be seen from Figs. 8 and 9 that when the scale factor is greater than 2, the value of MDE is less than RCMDE. Because DE is a nonlinear dynamic method to characterize the irregularity and complexity of time series, it can be judged that MDE has missed some information in the process of multi-scaling. In addition, the distinction between MDE and RCMDE in the three machining states is the largest, when the scale factor is 4. Calculate the distance between the entropy curves of MDE and RCMDE when the scale factor is 4, and compare the distinction between them. The results are shown in Table 5. RCMDE has a significantly higher distinction in the machining state than MDE. Therefore, RCMDE has a better performance in chatter detection than MDE.

Fig. 9
figure 9

Changes of MDE with the scale factor of signal decomposed and reconstructed by GA-SE-SCK-VMD

Table 5 Comparison of distinguishing effects between MDE and RCMDE

To verify the superiority of the proposed algorithm, the algorithm proposed is compared with the non-optimized VMD algorithm. The VMD parameters are initialized randomly by the GA algorithm. The results are shown in Table 6. The data in the table are taken as the parameters of the non-optimized VMD to avoid the influence of human factors. The value of RCMDE of the reconstructed signal is extracted by the above method, and its change with the scale factor is analyzed. The results are shown in Fig. 10. The severe chatter state can be analyzed at a large scale, but the entropy values of the stable state and the slight chatter state are mixed and cannot be effectively distinguished. In summary, the effect of OVMD-RCMDE is better than that of non-optimized VMD-RCMDE. From the above comparison, it can be seen that the penalty factor \(\alpha\) and the decomposition level K in VMD have a significant impact on the decomposition effect and then affect the accuracy of chatter feature extraction.

Table 6 GA algorithm randomly initializes VMD parameters
Fig. 10
figure 10

The RCMDE of the signal decomposed and reconstructed by unoptimized VMD changes with the scale factor

The GA-SE-SCK-VMD algorithm proposed is used to calculate the RCMDE with the scale factor is 4, and the length of signal analysis is set to 1024. The results are shown in Fig. 11. It can be seen from the figure that RCMDE is divided into three parts due to the sudden change in its value. There is an obvious sudden change in the value of RCMDE at about 6.14 s, with a difference of about 0.48, and at about 31.74 s, the value of RCMDE changed significantly, with a difference of about 0.47. These two obvious sudden changes indicate that the machining state changes from a stable state to a slight chatter state and then to a severe chatter state.

Fig. 11
figure 11

Change of RCMDE with time

To verify the accuracy of judging the change time point of the machining state through the entropy diagram, the machining state is further divided according to the surface quality of the workpiece, and the machining quality of the workpiece is photographed with the microscope. The results are shown in Fig. 12. The surface quality of surface quality of the workpiece is good in the stable state, without obvious vibration marks. In the slight chatter state, due to the self-excited vibration between the tool and the workpiece, the workpiece surface has a slight vibration mark with small spacing. In the severe chatter state, as the self-excited vibration becomes more and more intense, the surface quality of the workpiece becomes worse, severe vibration marks appear, and the spacing increases significantly. According to the cutting parameters, the machining state is stable in 0–6.14 s, slight chatter in 6.14–31.74 s, and severe chatter in 31.74–40 s, which are roughly the same as the time point for judging the change of the machining state according to entropy value, which proves the accuracy of the chatter feature extracted in this paper; comparing this time point with the time point that the machining state is divided according to the time domain signal in sect. 4.2, it can be found that there is a lag phenomenon in judging the machining state according to the time domain signal, so it is not the best choice to divide the machining state according to the time domain signal. Therefore, it is particularly important to select the chatter feature that can reflect the machining state and detect the change in the machining state in time. Through the experiments in this section, it can be found that the algorithm proposed can detect the occurrence of chatter in the early chatter, which lays a foundation for subsequent machining state identification and chatter suppression. Through the comparison of RCMDE with different scale factors, it is proved that RCMDE is more conducive to chatter detection than single-scale DE.

Fig. 12
figure 12

Workpiece surface quality and surface roughness

5 Conclusions

A new method for detecting milling chatter is proposed in this paper. Firstly, the parameters of VMD are optimized by GA, and a new index SE-SCK is introduced as the function for judging the optimal solution of parameter optimization. The milling force signals collected in the experiment are decomposed by OVMD, and the first three sub-components with high energy ratios are selected for signal reconstruction. RCMDE features are extracted from the reconstructed signals for chatter identification, and the change of RCMDE is used to replace the absolute threshold to detect the occurrence of chatter. The results of the experiment show that the OVMD algorithm proposed can separate the chatter frequency band separately, and its performance is better than the EMD algorithm. The performance of RCMDE in chatter detection is better than MDE. Compared with RCMDE extracted from the reconstructed signal without optimized VMD decomposition, the superiority of the OVMD algorithm proposed is proven. And the proposed algorithm can identify the occurrence of chatter in the early stage of chatter and lay the foundation for subsequent chatter suppression. In this paper, the method presented provides a new method for detecting milling chatter.