1 Introduction

Clearance-type nonlinearities such as gear backlash are related to convergence problems when nonlinear dynamic responses in a practical system are simulated. The clearance itself contains discontinuities due to contact or non-contact behaviors [1, 2]. Much prior research has been done to develop time-varying stiffness models or clearance-type nonlinear models with smoothening functions [312]. For example, Shen et al. [3] established a dynamical model of a spur gear pair by including the backlash, time-varying stiffness, and static transmission error. Rao et al. [4] studied the torsional instabilities in a two-stage gear system by considering the torsional flexibility of the shafts and the meshing time-varying stiffness of the gears.

Al-shayyab and Kahraman [5] investigated sub-harmonic and chaotic motions in a multi-mesh gear train using a nonlinear time-varying dynamic model. Raghothama and Narayanan [6] used the incremental harmonic balance method to obtain the periodic motions of a 3-degree-of-freedom (DOF) nonlinear model of a geared rotor system subjected to parametric excitation under sinusoidal excitation. Wong et al. [7] presented the non-linearities in the restoring force by employing the incremental harmonic balance method. Kim et al. [8] showed the differences in the nonlinear frequency response characteristics when selected smoothening functions were employed by different smoothening factors. Smoothening functions have also been used to simulate clearance or piecewise nonlinearities of the mechanical systems in many prior studies [912].

The examination by Kim et al. [8] of the effect of the smoothening functions was limited to a simple torsional model with a single DOF with idealized system parameters. In general, the parameters from a practical system can cause more difficulties in estimation, especially when the dynamic conditions of the system are affected by vibro-impacts, such as gear rattle in the wide-open-throttle (WOT) condition [1, 2, 9, 10]. Under severe gear impact conditions, the gear mesh stiffness changes abruptly from 0 to \(2.7\times 10^{8}\;\hbox {N}\;\hbox {m}^{-1}\) (or from \(2.7\times 10^{8}\) to \(0\;\hbox {N}\;\hbox {m}^{-1}\)). In addition, the prior studies focused on specific smoothening functions using hyperbolic tangent or arctangent functions [1, 2, 812, 21, 24], which are not enough to overcome the convergence problems with respect to the gear impact behaviors in a practical system. Thus, the main objectives of this study contain: (1) suggesting a new smoothening function model using an nth order polynomial function by comparing with prior models [8]; (2) examining the key factors of new smoothening function model along with certain ranges; (3) and investigating the limits of the simulation based on the harmonic balance method (HBM) by focusing on gear impact conditions.

2 Modeling of the practical system and nonlinearity

2.1 Physical system and its parameters

Figure 1a depicts the physical system based on a front-engine and front-wheel driveline with a manual transmission [10]. Based on the practical system shown in Fig. 1a, a schematic diagram can be constructed as shown in Fig. 1b. All of the loaded gears are assumed to be lumped into the input shaft without changing the dynamic characteristics of the system, and focus is centered on only one unloaded gear pair to examine the vibro-impact phenomenon [1, 2]. In addition, the employed gear pairs are assumed to be geometrically ideal without any errors under the dynamic conditions. Thus, the reduced order of the lumped system model with 4 DOF will be used to investigate the dynamic behaviors with smoothening functions.

The symbols used and their parameter values shown in Fig. 1 are described in Tables 1 and 2, where the given properties are measured and given from the relevant industry with a practical driveline. Here, the drag torques are assumed to be constant values in the given engine operating condition, and the damping of the clutch damper \(c_{\mathrm{f}}\) and the damping of the drive shaft \(c_{\mathrm{ve}}\) are estimated based on the 5 % modal damping ratio [1, 2, 10]. The scope of this study is limited to the 3rd gear engaged and 5th gear unloaded status under severe driving conditions such as the wide-open-throttle (WOT) condition.

Fig. 1
figure 1

A torsional system with a front-engine and front-wheel layout: a a practical driveline with a manual transmission; b a schematic diagram with gear mesh force

2.2 Basic equations and their nonlinearities

Based on the system in Fig. 1, the matrix formulation of the basic equations is expressed as follows.

$$\begin{aligned}&{\underline{\underline{\mathbf{M}}}} {\ddot{\underline{\varvec{\uptheta }}}}\left( \mathbf{t }\right) + {\underline{\underline{\mathbf{C }}}} {\underline{\dot{\varvec{\uptheta }}}} \left( \mathbf{t }\right) +{\underline{\underline{{\mathbf{K }}}}} {\underline{\varvec{\uptheta }}}\left( \mathbf{t }\right) + \underline{{\mathbf{f }}_{{\mathbf{n }}} }\left( {\underline{\varvec{\uptheta }}} \right) =\underline{{\mathbf{T }}_{{\mathbf{E }}} }\left( \mathbf{t }\right) ,\end{aligned}$$
(1a)
$$\begin{aligned}&{\underline{\underline{{\mathbf{M }}}}} = \hbox {diag}\left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} I_\mathrm{f} ,&I_{\mathrm{ie}} ,&I_{\mathrm{ou}} ,&I_{\mathrm{ve}} \end{array} \right] , \end{aligned}$$
(1b)
$$\begin{aligned}&{\underline{\underline{{\mathbf{C }}}}} =\left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} {c_\mathrm{f} }&{} {-c_\mathrm{f} }&{} 0&{} 0 \\ {-c_\mathrm{f} }&{} {c_\mathrm{f} +c_\mathrm{g} R_{\mathrm{iu}}^2 +c_{\mathrm{ve}} }&{} {c_\mathrm{g} R_{\mathrm{iu}} R_{\mathrm{ou}} }&{} {-c_{\mathrm{ve}} } \\ 0&{} {c_\mathrm{g} R_{\mathrm{iu}} R_{\mathrm{ou}} }&{} {c_\mathrm{g} R_{\mathrm{ou}}^2 }&{} 0 \\ 0&{} {-c_{\mathrm{ve}} }&{} 0&{} {c_{\mathrm{ve}} } \\ \end{array} \right] , \quad \end{aligned}$$
$$\begin{aligned}&{\underline{\underline{{\mathbf{K }}}}} =\left[ \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} {k_\mathrm{f} }&{} {-k_\mathrm{f} }&{} 0&{} 0 \\ {-k_\mathrm{f} }&{} {k_\mathrm{f} +k_{\mathrm{ve}} }&{} 0&{} {-k_{\mathrm{ve}} } \\ 0&{} 0&{} 0&{} 0 \\ 0&{} {-k_{\mathrm{ve}} }&{} 0&{} {k_{\mathrm{ve}} } \\ \end{array} \right] , \end{aligned}$$
(1c,d)
Table 1 Employed parameters of the torsional system
$$\begin{aligned}&{\underline{{\mathbf{f }}_{{\mathbf{n }}}}} \left( {\underline{{\varvec{\uptheta }}}} \right) =\left[ \begin{array}{c} 0 \\ R_{\mathrm{iu}} F_{\mathrm{gu}} \left( {\rho _\mathrm{u} } \right) +T_{\mathrm{Di}} \\ R_{\mathrm{ou}} F_{\mathrm{gu}} \left( {\rho _\mathrm{u} } \right) -T_{\mathrm{Du}} \\ T_{\mathrm{Dve}} \end{array} \right] , \end{aligned}$$
$$\begin{aligned}&{\underline{{\mathbf{T }}_{{\mathbf{E }}}}}\left( \mathbf{t }\right) = \left[ \begin{array}{c} T_\mathrm{E} \left( t \right) \\ 0 \\ 0 \\ 0 \\ \end{array} \right] . \end{aligned}$$
(1e,f)
$$\begin{aligned}&F_{\mathrm{gu}} \left( {\rho _\mathrm{u} } \right) \\&\quad =k_\mathrm{g} \delta \left( {\rho _\mathrm{u} } \right) ,\,\, \rho _\mathrm{u} =R_{\mathrm{iu}} \theta _{\mathrm{ie}} \left( t\right) +R_{\mathrm{ou}} \theta _{\mathrm{ou}} \left( t \right) . \end{aligned}$$
(1g,h)

The angular displacements \({\underline{{\varvec{\uptheta }}}}\left( \mathbf{t }\right) \!=\!\left[ {{\begin{array}{c@{\quad }c@{\quad }c@{\quad }c} {\theta _\mathrm{f} }&{\theta _{\mathrm{ie}} }&{\theta _{\mathrm{ou}} }&{\theta _{\mathrm{ve}} } \end{array} }} \right] ^{\mathrm{T}}\) are defined as the absolute motions of the flywheel, input shaft, unloaded gear, and vehicle, respectively. \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) is defined as the gear mesh force between the input shaft and unloaded gear in terms of the translational relative displacement \((\rho _{\mathrm{u}})\) of the unloaded gear pair. \(\delta (\rho _{\mathrm{u}})\) reflects the discontinuous characteristics of the gear motions. To simulate the torsional system using the HBM, non-dimensionalization of the time scale was conducted by employing a new variable \(\psi =\omega t\) and the time range \(0\le t\le \upsilon \tau \) for one period \(\tau \) can be mapped onto \(0\le \psi \le 2\pi \) [9]. Here, \(\upsilon \) indicates the sub-harmonic index [11, 12]. Also, when \(\omega \) is parameterized, the normalized value \(\bar{\omega }\left( \omega /{\omega _N} \right) \) is employed in order to avoid possible convergence problems. Here, \(\omega _N\) is the natural frequency relevant to vibro-impacts on the system shown in Fig. 1. In this study, 47.6 Hz is used for \(\omega _N\). Profound studies with respect to the relationship of simulation convergence to multiple choices of non-dimensionalization are beyond the scope of this research.

Table 2 Employed properties for the drag torques on the relevant sub-system

To calculate the system response, the input torque \(T_{\mathrm{E}}(t)\) is assumed as a sinusoidal excitation as follows.

$$\begin{aligned} T_\mathrm{E} \left( t \right) =T_\mathrm{m} +\sum _{i=1}^{N_{\max }} {T_{\mathrm{pi}} \cos \left( {i\omega _p t+\phi _{\mathrm{pi}} } \right) }. \end{aligned}$$
(2)

The employed values of the input torques are as follows: the mean torque is \(T_{\mathrm{m}} = 168.9\;\hbox {N}\;\hbox {m}\); the alternating torque is \(T_{\mathrm{p1}} = 251.53\;\hbox {N}\;\hbox {m}\); and the phase is \(\phi _{\mathrm{p1}} = -1.93\). The system is assumed to be under steady state conditions with constant velocity. Thus, the drag torques of each sub-system shown in Fig. 1 are estimated by assuming that the summation of torques is equal to the mean torque value \(T_{\mathrm{m}}\) as follows [1, 2, 9, 10]:

$$\begin{aligned} T_\mathrm{m} =T_{\mathrm{Di}} +T_{\mathrm{Dve}} +T_{\mathrm{Du}} \left( {{R_{\mathrm{iu}} }/{R_{\mathrm{ou}}}} \right) . \end{aligned}$$
(3)
Fig. 2
figure 2

Nonlinear behaviors of the gear pairs under the engine excitation condition: a gear contact conditions under vibro-impacts; b expected gear mesh force along with gear backlash regime

Figure 2a illustrates three different conditions of the gear pair for “contact on the driving side”, “no contact”, and “contact on the driven side”. When the dynamic behaviors of the gear pair show repetitive motions between “contact on the driving side” and “no contact” (or “contact on the driving side” and “contact on the driven side), the torsional system is severely affected by vibro-impacts such as gear rattle [1, 2]. In general, this nonlinear dynamic effect is caused by the clearance between the driving and driven gears called backlash. The dynamic motions of the gear pairs are mathematically described as follows.

$$\begin{aligned} \delta \left( {\rho _\mathrm{u} } \right) =\left\{ \begin{array}{l@{\quad }l} \rho _\mathrm{u} +\frac{b}{2},&{} \rho _\mathrm{u} \le -\frac{b}{2} \\ 0,&{} -\frac{b}{2}<\rho _\mathrm{u} <\frac{b}{2} \\ \rho _\mathrm{u} -\frac{b}{2},&{} \rho _\mathrm{u} \ge \frac{b}{2} \end{array} \right. . \end{aligned}$$
(4)

The gear backlash is defined as b with a value of 0.1 mm. The backlash is a piecewise nonlinearity expressed using a unit step function \(U\left( {\rho _\mathrm{u} } \right) \) as follows:

$$\begin{aligned} \delta \left( {\rho _\mathrm{u} } \right)= & {} \left( {\rho _\mathrm{u} +\frac{b}{2}} \right) +\left[ \left( {\rho _\mathrm{u} -\frac{b}{2}} \right) \times \,U\left( {\rho _\mathrm{u} -\frac{b}{2}} \right) \right. \nonumber \\&\left. -\left( {\rho _\mathrm{u} +\frac{b}{2}} \right) \times U\left( {\rho _\mathrm{u} +\frac{b}{2}} \right) \right] . \end{aligned}$$
(5)

Based on a prior study [8] and Eqs. (4, 5), the dynamic gear mesh forces are estimated using the Model I and II smoothening functions, which are mathematically described as follows. Here, the relative motions are defined as \(\theta _{\mathrm{r1}} =\rho _\mathrm{u} -\frac{b}{2}\) and \(\theta _{\mathrm{r2}} =\rho _\mathrm{u} +\frac{b}{2}\).

Model I

$$\begin{aligned}&U_1 \left( {\theta _{\mathrm{r1}} } \right) \cong \frac{1}{2}\left[ {\tanh \left( {\sigma _g \theta _{\mathrm{r1}} } \right) +1} \right] ,\end{aligned}$$
(6a)
$$\begin{aligned}&U_1 \left( {\theta _{\mathrm{r2}} } \right) \cong \frac{1}{2}\left[ {\tanh \left( {\sigma _g \theta _{\mathrm{r2}} } \right) +1} \right] , \end{aligned}$$
(6b)
$$\begin{aligned}&\delta _{\mathrm{gu1}} \left( {\rho _\mathrm{u} } \right) \\&\quad =\theta _{\mathrm{r2}} +\left[ {\theta _{\mathrm{r1}} U_1 \left( {\theta _{\mathrm{r1}} } \right) -\theta _{\mathrm{r2}} U_1 \left( {\theta _{r2} } \right) } \right] , \quad \\&F_{\mathrm{gu1}} \left( {\rho _\mathrm{u} } \right) =k_\mathrm{g} \delta _{\mathrm{gu1}} \left( {\rho _\mathrm{u} } \right) . \end{aligned}$$
(6c,d)

Model II

$$\begin{aligned}&U_2 \left( {\theta _{\mathrm{r1}} } \right) \cong \frac{1}{2}\left[ {\frac{2}{\pi }\arctan \left( {\sigma _g \theta _{\mathrm{r1}} } \right) +1} \right] ,\end{aligned}$$
(7a)
$$\begin{aligned}&U_2 \left( {\theta _{\mathrm{r2}} } \right) \cong \frac{1}{2}\left[ {\frac{2}{\pi }\arctan \left( {\sigma _\mathrm{g} \theta _{\mathrm{r2}} } \right) +1} \right] , \end{aligned}$$
(7b)
$$\begin{aligned}&\delta _{\mathrm{gu2}} \left( {\rho _\mathrm{u} } \right) \\&\quad =\theta _{\mathrm{r2}} +\left[ {\theta _{\mathrm{r1}} U_2 \left( {\theta _{\mathrm{r1}} } \right) -\theta _{\mathrm{r2}} U_2 \left( {\theta _{\mathrm{r2}} } \right) } \right] , \quad \\&F_{\mathrm{gu2}} \left( {\rho _\mathrm{u} } \right) =k_\mathrm{g} \delta _{\mathrm{gu2}} \left( {\rho _\mathrm{u} } \right) . \end{aligned}$$
(7c,d)

where \(k_{\mathrm{g}}(= 2.7\times 10^{8}\;\hbox {N}\;\hbox {m}^{-1})\) is the gear mesh stiffness, and \(b(= 0.1\;\hbox {mm})\) is the gear backlash. Also, \(1\times 10^{10}\) is employed for the value of \(\sigma _{\mathrm{g}}\) for both Models I and II in order to simulate the sudden change of gear mesh forces under vibro-impact conditions in a practical system [1, 2]. Figure 3 compares the difference between two numerical models for unit step functions using \(U_1 \left( {\theta _{\mathrm{r1}} } \right) \) and \(U_2 \left( {\theta _{\mathrm{r1}} } \right) \). When those models are compared, \(U_1 \left( {\theta _{\mathrm{r1}} } \right) \) shows steeper change rather than \(U_2 \left( {\theta _{\mathrm{r1}} } \right) \). In order to compare two numerical unit step functions, \(1\times 10^{3}\) is employed for the value of \(\sigma _{\mathrm{g}}\).

Fig. 3
figure 3

Comparison of two numerical unit step functions. Solid blue line \(U_1 \left( {\theta _{\mathrm{r1}} } \right) \); dotted red line \(U_2 \left( {\theta _{\mathrm{r1}} } \right) \). (Color figure online)

Fig. 4
figure 4

Smoothening function Model III for different orders: a comparison of Model III with different orders; b \(\varvec{\upvarepsilon }_{1}\) with different orders. Dotted red line \(\hbox {n} = 2\); dashed with dot green line \(\hbox {n} = 4\); dashed black line \(\hbox {n} =6\). (Color figure online)

Figure 2b indicates the expected gear mesh force with the relationship of \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) versus \(\rho _{\mathrm{u}}\) using Model I or II. Based on the expected gear mesh force shown in Fig. 2b, the dynamic conditions of the forces are abruptly changed at \(b/2(= 0.05\,\hbox {mm})\) or \(-b/2\), which causes strong stiffness problems in the simulation with sudden changes of the values for the gear mesh force estimation. Differences between the two models will be explained with a new smoothening function model.

3 Mathematical model of the smoothening function for gear mesh force

3.1 Development of the new smoothening function

Based on the prior smoothening function models, it is not possible to change the smoothened areas and values on the left (or right) side from the gear backlash b / 2. The smoothening effect from the prior models given in Eqs. (4)–(7) are only dependent on the value of \(\sigma _{\mathrm{g}}\). Also, several problems have been observed: (1) unrealistic gear mesh forces by using Model I can be expected contrary to the measured ones which will be explained later; (2) computational problems using Model II can be caused with respect to the numerical convergence since fractional functions should be used when the Jacobin matrix is employed. Thus, the new smoothening function model should be considered to avoid those problems observe in the prior studies as well as to have more flexibility to adjust the smoothening areas with asymmetrical way for both left and right sides from b / 2. For satisfying this conditions, this study employs an nth order polynomial function since it is easy to adapt more flexible smoothening effects than the models in Eqs. (4)–(7). Here, small amounts of the displacements \(\varepsilon _{1}\) and \(\varepsilon _{2}\) at b / 2 or \(-b/2\) can be defined for gear mesh force estimation under smoothening conditions. These factors also lead to the smoothening change of \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) in the stiff regimes, as illustrated in Fig. 4a. Model III is mathematically described as follows:

$$\begin{aligned} \delta \left( {\rho _\mathrm{u} } \right) = \left\{ \begin{array}{l@{\quad }l} {\rho _\mathrm{u} +\frac{b}{2}},&{} {\rho _\mathrm{u} <-B_2 } \\ {-\alpha \left( {\rho _\mathrm{u} +B_1 } \right) ^{n}},&{} {-B_2 \le \rho _\mathrm{u} <-B_1 } \\ 0,&{} {-B_1 \le \rho _\mathrm{u} <B_1 } \\ {\alpha \left( {\rho _\mathrm{u} -B_1 } \right) ^{n}},&{} {B_1 \le \rho _\mathrm{u} <B_2 } \\ {\rho _\mathrm{u} -\frac{b}{2}},&{} {\rho _\mathrm{u} \ge B_2 } \\ \end{array} \right. ,\nonumber \\ \end{aligned}$$
(8a)
$$\begin{aligned}&B_2 =\frac{b}{2}+\varepsilon _2,\,\,\, B_1 =B_2 -n\varepsilon _2 ,\,\,\, \varepsilon _1 =\frac{b}{2}-B_1 ,\,\,\,\\&\alpha =\frac{\varepsilon _2 }{\left( {B_2 -B_1 } \right) ^{n}}. \end{aligned}$$
(8b-e)

where \(\varepsilon _{1}\) and \(\varepsilon _{2}\) are the distances from b / 2 or \(-b/2\), as shown in Fig. 4a. To create the smoothening changes in the area of b / 2 or \(-b/2\), the nth order polynomial functions \(\alpha \left( {\rho _\mathrm{u} -B_1 } \right) ^{n}\) or \(-\alpha \left( {\rho _\mathrm{u} +B_1 } \right) ^{n}\) are used, respectively.

Fig. 5
figure 5

Smoothening function Model III with \(\varepsilon _{2}\): a comparison of Model III with different values of \(\varepsilon _{2}\); b \(\varepsilon _{1}\) versus \(\varepsilon _{2}\). Dotted red line \(\varepsilon _{2} = 5\times 10^{-4}\); dashed with dot green line \(\varepsilon _{2} = 8\times 10^{-4}\); dashed black line \(\varepsilon _{2} = 1\times 10^{-3}\). (Color figure online)

As described in Eq. (8b), \(B_{2}\) is estimated simply by adding \(\varepsilon _{2}\) to b / 2, where \(\varepsilon _{2}\) can be determined arbitrarily. Thus, if \(\varepsilon _{2}\) is small, \(\varepsilon _{1}\) becomes small and makes the gear contact motions stiff. In Eqs. (8a)–(8e), the properties \(B_{1}\) and \(\alpha \) are determined based on several conditions. For example, the polynomial function should first contact the linear line of \(k_\mathrm{g} \left( {\rho _\mathrm{u} -\frac{b}{2}} \right) \) (or \(k_\mathrm{g} \left( {\rho _\mathrm{u} +\frac{b}{2}} \right) )\) at the range of \(\rho _\mathrm{u} \ge B_2 \) (or \(\rho _\mathrm{u} <-B_2\)) tangentially. Thus, the derivative value at the contact point is the same as the slope of tangent line \(k_{\mathrm{g}}\), which is the gear mesh stiffness. Second, the polynomial function \(\alpha \rho _\mathrm{u}^n \) (or \(-\alpha \rho _\mathrm{u}^n )\) is shifted with the amount of \(B_{1}\) (or \(-B_{1}\)) on the \(\rho _{\mathrm{u}}\) axis. Third, the order n must be even, since an odd number can cause \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) to be estimated below 0 N in the area between \(-b/2\) and b / 2, which does not occur in a practical system. Thus, from Eqs. (8a)–(8e), the gear mesh force \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) is derived as follows. Here, the relative motions are defined as \(\theta _{\mathrm{r1}} =\rho _\mathrm{u} -\frac{b}{2}\), \(\theta _{\mathrm{r2}} =\rho _\mathrm{u} +\frac{b}{2}\), \(\theta _{\mathrm{p1}} =\rho _\mathrm{u} -B_1 \), \(\theta _{\mathrm{p2}} =\rho _\mathrm{u} -B_2 \), \(\theta _{\mathrm{n1}} =\rho _\mathrm{u} +B_1 \) and \(\theta _{\mathrm{n2}} =\rho _\mathrm{u} +B_2 \).

$$\begin{aligned}&U_3 \left( {\theta _{\mathrm{n1}} } \right) \nonumber \\&\quad \cong \frac{1}{2}\left[ {\tanh \left( {\sigma _\mathrm{g} \theta _{\mathrm{n1}} } \right) +1} \right] , \quad \nonumber \\&U_3 \left( {\theta _{\mathrm{n2}} } \right) \nonumber \\&\quad \cong \frac{1}{2}\left[ {\tanh \left( {\sigma _g \theta _{\mathrm{n2}} } \right) +1} \right] ,\end{aligned}$$
(9a)
$$\begin{aligned}&U_3 \left( {\theta _{\mathrm{p1}} } \right) \nonumber \\&\quad \cong \frac{1}{2}\left[ {\tanh \left( {\sigma _\mathrm{g} \theta _{\mathrm{p1}} } \right) +1} \right] , \quad \nonumber \\&U_3 \left( {\theta _{\mathrm{p2}} } \right) \nonumber \\&\quad \cong \frac{1}{2}\left[ {\tanh \left( {\sigma _\mathrm{g} \theta _{\mathrm{p2}} } \right) +1} \right] ,\end{aligned}$$
(9b)
$$\begin{aligned}&\delta _{\mathrm{gu3}} \left( {\rho _\mathrm{u} } \right) \nonumber \\&\quad =\theta _{\mathrm{r2}} +\left[ {\theta _{\mathrm{r1}} \cdot U\left( {\theta _{\mathrm{p2}} } \right) -\theta _{\mathrm{r2}} \cdot U\left( {\theta _{\mathrm{n2}} } \right) } \right] \nonumber \\&\qquad -\,\alpha \theta _{\mathrm{n1}}^n \left[ {U\left( {\theta _{\mathrm{n2}} } \right) -U\left( {\theta _{\mathrm{n1}} } \right) } \right] +\alpha \theta _{\mathrm{p1}}^n \left[ U\left( {\theta _{\mathrm{p1}} } \right) \right. \nonumber \\&\left. \qquad -\,U\left( {\theta _{\mathrm{p2}} } \right) \right] ,\end{aligned}$$
(9c)
$$\begin{aligned}&F_{\mathrm{gu3}} \left( {\rho _\mathrm{u} } \right) \nonumber \\&\quad =k_\mathrm{g} \delta _{\mathrm{gu3}} \left( {\rho _\mathrm{u} } \right) . \end{aligned}$$
(9d)

Figure 4 compares the smoothening effects with regard to n. As shown in Fig. 4a, when n is increased to a certain value such as 6, \(\varepsilon _{1}\) becomes large, as shown with a dashed line in Fig. 4a. This indicates that the smoothening effect starts more gradually than when n is reduced to 2, as shown with the dotted line. Thus, increasing the smoothening factor \(\varepsilon _{1}\) leads to improved numerical convergence conditions by avoiding sudden changes of the gear mesh stiffness from \(0\;\hbox {N}\;\hbox {m}^{-1}\) to \(2.7\times 10^{8}\;\hbox {N}\;\hbox {m}^{-1}\).

Fig. 6
figure 6

Comparison of three smoothening functions: a comparison of the smoothening function Models I, II, and III; b smoothening effect with Models I, II, and III in the area of \(-b/2\); c smoothening effect with Models I, II, and III in the area of the b / 2. Dotted blue line Model I; dashed black line Model II; dashed with dot red line Model III. (Color figure online)

Accordingly, Fig. 4b shows the relationship between \(\varepsilon _{1}\) and n. When n becomes large, the distance \(\varepsilon _{1}\) from b / 2 or \(-b/2\) increases. As explained previously, \(\varepsilon _{2}\) also affects the smoothening change of the gear contact motion. Figure 5 illustrates the relationship of \(\varepsilon _{2}\) to the smoothening effect. Figure 5a shows that \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) becomes stiff, as indicated by , when the value of the \(\varepsilon _{2}\) is reduced, indicated by . When \(\varepsilon _{2}\) is a small value such as \(1\times 10^{-5}\), the dynamic characteristics of the gear contact are closer to the practical motions than when \(\varepsilon _{2}\) is a large value such as \(1\times 10^{-3}\). Also, \(\varepsilon _{1}\) is estimated based on \(\varepsilon _{2}\) as described in Eqs. (8b)–(8e), from which the smoothening effect can be mathematically determined.

3.2 Comparison of the smoothening function models

Figure 6 shows the differences between Models I, II, and III. First, Model I uses a hyperbolic tangent function, as described in Eq. (6). This function shows a negative or positive overshoot in the area of b / 2 or \(-b/2\). Thus, Model I includes severe error since the gear mesh forces never fall into the negative or positive values before the gear contact occurs at b / 2 or \(-b/2\). Models II and III do not show the same problem as Model I, which is shown in Fig. 6. However, Models II and III slightly differ with respect to the areas where the smoothening changes of \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) occur. For example, the smoothened \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) in Model II is determined only by the factor \(\sigma _{\mathrm{g}}\) described in Eq. (7), but it is difficult to anticipate the location of \(\varepsilon _{1}\) and \(\varepsilon _{2}\). In contrast, \(\varepsilon _{1}\) with Model III is exactly estimated as long as \(\varepsilon _{2}\) is defined by Eq. (8). Moreover, \(\varepsilon _{1}\) is determined based on the order n from Eqs. (8c) and (8d). Thus, the dynamic characteristics with the smoothening effect in Model III are more definite than in Models I and II.

HBM was used to investigate the effect of the smoothening functions with different models [3, 525]. The development and basic process of the HBM for systems with one or more DOF have been introduced in prior studies [9, 10]. Figure 7 compares the HBM results using Models I, II, and III. The system responses in the figure were estimated by the maximum number of harmonics \(N_{\max } = 6\). The value of \(\sigma _{\mathrm{g}}\) is \(1\times 10^{10}\) for all of the models as described in Sect. 2.2, and \(\varepsilon _{2}\) and n for Model III are \(1\times 10^{-5}\) and 20, respectively. Here, \(\delta _{2}(t)=R_{\mathrm{iu}}\theta _{\mathrm{ie}}(t)+R_{\mathrm{ou}}\theta _{\mathrm{ou}}(t)\) is the relative displacement between the input shaft and unloaded gear, and \(\delta _{2(\max )}\), \(\delta _{2(\mathrm{mean}),}\) and \(\delta _{2(\mathrm{min})}\) are the maximum, mean, and minimum values of \(\delta _{2}(t)\), respectively, in one period of time responses, as shown in Fig. 7 with frequency sweeping conditions.

Fig. 7
figure 7

Comparison of the HBM (\(N_{\max } = 6\)) using smoothening function Models I, II, and III. Dotted blue line Model I; dashed black line Model II; dashed with dot red line Model III. (Color figure online)

Table 3 Feasibility analysis of the smoothening functions with the number of harmonics

The results for all the models correlated well with each other. Also, single-sided and double-sided vibro-impacts were observed clearly between \(\bar{{\omega }} = 0.775\) and \(\bar{{\omega }} = 1.175\). \(\bar{{\omega }}\) is the frequency normalized using the natural frequency \(\omega _N = 47.6\;\hbox {Hz}\). When the number of harmonics is increased, the simulation using HBM shows many differences between Models I, II, and III.

Table 3 describes the feasibility of the simulation using each smoothening function model. The nonlinear responses with \(N_{\max } = 6\) can be successfully simulated by using HBM. However, when the number of harmonics increased to more than 6, the simulation never accomplished calculating the nonlinear responses using Models I and II with any values of \(\sigma _{\mathrm{g}}\), as indicated in Table 3. However, the simulation with Model III was successfully conducted with all numbers of harmonics with some discrepancies. Based on the results described in Table 3, the simulation cases of Model III are more feasible for adaption with different harmonic conditions than Models I and II, since Model III includes more factors to manage the smoothening conditions, such as \(\varepsilon _{2}\), n, and \(\sigma _{\mathrm{g}}\) based on Eq. (8).

4 Results

4.1 Comparison of the numerical analysis and HBM using the new smoothening function model

Figure 8 shows the comparison of the simulated results using numerical analysis (NS) and HBM in the frequency domain. When the computational time is compared, NS needs 2 h for both up-(or down-)frequency sweeping. On the other hand, the calculation time with the HBM is 20 min. Both results reflect the vibro-impacts well. However, the HBM does not follow the NS results for the gear impact conditions. When the system has double-sided impacts between \(\bar{{\omega }} = 0.8403\) and \(\bar{{\omega }} = 1.122\), as indicated with a solid line in Fig. 8, severe differences between NS and HBM at \(\delta _{2(\max )}\) are observed. This is caused by the limits of the HBM, since the number of harmonics is 6, which was examined in a prior study [9]. Since the vibro-impacts contain inherently impulsive responses, they include multiple numbers of harmonics and higher super-harmonic components than \(N_{\max } = 6\) or 12 [2]. Also, the limit of the number of harmonics with HBM causes a discrepancy of the phase in the Fourier components in comparison with the FFT results from NS. These behaviors are shown in Figs. 91011 and 12.

Fig. 8
figure 8

Comparison of the HBM and NS results. Solid blue line HBM with Model III; circled red line NS. (Color figure online)

Fig. 9
figure 9

Comparison of the relative displacements with HBM and NS under different excitation conditions in the time domain: a time histories of \(\delta _{1}(t)\) and \(\delta _{2}(t)\) at \(\bar{{\omega }} = 0.825\); b time histories of \(\delta _{1}(t)\) and \(\delta _{2}(t)\) at \(\bar{{\omega }} = 1.0\). Solid blue line NS; dashed red line HBM. (Color figure online)

Fig. 10
figure 10

Comparison of the gear mesh forces with HBM and NS under different excitation conditions in the time domain: a time histories of \(F_{\mathrm{gu}}(t)\) at \(\bar{{\omega }} = 0.825\); b time histories of \(F_{\mathrm{gu}}(t)\) at \(\bar{{\omega }} = 1.0\). Solid blue line NS; dashed red line HBM. (Color figure online)

Fig. 11
figure 11

Comparison of the relative displacements with HBM and NS under different excitation conditions in the frequency domain: a FFT results of \(\delta _{2}(t)\) at \(\bar{{\omega }} = 0.825\); b FFT results of \(\delta _{2}(t)\) at \(\bar{{\omega }} = 1.0\). Line with square red line NS; line with circle blue line HBM. (Color figure online)

Fig. 12
figure 12

Comparison of the gear mesh forces with HBM and NS under different excitation conditions in the frequency domain: a FFT results of \(F_{\mathrm{gu}}(t)\) at \(\bar{{\omega }}= 0.825\); b FFT results of \(F_{\mathrm{gu}}(t)\) at \(\bar{{\omega }} = 1.0\). Line with square red line NS; line with circle blue line HBM. (Color figure online)

Figure 9 compares the relative displacements from HBM and NS in the time domain. Figure 9a shows the results estimated at the excitation frequency of \(\bar{{\omega }} = 0.825\), and Fig. 9b shows the results at \(\bar{{\omega }} = 1\). Here, \(\delta _{1}(t) = \theta _{\mathrm{f}}(t) -\theta _{\mathrm{ie}}(t)\) is the relative displacement between the flywheel and input shaft. Figure 9a shows the comparisons of time histories under single-sided impact. The results of \(\delta _{1}(t)\) from HBM and NS correlated well with each other. However, \(\delta _{2}(t)\) shows minor differences, especially when the gear impact occurs at b / 2, as indicated by the dotted circle. In general, NS includes the impulsive responses well, since it is based on time domain analysis. This discrepancy of HBM is clearly observed when the system has double-sided impacts, as shown in Fig. 9b. Since the double-sided impact conditions are worse than the single-sided impacts, the impulsive responses are more dominant than the single-sided impacts, as indicated with dotted lines.

Figure 10 compares the gear mesh forces \(F_{\mathrm{gu}}(t)\) at two different excitation conditions at \(\bar{{\omega }} = 0.825\) and \(\bar{{\omega }} = 1\). Figure 10a, b simulates the single-sided and double-sided impacts, respectively. For example, the dynamic behaviors of \(F_{\mathrm{gu}}(t)\) shown in Fig. 10a follow the zero values at the first time, and then they go up to high values around 370 N on only the positive side. However, the NS results show more density of the impulsive force responses than the HBM results. The result of \(F_{\mathrm{gu}}(t)\) shown in Fig. 10b depicts the double-sided impacts. In a similar manner to the single-sided impacts in Fig. 10b, \(F_{\mathrm{gu}}(t)\) from NS shows more density at the moment when the gear impacts occur. The dynamic behaviors of \(F_{\mathrm{gu}}(t)\) for the double-sided impacts show repetitive motions on both the positive and negative sides. Also, the peak values for the double-sided impacts are higher than those for the single-sided impacts. Each local maxima is 1200 N, and the minima are \(-1100\;\hbox {N}\). Overall, both comparisons between HBM and NS show that NS results contain more super-harmonic components than the HBM results. These discrepancies between the different simulations were found based on the FFT results as well.

Figures 11 and 12 compare the FFT results of \(\delta _{2}(t)\) and \(F_{\mathrm{gu}}(t)\) with 6 harmonics. Figure 11 shows the comparison of the FFT results from \(\delta _{2}(t)\) based on HBM and NS. When the single-sided impact occurs, severe discrepancies between the two simulations are not observed, as shown in Fig. 11a. However, the phase values at the 4th, 5th, and 6th harmonics do not correlate well. The FFT results for the double-sided impact show severe differences in the phases between HBM and NS for all ranges of harmonic components, which is clearly observed in Fig. 11b.

Figure 12 compares the FFT results of \(F_{\mathrm{gu}}(t)\) for both single- and double-sided impacts. In a similar manner to the FFT results of \(\delta _{2}(t)\), the FFT results of \(F_{\mathrm{gu}}(t)\) for the single-sided impact in Fig. 12a show that the phase of the 4th, 5th, and 6th harmonics from HBM is not correlated well with those of the NS. Also, the harmonic spectra of the phases for the double-sided impact for HBM show severe differences from the results of the NS, except for the phase component at the 1st harmonic. Thus, based on the results in Figs. 11 and 12, the limit number of harmonics employed for HBM causes discrepancies for the time response estimations in which high super-harmonic components such as those beyond 500 Hz are not included [2]. However, the HBM can still be used to assess the vibro-impact phenomena in frequency sweeping conditions in both frequency and time domains, as shown in Figs. 8 and 9.

4.2 Effect of the \(\varepsilon _{2}\) values in HBM with respect to vibro-impacts

In general, clearance nonlinearity inherently contains discontinuity, which makes gear mesh forces suddenly change from zero to high stiffness levels, or vice versa. These dynamic characteristics can cause convergence problems or consume calculation time. Thus, the smoothening functions are employed to resolve these problems. Based on Eqs. (8) and (9), smoothening effects can be adapted using different levels of \(\varepsilon _{2}\). For example, as the level of \(\varepsilon _{2}\) is increased, the relative displacements \(\delta _{2}(t)\) or gear mesh forces \(F_{\mathrm{gu}}(\rho _{\mathrm{u}})\) of the unloaded gear pair are estimated more smoothly than when \(\varepsilon _{2}\) is reduced.

Figure 13 compares the dynamic behaviors of \(\delta _{2}(t)\) estimated with different \(\varepsilon _{2}\). The simulations with \(\varepsilon _{2} = 5\times 10^{-5}\) and \(1\times 10^{-4}\) show discrepancies with the results of \(\varepsilon _{2} = 1\times 10^{-5}\). The HBM results with \(\varepsilon _{2} = 1\times 10^{-5}\) are assumed to simulate the vibro-impacts reasonably except for the impulsive responses, as described in Sect. 4.1. As shown in Fig. 13, the effect of \(\varepsilon _{2}\) is observed clearly in both the \(\delta _{2(\max )}\) and \(\delta _{2(\mathrm{RMS})}\) results, where \(\delta _{2\left( {\hbox {RMS}} \right) } =\sqrt{\frac{1}{m}\left\{ {\delta _2^2 \left( {t_0 } \right) +\delta _2^2 \left( {t_1 } \right) +\cdots +\delta _2^2 \left( {t_{m-1} } \right) } \right\} }\), and m is the number of steps in one period of the time history.

Fig. 13
figure 13

Comparison of the HBM results under different values of \(\varepsilon _{2}\) with the smoothening function Model III: a max, mean, and min values of \(\delta _{2}(t)\) with \(\varepsilon _{2}\); b RMS values of \(\delta _{2}(t)\) with \(\varepsilon _{2}\). Solid blue line \(\varepsilon _{2} =1\times 10^{-5}\); dotted red line \(\varepsilon _{2} =5\times 10^{-5}\); dashed black line \(\varepsilon _{2} =1\times 10^{-4}\). (Color figure online)

Fig. 14
figure 14

Effect of \(\varepsilon _{2}\) with number of harmonics: a comparison of the max, mean, and min values of \(\delta _{1}(t)\) with different \(\varepsilon _{2}\) values and numbers of harmonics; b number of harmonics versus available minimum values of \(\varepsilon _{2}\). Solid blue line \(N_{\max } = 6\); dotted red curve \(N_{\max } = 8\); dashed black line \(N_{\max } = 10\); dashed with dot green line \(N_{\max } = 12\). (Color figure online)

As indicated with the solid line in Fig. 13a, b, the vibro-impacts occur between \(\bar{{\omega }} = 0.775\) and \(\bar{{\omega }} = 1.175\). However, the simulation results with \(\varepsilon _{2} = 1\times 10^{-4}\) show severe discrepancy, since \(\delta _{2(\max )}\) has values below \(b/2 = 0.05\;\hbox {mm}\) in the frequency ranges with \(\bar{{\omega }} < 0.775\) and \(\bar{{\omega }} > 1.175\). The results of \(\delta _{2(RMS)}\) with both \(\varepsilon _{2} = 5\times 10^{-5}\) and \(\varepsilon _{2} = 1\times 10^{-4}\) are very different from those with \(\varepsilon _{2} = 1\times 10^{-5}\) at \(\bar{{\omega }} < 0.775\) and \(\bar{{\omega }} > 1.175\). In the simulated result of \(\delta _{2(\max )}\) with \(\varepsilon _{2} = 1\times 10^{-4}\) shown in Fig. 13a, \(\delta _{2(\max )}\) levels below \(b/2 = 0.05\;\hbox {mm}\) reflect that the gear pairs never contact each other, which is not practically reasonable for the physical system. Thus, to achieve reasonable simulation results, \(\varepsilon _{2}\) should be determined within a certain range. For this study, all of the HBM results are estimated with \(\varepsilon _{2} = 1\times 10^{-5}\).

Table 4 Employed minimum values of \(\varepsilon _{2}\) for the smoothening function Model III with the number of harmonics

Figure 14 and Table 4 show the relationship of \(\varepsilon _{2}\) values considered with the number of harmonics. Figure 14a shows the HBM results with different numbers of harmonics \(N_{\max }\). As \(N_{\max }\) is increased, the discrepancies become severe compared with the HBM results with \(N_{\max } = 6\). The reason why the accuracy of the HBM results dropped is the \(\varepsilon _{2}\) value. The \(\varepsilon _{2}\) values versus \(N_{\max }\) are indicated in Fig. 14b and described in Table 4. To overcome the convergence problems as \(N_{\max }\) is increased, \(\varepsilon _{2}\) must be larger than \(1\times 10^{-5}\) with \(N_{\max }= 6\). For example, the minimum value of \(\varepsilon _{2}\) considered for \(N_{\max }= 6\) is \(1\times 10^{-7}\). However, \(\varepsilon _{2}\) should be equal to or larger than \(1\times 10^{-3}\), when \(N_{\max }\) is increased to more than 6, as described in Table 4. Thus, due to the increase in \(\varepsilon _{2}\), the convergence problems are resolved. On the other hand, large \(\varepsilon _{2}\) causes the discrepancies shown in Fig. 14a.

5 Conclusion

New smoothening functions have been suggested and compared with other models in prior studies [8]. Using this model, the limits of HBM have been examined with respect to the impulsive responses in the time histories of \(\delta _{2}(t)\), \(F_{\mathrm{gu}}(t)\), and the FFT results. The specific contributions of this study are summarized as follows. First, the new smoothening function model has been suggested by using an nth order polynomial function. This model could achieve smoothening changes of the gear mesh force in the discontinuous regime, which leads to solving the convergence problems. Second, the feasibility of the smoothening factor \(\varepsilon _{2}\) has been examined. Increasing the value of \(\varepsilon _{2}\) assured that the convergence problems were removed. However, \(\varepsilon _{2}\) beyond certain values caused severe errors, especially in the frequency ranges where the vibro-impacts do not occur. Third, the limit of the HBM with respect to the impact phenomena such as gear rattle has been investigated. In the comparison of the HBM with NS, the vibro-impact behaviors simulated using HBM were still revealed, even though all of the impulsive response components were not included, which is directly due to the limited number of harmonics.

Based on the study of \(N_{\max }\) versus \(\varepsilon _{2}\), the Model III smoothening function still has a discrepancy, even though it successfully overcomes the convergence problems. Thus, the development of multiple non-dimensionalization methods or the investigation of analytical formulations for clearance-type nonlinearities for HBM is the potential subjects for further study.