1 Introduction

It is well known that the study of the nonlinear wave equation is more and more important in many fields of physics. Finding their traveling solutions of these equations has become a hot research topic for many scholars. Many methods have been used to investigate these types of equations, such as tanh–sech method [1], Lie group method [2], exp-function method, bifurcation method [3–6] and sine–cosine method.

In the shallow-water fields, many two-component systems already have increasing been payed attention and studied. For example, one is the well-known generalized two-component Camassa–Holm system:

$$\begin{aligned} \left\{ \begin{array}{l} u_t-u_{xxt}-Au_x+3uu_x+e\rho \rho _x\\ \quad -\,\sigma (2u_xu_{xx}+uu_{xxx})=0,\\ \rho _t+(\rho u)_x=0. \end{array}\right. \end{aligned}$$
(1.1)

where \(u(t, x)\) represents the horizontal velocity of the fluid, \(\sigma \) is a new free parameter and \(A>0\). The researchers have studied the solitary wave solutions, cusp wave solutions, periodic wave solutions, kink and anti-kink wave solution and breaking wave solutions of the system with \(e=\pm 1\) [7–9]. Hitherto this equation has already attracted a lot of attention all around the world.

In [10–12], Moon and Wu considered the generalized two-component Hunter–Saxton system with \(k=1\)

$$\begin{aligned} \left\{ \begin{array}{l} u_{xxt}+2\sigma u_xu_{xx}+\sigma uu_{xxx}-k\rho \rho _x+Au_x=0,\\ \rho _t+(\rho u)_x=0, \end{array}\right. \nonumber \\ \end{aligned}$$
(1.2)

where \(\sigma \) is a new free parameter and \(A\ge 0\). The system (1.2) is the short wave limit \(((t,x)\mapsto (\epsilon t, \epsilon x), \epsilon \rightarrow 0)\) of the generalized two-component Camassa–Holm system (1.1). Wu and Wunsh have gotten the global existence of solutions to the system (1.2) with \(\sigma =1, k=1\) in the periodic setting [12]. Moon and Liu have further obtained the global existence of solutions to the system (1.2) with \(\sigma \in R,k=1\) using by the localization analysis in [10]. Also, Moon has obtained some soliton wave solutions and peak solitary solutions of the equation using by the small perturbations method [11].

When \((\sigma , A)=(1,0)\), system (1.2) is become the two-component Hunter–Saxton system:

$$\begin{aligned} \left\{ \begin{array}{l} u_{xxt}+2u_xu_{xx}+uu_{xxx}-k\rho \rho _x=0,\\ \rho _t+(\rho u)_x=0. \end{array}\right. \end{aligned}$$
(1.3)

where \(k=\pm 1\). This system is a special case of Green–Naghdi system modeling the non-dissipative dark matter [13]. Many mathematical properties of this system have been also studied further in many works [14–16]. In [17], the scholars have obtained a smooth periodic solution traveling wave of the two-component Hunter–Saxton equation and mentioned no bounded traveling waves of Hunter–Saxton equation.

The aim of this paper is to study the bifurcations of traveling solutions and exact traveling solutions of system (1.2) by using the bifurcation method of dynamical system.

Let \(u(x,t)=\phi (x-ct)=\phi (\xi ), \rho (x,t)=\nu (x-ct)=\nu (\xi )\), where \(c\) is the speed of waves. Then, the second equation of system (1.2) was written as:

$$\begin{aligned} -c\nu ^{\prime }+(\phi \nu )^{\prime }=0, \end{aligned}$$

where “\(\prime \)” is the derivative with respect to \(\xi \). Integrating this equation, we have

$$\begin{aligned} \nu (\xi )=\frac{B}{\phi -c}. \end{aligned}$$
(1.4)

where \(B\) is an integral constant and \( B\ne 0\). Substituting (1.4) into the first equation of (1.2), then we have the following ordinary equation of the first equation of (1.2):

$$\begin{aligned} -c\phi ^{\prime \prime \prime }+2\sigma \phi ^{\prime }\phi ^{\prime \prime }+ \sigma \phi \phi ^{\prime \prime \prime }-k\nu \nu ^{\prime }+A\phi ^{\prime }=0. \end{aligned}$$

Once integrating this equation, we obtain

$$\begin{aligned} (\sigma \phi -c)\phi ^{\prime \prime }\!=\!-\frac{1}{2}\sigma (\phi ^{\prime })^2-A\phi +\frac{kB^2}{2(\phi -c)^2}+\frac{g}{2}, \end{aligned}$$
(1.5)

where \(\frac{g}{2}\) is an integral constant. Equation (1.5) is equivalent to the two-dimensional planar system:

$$\begin{aligned} \left\{ \begin{array}{l} \frac{\hbox {d}\phi }{\hbox {d}\xi }=y,\\ \frac{\hbox {d}y}{\hbox {d}\xi }=\frac{-\sigma y^2(\phi -c)^2+(\phi -c)^2(g-2A\phi )+kB^2}{2(\sigma \phi -c)(\phi -c)^2}. \end{array}\right. \end{aligned}$$
(1.6)

The system has the first integral:

$$\begin{aligned} H(\phi ,y)=y^2(\sigma \phi -c)+A\phi ^2-g\phi +\frac{kB^2}{\phi -c}=h. \end{aligned}$$
(1.7)

Without loss of generality, let the speed of waves \(c>0\). On the singular straight lines \(\phi =c\) and \(\phi =\frac{c}{\sigma } \), the second equation in (1.6) is discontinuous. Such system (1.6) is called a singular traveling system [18–25]. In other words, \(\phi ^{\prime \prime }\) has not been defined on the straight lines in the phase plane \((\phi ,y)\). It derives that the differential system (1.2) could exist some non-smooth behavior or breaking properties of traveling wave solution.

This paper is organized as follows. In Sect. 2, we analyze the bifurcations of phase portraits of system (1.6) with \(k=\pm 1\). In Sect. 3, we give the parametric representations of the smooth solitary wave solutions, peakon solutions, cuspon solutions and compacton solutions of the (1.2). A short conclusion is given in Sect. 4.

2 Phase portraits of the system (1.6)

Let \(A>0\) and \(c>0.\) Making the transformation \(\hbox {d}\xi =(\sigma \phi -c)(\phi -c)^2 \hbox {d}\tau \) for \(\phi \ne c, \frac{c}{\sigma }\) on the system (1.6). Under this transformation, system (1.6) becomes its regular system:

$$\begin{aligned} \left\{ \begin{array}{l} \frac{\hbox {d}\phi }{\hbox {d}\tau }=(\phi -c)^2(\sigma \phi -c)y,\\ \frac{\hbox {d}y}{\hbox {d}\tau }=-\frac{1}{2}\sigma y^2(\phi -c)^2\\ \quad +\,\frac{1}{2}[(\phi -c)^2(g-2A\phi )+kB^2]. \end{array}\right. \end{aligned}$$
(2.1)

System (2.1) has same first integral as system (1.6). Consequently, expect for the singular straight line \(\phi =c\) and \(\phi =\frac{c}{\sigma }\), the system (2.1) has the same topological phase portraits as system (1.6). Clearly, two singular straight lines \(\phi =c\) and \(\phi =\frac{c}{\sigma }\) are two invariant straight line solutions for system (2.1). Close to the two straight lines, the system (2.1) and system (1.6) have different dynamics behaviors. The variable \(\tau \) is a fast variable while the variable \(\xi \) is a slow variable.

In order to find the equilibrium points of (2.1), we have

$$\begin{aligned}&f(\phi )=\frac{1}{2}[(\phi -c)^2(g-2A\phi )+kB^2].\end{aligned}$$
(2.2)
$$\begin{aligned}&f^{'}(\phi )=(\phi -c)(g-3A\phi +Ac).\end{aligned}$$
(2.3)
$$\begin{aligned}&f^{''}(\phi )=-6A\phi +4Ac+g. \end{aligned}$$
(2.4)

Apparently, \(f^{'}(\phi )\) has two zero at \(\phi =\phi _{s1}=c,\) and \(\phi =\phi _{s2}=\frac{g+Ac}{3A}\). So, we get \(f(c)=\frac{1}{2}kB^2, f^{'}(c)=0, f^{''}(c)=-2Ac+g,f(0)=\frac{1}{2}(kB^2+gc^2),\) and \(f^{''}(\phi _{s2})=2Ac-g.\)

In the \(\phi \)-axis, equilibrium points \(E_j(\phi _j,0)\) of (2.1) satisfy the function \(f(\phi _j)=0.\) Without loss of generality, assume \(c>0.\) The intersections of the hyperbola \(y=-\frac{kB^2}{(\phi -c)^2}\) and the straight line function \(y=g-2A\phi \) determine the real zero \(\phi _j(j=1,2\hbox { or }1,2,3)\) of the function \(f(\phi )\). If \(B\ne 0,\) there is no equilibrium point of (2.1) on the straight line \(\phi =c.\) If \(\sigma f(\frac{c}{\sigma })>0\), there are two equilibrium points \(S_{\pm }(\frac{c}{\sigma },\pm Y_s)\) of (2.1) on the straight line \( \phi =\frac{c}{\sigma }\), where \(Y_s=\sqrt{\frac{2f(\frac{c}{\sigma })}{\sigma (\frac{c}{\sigma }-c)^2}}.\)

2.1 Type 1: The case of \(k=1\)

Let \(k=1,\) then \(f(c)=\frac{1}{2}B^2>0.\) According to the following conditions, let us analysis the numbers and relative position of simple equilibrium points \(E_j(\phi _j,0)\) of system (2.1) .

(1). The case of \(g > 0.\) We know that when \(0<c<\frac{g}{2A}, f^{''}(c)>0,\) it has \(c< \phi _{s2}\), and when \(c>\frac{g}{2A}, f^{''}(c)<0,\) it has \(c> \phi _{s2}\).

  1. (i)

    Assume that \(c >\frac{g}{2A}>0.\) If \(f(\phi _{s2})<0,\) Eq. (2.1) has three simple equilibrium points \(E_j(\phi _j, 0), j=1, 2, 3\) and it satisfies \(\phi _1<\phi _{s2} <\phi _2 < c <\phi _3\). If \(f(\phi _{s2})=0,\) Eq. (2.1) has two simple equilibrium points \(E_j(\phi _j, 0), j=1, 2\) and it satisfies \(\phi _{1,2}=\phi _{s2}< c <\phi _3\), where equilibrium point \(\phi _{s2}\) is double root. If \(f(\phi _{s2})>0\), Eq. (2.1) only has one equilibrium point \(E_1(\phi _1, 0)\) satisfying \(\phi _1>c\).

  2. (ii)

    Assume that \(0 <c <\frac{g}{2A}.\) We always get \(f(\phi _{s2}) >0, \) then Eq. (2.1) has only one simple equilibrium point \(E_1(\phi _1, 0)\) satisfying \(\phi _1 <\phi _{s2}.\)

(2). The case of \(g<0\). In this case, we have \(c>0>\frac{g}{2A}\). This case is the same as the case \(c>\frac{g}{2A}>0\) with \(g>0\).

(3). The case of \(g=0\). In this case, we have the same conclusions as the case \(c>\frac{g}{2A}>0\) with \(g>0\).

2.2 Type 2: The case of \(k=-1\)

Then, we have \(f(c)=-\frac{1}{2}B^2<0\) and the following results.

(1). The case of \(g > 0.\) We know that when \(0<c<\frac{g}{2A}, f^{''}(c)>0,\) it has \(c< \phi _{s2}\), and when \(c>\frac{g}{2A}, f^{''}(c)<0,\) it has \(c> \phi _{s2}\).

  1. (i)

    Assume that \(c >\frac{g}{2A}>0.\) We always have \(f(\phi _{s2}) <0, \); then, Eq. (2.1) has only one simple equilibrium point \(E_1(\phi _1, 0)\) satisfying \(\phi _1 <\phi _{s2}.\)

  2. (ii)

    Assume that \(0<c<\frac{g}{2A}.\) If \(f(\phi _{s2})>0,\) Eq. (2.1) has three simple equilibrium points \(E_j(\phi _j, 0), j=1, 2, 3\) satisfying \(\phi _1< c <\phi _2 < \phi _{s2} <\phi _3\). If \(f(\phi _{s2})=0,\) Eq. (2.1) has two simple equilibrium points \(E_j(\phi _j, 0), j=1, 2\) satisfying \(\phi _1< c <\phi _{2,3}=\phi _{s2}\), where the equilibrium point \(\phi _{s2}\) is double root. If \(f(\phi _{s2})<0\), Eq. (2.1) has only one equilibrium point \(E_1(\phi _1, 0)\) satisfying \(\phi _1< c\).

(2). The case of \(g<0\). In this case, we have \(c>0>\frac{g}{2A}\). This case is the same as the case \(c>\frac{g}{2A}>0\) with \(g>0\).

(3). The case of \(g=0\). This case has the same conclusions as the case \(c>\frac{g}{2A}>0\) with \(g>0\).

Let \(M(\phi _j, y_j)\) is the coefficient matrix of the system (2.1) at an equilibrium point \(E_j(\phi _j, y_j)\), and \(J(\phi _j,y_j)\) is its Jacobian determinant. By the theory of planar dynamical system, we know that if \(J < 0,\) then the equilibrium point is a saddle point; if \(J > 0\) and \(Tr(M(\phi _j, y_j))=0,\) the equilibrium point is a center point; if \(J > 0\) and \((Tr(M(\phi _j, y_j)))^2-4J>0,\) the equilibrium point is a node; if \(J = 0\) and the index of equilibrium point is zero, then the equilibrium is a cusp; if \(J = 0\) and the index of equilibrium point is not zero, then the equilibrium is a high-order equilibrium point.

We get

$$\begin{aligned} J(\phi _j, 0&= detM(\phi _j,0)\nonumber \\&= -(\phi _i-c)^2(\sigma \phi _i-c)f'(\phi _i). \end{aligned}$$
(2.5)
$$\begin{aligned} J\left( \frac{c}{\sigma }, \pm Y_s\right)&= det M\left( \frac{c}{\sigma },\pm Y_s\right) \nonumber \\&= -\sigma ^2Y^2_s\left( \frac{c}{\sigma }-c\right) ^4. \end{aligned}$$
(2.6)

Let \(h_i=H(\phi _j, 0), h_s=H(\frac{c}{\sigma },\pm Y_s)\), where \(H\) is given by (1.7). For a given speed of wave \(c>0\), assume that the two condition holds as following:

  1. (1)

    \(k=1, g>0, A>0, c>\frac{g}{2A}\). For given \(A\) and \(g, f^{''}(c)<0,f^{''}(\phi _{s2})>0, f(c)>0\) and \(f(\phi _{s2})<0\). The points \(c\) and \(\phi _{s2}\) are a maximum point and minimum point of the function \(f(\phi )\), respectively. Three simple equilibrium points \(E_j(\phi _j,0), j=1, 2, 3,\) are satisfied \(\phi _1<\phi _{s2} <\phi _2 < c <\phi _3\).

  2. (2)

    \(k=-1, g>0, A>0, 0<c<\frac{g}{2A}.\) For given \( A\) and \(g, f^{''}(c)>0,f^{''}(\phi _{s2})<0,f(c)<0\) and \(f(\phi _{s2})>0.\) The points \(c\) and \(\phi _{s2}\) are a minimum point and maximum point of the function \(f(\phi )\), respectively. Three equilibrium points \(E_j(\phi _j,0), j=1,2,3,\) are satisfied \(\phi _1<c<\phi _2<\phi _{s2}<\phi _3.\)

And the every \(\phi _j\) is not depend on the parameter \(\sigma .\)

Assume \(\sigma \ne 0\). We make \(\sigma \) increase from \(\sigma <1\) to \(\sigma \ge 1\). Make the singular line \(\phi =\frac{c}{\sigma }\) move from right to left in the \((\phi ,y)\)-phase plane. By the qualitative analysis, we obtain the different topological phase portraits Eq. (2.1) shown in Figs. 1a–o and 2a–l, respectively.

Fig. 1
figure 1

The phase portraits of system (2.1) with \(k=1\). a \(h_1<h_2<h_3<h_s,\phi _3<\frac{c}{\sigma },\sigma <1.\) b \(h_1<h_2<h_3=h_s,c<\frac{c}{\sigma }=\phi _3,\sigma <1.\) c \(h_1<h_2<h_3<h_s,c<\frac{c}{\sigma }<\phi _3,\sigma <1.\) d \(h_1<h_2<h_3, c=\frac{c}{\sigma }<\phi _3,\sigma =1.\) e \(h_s<h_1<h_2<h_3,\phi _2<\frac{c}{\sigma }<c,\sigma >1.\) f \(h_1=h_s<h_2<h_3,\phi _2<\frac{c}{\sigma }<c,\sigma >1.\) g \(h_1<h_s<h_2<h_3,\phi _2<\frac{c}{\sigma }<c,\sigma >1.\) h \(h_1<h_s=h_2<h_3,\phi _2=\frac{c}{\sigma }<c,\sigma >1.\) i \(h_1<h_s<h_2<h_3,\phi _1<\frac{c}{\sigma }<\phi _2,\sigma >1.\) j \(h_s=h_1<h_2<h_3,0< \frac{c}{\sigma }<\phi _1,\sigma >1.\) k \(h_1<h_s<h_2<h_3,0< \frac{c}{\sigma }<\phi _1,\sigma >1.\) l \(h_1<h_2=h_s<h_3,0< \frac{c}{\sigma }<\phi _1,\sigma >1.\) m \(h_1<h_2<h_s<h_3,0< \frac{c}{\sigma }<\phi _1,\sigma >1.\) n \(h_1<h_2<h_s<h_3,\frac{c}{\sigma }< 0,\sigma <0.\) o \(h_1<h_2<h_3=h_s,\frac{c}{\sigma }< 0,\sigma <0.\)

Full size image
Fig. 2
figure 2

The phase portraits of system (2.1) with \(k=-1\). a \(h_3<h_s=h_2<h_1,\phi _3<\frac{c}{\sigma },\sigma <1.\) b \(h_3<h_s<h_2<h_1,\phi _3<\frac{c}{\sigma },\sigma <1.\) c \(h_3=h_s<h_2<h_1,\phi _3=\frac{c}{\sigma },\sigma <1.\) d \(h_3<h_s<h_2<h_1, \phi _2<\frac{c}{\sigma }<\phi _3,\sigma <1.\) e \(h_3<h_s=h_2<h_1,\phi _2=\frac{c}{\sigma }<\phi _3,\sigma <1.\) f \(h_3=h_s<h_2<h_1,c<\frac{c}{\sigma }<\phi _2,\sigma <1.\) g \(h_3<h_s<h_2<h_1,c<\frac{c}{\sigma }<\phi _2,\sigma <1.\) h \(h_3<h_2<h_1,\phi _1<c=\frac{c}{\sigma }<\phi _2,\sigma =1.\) i \(h_3<h_2<h_1<h_s,\phi _1<\frac{c}{\sigma }<c,\sigma >1.\) j \(h_3<h_2<h_1=h_s,0< \frac{c}{\sigma }=\phi _1,\sigma >1.\) k \(h_3<h_2<h_1<h_s,0< \frac{c}{\sigma }<\phi _1,\sigma >1.\) l \(h_3<h_2<h_1<h_s, \frac{c}{\sigma }<0<\phi _1,\sigma <0.\)

Full size image

3 Single peak solitary wave and compacton solutions

3.1 Single peak solitary wave solutions of system (1.2)

In this section, we study classification of single peak solitary wave solutions of Eq. (1.2) by using the phase portraits given in the Sect. 2. To study single peak solitary wave solutions, we impose the boundary condition

$$\begin{aligned} \lim _{\xi \rightarrow \pm \infty }\phi =p, \end{aligned}$$
(3.1)

where \(p\) is a constant. In fact, the constant \(p\) is equal to the horizontal coordinate of saddle point \(E(\varphi _e,0)\). Substituting the boundary condition (3.1) into (1.7), then the ODE (1.7) becomes

$$\begin{aligned} (\phi ^{\prime })^2=\frac{A(\phi -p)^2(q-\phi )}{(\phi -c)(\sigma \phi -c)}, \end{aligned}$$
(3.2)

where

$$\begin{aligned} q=\frac{kB^2+c(Ap-g)(p-c)}{Ap(c-p)}. \end{aligned}$$
(3.3)

Definition 3.1

A function \(\phi (\xi )\) is said to be a single peak solitary wave solution of the Eq. (1.2) if \(\phi (\xi )\) satisfies the following conditions:

(C\(_1\)):

\(\phi (\xi )\) is continuous on \(R\) and has a unique peak point \(\xi _0\), where \(\phi (\xi )\) attains its global maximum or minimum value;

(C\(_2\)):

\(\phi (\xi )\in C^3(R-\{\xi _0\})\) satisfies (1.5) on \(R-\{\xi _0\}\);

(C\(_3\)):

\(\phi (\xi )\) satisfies the boundary condition (3.1).

Definition 3.2

A wave function \(\phi \) is called smooth solitary wave solution if \(\phi \) is smooth locally on either side of \(\xi _0\) and \(\lim _{\xi \uparrow \xi _0}\phi ^{\prime }(\xi )=\lim _{\xi \downarrow \xi _0}\phi ^{\prime }(\xi )=0\).

Definition 3.3

A wave function \(\phi \) is called peakon if \(\phi \) is smooth locally on either side of \(\xi _0\) and \(\lim _{\xi \uparrow \xi _0}\phi ^{\prime }(\xi )=-\lim _{\xi \downarrow \xi _0}\phi ^{\prime }(\xi )=a,\ a\ne 0,\ a\ne \pm \infty \).

Definition 3.4

A wave function \(\phi \) is called cuspon if \(\phi \) is smooth locally on either side of \(\xi _0\) and \(\lim _{\xi \uparrow \xi _0}\phi ^{\prime }(\xi )=-\lim _{\xi \downarrow \xi _0}\phi ^{\prime }(\xi )=\pm \infty \).

Without any loss of generality, we choose the peak point \(\xi _0\) as vanishing, \(\xi _0=0\).

Theorem 3.1

Assume that \(u(x,t)=\phi (\xi )=\varphi (x-ct)\) is a single peak solitary wave solution of the Eq. (1.2) at the peak point \(\xi _0=0\), then \(\phi (0)=c\) or \(\varphi (0)=\frac{c}{\sigma }\) or \(\phi (0)=q\).

Proof

If \(\phi (0)\ne c\) and \(\phi (0)\ne \frac{c}{\sigma }\), then \(\phi (\xi )\ne c\) and \(\phi (\xi )\ne \frac{c}{\sigma }\) for any \(\xi \in R\) since \(\phi (\xi )\in C^3(R-\{0\})\). Differentiating both sides of Eq. (3.2) yields \(\phi \in C^{\infty }(R)\).

If \(\phi (0)\ne c\) and \(\phi (0)\ne \frac{c}{\sigma }\), then \(\phi \in C^{\infty }(R)\). By the definition of single peak solitary wave solution, we have \(\phi ^{\prime }(0)=0\). However, by Eq. (3.2), we must have \(\phi (0)=q\). This completes the proof of Theorem 3.1.

Now, we give the following theorem on the classification of single peak solitary wave solutions of (1.2). The idea is inspired by the study of the traveling waves of the Camassa–Holm Eq. [26]. \(\square \)

Theorem 3.2

Assume that \(u(x,t)=\varphi (x-ct)\) is a single peak solitary wave solution of the Eq. (1.2) at the peak point \(\xi _0=0\), then we have the following solution classification:

  1. (i)

    if \(\phi (0)=q, \phi (0)\ne c\) and \(\phi (0)\ne \frac{c}{\sigma }\), then \(\phi (\xi )\in C^{\infty }(R)\), and \(\phi \) is a smooth solitary wave solution.

  2. (ii)

    if \(\phi (0)=c=q\), then \(\phi \) is a peakon solution and

    $$\begin{aligned}&\phi (\xi )-c=\lambda _1|{\xi }|+O(|\xi |^2),\ \ \xi \rightarrow 0,\\&\phi ^{\prime }(\xi )=\lambda _1\, \mathrm {sgn}(\xi )+O(|\xi |),\ \ \xi \rightarrow 0, \end{aligned}$$

    where \(\lambda _1=\pm \frac{|p-c|\sqrt{A}}{\sqrt{|c(\sigma -1)|}}\).

  3. (iii)

    if \(\phi (0)=\frac{c}{\sigma }=q\), then \(\phi \) is a peakon solution and

    $$\begin{aligned}&\phi (\xi )-\frac{c}{\sigma }=\lambda _2|{\xi }|+O(|\xi |^2),\ \ \xi \rightarrow 0,\\&\phi ^{\prime }(\xi )=\lambda _2\, \mathrm {sgn}(\xi )+O(|\xi |),\ \ \xi \rightarrow 0, \end{aligned}$$

    where \(\lambda _2=\pm \frac{|p-\frac{c}{\sigma }|\sqrt{A}}{\sqrt{|c(\sigma -1)|}}\).

  4. (iv)

    if \(\phi (0)=c\ne q\), then \(\phi \) is a cuspon solution and \(\phi \) has the following asymptotic behavior

    $$\begin{aligned}&\phi (\xi )-c=\lambda _3\left| {\xi }\right| ^{2/3}+O(\left| \xi \right| ^{4/3}),\ \ \xi \rightarrow 0,\\&\phi ^{\prime }(\xi )=\frac{2}{3}\lambda _3 |\xi |^{-1/3}\mathrm {sgn}(\xi )+O(\left| \xi \right| ^{1/3}),\ \ \xi \!\rightarrow \! 0, \end{aligned}$$

    where \(\lambda _3=\pm \Big (\frac{9(c-p)^2|A(q-c)|}{4|c(\sigma -1)|}\Big )^{1/3}\).

  5. (v)

    if \(\phi (0)=\frac{c}{\sigma }\ne q\), then \(\phi \) is a cuspon solution and \(\phi \) has the following asymptotic behavior

    $$\begin{aligned}&\phi (\xi )-\frac{c}{\sigma }=\lambda _4\left| {\xi }\right| ^{2/3}+O(\left| \xi \right| ^{4/3}),\ \ \xi \rightarrow 0,\\&\phi ^{\prime }(\xi )=\frac{2}{3}\lambda _4|{\xi }|^{-1/3}\mathrm {sgn}(\xi )+O(|\xi |^{1/3}),\ \ \xi \rightarrow 0, \end{aligned}$$

    where \(\lambda _4=\pm \Big (\frac{9(\frac{c}{\sigma }-p)^2|A(q-\frac{c}{\sigma })|}{4|c(\sigma -1)|}\Big )^{1/3}\).

Proof

  1. (i)

    From the process of proofing of Theorem 3.1, we know that if \(\phi (0)\ne c\) and \(\phi (0)\ne \frac{c}{\sigma }\), then \(\phi \in C^{\infty }(R)\) and \(\phi \) is a smooth solitary wave solution.

  2. (ii)

    If \(\phi (0)=c=q\), then from Eq. (3.2), we obtain

    $$\begin{aligned} \phi ^{\prime }=\pm \frac{|\phi -p|\sqrt{A}}{\sqrt{|\sigma \phi -c|}}. \end{aligned}$$
    (3.4)

    Let \(l_1(\phi )=\frac{\sqrt{|\sigma \phi -c|}}{|\phi -p|\sqrt{A}}\), then \(l_1(c)=\frac{\sqrt{|c(\sigma -1)|}}{|c-p|\sqrt{A}}\) and

    $$\begin{aligned} \int l_1(\phi )d\phi =\pm \int d\xi . \end{aligned}$$
    (3.5)

    Inserting \(l_1(\phi )=l_1(c)+O(|\phi -c|)\) into (3.5) and using the initial condition \(\phi (0)=c\), we obtain

    $$\begin{aligned} l_1(c)(\phi -c)(1+O(|\phi -c))^{-1}=\pm |\xi |. \end{aligned}$$
    (3.6)

    Since

    $$\begin{aligned} \frac{1}{1+O(\phi -c)}=1+O(\phi -c), \end{aligned}$$
    (3.7)

    we get

    $$\begin{aligned} |\phi -c|=\frac{1}{l_1(c)}|\xi |(1+O(\phi -c)), \end{aligned}$$
    (3.8)

    which implies \(|\phi -c|=O(|\xi |)\). Therefore, we have

    $$\begin{aligned} \phi (\xi )=c+\lambda _1|\xi |+O(|\xi |^2),\ \ \xi \rightarrow 0, \end{aligned}$$
    (3.9)

    and

    $$\begin{aligned} \phi ^{\prime }(\xi )=\lambda _1\, \mathrm {sgn}(\xi )+O(|\xi |),\ \ \xi \rightarrow 0, \end{aligned}$$
    (3.10)

    where \(\lambda _1=\pm \frac{|p-c|\sqrt{A}}{\sqrt{|c(\sigma -1)|}}\)

  3. (iii)

    Similar to the proof of (ii), we ignore it in this paper.

  4. (iv)

    If \(\phi (0)=c\ne q\), then by the definition of single peak solitary wave solution, we have \(p\ne c\). From Eq. (3.2), we obtain

    $$\begin{aligned} \phi ^{\prime }=\pm \frac{|\phi -p|\sqrt{|A(q-\phi )|}}{\sqrt{|(\phi -c)(\sigma \phi -c)|}}. \end{aligned}$$
    (3.11)

    Let \(l_2(\phi )=\frac{\sqrt{|\sigma \phi -c|}}{|\phi -p|\sqrt{|A(q-\phi )|}}\), then \(l_2(c)=\frac{\sqrt{|c(\sigma -1)|}}{|c-p|\sqrt{|A(q-c)|}}\), and

    $$\begin{aligned} \int l_2(\phi )\sqrt{|\phi -c|}d\phi =\pm \int d\xi . \end{aligned}$$
    (3.12)

    Inserting \(l_2(\phi )=l_2(c)+O(|\phi -c|)\) into (3.12) and using the initial condition \(\phi (0)=c\), we obtain

    $$\begin{aligned} \frac{2l_2(c)}{3}|\phi -c|^{3/2}(1+O(|\phi -c|))=|\xi |, \end{aligned}$$
    (3.13)

    thus

    $$\begin{aligned} \phi \!-\!c&= \pm \Big (\frac{3}{2l_2(c)}\Big )^{2/3}|\xi |^{2/3}(1+O(|\phi -c|))^{-2/3}\nonumber \\&= \pm \Big (\frac{3}{2l_2(c)}\Big )^{2/3}|\xi |^{2/3}(1+O(|\phi -c|)),\nonumber \\ \end{aligned}$$
    (3.14)

    which implies \(\phi -c=O(|\xi |^{2/3})\). Therefore, we have

    $$\begin{aligned} \phi (\xi )&= c\pm \Big (\frac{3}{2l_2(c)}\Big )^{2/3}|\xi |^{2/3}+O(|\xi |^{4/3})\\&= c+\lambda _3|\xi |^{2/3}+O(|\xi |^{4/3}),\ \ \xi \rightarrow 0,\\ \lambda _3&= \pm \Big (\frac{3}{2l_2(c)}\Big )^{2/3}\\&= \pm \Big (\frac{9(c-p)^2|A(q-c)|}{4|c(\sigma -1)|}\Big )^{1/3}, \end{aligned}$$

    and

    $$\begin{aligned} \phi ^{\prime }(\xi )=2/3\lambda _3|{\xi }|^{-1/3}\mathrm {sgn}(\xi )+O(|\xi |^{1/3}),\ \ \xi \rightarrow 0. \end{aligned}$$
    (3.15)
  5. (v)

    Similar to the proof of (iv), we ignore it in this paper. This completes the proof of Theorem 3.2. By virtue of Theorem 3.1, any single peak solitary wave solution of the Eq. (1.2) must satisfy the following initial and boundary values problem

    $$\begin{aligned} \left\{ \begin{array}{l} (\phi ^{\prime })^2=\frac{A(\phi -p)^2(q-\phi )}{(\phi -c)(\sigma \phi -c)}:=F(\phi ), \\ \phi (0)\in \{c,\frac{c}{\sigma },q\}, \\ \lim _{|\xi |\rightarrow \infty }\phi (\xi )=p. \end{array}\right. \end{aligned}$$
    (3.16)

    \(\square \)

Theorem 3.3

When \(\phi \) approaches the double zero \(p\) of \(F(\phi )\) so that \(F^{\prime }(p)=0, F^{\prime \prime }(p)\ne 0\), then the solution \(\phi \) satisfies

$$\begin{aligned} \phi (\xi )-p\sim a\exp (-|\xi |\sqrt{|F^{\prime \prime }(p)|}),\ \ \xi \rightarrow \pm \infty \end{aligned}$$
(3.17)

for some constant \(a\), thus \(\phi \rightarrow p\) exponentially as \(\xi \rightarrow \pm \infty \).

Proof

Because \(p\) is a double zero of \(F(\phi )\), we have

$$\begin{aligned} \phi _\xi ^2=(\phi -p)^2F^{\prime \prime }(p)+O((\phi -p)^3),\ \ \phi \rightarrow p. \end{aligned}$$
(3.18)

Furthermore, we get

$$\begin{aligned} \frac{d\xi }{d\phi }=\frac{1}{\sqrt{(\phi -p)^2F^{\prime \prime }(p)+O((\phi -p)^3)}}. \end{aligned}$$
(3.19)

Since

$$\begin{aligned}&\sqrt{(\phi -p)^2F^{\prime \prime }(p)+O((\phi -p)^3)}\nonumber \\&\quad =|\phi -p|(\sqrt{|F^{\prime \prime }(p)|}+O(\phi -p)) \end{aligned}$$
(3.20)

and

$$\begin{aligned} \frac{1}{\sqrt{|F^{\prime \prime }(p)|}+O(\phi -p)}=\frac{1}{\sqrt{|F^{\prime \prime }(p)|}}+O(\phi -p), \end{aligned}$$
(3.21)

we get

$$\begin{aligned} \frac{d\xi }{d\phi }=\frac{1}{|\phi -p|\sqrt{F^{\prime \prime }(p)}}+O(1). \end{aligned}$$
(3.22)

Integration gives Eq. (3.17). This completes the proof of Theorem 3.3. \(\square \)

Below, we will present some implicit formulas for the single peak solitary wave solutions for some specific cases.

Type 1: Smooth solitary wave solutions of system (1.2)

At first, suppose \(\sigma =1\). There exists smooth solitary wave solutions of system (1.2), which corresponds to the homoclinic orbits defined by \(H(\phi , y)=h_1\) in the Fig. 1d. From (3.2), we have

$$\begin{aligned} \begin{array}{l} \frac{1}{\sqrt{A}}\int _{\phi }^{q}\frac{c-\phi }{(\phi -p)\sqrt{q-\phi }}d\phi \\ \quad =\frac{1}{\sqrt{A}}\int _{\phi }^{q}(\frac{-1}{\sqrt{q-\phi }} +\frac{c-p}{(\phi -p)\sqrt{q-\phi }})d\phi . \end{array} \end{aligned}$$
(3.23)

So, we have the parametric representation of solitary wave solution of (1.2) as following:

$$\begin{aligned} \begin{array}{l} \phi (\chi )=q-(q-p)\tanh ^2(\frac{\sqrt{q-p}}{2(c-p)}\chi ),\\ \xi (\chi )=\chi -2\sqrt{q-\phi (\chi )}. \end{array} \end{aligned}$$
(3.24)

The profile of smooth solitary wave is shown in Fig. 3.

Fig. 3
figure 3

The profile of smooth solitary wave of \(\phi (\xi )\) of system (1.2)

Full size image

Type 2: Peakon solutions of system (1.2)

Corresponding to the heteroclinic loop defined by \(H(\phi , y)=h_s=h_1\) in the Fig. 1f, we have a peakon solution. Let \(\phi (0)=\frac{c}{\sigma }\), along with the heteroclinic orbits \(E_3S_+\) and \(E_3S_-\) to do integration, we have

$$\begin{aligned} \begin{array}{l} \xi =\sqrt{\frac{\sigma }{A}}\int _{\phi }^{\frac{c}{\sigma }}\sqrt{\frac{(\frac{c}{\sigma } -\phi )(c-\phi )}{(\phi -p)^2(\frac{c}{\sigma }-\phi )}}d\phi \\ \quad =\sqrt{\frac{\sigma }{A}}\int _{\phi }^\frac{c}{\sigma }[ \frac{-1}{\sqrt{c-\phi }} +\frac{c-p}{(\phi -p)\sqrt{c-\phi }} ]d\phi , \end{array} \end{aligned}$$
(3.25)

and its parametric representation of solution as following:

$$\begin{aligned} \begin{array}{l} \phi (\chi )=c-(c-p)\tanh ^2(\frac{\chi }{2\sqrt{c-p}}), \\ \xi (\chi )=\sqrt{\frac{\sigma }{A}}[\chi -2\sqrt{c-\phi (\chi )}\\ \quad +\,2\sqrt{c-\frac{c}{\sigma }}-2\sqrt{c-p}\tanh ^{-1} \sqrt{\frac{c-\frac{c}{\sigma }}{c-p}}]. \end{array} \end{aligned}$$
(3.26)

The profile of peakon is shown in Fig. 4.

Fig. 4
figure 4

The profile of peakon of \(\phi (\xi )\) of system (1.2)

Full size image

Type 3: Cuspon solutions of system (1.2)

If \(H(\phi , y)=h=h_1,\) the equilibrium point \(E_1(\phi _1, 0)\) is a saddle point. We note that \(l^u\) and \(l^s\) are the unstable and stable manifold of saddle point \(E_1(\phi _1, 0)\) and close to the singular straight line \(\phi =\frac{c}{\sigma }\)(see Fig. 1g). Let \(\phi (0)=\frac{c}{\sigma },\) we have

$$\begin{aligned} \begin{array}{l} \sqrt{\frac{\sigma }{A}}\int _{\phi }^{\frac{c}{\sigma }}\frac{\sqrt{(c-\phi ) (\frac{c}{\sigma }-\phi )}}{(\phi -p)(\sqrt{q-\phi })}d\phi ,\\ \quad =\sqrt{\frac{\sigma }{A}}\int _{\phi }^{\frac{c}{\sigma }}[\frac{\phi }{\sqrt{F_1(\phi )}}+\frac{A_{11}}{\sqrt{F_1(\phi )}} +\frac{A_{12}}{(\phi -p)\sqrt{F_1(\phi )}}]d\phi , \end{array} \end{aligned}$$
(3.27)

where \(F_1(\phi )=(c-\phi )(q-\phi )(\frac{c}{\sigma }-\phi ), A_{11}=p-c(1+\frac{1}{\sigma }), A_{12}=\frac{c^2}{\sigma }+p[p-(c+\frac{c}{\sigma })].\)

Integrating above equation, we have the parametric representation of the cuspon solutions as following:

$$\begin{aligned} \begin{array}{l} \phi (\chi )=q-\frac{q-\frac{c}{\sigma }}{1-sn^2(\chi ,k)},\\ \xi (\chi )=\sqrt{\frac{\sigma }{A}}g[(A_{11}+q+\frac{A_{12}}{q-p})\chi \\ \quad -(q-\frac{c}{\sigma })\Pi (arcsin(sn(\chi ,k)),1,k) \\ \quad +\frac{(q-\frac{c}{\sigma })A_{12}}{(\frac{c}{\sigma }-p) (q-p)}\Pi (arcsin(sn(\chi ,k)),\alpha ^2_3,k)], \end{array} \end{aligned}$$
(3.28)

where \(g=\frac{2}{\sqrt{c-\frac{c}{\sigma }}}, k^2=\frac{c-q}{c-\frac{c}{\sigma }}\) and \(\alpha ^2_3=\frac{q-p}{\frac{c}{\sigma }-p}, \Pi (\cdot \ \cdot \ \cdot )\) is the elliptic integral of the third kind. \(sn(\chi ,k)\) is the Jacobian elliptic function. The profile of cuspon is shown in Fig. 5.

Fig. 5
figure 5

The profile of cuspon of \(\phi (\xi )\) of system (1.2)

Full size image

3.2 Compacton solutions of system (1.2)

If \(H(\phi , y)=h_s=h_2,\) we have the homoclinic orbits which is tangent to the singular straight line \(\phi =\frac{c}{\sigma }\) at point \(E_3(\phi _3, 0)\)(see Fig. 2e). From \(H(\phi , y)=h_s,y=\phi ^{\prime }(\xi )\), given by (1.7), we have

$$\begin{aligned} (\phi ^{\prime })^2=\frac{A(\phi _M-\phi )(\phi -\frac{c}{\sigma })^2}{(\phi -c)(\sigma \phi -c)}. \end{aligned}$$
(3.29)

Solving Eq. (3.29), we get the following exact parametric representations of compacton solutions of system (1.2)

$$\begin{aligned} \begin{array}{l} \phi (\chi )=\phi _M-(\phi _M-\frac{c}{\sigma })sn^2(\chi , k), \\ \xi (\chi )=g\sqrt{\frac{\sigma }{A}}(\phi _M-\frac{c}{\sigma })E(arcsin(sn(\chi ,k)), k) \end{array} \end{aligned}$$
(3.30)

for \(\xi \in [-\xi _l,\xi _l]\), and \(\phi (\xi )\equiv \frac{c}{\sigma }\) for \(\xi \in (-\infty ,-\xi _l)\cup (\xi _l,+\infty )\), where \( g=\frac{2}{\sqrt{\phi _M-\frac{c}{\sigma }}}, k^2=\frac{\phi _M-\frac{c}{\sigma }}{\phi _M-c}, \xi _l=g\sqrt{\frac{\sigma }{A}}(\phi _M-\frac{c}{\sigma })E(\frac{\pi }{2},k), E(\cdot \ \cdot )\) is the elliptic integral of the second kind. The profile of compacton is shown in Fig. 6.

Fig. 6
figure 6

The profile of compacton of \(\phi (\xi )\) of system (1.2)

Full size image

4 Conclusion

In this paper, we study the generalized two-component Hunter–Saxton system. By using the method of dynamical system, we have analyzed the numbers and relative position of the equilibrium points. Furthermore, we obtain the parametric representations of single peak solitary wave and compacton solutions for the generalized two-component Hunter–Saxton system. Asymptotic analysis and numerical simulations are provided for smooth solitary wave, peakon, cuspon and compacton solutions of the generalized two-component Hunter–Saxton system.

It is a very interesting topic to find how many periodic traveling waves exist under some perturbation, and when the solitary wave still exists with one or two periodic traveling waves. This phenomenon was considered in [27]. The first-order Melnikov function was used to find the isolated zeros of the Melnikov function. The Melnikov method is the essential way to answer the above questions. For the Hunter–Saxton system under perturbation, the existences of isolated periodic traveling wave solutions deserve to study.