1 Introduction

When a new partner joins a mutualism association, it brings benefits to the association while it may result in conflicts and consequently lead to mutualism breakdown. This is the case for the emergence of ants in plant–pollinator mutualism.

In a plant–pollinator system, plants provide nectar, pollen, and other resources for pollinators. The pollinators transport pollen for their host plant in return, which enables the plant to outcross with other plants and produce offspring. Thus, the plant–pollinator interaction is mutualistic, and plants share an interest in reproduction from pollination mutualism (May 2001). Ants can provide a different benefit for plants. When they emerge in the plant–pollinator system, ants protect plants from herbivores such as beetles, while the plants provide the ants with several resources including nectar, food bodies, and nesting sites (Rickson and Risch 1984). Thus, in plant–ant mutualism, plants share an interest in growth but not in reproduction.

There exists a conflict between ants and plants, which is known as plant castration. In the plant castration, ants attack and destroy floral buds and flowers of their host plant (Janzen 1966). For example, the ant Crematogater nigrices destroys the axillary shoots of the swollen-thorn acacia, which leads to an increase in the net rate of domatia production (Young et al. 1997). However, the destruction prevents fruiting in the host plant and subsequently reduces plant reproduction. Therefore, ants have both positive and negative effects on plants by promoting plant growth but decreasing plant reproduction. The relationship between ants and plants exhibits a unidirectional consumer–resource (C–R) interaction, in which one species acts as a resource and the other as both a resource and a consumer (Holland and DeAngelis 2009).

There exists another conflict between ants and plants. Since ants are only concerned with the individual plant they live, they can prevent pollinators from accessing plants, which is detrimental to plant reproduction. Since plant–pollinator–ant communities are highly common and geographically widespread, an interesting question has puzzled biologists for years that under which conditions pollination mutualism can persist in the presence of ants (Oña and Lachmann 2011).

The three-species systems were characterized by May (2001) in a structure of classical models. The models were transformed into nondimensional differential equations by Murray (2003), in which the functional responses are either linear or Holling type II. In 2011, O\(\tilde{\mathrm{N }}\)A and Lachmann considered a plant–pollinator–ant system where functional responses are linear. Both theoretical analysis and numerical simulations demonstrate a novel threshold in ant aggressiveness against pollinators. When the level of aggressiveness is less than the threshold, the three species coexist. Otherwise, pollinators will be driven into extinction by ants. Oña and Lachmann (2011) also analyzed a plant–pollinator–ant system where functional responses are Holling type II. Numerical simulations demonstrate conditions under which the three species could coexist. For more relative works, we refer to Jang (2002), Cantrell et al. (2004), Cushing (2009), Hsu et al. (2013), Wang and Wu (2013), and Huang et al. (2014).

Fishman and Hadany (2010) demonstrated that the plant–pollinator interaction can be described by a Beddington–DeAngelis (BD) functional response. In their study, Fishman and Hadany considered interactions between a plant species and a social insect (the honeybee), and derived an analytical expression for population-level plant–pollinator interactions. Furthermore, they showed that the analytical expression can be approximated by a BD formula (Beddington 1975; DeAngelis et al. 1975). In the BD formula, two traits in plant–pollinator interactions are characterized. One is the time pollinators spend on plants in their individual interactions, and the other is the exploitation competition between pollinators. When ants emerge in the plant–pollinator system, they prevent pollinators from accessing plants by scaring them off. Since the ants act as interferers but not as predators, the ant interference leads to a loss in time but no loss in biomass to the pollinators. Therefore, the BD formula, which characterizes plant–pollinator interactions, should be extended to describe the time caused by ant interference when there exist ants. The plant–pollinator–ant systems with the extended BD functional responses can demonstrate fundamentally different properties from those with linear or Holling type II formulas.

In this paper, we consider plant–pollinator–ant systems in which the plant–pollinator interaction is mutualistic but ants have both positive and negative effects on plants. The ants also interfere with pollinators by preventing them from accessing plants. While a Beddington–DeAngelis formula can describe the plant–pollinator interaction, the formula is extended in this paper to characterize the pollination mutualism under the ant interference. Then, a plant–pollinator–ant system with the extended BD functional response is discussed, and global dynamics of the model demonstrate the mechanisms by which pollination mutualism can persist in the presence of ants. When the ant interference is strong, it can result in extinction of pollinators. Moreover, if the ants depend on pollination mutualism for survival, the strong interference could drive pollinators into extinction, which consequently lead to extinction of the ants themselves. When the ant interference is weak, a cooperation between plant–ant and plant–pollinator mutualisms could occur, which promotes the survival of both ants and pollinators, especially in the case that ants (respectively, pollinators) cannot survive in the absence of pollinators (respectively, ants). Even when the level of ant interference remains invariant, varying ants’ negative effect on plants can result in survival/extinction of both ants and pollinators. Therefore, our results provide an explanation for the persistence of pollination mutualism when there exist ants.

The paper is organized as follows. The three-species model is described in Sect. 2. Section 3 exhibits the dynamics of subsystems. Section 4 demonstrates the persistence of the whole system. Section 5 shows the stability and bifurcation of the interior equilibrium. Discussions are in Sect. 6.

2 A Plant–Pollinator–Ant Model

The model derivation in this section is based on the work by Fishman and Hadany (2010). In their work, Fishman and Hadany derived that the analytical expression for plant–pollinator interactions can be approximated by a Beddington–DeAngelis formula. We assume that plants can survive in the absence of pollinators (e.g., by selfing) and pollinators depend on plants for survival (Jang 2002; Soberon and Rio 1981). Then, the plant–pollinator system can be described by

$$\begin{aligned} \frac{\hbox {d}x_1}{\hbox {d}t}&= x_1 \left( r_1- d_1 x_1 + \frac{e_{12}x_2}{1+\alpha x_1 + \beta x_2}\right) \nonumber \\ \frac{\hbox {d}x_2}{\hbox {d}t}&= x_2 \left( -r_2 + \frac{e_{21}x_1}{1+\alpha x_1+\beta x_2}\right) \end{aligned}$$
(1)

where \(x_1\) and \(x_2\) are population densities of plants and pollinators, respectively. The parameter \(r_1\) is the intrinsic growth rate of plants and \(d_1\) the self-incompatible degree. \(\alpha \) represents the time pollinators spend on central place foraging and individual-level plant–pollinator interactions (Fishman and Hadany 2010), while \(\beta \) is the intensity of exploitation competition between pollinators (Pianka 1974). \(e_{12}\) denotes the plants’ efficiency in translating plant–pollinator interactions into fitness (Fishman and Hadany 2010), and \(e_{21}\) is the corresponding value for pollinators. \(r_2\) represents the per capita death rate of pollinators.

When ants emerge in the plant–pollinator system, they interfere with pollinators by scaring them off, which leads to a loss in time to the pollinators. Thus, the BD formula describing plant–pollinator interactions should be extended to include the ant interference. Meanwhile, the ants have both positive and negative effects on plants. The plant–ant mutualism has the same traits as those of plant–pollinator mutualism, so that it can be described by a BD functional response. In plant castration, ants only destroy buds without eating them. Thus, there is no saturation in the destruction, and the ants’ negative effect on plants should be proportional to their density.

We assume that ants depend on plants for survival (e.g., Oña and Lachmann (2011). Then, the plant–pollinator–ant system can be depicted by

$$\begin{aligned} \frac{\hbox {d}x_1}{\hbox {d}t}&= x_1\left( r_1- d_1 x_1 + \frac{e_{12}x_2}{1+\alpha x_1+\beta x_2 + \gamma x_3} + \frac{e_{13} x_3}{ 1+\bar{\alpha } x_1 + \bar{\beta } x_3 } - \bar{\gamma } x_3\right) \nonumber \\ \frac{\hbox {d}x_2}{\hbox {d}t}&= x_2 \left( -r_2 + \frac{e_{21}x_1}{1+\alpha x_1+\beta x_2 + \gamma x_3}\right) \nonumber \\ \frac{\hbox {d}x_3}{\hbox {d}t}&= x_3 \left( -r_3 + \frac{e_{31}x_1}{1+\bar{\alpha } x_1 + \bar{\beta } x_3}\right) \end{aligned}$$
(2)

where \(x_3\) be the population density of ants. The parameter \(\gamma \) denotes the level of ant interference, characterizing the loss in time to pollinators due to ants’ scaring. \(e_{13}\) represents the plants’ efficiency in translating plant–ant interactions into fitness, and \(e_{31}\) is the corresponding value for ants. \(\bar{\alpha }\) represents the time ants spend on plants, while \(\bar{\beta }\) is the intensity of exploitation competition between ants. \(\bar{\gamma }\) represents the degree of plant castration by ants, which can be measured by the biomass of buds and flowers destroyed by one ant. \(r_3\) is the per capita death rate of ants.

When there is no pollinator, system (2) becomes a plant–ant model

$$\begin{aligned} \frac{\hbox {d}x_1}{\hbox {d}t}&= x_1\left( r_1- d_1 x_1 + \frac{e_{13} x_3}{ 1+\bar{\alpha } x_1 + \bar{\beta } x_3 } - \bar{\gamma } x_3\right) \nonumber \\ \frac{\hbox {d}x_3}{\hbox {d}t}&= x_3\left( -r_3 + \frac{e_{31}x_1}{1+\bar{\alpha } x_1 + \bar{\beta } x_3}\right) \end{aligned}$$
(3)

where ants have both positive and negative effects on plants.

When \(e_{21} \le r_2 \alpha \), we have \(dx_2/dt \le 0\) by the second equation of (2), so that the Liapunov Theorem (Hofbauer and Sigmund 1998) implies that \(\lim _{t \rightarrow \infty }x_2(t) =0\). Similarly, when \(e_{31} \le r_3 \bar{\alpha }\), we obtain \(\lim _{t \rightarrow \infty }x_3(t) =0\). Since we are concerned with the coexistence of the three species, we assume \(e_{21}>r_2 \alpha \) and \( e_{31}>r_3 \bar{\alpha }\) in this paper.

Based on the description by Fishman and Hadany (2010) and Oña and Lachmann (2011), the parameters with their dimensions are summarized in Table 1.

Table 1 Summary of key model parameters
Full size table

3 Three Subsystems

There are three subsystems of (2). In the pollinator–ant subsystem, we can see that both species go to extinction since they cannot survive in the absence of plants. Dynamics of the plant–pollinator and plant–ant subsystems have been studied by Wang et al. (2012), which are cited as follows.

Let \(E(x_1,x_2)\) be an interior equilibrium of system (1). Then, \(E\) satisfies

$$\begin{aligned} x_2 = \frac{1}{\beta x_1^0}\left( x_1 -x_1^0\right) , \quad g_1(x_1) = l_1(x_1) \end{aligned}$$

where

$$\begin{aligned} x_1^0 = \frac{r_2}{e_{21} -r_2 \alpha },\quad g_1(x_1)= x_1 ( r_1- d_1x_1),\quad l_1(x_1)= -\frac{e_{12} r_2 }{e_{21}\beta x_1^0} \left( x_1 -x_1^0\right) . \end{aligned}$$
(4)

Suppose \(r_1/d_1 < x_1^0 \). Let \(k_{1}\) be the slope of line \(l_1\), then \(\lim _{e_{12} \rightarrow +\infty } k_{1} = -\infty \) while \(l_1\) passes through point \((x_1^0,0)\). Since the parabolic curve \(v=g_1(x_1)\) is convex upward and passes through points \((0,0)\) and \((r_1/d_1,0)\), there exists \(e_{12}^0>0 \) such that the curves \(g_1(x_1)\) and \(l_1(x_1)\) are tangent in the region \(x_1>x_1^0\) when \(e_{12}=e_{12}^0\). Hence, when \(e_{12} \ge e_{12}^0\), there are two intersection points of \(g_1(x_1)\) and \(l_1(x_1)\) in the region \(x_1>x_1^0\), which correspond to two interior equilibria \(E_{12}^+(x_1^+, x_2^+)\) and \(E_{12}^-(x_1^-, x_2^-)\) of (1). When \(e_{12} < e_{12}^0\), there is no interior equilibrium of (1). The computation of \(e_{12}^0\) and \(E_{12}^\pm \) is shown in “Appendix 1.” Therefore, we have the following result.

Theorem 3.1

(Wang et al. 2012)

  1. (i)

    System (1) admits no periodic orbit.

  2. (ii)

    When \(r_1/d_1>x_1^0 ,\) \(E_{12}^+(x_1^+, x_2^+)\) is the unique interior equilibrium of (1) and is globally asymptotically stable in the interior of the \((x_1,x_2)\)-plane.

  3. (iii)

    When \(r_1/d_1<x_1^0\) and \(e_{12} \ge e_{12}^0\), \(E_{12}^-(x_1^-,x_2^-)\) and \(E_{12}^+(x_1^+, x_2^+)\) are interior equilibrium of (1). \(E_{12}^-\) is a saddle point while \(E_{12}^+\) and \(E_1(r_1/d_1,0)\) are locally asymptotically stable. The separatrices of \(E_{12}^-\) divide the interior of the \((x_1,x_2)\)-plane into two regions: one is the basin of attraction of \(E_1\) while the other is that of \(E_{12}^+\).

  4. (iv)

    When \(r_1/d_1<x_1^0\) and \(e_{12} < e_{12}^0\), \(E_1(r_1/d_1,0)\) is globally asymptotically stable in the interior of the \((x_1,x_2)\)-plane.

Let \(\bar{E}(x_1,x_3)\) be an interior equilibrium of (3). Then, \(\bar{E}\) satisfies

$$\begin{aligned} x_3 = \frac{1}{\bar{\beta } \bar{x}_1^0} (x_1 -\bar{x}_1^0),\quad g_2(x_1) = l_2(x_1), \end{aligned}$$

where

$$\begin{aligned} \bar{x}_1^0&= \frac{r_3}{e_{31} -r_3 \bar{\alpha }}, \quad g_2(x_1)= x_1 \left[ r_1- d_1x_1 - \frac{\bar{\gamma }}{ \bar{\beta } \bar{x}_1^0} \left( x_1 - \bar{x}_1^0\right) \right] , l_2(x_1)\nonumber \\&= -\frac{e_{13} r_3 }{e_{31}\bar{\beta } \bar{x}_1^0} \left( x_1 -\bar{x}_1^0\right) . \end{aligned}$$
(5)

Suppose \(r_1/d_1 < \bar{x}_1^0 \). Then, the curve \(v=g_2(x_1)\) is a parabola and \(g_2(0)=0, g_2(\bar{x}_1^0)<0\). Thus, the roots of \(g_2(x_1)=0\) are in the region \(x_1<\bar{x}_1^0.\) Let \(k_{2}\) be the slope of line \(l_2\), then \(\lim _{e_{13} \rightarrow +\infty } k_{2} = -\infty \) while \(l_2\) passes through point \((\bar{x}_1^0,0)\). Since the parabolic curve \(v=g_2(x_1)\) is convex upward and the roots of \(g_2(x_1)=0\) are in the region \(x_1<\bar{x}_1^0,\) there exists \(e_{13}^0>0 \) such that the curves \(g_2(x_1)\) and \(l_2(x_1)\) are tangent in the region \(x_1>\bar{x}_1^0\) when \(e_{13}=e_{13}^0\). Hence, when \(e_{13} \ge e_{13}^0\), there are two intersection points of \(g_2(x_1)\) and \(l_2(x_1)\) in the region \(x_1>\bar{x}_1^0\), which correspond to two interior equilibria \(E_{13}^+(\bar{x}_1^+, x_3^+)\) and \(E_{13}^-(\bar{x}_1^-, x_3^-)\) of (3). When \(e_{13} < e_{13}^0\), there is no interior equilibrium of (3). The computation of \(e_{13}^0\) and \(E_{13}^\pm \) is shown in “Appendix 1.” Thus, dynamics of system (3) can be described as follows.

Theorem 3.2

(Wang et al. 2012)

  1. (i)

    System (3) admits no periodic orbit.

  2. (ii)

    When \(r_1/d_1>\bar{x}_1^0 ,\) \(E_{13}^+(\bar{x}_1^+, x_3^+)\) is the unique interior equilibrium of (3) and is globally asymptotically stable in the interior of the \((x_1,x_3)\)-plane.

  3. (iii)

    When \(r_1/d_1<\bar{x}_1^0\) and \(e_{13} \ge e_{13}^0\), \(E_{13}^-(\bar{x}_1^-,x_3^-)\) and \(E_{13}^+(\bar{x}_1^+, x_3^+)\) are interior equilibrium of (3) as shown in Fig. 1. \(E_{13}^-\) is a saddle point while \(E_{13}^+\) and \(\bar{E}_1(r_1/d_1,0)\) are locally asymptotically stable. The separatrices of \(E_{13}^-\) divide the interior of the \((x_1,x_3)\)-plane into two regions: one is the basin of attraction of \(\bar{E}_1\) while the other is that of \(E_{13}^+\).

  4. (iv)

    When \(r_1/d_1<\bar{x}_1^0\) and \(e_{13} < e_{13}^0\), \(\bar{E}_1(r_1/d_1,0)\) is globally asymptotically stable in the interior of the \((x_1,x_3)\)-plane.

Fig. 1
figure 1

Dynamics of subsystem (3) with two interior equilibria \(E_{13}^-\) and \(E_{13}^+\), as shown in Theorem 3.2. Vector fields are displayed by gray arrows. Stable and unstable equilibria are represented by solid and open circles, respectively. The separatrices (the black line) of \(E_{13}^-\) divide the plane into two regions: one is the basin of attraction of \(\bar{E}_1(r_1/d_1,o)\) and the other is that of \(E_{13}^+\)

Full size image

4 Persistence

We show uniform persistence of system (2) by applying the acyclicity theorem by Butler et al. (1986) and Butler and Waltman (1986). First, we demonstrate that system (2) is dissipative. Then, we show the boundary equilibria of (2) cannot form a heteroclinic cycle, which is the acyclicity condition in the uniform persistence theorem (Butler et al. 1986; Butler and Waltman 1986) (Figs. 2, 3).

Fig. 2
figure 2

Variation in interaction outcomes in system (2) when degrees of plant castration \(\bar{\gamma }\) and ant interference \(\gamma \) change. Solutions of plants (\(x_1\)), pollinators (\(x_2\)), and ants (\(x_3\)) are denoted by red, blue, and black lines, respectively. a When the plant castration is strong (\(\bar{\gamma } = 0.004\)) but ant interference is weak (\(\gamma = 0.2\)), both pollinators and ants go to extinction. b, c When both the plant castration and ant interference are weak (\(\bar{\gamma } = 0.001\), \(\gamma = 0.2\)), the three species coexist if their initial densities are large. Otherwise, both pollinators and ants cannot survive. d When the plant castration is weak (\(\bar{\gamma } = 0.001\)) but ant interference is strong (\(\gamma = 0.4\)), both pollinators and ants go to extinction (Color figure online)

Full size image
Fig. 3
figure 3

a In Lemma 4.2, when \(b/a > \bar{u}_1^+ > \bar{u}_1^0\), the parabola \(g(x_1)\) and line \(l(x_1)\) have intersections in the region \(x_1 > b/a\) if the slope \(k_l\) is sufficiently small. b In Lemma 4.5, when \(G(0)>0\) and \( G(\bar{u}_1^0)<0\), the parabola \(G(x_1)\) and line \(L(x_1)\) have intersections in the region \(x_1 > \bar{u}_1^0\) if the slope \(k_L\) is sufficiently small. c In Lemma 4.5, when \(b/a > \bar{u}_1^0 > \bar{u}_1^+\), the parabola \(g(x_1)\) and line \(l(x_1)\) have no intersection in the region \(x_1 > b/a\). d In Lemma 4.9, when \(\bar{\gamma } < \bar{\gamma }_0\) and \(\gamma \le \gamma _0\), the parabola \(G(x_1)\) and line \(L(x_1)\) have intersections in the region \(x_1 > \bar{u}_1^0\)

Full size image

We are concerned with the solutions of (2) with initial values \(x(0) \ge 0\). It can be verified that these solutions are nonnegative. The following result shows that the solutions of system (2) are bounded, while the proof is in “Appendix 2.”

Lemma 4.1

System (2) is dissipative.

Stability of the boundary equilibria is shown by eigenvalues of Jacobian matrices of (2) at the equilibria. Indeed, the equilibrium \(O(0,0,0)\) has eigenvalues \(r_1,-r_2\) and \(-r_3\), which implies that it is a saddle point and is stable in the \(x_2\)- and \(x_3\)-directions but unstable in the \(x_1\)-direction. The equilibrium \(P_1(r_1/d_1,0,0)\) has eigenvalues

$$\begin{aligned} \lambda _1^{(1)} = -r_1,\quad \lambda _1^{(2)} = -r_2 + \frac{e_{21}r_1}{d_1+\alpha r_1}, \quad \lambda _1^{(3)} =-r_3 + \frac{e_{31}r_1}{d_1+\bar{\alpha } r_1} \end{aligned}$$
(6)

where \(\lambda _i^{(j)}\) is the eigenvalue of equilibrium \(P_i\) in the \(x_j\)-direction.

When \(P_{12}^+(x_1^+, x_2^+,0)\) and \(P_{12}^-(x_1^-, x_2^-,0)\) are boundary equilibria of (2), \(P_{12}^+\) is locally asymptotically stable and \(P_{12}^-\) is a saddle point on the \((x_1,x_2)\)-plane with \(x_1^+ >x_1^- >r_1/d_1\). Their eigenvalues in the \(x_3\)-direction are

$$\begin{aligned} \lambda _{12}^{+} = -r_3+ \frac{e_{31}x_1^+}{1+\bar{\alpha } x_1^+}, \quad \lambda _{12}^{-} = -r_3+ \frac{e_{31}x_1^-}{1+\bar{\alpha } x_1^-}. \end{aligned}$$
(7)

When \(P_{13}^+(\bar{x}_1^+,0, x_3^+)\) and \(P_{13}^-(\bar{x}_1^-,0, x_3^-)\) are boundary equilibria of (2), \(P_{13}^+\) is locally asymptotically stable and \(P_{13}^-\) is a saddle point on the \((x_1,x_3)\)-plane with \(\bar{x}_1^+ >\bar{x}_1^- >r_1/d_1\). Their eigenvalues in the \(x_2\)-direction are

$$\begin{aligned} \lambda _{13}^{+} = -r_2+ \frac{e_{21}\bar{x}_1^+}{1+\alpha \bar{x}_1^+ + \gamma x_3^+}, \quad \lambda _{13}^{-} = -r_2+ \frac{e_{21}\bar{x}_1^-}{1+\alpha \bar{x}_1^- + \gamma x_3^-}. \end{aligned}$$
(8)

The acyclicity condition for the uniform persistence of system (2) is considered in three cases such as: (1) \(\lambda _1^{(3)}>0\); (2) \(\lambda _1^{(3)}<0\) and \(e_{13}<e_{13}^0\); (3) \(\lambda _1^{(3)}<0\) and \(e_{13} \ge e_{13}^0\);

First, we consider the case of \(\lambda _1^{(3)}>0\), which implies that ants can survive in the absence of pollinators. The following Lemma 4.2 shows conditions under which system (2) has interior equilibria, while the proof is in “Appendices 3 and 4” exhibits the proof of \(e_{12}^*\) and \(e_{13}^*\) in “Appendix 3.”

Lemma 4.2

Assume \(\lambda _{13}^+<0\). There exists \(e_{12}^* > 0\) such that system (2) has interior equilibria if and only if \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\), where \(\gamma ^* =\bar{\beta } \bar{x}_1^0/x_1^0 \).

When pollinators can persist in the plant–pollinator system, Theorem 4.3 shows conditions under which pollinators can survive in the presence of ants, while the proof is in “Appendix 5.”

Theorem 4.3

Let \(\lambda _1^{(2)}>0, \lambda _1^{(3)}>0.\)

  1. (i)

    When \(\lambda _{13}^+ >0\), system (2) is uniformly persistent.

  2. (ii)

    When \(\lambda _{13}^+<0\), \(\gamma < \gamma ^*\) and \( e_{12} \ge e_{12}^*\), \(P_{13}^+\) is locally asymptotically stable with a basin of attraction \(\omega _{13}\) and system (2) restricted on int\(R_+^3 - \omega _{13}\) is uniformly persistent.

  3. (iii)

    When \(\lambda _{13}^+<0\), \(\gamma < \gamma ^*\), \( e_{12} < e_{12}^*\) or \(\lambda _{13}^+<0\), \(\gamma \ge \gamma ^*\), \(P_{13}^+\) is globally asymptotically stable in int\(R_+^3\).

A threshold \(\gamma ^*\) in the ant interference is defined in Theorem 4.3. In this section, we focus on the effect of ant interference on the persistence of (2), while the effect of plant castration is discussed in Sect. 5 of this paper. In the situation considered by Theorem 4.3, pollinators (respectively, ants) can survive in the absence of ants (respectively, pollinators). The condition \(\lambda _{13}^+ >0\) in Theorem 4.3(i) can be rewritten as \(e_{21} > \bar{e}_{21}\) with \( \bar{e}_{21} = r_2 (1+\alpha \bar{x}_1^++\beta x_3^+ )/\bar{x}_1^+ .\) Thus, if the pollinators’ efficiency is high (i.e., \(e_{21} > \bar{e}_{21}\)), Theorem 4.3(i) shows that they can persist in the presence of ants.

Assume the pollinators’ efficiency is low (i.e., \(e_{21} < \bar{e}_{21}\)). If the ant interference is weak (i.e., \(\gamma < \gamma ^*\)), Theorem 4.3(ii) shows that pollinators can survive when the plants’ efficiency is high (i.e., \(e_{12} \ge e_{12}^*\)) and pollinators’ initial density is large. However, if the ant interference is strong (i.e., \(\gamma \ge \gamma ^*\)), Theorem 4.3(iii) shows that pollinators will be driven into extinction. Since the pollinators can survive in the absence of ants, it is the strong ant interference that leads to the extinction of pollinators.

When pollinators with low initial densities cannot survive in the plant–pollinator system, Theorem 4.4 shows the way by which pollinators can persist in the presence of ants, while the proof is in “Appendix 6.”

Theorem 4.4

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}>0\).

  1. (i)

    When \(\lambda _{13}^+ >0\), system (2) is uniformly persistent.

  2. (ii)

    Assume \(\lambda _{13}^+<0.\) When \(\gamma < \gamma ^*\) and \( e_{12} \ge e_{12}^*\), \(P_{13}^+\) is locally asymptotically stable with a basin of attraction \(\omega _{13}\) and system (2) restricted on \(R_+^3 - \omega _{13}\) is uniformly persistent. Otherwise, \(P_{13}^+\) is globally asymptotically stable in int\(R_+^3\).

The threshold \(\gamma ^*\) is crucial to the survival of pollinators in the situation of Theorem 4.4, where the plant–ant system is persistent in the absence of pollinators, while pollinators cannot survive in the plant–pollinator system (\(e_{12} < e_{12}^0\)) or their persistence in the plant–pollinator system is density-dependent (\(e_{12} \ge e_{12}^0\)). If the pollinators’ efficiency is high (\(e_{21} > \bar{e}_{21}\)), Theorem 4.4(i) shows that pollinators can survive in the presence of ants.

Assume the pollinators’ efficiency is low (\(e_{21} < \bar{e}_{21}\)). When the ant interference is weak (\(\gamma < \gamma ^*\)), Theorem 4.4(ii) shows that the pollinators can survive in the presence of ants if the plants’ efficiency is high (\( e_{12} \ge e_{12}^*\)) and pollinators’ initial density is large. Otherwise, when the ant interference is strong (\(\gamma \ge \gamma ^*\)), Theorem 4.4(ii) shows that the pollinators will be driven into extinction. Since pollinators with large initial densities can survive in the plant–pollinator systems as \(e_{12} \ge e_{12}^0\), it is the strong ant interference that leads to the extinction of pollinators.

On the other hand, Theorem 4.4(ii) shows that when the ant interference is weak (\(\gamma < \gamma ^*\)), the pollinators, who cannot survive in the plant–pollinator system as \(e_{12} < e_{12}^0\), can survive in the plant–pollinator–ant system if the plants’ efficiency is high (\( e_{12} \ge e_{12}^*\)) and pollinators’ initial density is large. This means that plant–ant mutualisms could promote the survival of pollinators when the ant interference is weak.

Second, we consider the case of \( \lambda _1^{(3)}<0\) and \(e_{13} < e_{13}^0\), which implies that ants cannot survive in the absence of pollination mutualisms. When pollinators can persist in the plant–pollinator system, Lemma 4.5 shows conditions under which system (2) has interior equilibria, while the proof is in “Appendix 7.”

Lemma 4.5

Let \(\lambda _1^{(2)}>0, \lambda _1^{(3)}<0, \lambda _{12}^+ <0\) and \(e_{13} < e_{13}^0\). There exists \(e_{13}^*>0\) such that (2) has interior equilibria if and only if \(e_{13} \ge e_{13}^*\).

Based on the Lemma 4.5, we obtain Theorem 4.6 by a proof similar to that of Theorem 4.3, while we omit the proof.

Theorem 4.6

Let \(\lambda _1^{(2)}>0, \lambda _1^{(3)}<0\) and \(e_{13} < e_{13}^0\).

  1. (i)

    When \(\lambda _{12}^+ >0\), system (2) is uniformly persistent.

  2. (ii)

    Assume \(\lambda _{12}^+<0\). When \( e_{13} \ge e_{13}^*\), \(P_{12}^+\) is locally asymptotically stable with a basin of attraction \(\omega _{12}\) and system (2) restricted on int\(R_+^3 - \omega _{12}\) is uniformly persistent. Otherwise, \(P_{12}^+\) is globally asymptotically stable in int\(R_+^3\).

When ants cannot survive in the absence of pollination mutualisms, Theorem 4.6 shows that the ants’ efficiency in translating plant–ant interactions into fitness is important to their invasion in the plant–pollinator system. The condition \(\lambda _{12}^+ >0\) in Theorem 4.6(i) can be rewritten as \(e_{31} > \bar{e}_{31}\) with \( \bar{e}_{31} = r_3 (1+\alpha x_1^+ )/x_1^+.\) Theorem 4.6(i) demonstrates that when the ants’ efficiency is high (\(e_{31} > \bar{e}_{31}\)), ants can invade the plant–pollinator system and persist. When the ants’ efficiency is low but plants’ efficiency is large (\(e_{13} \ge e_{13}^*\)), Theorem 4.6(ii) shows that the ants with large initial densities can survive. Otherwise, the ants will go to extinction.

When pollinators cannot survive in the plant–pollinator system, we have \(\lambda _1^{(2)}<0\) and \( e_{12} < e_{12}^0\). When \(\gamma ^* \le \bar{\beta }\), by a proof similar to that of Lemma 4.2, we conclude that system (2) has interior equilibria if and only if \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\). When \(\gamma ^* > \bar{\beta }\), by a proof similar to that of Lemma 4.5, we conclude that system (2) has interior equilibria if and only if \(e_{13} \ge e_{13}^*\). Therefore, we have the following result.

Lemma 4.7

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, ~e_{12} < e_{12}^0,~ e_{13} < e_{13}^0.\)

  1. (i)

    When \(\gamma ^* \le \bar{\beta }\), system (2) has interior equilibria if and only if \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\).

  2. (ii)

    When \(\gamma ^* > \bar{\beta }\), system (2) has interior equilibria if and only if \(e_{13} \ge e_{13}^*\).

Based on the Lemma 4.7, we obtain Theorem 4.8 by a proof similar to that of Theorem 4.3, while we omit the proof.

Theorem 4.8

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, ~e_{12} < e_{12}^0,~ e_{13} < e_{13}^0.\)

  1. (i)

    Assume \(\gamma ^* \le \bar{\beta }\). If \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\), \(P_1\) is locally asymptotically stable with a basin of attraction \(\omega _1\) and system (2) restricted on \(R_+^3 - \omega _{1}\) is uniformly persistent. Otherwise, \(P_1\) is globally asymptotically stable in int\(R_+^3\).

  2. (ii)

    Assume \(\gamma ^* > \bar{\beta }\). If \(e_{13} \ge e_{13}^*\), \(P_1\) is locally asymptotically stable with a basin of attraction \(\omega _1\) and system (2) restricted on \(R_+^3 - \omega _{1}\) is uniformly persistent. Otherwise, \(P_1\) is globally asymptotically stable in int\(R_+^3\).

When pollinators (respectively, ants) cannot survive in the absence of ants (respectively, pollinators), Theorem 4.8 demonstrates that the plants’ efficiencies play a crucial role in the survival of both pollinators and ants. By the expression of \(\gamma ^*\) in Lemma 4.2, the assumption \(\gamma ^* \le \bar{\beta }\) in Theorem 4.8(i) is equivalent to the inequality \(\bar{x}_1^0 \le x_1^0,\) which implies that the pollinators’ efficiency is relatively low by the definition of \(\bar{x}_1^0\) and \(x_1^0\) in (4) and (5). Thus, when the pollinators’ efficiency is relatively low (\(\gamma ^* \le \bar{\beta }\)), Theorem 4.8(i) shows that the persistence of the three species is density-dependent if the ant interference is weak (\(\gamma < \gamma ^*\)) and plants’ efficiency is high (\(e_{12} \ge e_{12}^*\)). Otherwise, both ants and pollinators will go to extinction. When the pollinators’ efficiency is relatively high (\(\gamma ^* > \bar{\beta }\)), Theorem 4.8(ii) shows that persistence of the three species is density-dependent if the plants’ efficiency is high (\(e_{13} \ge e_{13}^*\)). Otherwise, both ants and pollinators will go to extinction.

When pollinators’ persistence in the plant–pollinator system is density-dependent, Lemma 4.9 shows conditions under which there are interior equilibria of system (2), while the proof is in “Appendix 8.”

Lemma 4.9

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, \lambda _{12}^- >0, ~e_{12} \ge e_{12}^0,~ e_{13} < e_{13}^0\).

  1. (i)

    Assume \(\gamma ^* \le \bar{\beta }\). There are interior equilibria of system (2) if and only if \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\).

  2. (ii)

    Assume \(\gamma ^* >\bar{\beta }\). There exist \(\bar{\gamma }_0>0\) and \(\gamma _0 >0\) such that when \(\bar{\gamma } < \bar{\gamma }_0\) and \(\gamma \le \gamma _0\), system (2) has interior equilibria. When \(\bar{\gamma } \ge \bar{\gamma }_0\) or \(\gamma > \gamma _0\), system (2) has interior equilibria if and only if \(e_{13} \ge e_{13}^*\).

Based on the Lemma 4.9, we obtain Theorem 4.10 by a proof similar to that of Theorem 4.3, while we omit the proof.

Theorem 4.10

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, \lambda _{12}^- > 0, ~e_{12} \ge e_{12}^0, ~e_{13} < e_{13}^0\). Then, \(P_1\) is locally asymptotically stable with a basin of attraction \(\omega _1\).

  1. (i)

    Assume \(\gamma ^* \le \bar{\beta }\). If \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\), then system (2) restricted on int\(R_+^3 - \omega _{1}\) is uniformly persistent. Otherwise, \(P_1\) is globally asymptotically stable in int\(R_+^3\).

  2. (ii)

    Assume \(\gamma ^* > \bar{\beta }\). If \(e_{13} \ge e_{13}^*\), or, \(\bar{\gamma } < \bar{\gamma }_0\) and \(\gamma \le \gamma _0\), then system (2) restricted on int\(R_+^3 - \omega _{1}\) is uniformly persistent. Otherwise, \(P_1\) is globally asymptotically stable in int\(R_+^3\).

When ants cannot survive in the absence of pollinators, Theorem 4.10 shows that sufficiently weak interference could lead to coexistence of the three species, although pollinators’ persistence in the plant–pollinator system is density-dependent. The condition \(\lambda _{12}^- >0\) in Theorem 4.10 can be rewritten as \( e_{31} > e_{31}^-\) with \( e_{31}^- = r_3(1+\bar{\alpha }x_1^- )/x_1^- . \) When the ant interference is sufficiently weak (\(\gamma < \gamma _0\)), Theorem 4.10(ii) shows that the three species could coexist if their initial densities are in an appropriate region and plant castration is weak (\(\bar{\gamma } < \bar{\gamma }_0\)). The reason is that under the sufficiently weak interference, plant–pollinator and plant–ant mutualisms can benefit each other, which promotes ants’ survival. However, when the ant interference is not sufficiently weak (\(\gamma > \gamma _0\)) but ants’ efficiency is small (\(e_{13} < e_{13}^*\)), Theorem 4.10(ii) shows that the ants can invade the plant–pollinator system and drive pollinators into extinction, which consequently results in the extinction of the ants themselves. A similar discussion can be given for the situation in Theorem 4.10(i), while more discussions are in Remark 4.17 at the end of this section.

When \(\lambda _{12}^+ < 0,\) we have \(x_1^0 < x_1^+ < \bar{x}_1^0 \) and \(\lambda _{12}^- < 0.\) Then, \(\gamma ^* > \bar{\beta }\) and \(P_{12}^+\) is locally asymptotically stable with a basin of attraction \(\omega _{12}\). By a proof similar to that of Lemma 4.5 and Theorem 4.6, we have Lemma 4.11 and Theorem 4.12.

Lemma 4.11

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, \lambda _{12}^+ <0, ~e_{12} \ge e_{12}^0, ~e_{13} < e_{13}^0.\) There are interior equilibria of system (2) if and only if \(e_{13} \ge e_{13}^*\).

Theorem 4.12

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, \lambda _{12}^+ <0, ~e_{12} \ge e_{12}^0, ~e_{13} < e_{13}^0.\) If \(e_{13} \ge e_{13}^*\), then system (2) restricted on int\(R_+^3 - \omega _{1}- \omega _{12}\) is uniformly persistent. Otherwise, solutions of (2) with \(x(0)>0\) converge to \(P_1 \cup P_{12}^\pm \).

When ants cannot survive in the absence of pollinators, Theorem 4.12 shows that plants’ efficiency is crucial to ants’ survival if pollinators’ persistence in the plant–pollinator system is density-dependent. When the plants’ efficiency is high (\(e_{13} \ge e_{13}^*\)), Theorem 4.12 shows that the three species could coexist if their initial densities are in an appropriate region. Otherwise, ants will go to extinction.

Assume \(\lambda _1^{(2)}<0, ~e_{12} \ge e_{12}^0\). If \(\lambda ^-_{12} < 0\) and \(\lambda _{12}^+ > 0\), then \(P_{12}^-\) has a two-dimensional stable manifold in int\(R_+^3\). Thus, there is a solution \(x(t)\) of (2) with \(x(0)>0\) such that \(\lim _{t \rightarrow \infty }x_2(t) = 0\), which implies that system (2) is not persistent. Therefore, we conclude the following result.

Theorem 4.13

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, ~e_{12} \ge e_{12}^0, ~e_{13} < e_{13}^0.\) When \(\lambda _{12}^- \cdot \lambda _{12}^+ <0,\) system (2) is not persistent.

Third, we consider the case of \( \lambda _1^{(3)}<0\) and \(e_{13} \ge e_{13}^0\), which implies that ants’ persistence in the plant–ant system is density-dependent. Thus, \(P_{13}^+\) and \(P_{13}^-\) are boundary equilibria of (2). On the \((x_1,x_3)\)-plane, the separatrices of \(P_{13}^-\) divide the interior of the plane into two regions \(\omega _1^0\) and \( \omega _{13}^0\): \(\omega _1^0\) is the basin of attraction of \(P_1\) and \(\omega _{13}^0\) is that of \(P_{13}^+\) as shown in Fig. 1.

Let \(x(t)\) be a solution of (2) with \((x_1(0),x_3(0)) \in \omega _{13}^0.\) Let \(\bar{x}(t)\) be a solution of (3) with \(\bar{x}_i(0)=x_i(0), i=1,3.\) By (2), \(x(t)\) satisfies

$$\begin{aligned} \frac{dx_1}{dt}&\ge x_1 ( r_1- d_1 x_1) + \frac{e_{13} x_1x_3}{1+\bar{\alpha } x_1 + \bar{\beta } x_3}- \bar{\gamma } x_1 x_3 \\ \frac{dx_3}{dt}&\ge x_3 \left( -r_3 + \frac{e_{31}x_1}{1+\bar{\alpha } x_1 + \bar{\beta } x_3}\right) \end{aligned}$$

so that the comparison theorem (Cosner 1996) implies \( x_i(t) \ge \bar{x}_i(t)\) as \(t>0, i=1,3.\) Since \(\bar{x}(t)\) converges to \(P_{13}^+\), we have \(\lim \inf _{t \rightarrow \infty } x_i(t) \ge \delta _0\) for some \(\delta _0>0.\) Thus, ants can persist if \(x(0)\) satisfies \((x_1(0),x_3(0)) \in \omega _{13}^0.\)

When \(\lambda _1^{(2)}>0,\) \(P_{12}^+\) is globally asymptotically stable in the interior of the \((x_1,x_2)\)-plane. By a proof similar to that of Theorem 4.3, we have the following result based on Lemma 4.2.

Theorem 4.14

Let \(\lambda _1^{(2)}>0, \lambda _1^{(3)}<0, ~e_{13} \ge e_{13}^0.\)

  1. (i)

    If \(\lambda _{13}^+ >0, \lambda _{13}^- >0\) and \(\lambda _{12}^+ >0,\) then system (2) is uniformly persistent.

  2. (ii)

    If \(\lambda _{13}^+ >0, \lambda _{13}^- >0\) and \(\lambda _{12}^+ <0,\) then \(P_{12}^+\) is locally asymptotically stable with a basin of attraction \(\omega _{12}\). System (2) restricted on int\(R_+^3 - \omega _{12}\) is uniformly persistent.

  3. (iii)

    If \(\lambda _{13}^+ <0, \lambda _{13}^- <0\) and \(\lambda _{12}^+ <0,\) then the equilibrium sets \(P_{12}^+\) and \(P_{13}^+ \cup P_{13}^-\) have basins of attraction \(\omega _{12}\) and \(\omega _{13}\) in \(R_+^3\), respectively. If \( \gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*,\) system (2) restricted on int\(R_+^3 - \omega _{12}- \omega _{13}\) is uniformly persistent. Otherwise, solutions of (2) with \(x(0) >0\) converge to \(P_{12}^+ \cup P_{13}^\pm .\)

  4. (iv)

    If \(\lambda _{13}^+ <0, \lambda _{13}^- <0\), \(\lambda _{12}^+ >0\), or, \(\lambda _{13}^+ \cdot \lambda _{13}^- <0\), then system (2) is not persistent.

When \(\lambda _1^{(2)}<0\) and \(e_{12} < e_{12}^0\), there is no equilibrium of (2) in the interior of the \((x_1,x_2)\)-plane. By a proof similar to that of Theorem 4.3, we have the following result based on Lemma 4.2.

Theorem 4.15

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0,~ e_{12} < e_{12}^0, ~e_{13} \ge e_{13}^0.\)

  1. (i)

    If \(\lambda _{13}^+ >0\) and \( \lambda _{13}^- >0\), then \(P_1\) has a basin of attraction \(\omega _{1}\) in \(R_+^3\) and system (2) restricted on int\(R_+^3 - \omega _{1}\) is uniformly persistent.

  2. (ii)

    When \(\lambda _{13}^+ <0\) and \( \lambda _{13}^- <0\), the equilibrium sets \(P_1\) and \(P_{13}^+ \cup P_{13}^-\) have basins of attraction \(\omega _{1}\) and \(\omega _{13}\) in \(R_+^3\), respectively. If \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\), then system (2) restricted on int\(R_+^3 - \omega _{1} - \omega _{13}\) is uniformly persistent. Otherwise, solutions of (2) with \(x(0) >0\) converge to \(P_1 \cup P_{13}^\pm .\)

  3. (iii)

    If \(\lambda _{13}^+ \cdot \lambda _{13}^- <0\), then system (2) is not persistent.

When \(\lambda _1^{(2)}<0\) and \(e_{12} \ge e_{12}^0\), \(P_{12}^+\) and \(P_{12}^-\) are boundary equilibria of (2). By a proof similar to that of Theorem 4.3, we have the following result based on Lemma 4.2.

Theorem 4.16

Let \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, ~e_{12} \ge e_{12}^0, ~e_{13} \ge e_{13}^0.\)

  1. (i)

    If \(\lambda _{12}^\pm >0\) and \(\lambda _{13}^\pm >0\), then system (2) restricted on int\(R_+^3 - \omega _{1} \) is uniformly persistent.

  2. (ii)

    If \(\lambda _{12}^\pm <0\) and \(\lambda _{13}^\pm >0\), then system (2) restricted on int\(R_+^3 - \omega _{1} - \omega _{12}\) is uniformly persistent.

  3. (iii)

    Assume \(\lambda _{12}^\pm >0\) and \(\lambda _{13}^\pm <0\). If \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\), then system (2) restricted on int\(R_+^3 - \omega _{1} - \omega _{13}\) is uniformly persistent. Otherwise, solutions of (2) with \(x(0) >0\) converge to \(P_1 \cup P_{13}^\pm .\)

  4. (iv)

    Assume \(\lambda _{12}^\pm <0\) and \(\lambda _{13}^\pm <0\). If \(\gamma < \gamma ^*\) and \(e_{12} \ge e_{12}^*\), then system (2) restricted on int\(R_+^3 - \omega _{1} - \omega _{12}- \omega _{13}\) is uniformly persistent. Otherwise, solutions of (2) with \(x(0) >0\) converge to \(P_1 \cup P_{12}^\pm \cup P_{13}^\pm .\)

  5. (v)

    If \(\lambda _{12}^+ ~ \cdot \lambda _{12}^- <0\) or \(\lambda _{13}^+ ~ \cdot \lambda _{13}^- <0\), then system (2) is not persistent.

When ants’ persistence in the plant–ant system is density-dependent, Theorems 4.14–4.16 demonstrate that levels of the ant interference could determine the persistence of the three-species system. In the situation considered by Theorem 4.14, pollinators can survive in the plant–pollinator system. Assume both the pollinators’ and ants’ efficiencies are low (i.e., \(\lambda _{13}^+ <0, \lambda _{13}^- <0,\lambda _{12}^+ <0\) as discussed above in this section). When the ant interference is weak (\(\gamma < \gamma ^*\)), Theorem 4.14(iii) shows that the three species could coexist if their initial densities are in an appropriate region and plants’ efficiency is high (\(e_{12} \ge e_{12}^*\)). However, when the ant interference is strong (\(\gamma \ge \gamma ^*\)), Theorem 4.14(iii) demonstrates that either pollinators or ants will go to extinction. Similar discussions can be given for other situations in Theorems 4.14–4.16.

Remark 4.17

The critical values \(e_{12}^*\) and \(e_{13}^*\) can vary with the ant interference \(\gamma \), which implies that the levels of ant interference could determine the survival of both pollinators and ants. Indeed, the slope of \(F(x_1)\) decreases monotonically with the increase in \(\gamma \) as \( x_1 >\bar{x}_1^0 \): from Eq. (20), we have \(\partial F /\partial x_1 <0, \partial ^2 F /\partial \gamma \partial x_1 <0\) as \( x_1 >\bar{x}_1^0 \), which implies

$$\begin{aligned} \frac{\partial e_{13}^*}{\partial \gamma }>0,\quad \lim _{\gamma \rightarrow \infty } e_{13}^* =+\infty . \end{aligned}$$

Similarly, from Eq. (19), we have

$$\begin{aligned} \frac{\partial e_{12}^*}{\partial \gamma }>0,\quad \lim _{\gamma \rightarrow 1/x_1^0-} e_{12}^* =+\infty . \end{aligned}$$

As an example, we consider the situation in Theorem 4.8(i), while similar discussions can be given for other situations in Theorems of this paper. Indeed, the condition \(e_{12} < e_{12}^0\) and \(e_{13} < e_{13}^0\) in Theorem 4.8 implies that there exists \(\hat{\gamma }>0\) such that when \(\gamma < \hat{\gamma }\), we have \(e_{12}^* < e_{12}^0,~ e_{13}^* < e_{13}^0\) and conditions in Theorem 4.8 can be effective. Moreover, as shown in Theorem 4.8(i), when the ant interference is small such that \(e_{12}^* < e_{12}\), the three species with appropriate initial densities could coexist. However, when the ant interference is large such that \(e_{12}^* > e_{12}\), both pollinators and ants will go to extinction.

Remark 4.18

The mathematical results of persistence are summarized in Table 2.

Table 2 Results of uniform persistence
Full size table

5 Stability of the Interior Equilibrium

When the three-species system (2) is uniformly persistent, there exists an interior equilibrium \(P^*(x_1^*, x_2^*, x_3^*)\) by the theorems of Butler et al. (1986), Butler and Waltman (1986), Freedman et al. (1994). By the right-hand sides of (2), \(P^*\) satisfies

$$\begin{aligned}&A_1 x_1^{*2} + B_1 x_1^* + C_1=0\nonumber \\&x_2^* = \left[ \frac{e_{21}-r_2 \alpha }{r_2 \beta } - \frac{\gamma (e_{31}-r_3 \bar{\alpha })}{r_3 \beta \bar{\beta }} \right] x_1^* - \frac{1}{\beta } + \frac{\gamma }{\beta \bar{\beta }}\nonumber \\&x_3^* = \frac{e_{31}-r_3 \bar{\alpha }}{r_3 \bar{\beta }} x_1^* - \frac{1}{\bar{\beta }} \end{aligned}$$
(9)

where

$$\begin{aligned} A_1&= -d_1, \quad C_1= \frac{ e_{12} r_2 \gamma }{e_{21} \beta \bar{\beta }} - \frac{ e_{12} r_2 }{e_{21} \beta } - \frac{ e_{13} r_3 - e_{31} \bar{\gamma } }{e_{31} \bar{\beta }}\nonumber \\ B_1&= r_1 + \frac{e_{12} (e_{21}-r_2 \alpha )}{e_{21} \beta } - \frac{e_{12} r_2 \gamma (e_{31}-r_3 \bar{\alpha })}{e_{21} r_3 \beta \bar{\beta }} + \frac{ (e_{13} r_3 - e_{31} \bar{\gamma }) (e_{31} - r_3 \bar{\alpha } ) }{e_{31} r_3 \bar{\beta } }.\nonumber \\ \end{aligned}$$
(10)

Since there exist at most two interior equilibria of (2), they are denoted by \(P^*(x_1^*, x_2^*, x_3^*)\) and \(\bar{P}^*(\bar{x}_1^*, \bar{x}_2^*, \bar{x}_3^*)\) with

$$\begin{aligned} x_1^*&= \frac{-B_1+\sqrt{B_1^2 - 4 A_1 C_1}}{2 A_1}, \quad x_2^* =x_2^* (x_1^*), \quad x_3^* =x_3^* (x_1^*)\\ \bar{x}_1^*&= \frac{-B_1 - \sqrt{B_1^2 - 4 A_1 C_1}}{2 A_1}, \quad \bar{x}_2^* =\bar{x}_2^* (\bar{x}_1^*), \quad \bar{x}_3^* =\bar{x}_3^* (\bar{x}_1^*). \end{aligned}$$

Let \(f,g,h\) represent the right-hand sides of (2), respectively. In the following discussion, we focus on equilibrium \(P^*\), while a similar discussion can be given for \(\bar{P}^*\). The Jacobian matrix of system (2) at \(P^*\) has the form

$$\begin{aligned} J(P^*) = \left[ \begin{array}{ccc} f_1^* &{}f_2^* &{}f_3^* \\ g_1^* &{}g_2^* &{}g_3^* \\ h_1^* &{}0 &{}h_3^* \end{array}\right] \!, \end{aligned}$$
(11)

where

$$\begin{aligned} f_1^*&= x_1^* \left[ -d_1 - \frac{e_{12} \alpha x_2^*}{(1+\alpha x_1^*+\beta x_2^* + \gamma x_3^*)^2} - \frac{e_{13} \bar{\alpha } x_3^*}{ (1+\bar{\alpha } x_1^* + \bar{\beta } x_3^*)^2 } \right] <0\\ f_2^*&= \frac{(1+\alpha x_1^* + \gamma x_3^*) e_{12} x_1^*}{(1+\alpha x_1^*+\beta x_2^* + \gamma x_3^*)^2}>0\\ f_3^*&= x_1^* \left[ - \frac{e_{12} \gamma x_2^*}{(1+\alpha x_1^*+\beta x_2^* + \gamma x_3^*)^2} + \frac{e_{13} (1+\bar{\alpha } x_1^* )}{( 1+\bar{\alpha } x_1^* + \bar{\beta } x_3^* )^2 } - \bar{\gamma }\right] \\ g_1^*&= x_2^* \frac{e_{21}(1 +\beta x_2^* + \gamma x_3^* )}{(1+\alpha x_1^*+\beta x_2^* + \gamma x_3^*)^2}>0, \quad g_2^* = - \frac{e_{21} \beta x_1^* x_2^*}{(1+\alpha x_1^*+\beta x_2^* + \gamma x_3^*)^2}<0\\ g_3^*&= - \frac{e_{21} \gamma x_1^* x_2^*}{(1+\alpha x_1^*+\beta x_2^* + \gamma x_3^*)^2}<0, \quad h_1^* = \frac{e_{31}x_3^* (1 + \bar{\beta } x_3^*)}{(1+\bar{\alpha } x_1^* + \bar{\beta } x_3^*)^2}>0\\ h_3^*&= - \frac{e_{31} \bar{\beta } x_1^* x_3^* }{(1+\bar{\alpha } x_1^* + \bar{\beta } x_3^*)^2}<0. \end{aligned}$$

The characteristic equation of \(J(P^*)\) is given by

$$\begin{aligned} \lambda ^3 + a_1 \lambda ^2 + a_2 \lambda + a_3 =0 \end{aligned}$$
(12)

where

$$\begin{aligned} a_1&= - ( f_1^* + g_2^* + h_3^*)>0,~~ a_2 = f_1^* (g_2^* + h_3^*) + f_2^* g_1^* + f_3^* h_1^* + g_2^* h_3^*\\ a_3&= -f_1^* g_2^* h_3^* + f_2^* (g_1^* h_3^* - g_3^* h_1^*) + f_3^* g_2^* h_1^*. \end{aligned}$$

Since \(a_1>0\), we conclude the following result by Routh–Hurwitz criterion and the bifurcation criterion by Yu (2005).

Theorem 5.1

  1. (i)

    If

    $$\begin{aligned} a_3>0, \quad a_1a_2-a_3>0 \end{aligned}$$

    , then the interior equilibrium \(P^*\) is locally asymptotically stable.

  2. (ii)

    If \(a_1a_2-a_3=0\), then Hopf bifurcation occurs at \(P^*\).

The Hopf bifurcation at \(P^*\) by varying the parameter \(e_{31}\) is shown as follows, which is an application of Proposition 3.5 by Ruan (2001). Indeed, when the condition in Theorem 5.1(i) is violated (e.g., \(a_1a_2-a_3<0\)), stability of \(P^*\) changes and the Hopf bifurcation occurs. Assume that there is a critical value \(e_{31}^0\) which satisfies \(a_3(e_{31}^0)>0, a_1(e_{31}^0)a_2(e_{31}^0)-a_3(e_{31}^0)=0\) and \([a_1(e_{31}) a_2(e_{31}) -a_3(e_{31})]'|_{e_{31} = e_{31}^0} <0\). When \(e_{31} = e_{31}^0\), the characteristic equation (12) can be written as

$$\begin{aligned} (\lambda +a_1) (\lambda ^2 +a_2 )=0 \end{aligned}$$

which has the roots of \(\lambda _1(e_{31}^0) = -a_1(e_{31}^0)<0\) and \(\lambda _{2,3}(e_{31}^0) = \pm i \sqrt{a_2(e_{31}^0)}\).

Let \(\lambda _{2,3}(e_{31}) = \mu (e_{31}) \pm i \nu (e_{31})\) be the pair of conjugate complex roots of (12). In order to show Hopf bifurcation at \(e_{31} = e_{31}^0\) , we should verify the transversality condition

$$\begin{aligned} \frac{d \mathrm{Re } \lambda _{2,3}}{d e_{31}}|_{e_{31} = e_{31}^0} = \frac{d \mu (e_{31})}{d e_{31}}|_{e_{31} = e_{31}^0} \ne 0. \end{aligned}$$
(13)

Replacing \(\lambda \) with \(\lambda _2(e_{31})\) in equation (12) and calculating the derivatives on \(e_{31}\), we obtain the following equations as \(e_{31} = e_{31}^0\)

$$\begin{aligned} c_0 \mu ^{\prime } -c_1 \nu ^{\prime } +c_2=0, \quad c_1 \mu ^{\prime } +c_0 \nu ^{\prime } +c_3=0 \end{aligned}$$
(14)

where

$$\begin{aligned} c_0&= -2a_2(e_{31}^0), \quad c_1= 2a_1(e_{31}^0) \sqrt{a_2(e_{31}^0)}\\ c_2&= a_3'(e_{31}^0) - a_1'(e_{31}^0) a_2(e_{31}^0), \quad c_3 = a_2'(e_{31}^0) \sqrt{a_2(e_{31}^0)}. \end{aligned}$$

From (14), we have

$$\begin{aligned} \frac{d \mathrm{Re } \lambda _{2,3}}{d e_{31}}|_{e_{31} = e_{31}^0} =\mu '|_{e_{31} = e_{31}^0} =- \frac{(a_1a_2-a_3)'}{a_1^2 +a_2}|_{e_{31} = e_{31}^0} >0. \end{aligned}$$

Thus, the real part of \( \lambda _{2,3}\) changes from a negative value to zero as \(e_{31}\) increases to \(e_{31}^0\), while it becomes positive as \(e_{31}>e_{31}^0\). Similar to Proposition 3.5 by Ruan (2001), we conclude the following result.

Proposition 5.2

Assume that there is a positive value \(e_{31}^0\) which satisfies

$$\begin{aligned} a_3(e_{31}^0)>0,\quad a_1a_2-a_3|_{e_{31} = e_{31}^0} =0, \quad \frac{d(a_1 a_2 -a_3)}{d e_{31}}|_{e_{31} = e_{31}^0} <0. \end{aligned}$$

When \(e_{31} < e_{31}^0\), the interior equilibrium \(P^*\) is asymptotically stable. When \(e_{31} = e_{31}^0\), \(P^*\) loses its stability and Hopf bifurcation occurs. When \(e_{31} > e_{31}^0\), \(P^*\) becomes unstable and a stable periodic orbit is bifurcated.

As an example, we fix \(r_1=r_2=r_3=1,d_1=0.01,e_{12}=e_{13}=0.1, \alpha =\bar{\alpha }=\beta =\bar{\beta }=0.1,\) \( e_{21}=0.204, \gamma =0.8\), \(\bar{\gamma }=0.5\), and let \(e_{31}\) vary. A direct computation shows that \(e_{31}^0 \approx 0.1328\). When \(e_{31}(=0.10)\) is extremely small, there is no interior equilibrium and all solutions converge to \(P_{12}^+\), as shown in Fig. 4a. When \(e_{31}(=0.13)\) is less than \(e_{31}^0\), interior equilibrium \(P^*\) is asymptotically stable, as shown in Fig. 4b. When \(e_{31}\)(\(=\)0.1452) is larger than \(e_{31}^0\), \(P^*\) becomes unstable and a stable periodic orbit is bifurcated, as shown in Fig. 4c. When \(e_{31}\)(\(=\)0.18) is extremely large, there is no interior equilibrium and all solutions converge to \(P_{13}^+\), as shown in Fig. 4d.

6 Discussion

This paper considers plant–pollinator–ant systems in which the plant–pollinator interaction is mutualistic, the plant–ant interaction is unidirectional, and ants interfere with pollinators. Dynamics of the model demonstrate that the emergence of ants in the plant–pollinator system play a role in the persistence of pollination mutualism.

Levels of the ant interference are crucial to the survival of pollinators. As shown in Theorem 4.3, a strong interference could result in the extinction of pollinators. When ants depend on pollination mutualism for survival, a strong interference could drive pollinators into extinction, which consequently leads to the extinction of the ants themselves as discussed in Theorem 4.10. On the other hand, when the ant interference is weak, a cooperation between plant–ant and plant–pollinator mutualisms can occur, which promotes the survival of both ants and pollinators as shown in Theorem 4.8.

Fig. 4
figure 4

Bifurcation in system (2) as shown in Theorem 5.1. Solutions of plants (\(x_1\)), pollinators (\(x_2\)), and ants (\(x_3\)) are denoted by red, blue, and black lines, respectively. Fix \(r_1=r_2=r_3=1,d_1=0.01,e_{12}=e_{13}=0.1, \alpha =\bar{\alpha }=\beta =\bar{\beta }=0.1,\) \( e_{21}=0.204, \gamma =0.8\), \(\bar{\gamma }=0.5\), and let \(e_{31}\) vary. a When \(e_{31}=0.10\), ants go to extinction while plants and pollinators coexist. b When \(e_{31}=0.13\), the three species coexist at a stable equilibrium. c When \(e_{31}=0.1452\), Hopf bifurcation occurs and the three species coexist in periodic oscillation. d When \(e_{31}=0.18\), pollinators go to extinction while plants and ants coexist (Color figure online)

Full size image

The levels of ant interference with pollinators can vary in the evolution of ants. As shown by Oña and Lachmann (2011), when ants can obtain a direct benefit from pollination mutualism, they would reduce their attack against pollinators. For example, epiphytic angiosperms are observed to grow in the arboreal nests of several species of ants (Amazonasgebiet 1901), which implies that the ants benefit by taking elaiosomes from seeds or fruits without damaging the seeds. Thus, the ants would ensure that their host plant is cross-fertilized and then reduce their interference with pollinators by discriminating them from herbivores. When the ant interference is sufficiently weak, both plant–ant and plant–pollinator mutualisms can benefit each other, which promotes the persistence of the whole system. Thus, our results related to the levels of ant interference are biologically feasible.

In the unidirectional interactions between plants and ants, although the plant–ant mutualisms are beneficial to the coexistence of the three species, the degree of plant castration by ants is important to the persistence of the whole system. As described in (2), the degree of plant castration is represented by \(\bar{\gamma }\). From (5), we have

$$\begin{aligned} \frac{d g_2(x_1)}{d x_1} = r_1- d_1- d_1x_1 - \frac{\bar{\gamma }}{ \bar{\beta } \bar{x}_1^0} (x_1 - \bar{x}_1^0 + 1) \end{aligned}$$

so that \(\partial [d g_2(x_1)/d x_1] /\partial \bar{\gamma } <0\) as \(x_1 >\bar{x}_1^0 \). Since \(g_2(\bar{x}_1^0)\) is irrelevant to \(\bar{\gamma }\), we have

$$\begin{aligned} \frac{\partial \bar{x}_1^+ }{\partial \bar{\gamma }} <0,\quad \frac{\partial x_3^+ }{\partial \bar{\gamma }} <0,\quad \frac{\partial \bar{x}_1^- }{\partial \bar{\gamma }} >0,\quad \frac{\partial x_3^- }{\partial \bar{\gamma } } >0,\quad \frac{\partial e_{13}^0 }{\partial \bar{\gamma } } >0. \end{aligned}$$

Similarly, from (19) and (20), we have

$$\begin{aligned} \frac{\partial e_{12}^* }{\partial \bar{\gamma }} >0,\quad \frac{\partial e_{13}^*}{\partial \bar{\gamma }} >0. \end{aligned}$$

Thus, the increase in \(\bar{\gamma }\) may lead to the extinction of ants and pollinators. For example, (a) when \(\bar{\gamma }\) is large such that \(e_{13}^0 > e_{13}\), Theorem 3.2 (iii) shows that ants cannot survive in the plant–ant system if \(r_1/d_1<\bar{x}_1^0\). (b) When \(\bar{\gamma }\) is large such that \(e_{12}^* > e_{12}\), Theorem 4.3 (iii) demonstrates that pollinators will be driven into extinction if \(\lambda _{13}^+<0\) and \(\gamma < \gamma ^*\). (c) When \(\bar{\gamma }\) is large such that \(e_{13}^* > e_{13}\), Theorem 4.6 (ii) shows that ants cannot invade the plant–pollinator system and will go to extinction if \(\lambda _{12}^+<0\). Similar discussions can be given for other situations in Theorems of this paper.

Conditions in theorems of this paper can be satisfied. We focus on the condition in Theorems 4.8 (ii), while similar simulations can be given for the others. Let \(r_1=1.0,d_1=0.01,e_{12}=0.2, e_{13}=0.76, \alpha =\bar{\alpha }=0.2,\) \(\beta =\bar{\beta }=0.2,\) \(r_2=r_3=0.5,\) \( e_{21}=0.104, e_{31}=0.102\), and let \(\bar{\gamma }\) and \(\gamma \) vary. Then, we have \(x_1^0 =125, \bar{x}_1^0 =250,\) \(e_{12}^0 =0.69,\) \(\lambda _1^{(2)}= -0.0044, \lambda _1^{(3)}=-0.0046\), \(\gamma ^*=0.4\) and \( \gamma ^* > \bar{\beta }\). Let \(\gamma =0.2, \bar{\gamma } =0.001\), we obtain \(e_{13}^0 =0.767,~e_{13}^* =0.756.\) Thus, the condition in Theorems 4.8(ii) holds. Since \(\lambda _1^{(2)}<0, \lambda _1^{(3)}<0, ~e_{12} < e_{12}^0,~ e_{13} < e_{13}^0,\) pollinators (respectively, ants) cannot survive in the absence of ants (respectively, pollinators). Numerical simulations in Fig. 2 show that when \(\gamma \) and \( \bar{\gamma }\) are small, the three species could coexist if their initial densities are in an appropriate region. Otherwise, both pollinators and ants will go to extinction.

The attraction basins of equilibria vary with key parameters in the model. We focus on the parameter \(\gamma \), while similar discussions can be given for the others. Let \(r_1=1.0,d_1=0.01,e_{12}=0.9, e_{13}=0.1, \alpha =\bar{\alpha }=0.1,\) \(\beta =\bar{\beta }=0.1,\) \(r_2=r_3=0.1,\) \( e_{21}=0.09, e_{31}=0.1\), \(\bar{\gamma }=0.001\), and let \(\gamma \) vary. When \(\gamma = 0.01\), the condition in Theorem 4.3(ii) is satisfied. Thus, boundary equilibria \(O\), \(P_1\), and \(P_{12}\) are unstable while \(P_{13}\) is asymptotically stable. Numerical simulations show that there are two interior equilibria \(\bar{P}^*\) and \(P^*\). \(\bar{P}^*\) is a saddle point with a two-dimensional stable manifold, as shown by the upper surface in Fig. 5. The surface divides int\(R_+^3\) into two regions, one is the basin of attraction of \(P^*\) while the other is that of \(P_{13}\). When \(\gamma \) decreases from 0.01, 0.009 to 0.008, the surface lowers monotonically, as shown in Fig. 5, that is, the attraction basin of \(P^*\) increases monotonically.

Fig. 5
figure 5

Variation in the attraction basin of \(P^*\) when parameter \(\gamma \) changes. Filled and open circles represent the stable and unstable equilibria, respectively. Let \(r_1=1.0,d_1=0.01,e_{12}=0.9, e_{13}=0.1, \alpha =\bar{\alpha }=0.1,\) \(\beta =\bar{\beta }=0.1,\) \(r_2=r_3=0.1,\) \( e_{21}=0.09, e_{31}=0.1\), \(\bar{\gamma }=0.001\), and let \(\gamma \) vary. When \(\gamma = 0.01\), there are two interior equilibria \(\bar{P}^*\) and \(P^*\). \(\bar{P}^*\) is a saddle point with a two-dimensional stable manifold, as shown by the upper surface. The surface divides int\(R_+^3\) into two regions, one is the basin of attraction of \(P^*\) while the other is that of \(P_{13}\). When \(\gamma \) decreases from 0.01, 0.009 to 0.008, the surface lowers monotonically, that is, the attraction basin of \(P^*\) increases monotonically

Full size image

In this paper, the negative effect of ants on plants (plant castration) is described by a Holling I functional response. When it is characterized by other forms such as Holling II function, the plant–pollinator–ant coexistence can be analyzed in a similar way. Although the model in this work is simple, it provides an explanation for the persistence of pollination mutualism when there exist ants, which may be helpful for understanding complexity in multiple-species mutualism association.