Introduction

Biodiversity has both fascinated and puzzled biologists (Wilson 1992). Early experimental and theoretical works have led to the principle of competitive exclusion, which states that two species with similar niches can hardly coexist (Gause 1934). However, such coexistence is readily observed in natural communities. This obvious contradiction between competitive exclusion and species coexistence has been a long-standing problem (Lundberg et al. 2000). In order to resolve this problem, several researchers have proposed various factors that promote species coexistence against competitive exclusion including niche partition, disturbance, habitat heterogeneity, and environmental fluctuation (Begon et al. 1996; Tokeshi 1999; Chesson 2000).

In this study, we propose an additional possible factor affecting species coexistence of animals. This factor is a behavioral interaction between individuals with regard to their spatial positions. In this paper, we propose a plausible hypothesis regarding this behavioral interaction, and then discuss the effect of the hypothesis on species coexistence.

Hypothesis

Suppose two competing species share the same niche. Males of these species establish a territory and breed. They attack and exclude conspecific males from their territories in order to secure their mating opportunities, paternity, and ecological resources such as food and nesting sites. They attack heterospecific males only to defend their ecological resources. Among males, since heterospecific males are not competitors with regard to mating and paternity, aggressive behavior against conspecific males is more important than that against heterospecific males.

When a male of one of these two species establishes his territory in the neighborhood of other territories of conspecific males, the given male reduces his reproductive success. This is because he wastes more time and energy on vigilance against neighborhood males who attempt to perform extra-pair copulations. Therefore, males are expected, through natural selection, to avoid establishing territories in the neighborhood of other conspecific males. In addition, males that already have territories are expected to attempt to deter other conspecific males from establishing territories nearby. On the other hand, males are also expected to “tolerate” heterospecific males having territories in their neighborhood because the reduction in reproductive success is less than that when conspecific males have territories nearby (perhaps, males may positively accept heterospecific territories in their neighborhood because heterospecific territories can function as “walls” or “alarm devices” against conspecific intruders). Such repulsive behavior (male–male repulsion) will thus result in the territories of the same species being distributed more discretely than those of different species. Thus, males of one species can establish their territories within spatial gaps between heterospecific territories and can use the resources contained therein. In this way, male–male repulsion facilitates the coexistence of competing species, even if their ecological niches completely overlap.

A simple theoretical example

The model

To show the effects of male–male repulsion on species coexistence, we present a simple cellular automaton model. We assumed two competing species (A and B) which completely share the same niche. The males of these species try to establish their territories. These territories are spatially arranged on a two-dimensional lattice (10×10), and each site has four neighboring sites. Each edge of this lattice joins the opposite side of the same lattice thereby eliminating the edge effect. One hundred males have their territories on this lattice. These 100 territorial males mate with females and produce offspring, while this is not possible with the non-territorial males. By the next breeding season, all the adults die (the territories on the lattice are cleared and the lattice returns to its original state), and all offspring become adults. In the next generation (in the next breeding season), these new adults, like their parents, try to establish territories and only territorial males mate and produce offspring. The population dynamics of females and the sex ratio are not considered for simplicity.

In order to describe the male–male repulsion, we assumed that the probability (P) of a male establishing a territory at a certain vacant site is influenced by the state of the four neighbor sites, i.e., whether the sites are occupied or unoccupied and, if occupied, male belonging to which species occupies the sites. Then P is described as follows:

$$ P = \frac{{\beta N_{c} }} {{\beta N_{c} + (1 - \beta )N_{h} }} $$
(1)
$$ \beta = \frac{{0.5c(1 - \alpha ) + 0.5h(1 + \alpha ) + 0.5v}} {4} = \frac{{4 + \alpha (h - c)}} {8} $$
(2)

where c and h are the number of neighboring sites already occupied by conspecific and heterospecific males, respectively (0 ≤ c≤ 4, 0 ≤ h≤4). v is the number of vacant neighboring sites (v = 4 – c – h). N c and N h are the number of conspecific and heterospecific males, which do not have territories yet and try to establish territories at vacant sites, (0 ≤ N c , 0 ≤ N h ) respectively. α is the strength of the conspecific male–male repulsion in both species (0 ≤ α ≤ 1). β indicates the sum effect of male–male repulsion of the four neighboring sites. The probability (P) of a male establishing a territory at a vacant site with conspecific neighbors decreases with an increase in α (When α=0, β=0.5 and P is determined according to N c and N h ).

Then, we assumed that pairs with conspecific neighbors have a more negative effect than that with heterospecific neighbors because it is responsible for the evolution of male–male repulsion. That is, we assumed that the reproductive success of a pair is determined by the number of conspecific and heterospecific neighbors of the pair. Then, the number of offspring (F) produced by one territorial male is described as follows:

$$ F = F_{{{\text{basic}}}} - c \times E_{c} - h \times E_{h} $$
(3)

where F basic is the expected basic number of offspring without any influence from the heterospecific and conspecific neighbors. E c and E h is the expected negative effect on the number of offspring by a conspecific neighbor and a heterospecific neighbor, respectively (the present hypothesis assumed that E c > E h ). In this model, since we assumed that these species completely share the same niche, the difference between these negative effects do not stem from the difference in niche but from the difference in the behavior (e.g., time and energy wasted on vigilance against neighborhood males).

The first and subsequent sites, which were to be occupied by males, were chosen randomly among the vacant sites, and the determination of the species occupying the site was based on the above probabilities. This procedure was repeated until all 100 sites were occupied by the 100 males.

At the beginning of the simulation, five individuals of species A and 100 individuals of species B were introduced. We then examined the occurrence of species coexistence, i.e., whether species A can invade the community and coexist with species B for 1,000 generations. Examination of the ability to invade has commonly been used to evaluate species coexistence (Chesson 2000). In order to examine the effects of male–male repulsion (α) on species coexistence with differences between E c and E h and also between F basic of species A and B, we conducted the simulation as follows: fixed the value of E c and varied the combination of α and E h (Fig. 1) and fixed the F basic of species A and varied the combinations of α and F basic of species B (Fig. 2). These simulations with the same sets of parameters were repeated 1,000 times, and the number of occurrences of coexistence was counted.

Results

Figure 1 shows that the number of occurrences of coexistence increased with an increase in α. This implies that male–male repulsion (α) does promote species coexistence. Figure 1 also shows that the coexistence increased with a decrease in E h . Although this coexistence mechanism is identical to that of the Lotka-Volterra model (if intraspecific competition is stronger than interspecific competition, two competing species can coexist), the coexistence is promoted more by the increase in α. Note, when E h = 0.10 (i.e., E h  = E c ), the male–male repulsion should not evolve due to lack of merit. Therefore, this result (when E h =0.10 and α=0.0, 0.1,..., 1.0) does not reflect nature. However, male–male repulsion certainly promotes species coexistence. We may note, in passing, the situation when α=0 and E h =E c =0.1. In this situation, the intensity of interspecific competition is equal to that of the intraspecific one (interspecific competition coefficient = intraspecific competition coefficient) and the behavior of the population dynamics of the two species is similar to that of the neutral model of molecular evolution (Kimura 1983). In this case, the coexistence does not occur because one species is “fixed” during 1,000 generations (i.e., occurrence of competitive exclusion).

Figure 2 shows that the number of occurrence of coexistence decreases with an increase in F basic of species B. This is because species A becomes the more inferior species in terms of reproductive success with the increase in F basic of species B. However, a sufficiently large α value promotes species coexistence against this effect. Thus, male–male repulsion creates space for even inferior species and promotes their coexistence.

Fig. 1
figure 1

Effects of conspecific male–male repulsion on the coexistence of two competing species under various combinations of α (0.0, 0.1,..., 1.0) and E h (0, 0.01,..., 0.1) with fixed E c at 0.1. F basic of both species were set at 2

Full size image
Fig. 2
figure 2

Effects of conspecific male–male repulsion on the coexistence of two competing species under various combinations of α (0.0, 0.1,..., 1.0) and F basic of species B (2.0, 2.2,..., 4.0) with the F basic of species A fixed at 2. E c and E h were set at 0.1 and 0.08, respectively

Full size image

This model shows that the species coexistence occurs even when two competing species’ ecological niches completely overlap. Naturally, the species coexistence is promoted to a greater extent if there is a difference in the niches.

Comparison with previous hypotheses

The present hypothesis does not conflict with the widely accepted classical competition theory (i.e., Lotka-Volterra model), in that two competing species can coexist if intraspecific competition is stronger than interspecific competition. The difference is that, in our hypothesis, the male–male competition for a conspecific female leads to stronger intraspecific competition (than interspecific competition) and this, as a result, creates a spatial gap for heterospecific males. In other words, the species coexistence, in the present hypothesis, is promoted not only by the strong intraspecific competition but also by the utilization of space, which arises due to this strong intraspecific competition.

Our hypothesis is similar to that of Janzen–Connell (Janzen 1970; Connnell 1971) that explained tree species diversity in tropical rain forests. According to that hypothesis, conspecific trees tend to be separate from each other, as species-specific predators extensively feed on seeds and seedlings of the adult trees. As a result, this leads to more space for other species of trees, which in turn leads to the coexistence of many tree species. The present hypothesis and that of Janzen–Connell differ in terms of the mechanism which creates space, but are similar with regard to the resulting spatial pattern of species distribution.

The present hypothesis is apparently inconsistent with the “aggregation model”. In the aggregation model, superior competitors create partial refuges by aggregating with conspecifics in a few patches, thus allowing inferior species to exist in patches. Subsequently, species coexistence is promoted (Atkinson and Shorrocks 1981,1984; Hartley and Shorrocks 2002; Rohlfs and Hoffmeister 2003). Conversely, in the present hypothesis it is repulsive behavior against conspecifics that promotes species coexistence. This apparent inconsistency stems from the difference in the scale in which vacant space is created: in aggregation model spaces are created in patch scale while in the present hypothesis space is created in territory scale. This is an interesting insight which shows that different phenomena at the individual level can lead to similar results at the community level.

Evidence supporting the hypothesis

The study by Kohda (1998) provides direct evidence which supports our hypothesis. The rocky shores of Lake Tanganyika are covered by territories of herbivorous fishes of the genus Petrochromis. Kohda showed that the territories of male P. polyodon were separated to a greater degree among conspecific males than among heterospecific males. The removal of some males occupying territories suggested that male-mating attacks are the cause of the discrete distribution of male P. polyodon territories. The vacant territories adjacent to those of male P. polyodon were immediately occupied by congeneric fishes other than male P. polyodon. In this case, although the degree to which these species shared the same niche was completely unknown, aggressive male–male mating behavior played a role in creating spatial gaps between conspecific males and in the promotion of the coexistence of multiple territorial species.

Currently, among other species, the following observations might be considered as evidence. The territories of some fishes form dense multispecies mosaics across shores (Genner et al. 1999a). In a certain bird species, males tend to have territories near those of males from the same species less frequently than expected from random distribution (Mikami and Kawata 2002). Males of another bird species establish their territories in neutral areas between the territories held by other species that arrived earlier (Kagawa 1989; Takagi and Ogawa 1995). However, these observations do not directly or explicitly support the present hypothesis. This is partly because such spatial patterns can be observed in the case of spatial heterogeneity of resources. In addition, such spatial patterns can also be observed if the intensity of the interspecific competition for resources is very low, with a resultant absence of interspecific territory. In this case, niche differentiation can fully explain the coexistence without the effect postulated by the present hypothesis.

How to test the hypothesis

The above studies certainly indicate that our hypothesis can be applied to certain species. However, it is still rather difficult to conclude that there are enough studies that support this hypothesis. Therefore, we herein propose a method to test this hypothesis in the future.

In the present hypothesis, an important phenomenon facilitating the coexistence of competing species is that males tend not to have territories near those of conspecific males, thus creating an opportunity for competing species to have territories and reproduce. We need to prove that this phenomenon is not caused by environmental heterogeneity but by interactions between individuals. This can be tested by the following experiments. Individuals of two territorial species whose niches almost completely overlap (e.g. sibling species in cichlid fish) are released into an enclosed field area or a large laboratory enclosure without environmental heterogeneity. Subsequently, if the results show that territories of the same species are separated to a greater extent than those of different species and that individuals of one species have their territories in the spatial gaps between heterospecific territories, the hypothesis would be supported.

Coexistence immediately after speciation

The mechanisms underlying the present hypothesis for promotion of coexistence can explain the coexistence of competing sibling species between which reproductive isolation has evolved. Recent theoretical studies have suggested that sexual selection can cause the evolution of premating isolation in sympatry without niche differentiation (Higashi et al. 1999; Kawata and Yoshimura 2000). Furthermore, an empirical study (Seehausen and van Alphen 1999) has provided compelling evidence for rapid sympatric speciation through disruptive sexual selection as in the case of cichlid species. In such speciation, after reproductive isolation, two reproductively isolated populations may not be able to coexist unless they diverge rapidly in resource or habitat use (Kawata and Yoshimura 2000). However, if male–male repulsion facilitates coexistence of the two species with little or no ecological differentiation, they will have sufficient time to diverge in ecological niches.

This fact suggests that the present hypothesis may be able to explain the coexistence of cichlid species. Although the cichlids have undergone explosive speciation in several lakes (Galis and Metz 1998; Turner et al. 2001), the coexistence of numerous cichlid fish species is yet to be explained (Genner et al. 1999a). Genner et al. (1999b) proposed that to explain such extensive coexistence, alternatives to niche differentiation should be considered, because some cichlid species seem to coexist without full niche differentiation. Males of most cichlid species hold territories to ensure mating opportunities and paternity (Barlow 2000). The rapidity of evolution through premating isolation without niche differentiation might be crucially involved in promoting a rapid increase in species diversity in the African lakes. Thus, the present hypothesis on the promotion of coexistence should be considered as an additional possible explanation for the coexistence of sibling species in African lakes.

Conclusion

Until now the effect of behavioral interaction between individuals on species coexistence has not been focused upon by ecologists. However, as indicated by Orians (2000), the influences of behavior on the structure of an ecological community may play an important role in ecology and ethology. We believe that local behavioral interactions between individuals could be important mechanisms for explaining species coexistence and diversity.