Introduction

In a forest community, most coexisting species persist with small populations, whereas only a few species have large populations (McGill et al. 2007). Exploring the causes of this pattern is vital to understanding community assembly. Asymmetry in conspecific negative density dependence (CNDD) is a possible mechanism for abundance differentiation. Species suffering stronger CNDD tend to decrease in abundance in a community (Comita et al. 2010; Johnson et al. 2012), although the strength of the relationships between CNDD and abundance can change with the latitude in which the communities are located (LaManna et al. 2017). Detto et al. (2019) further pointed out that it could be a spurious trend because the density measurements were error-prone proxies (EPP) which would result in disproportionate underestimation of the strength of CNDD for the more-abundant species than the less-abundant ones. Contradictory evidence also exists which indicates that less-abundant species suffer weaker CNDD (Zhu et al. 2015a). In this case, asymmetric CNDD enables the less-abundant species to persist in the community.

Differences in the ability of species to live in different environments could also shape the local abundance of those species. A species which exploits a narrower range of conditions (i.e., which has stronger habitat preference) should occupy fewer localities and be more rare (Gaston et al. 2000). This leads to a pattern of negative correlation between the strength of habitat preference and abundance. Empirical evidence of such pattern has emerged from forest ecosystems (Kolb et al. 2006). However, these studies mainly focused on the relationship between the abundance of a single species across a series of communities and the suitability of that species to the environments of those different communities, rather than on the relationship between the abundance of different species and their habitat suitability in the same community.

The mechanisms that shape relative species abundance favor dominant and common species, and disadvantage the less-abundant ones; thus, how can less-abundant species persist in a community? Is there a mechanism which shapes relative species abundance and simultaneously promotes coexistence? For example, CNDD, on the one hand, can shape relative abundance if it is disproportionately stronger for some species and restricts their populations (Comita et al. 2010). On the other hand, CNDD has long been regarded as an important coexistence mechanism (Janzen 1970; Connell 1971). This appears to be a dilemma. One explanation is that low abundance decreases the frequency of conspecific interactions sufficiently to outweigh the disadvantages of strong CNDD (Chisholm and Muller-Landau 2011; Fricke and Wright 2017). In this case, although the less-abundant species suffer stronger CNDD compared with the more-abundant ones, their populations can still perform better, which will lead to community compensatory trends (CCTs). In addition, stronger CNDD for the less-abundant species could facilitate coexistence through stronger stabilizing effects when their populations fall below their equilibrium abundances (Yenni et al. 2012). In this circumstance, asymmetric CNDD not only shapes relative abundance, but also preserves the less-abundant species and promotes coexistence.

It is also possible that the mechanisms which enable less-abundant species to coexist with more-abundant ones differ from those which shape their abundances. In addition to CNDD, heterospecific facilitation is another possible driving force for coexistence (Brooker et al. 2008). Heterospecific plant species can facilitate resource uptake (Richards and Caldwell 1987) and promote environment amelioration (Bertness and Callaway 1994) through underground mycorrhiza fungi networks (van der Heijden et al. 2015), or by reducing the prevalence of natural enemies (Wills 1996). They can also facilitate seed dispersal of co-fruiting-plant neighbors through shared frugivore seed dispersers (Carlo 2005). Facilitation can be detected when it overwhelms the negative effects of interspecific competition and damage by generalized natural enemies (Peters 2003). With stronger community-wide heterospecific facilitation, individuals of the less-abundant species will obtain more benefits compared with those of the more-abundant ones because the former tend to be surrounded by more heterospecific neighbors, which will help them to recover from rarity. Meanwhile, heterospecific neighbors usually comprise multiple species, and they might have different and even contradictory effects on focal individuals. Closely related species might exert negative effects because of severe niche overlap and resource competition (Burns and Strauss 2011), as well as a greater possibility of sharing natural enemies (Gilbert and Webb 2007; Liu et al. 2012), whereas distantly related species might have positive effects if they are able to obstruct the spread of species-specific pathogens of the focal species (Wills 1996). These phenomena indicate that the effects of heterospecific neighbors should be positively correlated to the phylogenetic distance to the focal individuals; thus, it is reasonable to consider the phylogenetic relativeness of heterospecifics when testing heterospecific neighbor effects (Webb et al. 2006; Zhu et al. 2015b; Comita et al. 2018).

The mechanisms promoting coexistence and shaping relative abundance might operate differently at different life-history stages. In earlier life-history stages, resource competition among understory seedlings is supposed to be negligible (Svenning et al. 2008), and CNDD is hypothesized to be the strongest effect (Harms et al. 2000; Zhu et al. 2015b) because of the prevalence of host-specific natural enemies (Janzen 1970; Connell 1971). Meanwhile, heterospecific facilitation will emerge as a result of herd protection effects (Wills 1996; Wills et al. 1997; Peters 2003). During the later life-history stages, self-thinning begins to take effect as trees grow. Neighbor effects will then be dampened or even reverse direction (Zhu et al. 2015b). Meanwhile, the intensity of competition for resources increases, such that niche partitioning will contribute more to community assembly. Although coexistence mechanisms have been tested across different life-history stages (Zhu et al. 2015b, 2018), tests for mechanisms shaping abundance have so far been restricted to early life-history stages (Comita et al. 2010; Johnson et al. 2012; Zhu et al. 2015a).

Thus, in the current study, we conducted a 10-year field survey to collect data from three 1-ha permanent plots in a subtropical forest to investigate whether asymmetry in CNDD, heterospecific facilitation and/or habitat preference affects relative species abundance. We also attempted to detect the mechanisms that promote coexistence, in particular, whether there are CCTs induced by CNDD, heterospecific facilitation, or phylogenetic neighbor effects. We tested these mechanisms across different plant life-history stages to see whether different mechanisms operate at different life-history stages.

Materials and methods

Study site and data collection

We conducted our field work in Heishiding Nature Reserve (Guangdong Province, China; 111°53′ E, 23°27 ′N, 150–927 m above sea level). The reserve comprises ~ 4200 ha of subtropical evergreen broad-leaved forests with the Tropic of Cancer running through its center. Most of the core area in the reserve was well protected and had no disturbance for more than one hundred years (Yu et al. 2000). The study site is located in a moist monsoon climate zone. Mean annual temperature is 19.6 °C, and mean monthly temperatures range from 10.6 °C in January to 28.4 °C in July. The average annual precipitation is ~ 1743.8 mm, with a humid season from April to September and a dry season from October to March. Dominant species of the forest are mainly from the Fagaceae and Lauraceae families.

During late 2007 to early 2008, we established three 1-ha permanent dynamic plots at average altitude of 300 m, two of which are adjacent (Liu et al. 2012). We tagged all free-standing woody individuals in the plots with diameter at breast height (DBH) ≥ 1 cm and identified them to species. We measured their DBHs and mapped their locations. In 2017, we repeated the census and recorded the survival status of previously tagged trees. In spring 2008, we also established 600 1 × 1-m2 seedling quadrats, which were spaced evenly within each plot, and all seedlings of woody plants (DBH < 1 cm) were surveyed (see Fig. S1 for details of seedling quadrat network design). We repeated the annual census in every spring of the following years until 2017.

In 2017, we measured the elevation of every seedling quadrat using a handheld GPS altitude meter (Kestrel 4000; Nielsen-Kellerman, Boothwyn, PA). We collected at least one soil sample at a depth of 5 cm from each of the 50 sites around the seed traps in each 1-ha plot. Twenty percent of the sites were randomly chosen to catch directional variations in soil properties at fine scales, where soil samples were collected at distance of 0.1, 0.3, 0.8, 2, and 8 m from each chosen location in a random direction (see Fig. S1 for details). We collected 300 soil samples in total. The samples were air-dried and the total N, P, and K were measured. We interpolated the soil nutrient content and elevation data to a 10 × 10 m2 resolution using the universal kriging method with a spherical variogram model (Wackernagel 2003). We then calculated the slopes and aspects of each 10 × 10 m2 quadrat based on the interpolated elevation data.

We constructed a metaphylogenetic tree with all the species recorded in the censuses (149 species from 89 genera and 46 families). The phylogenetic structure at the family level and divergent age data were acquired from Gastauer and Meira Neto (2017). We then used the Phylomatic program (Webb and Donoghue 2005) and the BLADJ algorithm of the Phylocom version 4.2 software package (Webb et al. 2008) to obtain an ultrametric tree at the species level with branch lengths scaled to divergence time.

Focal individuals and life-history stages

Newly germinated seedlings were considered to represent the seedling stage. All tagged trees in the first census were classified into three life-history stages on the basis of their DBH. Trees with 1 cm ≤ DBH < 5 cm, 5 cm ≤ DBH < 10 cm, and DBH ≥ 10 cm were regarded as saplings, juveniles, and adults, respectively (Peters 2003). The adults might have lived for so long before the census and what they had experienced in that period of time might exert continuing and accumulative effects on their survival. The predictors, such as neighbor densities, based on the recent 10-year census might not reflect the life-history differences of the adults. Hence, we did not include adults as focal individuals in our analysis.

Given that direct neighborhood interactions in tree communities are commonly found to occur within a radial distance of 20–30 m (Peters 2003; Comita et al. 2010), we included only woody plants located > 20 m to each side of each plot as focal individuals in the analysis (see Fig. S1 for details). We excluded shrubs from focal individuals but included them as heterospecific neighbors. We also excluded seedlings in the first and last census from being focal ones because either their ages or survival statuses were unknown.

We used the 2007 census data of plants with DBH ≥ 1 cm to calculate the densities and phylogenetic index (see below) of neighboring trees for the seedlings that germinated in 2009–2012, and the 2017 census data for those that germinated in 2013–2016.

In total, we included 96 focal species from 40 families in the analysis. We collected 6 life-history traits to see whether they were correlated with species abundance. Specifically, we collected leaf area, specific leaf area, leaf dry-matter content, wood density and wood dry-matter content of 67 species (He and Biswas 2019). We also collected data of dispersal mode of all 96 species according to Flora Reipublicae Popularis Sinicae (https://www.iplant.cn/) and the personal observations of our fieldwork staffs. We classified the plant species into three dispersal modes: gravity (i.e. unassisted) dispersal, animal dispersal, and wind dispersal (anemochory) according to Russo et al. (2007). We provided the trait data in Appendix I.

Data analysis

We used two metrics of species abundance: size-weighted abundance and numerical abundance. Size-weighted abundance of a given species was calculated as the sum of the basal area (BA) of individuals with DBH ≥ 1 cm within all three 1-ha plots in the 2007 census, while numerical abundance was the total number of conspecific individuals of the given species.

Neighboring seedlings with DBH < 1 cm in the seedling quadrats were assumed to only affect focal seedlings but not saplings or juveniles. Therefore, focal seedlings had both seedling neighbors in the corresponding seedling quadrats and neighboring trees (DBH ≥ 1 cm) within a 20-m radius, whereas focal saplings and juveniles only had neighboring trees (DBH ≥ 1 cm) in the analysis. To calculate the conspecific and heterospecific densities of the neighboring trees, we summed their inverse distance-weighted basal areas according to Canham et al. (2004). We calculated the conspecific and heterospecific densities of the neighboring seedlings as the numbers of living individuals within the same seedling quadrats. Since the measurements of densities could be error-prone proxies (EPP) as the true density that can reflect some important ecological processes may not be simply proportional to the sum of distance-weighted basal area (Detto et al 2019), we introduced a parameter c as the exponent of density measurements. To find the proper c for each life-history stage, we used logistic regression to fit the survival data and calculate the log-likelihoods when c was allotted with different values from 0.01 to 2. We selected the c value when the corresponding log-likelihoods reached maximum. For the density of neighbors with DBH ≥ 1 cm, we assigned c = 1.30, 1.20, and 0.23 at the juvenile, sapling and seedling stages, respectively. For the density of small neighbors (DBH < 1 cm) of focal seedlings, we selected c = 0.04 (Fig. S2; see Supplementary Method 1 for detail).

To test the effects of neighbor phylogenetic structures on the survival of focal individuals, we calculated the standardized average phylogenetic distance (APd’) of heterospecific neighbors to the corresponding focal individuals (Webb et al. 2006). Specifically, we first calculated the mean observed phylogenetic distance between the focal individual and all other species within the 20-m radius (or within the same seedling quadrat, for seedling neighbors). We then used a null model to generate 1,000 random communities for a given species richness, and calculated the mean and standard deviation of expected phylogenetic distances to each focal individual to correct for the effect of sample species richness. APd’ was then calculated as the difference between the observed mean phylogenetic distance and the mean of the null model, divided by the standard deviation of the null model (Webb et al. 2006; Zhu et al. 2015b). For the focal seedlings that had no heterospecific neighbors, we allotted 0 as their APd’.

To assess the susceptibility to environmental factors, we first estimated the habitat center of the population of a given species. We then calculated the distance of focal individuals to the estimated population habitat center. We used soil nitrogen (N), phosphorus (P), and potassium (K), elevation, slope and cosine of aspects as axes of the niche space, given that soil N, P, and K are the most important nutrients for plants, whereas elevation, slope, and cosine of aspects can reflect variations in soil water content, and thermal and light conditions. These abiotic environmental factors had considerable variation within the plots (Fig. S3). We used total N and P rather than available N and P because evidence has emerged that most tree species can associate with mycorrhizal fungi which can help them to exploit complex organic forms of N and P (George et al. 1995; Makarov 2019). As K was quite labile, we used total K as an estimate of what a plant can uptake.

The abiotic niche of a species was assumed to be an approximate Gauss hyper-volume (Pocheville 2015), which gave a Gaussian approximation for each axis (an abiotic resource or topographic variable). The standardized Euclidean distance to the habitat center was regarded as the degree of departure from the favorite habitat. For convenience, the process started with a 1-dimension resource space r, for species s. First, we estimated the habitat center of a given species as the basal-area-weighted mean value of the resource at the locations where the individuals of the species colonized

$$M_{{sr}} = \frac{{\sum\nolimits_{i} {\left( {{\text{BA}}_{i} \cdot r_{i} } \right)} }}{{\sum\nolimits_{i} {{\text{BA}}_{i} } }}.$$

We also calculated the BA-weighted deviation in r of the species as

$$D_{s r} = \sqrt {\frac{{\mathop \sum \nolimits_{i} \left[ {{\text{BA}}_{i} \cdot\left( {r_{i} - M_{s r} } \right)^{2} } \right]}}{{\mathop \sum \nolimits_{i} {\text{BA}}_{i} }}}$$

Then, we calculated the habitat departure of an individual of the species as:

$$CV_{{s_{i} r_{i} }}^{2} = \frac{{(r_{i} - M_{sr} )}}{{D^{2}_{sr} }}.$$

where CVsi ri represented the departure of the individual i of species s located at ri from the population habitat center Ms r.

Finally, when extending to the multidimensional environmental space R, habitat departure of the ith individual of species s could be regarded as the Euclidean distance to the habitat center:

$$CV_{{s_{i } R_{i} }}^{2} = \sum\nolimits_{r}^{R} C V_{{s_{i} r_{i} }}^{2} = \sum\nolimits_{r}^{R} {\frac{{\left( {r_{i} - M_{s r} } \right)^{2} }}{{D_{s r}^{2} }}} .$$
$$CV_{{s_{i} R_{i} }} = \sqrt {\sum\nolimits_{r}^{R} {\frac{{\left( {r_{i} - M_{s r} } \right)^{2} }}{{D_{s r}^{2} }}} } .$$

CV of an individual was regarded as its departure from the habitat center of the given species, and its coefficient in the regression (see models below) was referred to as habitat preference of the species. A greater absolute negative value of the coefficient indicated that the species had stronger preference for the environment of its habitat center and that it was more susceptible to heterogeneous environments. We used individuals with DBH ≥ 1 cm in the 2007 census to calculate CV of each focal species. We excluded the species whose population sizes were less than 5 individuals in the 2007 census from the subsequent analyses to avoid large bias in the measure of CV. Note that the estimated habitat center is based on the sample of the population of a given species in the study area. It is just an approximate of the habitat center of the particular population rather than a species. We think it is reasonable to estimate habitat preference based on the population-level habitat center because we focus on the mechanisms of community assembly at a local scale where the participants are particular populations of different species. We also realize that the population of a given species may not occupy the most suitable environments in the community; thus, there is still bias between the estimated population habitat center and the true one. This will in turn induce bias in the measure of habitat departure (standardized distance to the habitat center). Based on the theory of regression dilution (Detto et al. 2019), we will underestimate habitat preference of the population of a given species.

We used hierarchical Bayesian models that allowed for variation among species to analyze how species abundance was affected by conspecific and heterospecific density dependence, phylogenetic effects, and habitat preference for individuals of the three different life-history stages. This set of models included both individual- and species-level regressions. In the individual-level regression, survival (p) of an individual seedling i, of species s, in quadrat q, of census year y (for seedlings only) was modeled as a function of conspecific (Con) and heterospecific densities (Het) and APd’ (APd) of large neighbors with DBH ≥ 1 cm (T) and seedling neighbors (S; for focal seedlings only), departure (Hab_d) from the habitat center, and initial basal area (BA; not for seedlings):

$${\text{surv}}_{isqy} = {\text{Bernoulli}} \left( { p_{sqy} } \right){ },$$
$$\begin{aligned} {\text{logit}} \left( {p_{sqy} } \right) & = \beta_{0s} + \beta_{1s} \cdot{\text{ConT}}_{sqy} \beta_{2s} \cdot{\text{HetT}}_{sqy} \beta_{3s} \cdot{\text{APdT}}_{sqy} \\ + \beta_{4s} \cdot{\text{ConS}}_{sqy} + \beta_{5s} \cdot{\text{HetS}}_{sqy} + \beta_{6s} \cdot{\text{APdS}}_{sqy} \\ \beta_{7s} \cdot{\text{Hab}}\_{\text{d}}_{sq} + \beta_{8s} \cdot{\text{BA}}_{s} + \varphi_{q} + \varphi_{y} , \\ \end{aligned}$$

which included random effects for quadrats φq and census period φy (for seedlings only) to control for spatial and temporal autocorrelation in the survival of seedlings within the 1-m2 quadrats, or that of saplings and juveniles within 100-m2 quadrats.

In the species-level regression, the coefficients (β0–7) of each species s were modeled as functions of the species log transformed abundance (Abund) in the community

$$\beta_{ms} \sim N(\gamma_{m0} + \gamma_{m1} \cdot{\text{Abund}}_{s} , \sigma_{m}^{2} ),$$

where all individual- and species-level coefficients were assigned weakly informative priors. Specifically, we used Cauchy (0, 5) as the priors of the regression coefficients and scale parameters. The priors of the scale parameters were truncated to half-Cauchy distributions implicitly because the scale parameters were declared to be positive (see Supplementary Code 1 for details). All explanatory variables were standardized with their means and standard deviations before entering the models.

To investigate the community-wide average effects, we simplified the species-level regression by reducing the term γm1 × Abunds.

To test CCTs in the survival of different life-history stages, we simplified the individual-level model only to retain the intercept β0s. Correspondingly, only the intercept (β0s) of each species s was modeled as a function of the species log transformed abundance in the species-level regression.

We performed Bayesian inferences with Stan (Stan Development Team 2019) in R version 3.6.1 (R Core Team 2019). We ran two independent chains with different initial values, and used the Gelman–Rubin statistics to assess convergence (Brooks and Gelman 1998). All the models were run for 40,000 iterations, within which there were 20,000 warm-ups. Convergence was ensured and no transitions existed after warming up in any of the Bayesian sampling chains. We used the R function hdi of package HDInterval (Meredith and Kruschke 2020) to calculate the highest posterior density intervals (HPDI) of the parameters. An effect was regarded to get strong evidence if its 95% HPDI did not encompass 0. It was regarded to get weak evidence if 95% HPDI encompassed 0 but 90% HPDI did not encompass 0.

Results

The relationship between species abundance and neighbor effects / habitat preference

In the study plots, the top 10% species in size-weighted abundance rank held 74.6% of the total basal area and top 10% species in numerical abundance rank held 61.4% of the total individuals, which demonstrated that only a few species had very high abundance while most of the others had very low abundance (Fig. S4). Most of the tested life-history traits, such as leaf area, specific leaf area, leaf dry-matter content, wood density or wood dry-matter content, had no strong correlation with size-weighted or numerical abundance (Table S1; see Supplementary Method 2 for details) except dispersal modes. Wind-dispersed species tended to occupy higher ranks in size-weighted abundance than gravity-dispersed and animal-dispersed species (Fig. S5).

We did not find strong evidence for correlations between conspecific density effects and size-weighted abundance across the seedling [mean = − 0.526 and 95% HPDI (− 1.235, 0.177)], sapling [mean = − 0.122 and 95% HPDI (− 0.39, 0.156)], or juvenile stage [mean = − 0.385 and 95% HPDI (− 0.867, 0.093); Table 1]. We did not find evidence for the correlation between size-weighted abundance and the effects of heterospecific density or their phylogenetic structure, APd’ (Table 1). We observed similar results when numerical abundance was used in the analysis (Table S2).

Table 1 Results from two-level Bayesian analysis examining the relationship between survival at different life stages, species size-weighted abundance and the effects of conspecific and heterospecific neighbor density, neighbor phylogenetic structure, and habitat
Full size table

We found a consistently positive relationship between habitat preference and size-weighted abundance across all of the three life-history stages (Table 1) although it only got weak evidence at the juvenile stage [mean = 0.189, 95% HPDI (− 0.030, 0.415), 90% HPDI (0.005, 0.372); Table 1 and Fig. 1]. In contrast to more-abundant species, less-abundant ones had higher mortality risks when they inhabited environments different from their corresponding habitat centers. When we used numerical abundance, a weak evidence for a positive relationship between habitat preference and species abundance emerged at the seedling stage [mean = 0.274, 95% HPDI (− 0.008, 0.561), 90% HPDI (0.031, 0.506)] but not across the other life-history stages (Table S2).

Fig. 1
figure 1

The relationship between habitat preference at the juvenile stage and size-weighted species abundances in the Heishiding plot. Solid points and grey bars represent species-level means and standard deviations, respectively. The overall positive relationship fitted by the hierarchical Bayesian model got weak evidence [the solid-grey line; mean = 0.189, 95% HPDI (− 0.030, 0.415), 90% HPDI (0.005, 0.372)]

Full size image

Phylogenetic neighbor effects across life-history stages

Large conspecific neighbors (DBH ≥ 1 cm) exerted negative effects on survival consistently during the seedling, sapling, and juvenile stages although the effects were not strongly supported as their 90% HPDIs encompassed 0 (Fig. 2a and Table S3). Specifically, the effect was − 0.150 [95% HPDI (− 0.409, 0.106)], − 0.056 [95% HPDI (− 0.248, 0.134)], and − 0.346 [95% HPDI (− 0.966, 0.285)] on survival of seedlings, saplings and juveniles, respectively. Conspecific seedling neighbors had a negative effect on focal seedling survival [posterior mean = − 0.180, 95% HPDI (− 0.343, − 0.026); Fig. 2d and Table S3]. It meant seedling survival chance would decrease by 4.3% from 60.6% to 56.3% on average with one standard deviation increase in seedling neighbor density.

Fig. 2
figure 2

Community-level effects on individual survival across different life-history stages. The effects of (a) conspecific (Con) and (b) heterospecific density (Het) of neighboring trees with DBH ≥ 1 cm (T), and (c) their average phylogenetic diversity (APd) were acquired from the posterior samples of Bayesian models for the juvenile, sapling, and seedling stages. d Conspecific and heterospecific density, as well as APd’ of seedling neighbors (S) in the 1-m2 seedling quadrats were thought to only affect focal seedlings. The coefficients of habitat departure were (e) habitat preference. We took account of (f) the initial basal area effects of focal saplings and juveniles. Estimated coefficients are shown in the map with posterior means (points), and 90% (dark-grey bars) and 95% (light-grey bars) highest posterior density intervals (HPDI). Solid-black points indicate that 95% HPDIs did not encompass 0 (strong evidence), whereas solid-grey points indicate that 90% HPDIs did not encompass 0 but that 95% HPDIs did (weak evidence). Unfilled points indicate that 90% HPDIs encompassed 0 (no evidence)

Full size image

Heterospecific neighbors (DBH ≥ 1 cm) had positive effects on individual survival across all of the tested life-history stages (Fig. 2b and Table S3). Specifically, the effect was 0.162 [95% HPDI (− 0.009, 0.329), 90% HPDI (0.023, 0.304)] at the seedling stage, 0.279 (95% HPDI [0.155, 0.404]) at the sapling stage, and 0.286 [95% HPDI (0.040, 0.544)] at the juvenile stage. We found no evidence for the effect of standardized average phylogenetic distance (APd’) of these heterospecific neighbors on survival of focal individuals across any of the tested life-history stages (Fig. 2c and Table S3). We did not find evidence for the effects of heterospecific seedling neighbors or their APd’ on seedling survival (Fig. 2d and Table S3).

Habitat preference across life-history stages

In the study community, the estimated habitat centers of different species differed (Fig. S6). We did not find evidence for habitat preference at the seedling stage [posterior mean = − 0.114 and 95% HPDI (− 0.361, 0.154); Fig. 2e and Table S3]. In contrast, strong evidence for habitat preference emerged during the sapling and juvenile stages (Fig. 2e and Table S3). Specifically, habitat preference was − 0.257 [95% HPDI (− 0.421, − 0.101)] at the sapling stage, indicating sapling survival chance decreased by 6.2% on average from 61.4 to 55.2% with one standard deviation increase of habitat departure. It was − 0.314 [95% HPDI (− 0.533, − 0.094)] at the juvenile stage, indicating juvenile survival chance decreased by 6.6% on average from 73.0 to 66.4% with one standard deviation increase of habitat departure.

Community compensatory trends across life-history stages

We found a CCT in survival during the seedling stage, when seedling survival odds were negatively correlated with size-weighted abundance [mean = − 0.396, 95% HPDI (− 0.650, − 0.141); Fig. 3d, Table S4], but we did not find evidence for such trends during the sapling or juvenile stage (Fig. 3a, b, c and Table S4). When we used numerical abundance for the analysis, we did not find evidence for CCT at any of the tested life-history stages (Fig. S7 and Table S4).

Fig. 3
figure 3

The relationships between size-weighted abundances and species log survival odds at a juvenile, b sapling, or c seedling life-history stage. Lines fitted by the hierarchical Bayesian models represent the relationship with strong evidence to be different from 0 (the 95% HPDIs do not encompass 0; solid) or with no evidence (the 90% HPDIs encompass 0; dashed). The vertical grey lines show ± 1 SD of the posteriors of species-specific log survival odds

Full size image

Discussion

The current study revealed that the mechanisms shaping species abundances and those promoting coexistence operate differently at different life-history stages in the subtropical forest. Less-abundant species were rare because they had stronger habitat preference (i.e., narrower niches) as compared to the more-abundant ones especially at the juvenile stage, not because they suffered stronger CNDD at any of the tested life-history stages. The relationship between habitat preference and size-weighted abundance across different life-history stages was consistent, although it was only evident during the juvenile stage. By contrast, CNDD was uncorrelated to the abundance of seedlings, saplings or juveniles. The CCT in seedling survival, which was probably induced by CNDD of seedling neighbors and facilitation of heterospecific neighbors with DBH ≥ 1 cm, enabled less-abundant species to recover from rarity and promoted coexistence. At the sapling or juvenile stage, there was no strong evidence for community-level CNDD. Habitat preference and heterospecific facilitation, which could promote species coexistence, were detected at these life-history stages, but they did not result in CCTs.

Factors shaping plant species' abundance

Our results supported the hypothesis that differences in habitat preference shaped species' relative abundance. Species that had stronger habitat preference at the juvenile stage tended to be less abundant when size-weighted abundance was used as the metric of abundance (Fig. 2 and Table 1), and this pattern emerged at the seedling stage if numerical abundance was applied (Table S2). It is important to point out the bias in the estimation of habitat preference of species. For instance, a species with low abundance had small population size, and the sample for estimation of its habitat center was small. Thus, its estimated habitat center, compared to the more-abundant species, was more likely to have larger bias from the “true” habitat center, which then would result in larger bias of CV measurements. Based on the theory of regression dilution (Detto et al. 2019), larger bias of CV would result in the more severe underestimation of the strength of habitat preference for the less-abundant species, therefore producing a spurious negative relationship between species abundance and habitat preference. We used 1000 sets of simulated data and confirmed a weak tendency in detection of the spurious negative relationship when there were actually none (Supplementary Method 3 and Code 2). But in the analysis of the real forest data, we detected a positive rather than negative relationship between habitat preference and species abundance at the juvenile stage (Fig. 1). We believe this is a robust result that is real and not caused by differences in sample sizes among plant species.

Strong habitat preference implies narrow ecological niche of the population of a given species. The positive relationship between habitat preference and species abundance indicates that species with broader niche have advantage over those with narrower niche, and consequently become more abundant in the community. Previous research has found that species occupying the most sites also had the highest average abundance within those sites at a large geographical scale (Gaston et al. 2000). At a local community level, a similar pattern was found in a nitrogen-limited Arctic tundra, where the most productive species utilized the most available chemical forms of nitrogen (McKane et al. 2002). However, such evidence has been lacking in forests. To the best of our knowledge, the results of the current study are the first to demonstrate how asymmetric habitat preference shapes species abundances in diverse forest communities.

We did not find evidence to support the hypothesis that asymmetric CNDD shapes relative species abundance. Previous research found that less-abundant species suffer stronger CNDD other forests (Comita et al. 2010; Johnson et al. 2012). But a recent review pointed out that it could be a spurious relationship due to disproportionate underestimation of CNDD for the more-abundant species (Detto et al. 2019). According to Detto et al. (2019), we selected a proper exponent parameter c of the measured density for each of the three life-history stages (see Supplementary Method 1) to avoid the “spurious” trend. There were also some studies which reported negative relationship between CNDD and abundance (Zhu et al. 2015a). In this circumstance, asymmetric CNDD can benefit less-abundant species by suppressing the recruitment of abundant species disproportionately, promoting species coexistence.

Interestingly, although there was no evidence for the positive relationship between CNDD and species abundance, we did found strong support for CNDD of seedling neighbors at the seedling stage. We also found wind-dispersed species tended to be more-abundant than gravity-dispersed and animal-dispersed species (Fig. S5). A possible explanation is that compared to gravity-dispersed species, wind-dispersed species might have more dispersal advantages, which would result in lower local densities of recruitments and longer average distance to the parent trees, and this could help the seedlings of wind-dispersed species escape from strong CNDD (Howe and Miriti 2004). Seeds dispersed by animals are probably deposited very patchily driven by animal behavior, and seed cluster increases local density, which causes higher seedling mortality due to CNDD (reviewed by Schupp et al. 2002).

Species coexistence mechanisms

CNDD is widely known as an important coexistence mechanism, which can help a species recover from rarity (Johnson et al. 2012; Zhu et al. 2015a). CNDD of seedling neighbors was detected at the seedling stage. This is consistent with most empirical studies in forest communities (e.g., Chen et al. 2010; Zhu et al. 2015b, 2018). The large conspecific neighbors (DBH > 1 cm) exerted overall negative but slight effects on survival at the seedling, sapling and juvenile stages (Fig. 2a and Table S3), but the estimation of the effects had large uncertainty as their 95% HPDIs encompassed 0. Based on the posterior mean values of the species-level conspecific neighbor effects, we found most of the species tended to suffer CNDD, especially at the juvenile and seedling stages (Fig. S8 e and m). A few previous studies also found CNDD across these life-history stages (Zhu et al. 2015b and 2018). It implied that CNDD might have continuing effects through seedling to juvenile stages.

Heterospecific facilitation is also regarded as an important coexistence mechanism to protect rare species from local extinction through a frequency-dependent process (Wills 1996). The current study found pervasive heterospecific facilitation across the three tested life-history stages, which is consistent with the results from other studies focusing on tree species of different size classes (Wills et al. 1997; Comita et al. 2010; Zhu et al. 2018). We did not find evidence for the effects of the heterospecific phylogenetic structure (APd’) across any of the tested life-history stages. Based on the finding that closely related species tend to share more natural enemies (Gilbert and Webb 2007; Liu et al. 2012) and suffer stronger competition due to similar functional traits and resource demands (Cahill et al. 2008), positive phylogenetic effects had been expected. However, functional traits and fundamental niches are not always phylogenetically conserved (Cadotte et al. 2017). Moreover, even if the functional traits are phylogenetically conserved but they are concerned with competitive ability rather than resource partitioning, habitat filtering will prefer closely related competitors (Mayfield and Levine 2010), thus resulting in a negative correlation between phylodiversity and survival. These opposite processes might operate simultaneously in the study forest, resulting in no correlation between phylodiversity and survival. In addition, our phylogenetic tree, which was built based on APG IV, was not well-resolved below the family level. Thus, the coarse measure of relatedness could hinder the detection of phylogenetic effects.

Niche partitioning is a classical ecological process determining the coexistence of competing plants (McKane et al. 2002; Silvertown 2004). In the current study forest, saplings and juveniles showed strong habitat preference (Fig. 2e). They had higher survival odds when inhabiting abiotic environments that were similar to their habitat centers. Meanwhile, different species showed considerable variation in habitat centers (Fig. S6). However, in the current study, there is no strong evidence for habitat preference at the seedling stage. One possible reason is that differences exist between the abiotic niches of mature plants and the regeneration niches of seedlings (Grubb 1977). In the current study, the habitat centers of different species were calculated based on the spatial distribution of individuals of larger size classes (DBH ≥ 1 cm), which might not accurately reflect the resource demands of newly germinated seedlings.

Neighbor effects and CCT

If we used size-weighted abundance as the measure of species abundance, a CCT was detected at the seedling stage, indicating that less-abundant species have better opportunities for recruitments and, thus, can persist in the community. According to the original theory that a CCT would emerge if species shared a similar strength of CNDD but differed in their abundances (Connell et al. 1984), most CCT tests have detected CCTs and community-wide CNDD simultaneously (Queenborough et al. 2007; Chen et al. 2010; Yan et al. 2015). The current study found evidence not only for CNDD, but also for heterospecific facilitation. In addition, no evidence supported a correlation between species abundance and CNDD or heterospecific facilitation at the seedling stage (Table 1). Thus, we believe both CNDD and heterospecific facilitation contribute to the CCT in seedling survival. But if we used numerical abundance as the measure of abundance, CCT in seedling survival was not detected. A previous research also reported inconsistent relationship between seedling survival and abundance when basal area and individual number were used as different measures of species abundance, respectively (Chen et al. 2010). This inconsistency may result from the mismatch between the rank of a species in numerical abundance and the rank in size-weighted abundance.

Either numerical or size-weighted abundance was used in the analysis, there was no evidence for CCTs in the survival of saplings or juveniles (Fig. 3 and Fig. S7). Across these life-history stages, we detected community-level heterospecific facilitation and habitat preference (Fig. 2b, e). In theory, these processes should induce CCTs and promote coexistence. However, we did not detect CNDD across these life-history stages (Fig. 2a). Meanwhile, less-abundant species expressed disproportionately stronger habitat preference (Table 1) which implied they had narrower niches. These processes might eliminate the expectant compensatory trends.

Conclusion

Our research has broadened our understanding of community assembly in terms of the different mechanisms that shape relative species abundance and promote coexistence at different life-history stages in subtropical forests. The disproportionately stronger habitat preference of some species at the juvenile stage decreases their abundance and, thus, shapes the relative species abundance in this community. A CCT at the seedling stage, induced by local CNDD and heterospecific facilitation, enables less-abundant species to persist in the community, thus promoting coexistence. Given the complexity of natural forest communities, we suggest that further studies should take different processes and different life-history stages into consideration when investigating the mechanisms of community assembly.