Introduction

Understanding the mechanisms that yield biodiversity patterns is one of the most intriguing and longest-standing challenges in ecology (Whittaker 1960, 1972). Different factors can influence local diversity (α diversity) and its variation (β diversity) within regional species pools (ɣ diversity) at different scales of space and time (Fukami 2004). Investigations that simultaneously consider various components of diversity and analyze their variation in response to environmental factors can reveal processes shaping ecological communities (Leibold and Mikkelson 2002; Fukami 2004; Kraft et al. 2011; Mittelbach and Schemske 2015). An important environmental influence on species diversity is habitat structural complexity (MacArthur and MacArthur 1961; Tews et al. 2004; Kovalenko et al. 2012; Stein and Kreft 2015). Habitat structural complexity is commonly characterized as the density or diversity of physical structures that influence habitat suitability for organisms (MacArthur and MacArthur 1961; McCoy and Bell 1991; Tokeshi and Arakaki 2012). Habitat structural complexity can enhance fitness by increasing availability of food resources and/or refuges from predation (Gause 1936; Huffaker 1958; MacArthur and MacArthur 1961; Sánchez-Botero et al. 2007), which should result in a positive relationship between α diversity and habitat structural complexity. In some habitats, however, greater structural complexity may reduce useable space for large organisms (Gibb and Parr 2010; Strayer and Findlay 2010; Yeager and Hovel 2017). In these cases, α diversity should reveal a negative relationship with habitat structural complexity, or a hump-shaped pattern that derives from the influence of two opposing mechanisms (Crowder and Cooper 1979; Heck and Orth 1980; Diehl and Kornijow 1998; Tokeshi and Arakaki 2012; Paxton et al. 2017).

Gradients of habitat structural complexity are common in many landscapes (August 1983; Fahr and Kalko 2011). A single lake, for example, can show patches with different levels of structural complexity that range from bare sediment to dense beds of submerged vegetation (St. Pierre and Kovalenko 2014). Because species respond differentially to levels of structural complexity, patterns of coexistence may be clumped with species turnover along complexity gradients (Leibold and Mikkelson 2002; Leibold et al. 2004; Strayer and Findlay 2010). Limiting factors at lowest and highest levels of habitat structural complexity, such as predation risk (low complexity) and lack of space for movement (high complexity), would further influence assemblage nestedness (Leibold and Mikkelson 2002; Leibold et al. 2004; Hylander et al. 2005).

Whereas variation in α diversity may reveal insights about mechanisms for coexistence of species at local scales, β diversity provides insight about community assembly in response to environmental gradients (Hylander et al. 2005; Petsch et al. 2017). Although these diversity components are complementary, most studies investigating changes in assemblage structure in relation to habitat structural complexity have only examined α diversity (e.g., Newman et al. 2015). Few studies have attempted to analyze variation in β diversity across gradients of structural complexity, and studies are needed that jointly investigate both diversity components (Petsch et al. 2017). Moreover, analysis of species turnover and nestedness can provide further insights about mechanisms of community assembly (Hylander et al. 2005; Fahr and Kalko 2011).

Here, we conducted a field survey to investigate the role of structural complexity as a function of vegetation density regulating small fish α and β diversity in lakes of a floodplain in Brazil. Several studies investigating the effects of habitat structural complexity on diversity have been performed in tropical aquatic ecosystems (Arrington et al. 2005; Willis et al. 2005; Dibble and Pelicice 2010). Fishes in general, and freshwater fishes in particular, respond strongly to habitat structural complexity (Strayer and Findlay 2010), with many species relying on submerged structures (e.g., aquatic vegetation) for feeding, refuge from predation, spawning or nesting (Rozas and Odum 1988; Rossier et al. 1996; Santos et al. 2011; Yeager and Hovel 2017). In the tropics, diverse fishes of small body size take refuge in beds of aquatic vegetation (Lopes et al. 2015). In tropical floodplains, the density of aquatic vegetation generally varies along water depth gradients, with higher plant density and smaller and more numerous interstices in shallow water near shore (Lopes et al. 2015). In addition, dissolved oxygen concentration often varies as a function of plant density and distance from shore, with dense patches in shallow water sometimes experiencing aquatic hypoxia during the night (Miranda and Hodges 2000; Miranda et al. 2000). Therefore, aquatic vegetation in floodplain habitats creates environmental gradients of both structural complexity and environmental stress.

We formulated a series of questions and conducted a deductive approach to test a hypothesis about the effects of habitat structural complexity regulating small fish α and β diversity. Our questions included: Do α and β diversity of fish assemblages differ from diversity derived from a random partitioning of regional diversity? If yes, how is habitat complexity related to species diversity components? What is the role of oxygen concentration in explaining the relationship between diversity components of fish assemblages and habitat structural complexity? Considering previous theory on habitat structural complexity, we hypothesized that α diversity is related to habitat structural complexity in a hump-shaped fashion, while habitat structural complexity leads to changes in β diversity considering both turnover and nestedness components. We further predicted that (1) α and β diversity patterns are different from those expected in random assemblages; (2) along a gradient of habitat structural complexity, α diversity increases until an optimum level but decreases after that; and (3) a gradient of habitat structural complexity leads to changes in β diversity accompanied by increases in species turnover and nestedness at lowest and highest levels of habitat structural complexity. We also investigated the potential influence of fish size and species tolerance to hypoxic condition on α and β diversity patterns in relation to gradients of structural complexity and dissolved oxygen concentration. Aspects of β diversity were investigated using the concepts extended to abundance data described by Podani et al. (2013), who refer to (1) β diversity as differences in abundance of particular species along ecological gradients (hereafter total β diversity), (2) turnover as the replacement of abundances of particular species with the same abundances of different species along ecological gradients, and (3) nestedness as reductions in the abundances of particular species that yields assemblage subsets along ecological gradients.

Materials and methods

Study area

Surveys were performed in natural lakes in the floodplain of the Baía River, a black-water tributary of the Upper Paraná River in Brazil. The Baía and Paraná rivers meander over broad alluvial plains within an area of 28,106 km2 (Stevaux 1994; Orfeo and Stevaux 2002; Agostinho et al. 2004c) and have seasonal patterns of discharge (Agostinho et al. 2000, 2004c). Geology of the region is dominated by sedimentary and volcanic rocks (Stevaux 1994; Orfeo and Stevaux 2002), and the floodplains contain numerous secondary channels, lakes, and wetlands (Stevaux 1994; Agostinho et al. 2004c; Stevaux and Souza 2004). Within the Baía floodplain, lakes and flooded areas are highly connected during the wet season. Most lakes are shallow [depth < 3 m (Stevaux and Souza 2004)] with extensive littoral zones containing extensive beds of aquatic vegetation. Dominant species of aquatic vegetation are Eichhornia azurea (Sw.) Kunth and Eichhornia crassipes (Mart.) Solms. These floating vegetation have similar morphology consisting of networks of submerged stems and roots, with emergent stems and leaves that increase habitat complexity for many small fish species (Delariva et al. 1994; Gomes et al. 2012).

The regional fish fauna has > 270 species (Agostinho et al. 2004a, 2007a), with the Baía River generally having greater fish densities and diversity compared to similar habitats in flowing reaches of the nearby Paraná River (Fernandes et al. 2009). Several piscivorous species are common, including Pseudoplatystoma corruscans (Spix and Agassiz 1829), Cichla kelberi Kullander and Ferreira, 2006, Hemisorubim platyrhynchos (Valenciennes 1840), Rhaphiodon vulpinus Spix and Agassiz 1829 and Astronotus crassipinnis (Heckel 1840) (Agostinho et al. 2004b, 2007a; Gomes et al. 2012). These relatively large species are highly mobile and move among habitats throughout the landscape. Many small species (≤ 4 cm total length; e.g., Hyphessobrycon, Moenkhausia and Serrapinus spp.) and juveniles of larger species are largely restricted to stands of submerged vegetation where they feed on aquatic invertebrates and benthic microalgae while also obtaining cover that reduces exposure to predators. Given their small size and vulnerability to piscivorous fishes, dispersal by these small fishes is limited compared to that by larger fishes (Delariva et al. 1994; Agostinho et al. 2007b; Gomes et al. 2012; Lopes et al. 2015).

Sampling procedure

In December 2011, field surveys were performed at 48 locations distributed in a nested design within four lakes in the floodplain of the Baía River. In each lake, we randomly selected two vegetation mats dominated by mixtures of E. azurea and E. crassipes, with the requirement that there was variation in the density of vegetation between and within stands. In general, the area of each vegetation stand was ≥ 400 m2, and stands were separated from each other by at least 100 m. We sampled six locations along 15-m transects within each stand. Each transect began at the stand margin with open water and ran towards the shoreline, with collecting locations positioned 3 m apart. Variation in vegetation density was observed both within and between transects, so that sample collection spanned a range of levels of structural complexity as a function of vegetation density.

For sampling small fishes, we used 0.3 m × 0.3 m × 0.3 m box-trap made with plexiglass [details appear in Dibble and Pelicice (2010); Lopes et al. (2015)]. These traps have floats that keep them near the water surface and embedded within the aquatic vegetation. This device captures fishes with lengths up to c.a. 10 cm and have minimal impact to habitat structure during deployment (Ribeiro and Zuanon 2006). Fishes living within plant structures are small (median: 3.1 and 95% CI 2.3–6.2 cm); larger species are sometimes encountered in vegetated areas, but these generally occupy open-water spaces where their movements are not impeded. To test for potential bias associated with gear selectivity, fish capture data were compared for the traps and surveys conducted using seine nets in floating vegetation stands in the same lakes. Fish community structure or body length distributions were not significantly different for samples obtained by the two methods (details of this analysis appear in the supplementary materials). Traps were deployed in each survey site at 0700 h and then inspected for fish after 12 and 24 h. Fishes from each trap survey were counted, identified to species level with the aid of taxonomic keys (Graça and Pavanelli 2007; Ota et al. 2018). α diversity at each site along a transect was recorded as species richness per trap day.

To characterize physical and chemical aspects of habitat at each trap location, we measured the following parameters at 0.4 m depth using a YSI probe and digital meter: dissolved oxygen (mg L− 1), pH, conductivity (µS m− 1), and temperature (°C). Water depth (m) was measured using a meter stick. These measures were taken twice at each trap location on the date of the survey (morning and evening) and averaged. Although averaging obscures minimum and maximum values that could have ecological implications beyond the average (e.g., minimum dissolved oxygen), averages nonetheless effectively revealed spatial gradients for environmental factors. Because dissolved oxygen concentration sometimes has an influence on fish distributions within vegetation stands (Miranda and Hodges 2000), we included this environmental factor as an explanatory variable in our statistical analysis. Plant density was estimated at each trap location. After each trap was removed, all plant material within a 0.5 × 0.5 m quadrat adjacent to the trap location was collected, separated into emergent and submerged components, and each component was dried (60 °C) until a constant dry weight (DW) was achieved. To provide an estimate of habitat structural complexity, submerged plant DW was converted to DW per unit volume of habitat (Kg DW L− 1), with habitat volume calculated from measurements of water depth and the area sampled.

Statistical analysis

Additive partitioning of species diversity

We partitioned the total diversity of samples into α and β components and used null models to assess if the observed partition is structurally different from random expectations. Nonrandom species distributions would suggest that ecological factors have a significant influence on diversity patterns (Crist et al. 2003). These results were used to test our hypothesis that habitat structural complexity and perhaps dissolved oxygen concentrations influence spatial patterns of fish diversity in vegetation stands. Diversity partitioning was performed using the additive framework and randomization method of Crist et al. (2003). Null models were derived from 9999 randomizations and compared with observed values. Analyses were performed in the vegan package (Oksanen et al. 2019) in R.

α diversity and explanatory variables’ associations

We used Linear Mixed-Effect Models (LMM) to test the hypothesis that α diversity is related to habitat structural complexity in a hump-shaped fashion, and to test for a potential influence of dissolved oxygen concentration in the relationship. Fixed effects were vegetation density, [vegetation density]2 (to enable evaluation of hump-shaped relationships of diversity with structural complexity) and dissolved oxygen concentration. We used a hierarchical random structure to account for the nested structure of samples and to ensure the assumption of independence among samples (see the work of Millar and Anderson (2004) for a discussion on nested designs in fisheries field experiments). Random effects included nested effects of lake identity and transect identity, in addition to the distance from the survey location to the margin of the vegetation patch with open water. We tested for multicollinearity by estimating a variance inflation factor (VIF), and all VIFs were < 3 (between-variable |r| < 0.65), indicating no collinearity (Zuur et al. 2010). Model selection based on Akaike Information Criterion (AICC, corrected second-order criterion) was done on data subsets with different combinations of explanatory variables. Parsimonious models were those with Δi AICC < 2 (Burnham and Anderson 2002). Model selection only supported inclusion of the second-order term for vegetation density when the model also included the first-order term for vegetation density. We used Wald χ2 tests and conditional and marginal R2 for validating model fitting. Analyses were performed in R using vegan, MuMIn and lme4 packages (Bartón 2018; Bates et al. 2014).

β diversity and explanatory variables’ associations

Pairwise measurements of β diversity were estimated using dissimilarity indexes based on species abundance data. β diversity was expressed as the Ružička dissimilarity index (quantitatively equivalent to Jaccard) that measures changes in assemblage composition between two sites with particular emphasis on differences in species abundance. The Ružička dissimilarity index was calculated as (B + C)/(A + B + C), where A is the sum of abundances for each species that are shared by sites 1 and 2, B is the sum of abundances for each species that are exclusive to site 1, and C is the sum of abundances for each species that are exclusive to site 2. Because this measure of β diversity incorporates variation due to species turnover and nestedness, we used the decomposition of Ružička dissimilarity index to investigate variation in these components. In general, two methods for decomposing β diversity have been promoted (see Legendre 2014), and the method used here is the one proposed in Podani et al. (2013). Their method provides a robust measure of nestedness that differs fundamentally from other popular methods (e.g., Baselga 2010) that may bias interpretations of nestedness (Schmera and Podani 2011). Species turnover was calculated as the relativized abundance replacement, which measures the replacement of abundances of particular species in one site by the same abundances of completely different species in another site (Podani et al. 2013). Relativized abundance replacement is measured as 2 min(B, C)/(A + B + C), and the maximum relativized abundance replacement is achieved when two sites show the same total abundance but share no species. Nestedness was calculated using relativized nestedness that measures the extent to which species abundances at one site are a subset of species abundances at another site (Podani et al. 2013). Relativized nestedness is measured as (A + |B − C|)/(A + B + C) when A > 0, otherwise it equals to 0. Relativized nestedness yields its maximum value when abundances for every species at one site do not exceed their respective abundances at the other site.

We used distance-based Redundancy Analysis to test the hypothesis that habitat structural complexity leads to changes in β diversity considering both turnover and nestedness components, and to test for potential influence of dissolved oxygen concentration on the relationship. Explanatory variables were vegetation density, [vegetation density]2, and dissolved oxygen concentration. We incorporated nested structure of sampling design by conditioning the distance from the survey location to open water and by using null models that constrain permutations under a hierarchical design for significance tests (Winkler et al. 2015). Similar to α diversity analysis, model selection was conducted considering Akaike Information Criterion (AICC, corrected second-order criterion). Model selection was done on data subsets with different combinations of explanatory variables, and only supported inclusion of the second-order term for vegetation density when the model also included the first-order term for vegetation density. The most parsimonious model was that with the smallest AICC. To provide further support for interpretations, we calculated the index for local contribution to β diversity (LCBD), an estimate of the ecological uniqueness of a particular local assemblage (Legendre and De Caceres 2013). LCBD measures the proportion of variation in β diversity attributed to a particular local assemblage, and can be obtained by decomposing the sums of squares of a dissimilarity matrix. We calculated LCBD from β diversity measurements and its components and used them as a basis to weight the sizes of circles representing local assemblages in ordination plots. Analyses were performed using the vegan package (Oksanen et al. 2019) in R, including functions provided by Pierre Legendre (2014).

Biological features

Because one of our hypotheses predicts that α diversity declines beyond an optimum level of habitat structural complexity due to reduction of interstitial spaces, we evaluated relationships between fish size and vegetation density. We used the median fish standard length and the variation in fish standard length in each sample to characterize fish size. Not all individuals were measured; therefore, bootstrapping was performed to estimate the median and the variation in fish standard length within local assemblages. Bootstraps involved resampling standard length measurements for the species abundances in a sample from pools of species standard length measurements. Pools of species standard length measurements considered individuals measured in the whole data set. Estimates for the median and the variation in fish standard length were obtained by median, minimum and maximum values after 999 simulations. We used Linear Mixed-Effect Models to test whether median and variation in fish standard length were constrained by habitat structural complexity. Vegetation density was treated as a fixed effect, and random effects included nested effects of lake identity and transect identity, in addition to the distance from the open-water end of the transect. Wald χ2 tests and conditional and marginal R2 were used to assess model fit. Power function and logarithmic transformations were applied to vegetation density in order to linearize relationships. Analyses were performed in R using lme4 and MuMIn packages (Bartón 2018; Bates et al. 2014).

We also evaluated the hypothesis that interspecific differences in adaptation for resistance to aquatic hypoxia yields a pattern of species filtering along gradients of dissolved oxygen. We tested the proportional contribution of species possessing auxiliary respiratory adaptations or documented tolerance to hypoxia to α diversity and total fish abundance in relation to dissolved oxygen gradients. Species were classified based on previous published literature reporting adaptations such as presence of dermal lip protuberances that enhance aquatic surface respiration, aerial respiration using the gut or swim bladder for gas exchange, capacity for anaerobic respiration and hypoxia tolerance. We used Generalized Linear Mixed-Effect Models to test the relationship between the percentage of hypoxia-tolerant fishes in the local assemblage and the dissolved oxygen concentration. The fixed effect was oxygen concentration, and random effects included nested effects of lake identity, transect identity and distance from the open-water end of the transect. Residuals were modeled following a binomial distribution using a log-link function to handle the percentage data. Wald χ2 tests and conditional and marginal R2 were used to assess model fit. Analyses were performed in R using lme4 and MuMIn packages (Bartón 2018).

Results

A total of 1832 small fishes belonging to 24 species was collected during the study. Most species were Characiformes, and the most abundant species were Moenkhausia bonita Benine, Castro & Sabino 2004, Hyphessobrycon eques, Serrapinnus heterodon (Steindachner 1882) and Serrapinnus notomelas (Eigenmann 1915) (210–351 individuals per species) (Fig. 1). More than half of the collected species lack known adaptations for resisting aquatic hypoxia, except for aquatic surface respiration during extreme hypoxia, a nearly universal behavior in teleost fishes. The number of fish caught per trap varied from to 1 to 137 individuals day− 1, except for 2 traps from which no individuals were captured (Table 1). Percent richness and abundance of hypoxia-tolerant species varied between 0 and 100%. Median standard length was 3.2 cm (minimum: 2.8; maximum: 7.20 cm), and mean SL range was c.a. 3.9 cm (minimum: 0.0; maximum: 5.4). Aquatic vegetation density, our proxy for habitat structural complexity, varied from 0 to 2.5 10− 3 Kg DW L− 1. Dissolved oxygen concentration varied from nearly 0 to 6 mg L− 1.

Fig. 1
figure 1

Relative abundance (diamonds) of fish species at survey locations across a gradient of aquatic vegetation density. Reciprocal averaging was used for ranking species and sites in order to emphasize the gradient changes in species composition among sites. Arrows in the y-axis indicate direction of changes in vegetation density

Full size image
Table 1 Fish assemblage attributes and habitat environmental variables measured to investigate α and β diversity in relation to a gradient of aquatic vegetation density
Full size table

Additive partitioning of species diversity

As expected, patterns of both α and β diversity differed significantly from random (p < 0.001 for both α and β diversity; Fig. 2a), suggesting that one or more non-random mechanisms drive diversity patterns. Whereas α diversity was lower than expected at random (8.1 vs. 10.6 species), β diversity was greater (18.9 vs. 16.4 species). Decomposition of the β diversity according to spatial scales revealed significant differences at trap (p = 0.001) and transect scales (p = 0.003) but not among lakes (p = 0.43), indicating that variation in β diversity was mostly associated with small spatial scales.

Fig. 2
figure 2

Additive partitioning of α and β diversity of fishes inhabiting vegetation stands. Observed α and β are contrasted with expected values obtained by randomizations. Analyses were performed with (a) and without considering nested sampling structure (b) (i.e., traps and transects within lakes). *Significant differences from the null model

Full size image

α diversity and explanatory variable associations

The best model for fish α diversity (Δ AICC < 2, ωi = 0.69) explained 46% of the variation (conditional R2 = 0.46), with fixed effects accounting for 30% of the variation (marginal R2 = 0.30; Fig. 3). Only vegetation density was included in the best model, with diversity having a significant hump-shaped response to vegetation density (χ2 = 21.46; p < 0.001; Fig. 3).

Fig. 3
figure 3

Partial residuals of the relationship between fish α diversity and vegetation density for fishes inhabiting vegetation stands

Full size image

β diversity and explanatory variable associations

Variation in total β diversity was most strongly related to vegetation density, [vegetation density]2 and oxygen content (F = 2.39; p < 0.001), suggesting that habitat structural complexity and dissolved oxygen concentration regulate spatial variation in species occurrence (Fig. 4a). Analysis of individual components of β diversity showed that species turnover was most strongly correlated with vegetation density and [vegetation density]2 (F = 2.53; p < 0.001). Complementary to this result, LCBDrepl values tended to increase towards lower and upper limits of vegetation density, suggesting that turnover rates are even greater towards the highest and the lowest levels of habitat structural complexity (Fig. 4b). For the nestedness component of β diversity, dissolved oxygen was the only correlated variable (F = 1.95; p = 0.04; Fig. 4c).

Fig. 4
figure 4

Distance-based Redundancy Analyses (dbRDA) assessing the relationship of variation in aspects of fish β diversity with vegetation density and dissolved oxygen concentration. a Total β diversity, b species turnover, c nestedness, VegDen vegetation density, VegDen2 [vegetation density]2

Full size image

Biological features

Vegetation density was significantly and negatively correlated with fish standard length (χ2 = 35.72; p < 0.001; Fig. 5a), explaining 44% of variation (conditional R2 = 0.48; marginal R2 = 0.44). Vegetation density also significantly correlated with the variation in fish standard length, but with the variation in fish standard length having a significant hump-shaped response to vegetation density (χ2 = 6.35; p = 0.01; conditional R2 = 0.47; marginal R2 = 0.11; Fig. 5b). Together, fish standard length and variation in fish standard length suggests that the size of interstitial spaces within highly complex habitats constrains fish body sizes.

Fig. 5
figure 5

Relationships of the median (a) and the range (b) of fish standard length with vegetation density for fishes inhabiting vegeation stands

Full size image

The percentage of fish specimens in assemblages that were categorized as hypoxia-tolerant was negatively correlated with dissolved oxygen concentration (χ2 = 6.84; p < 0.001; conditional R2 = 0.23; marginal R2 = 0.23; Fig. 6a). Similarly, the percentage of fish species categorized as hypoxia-tolerant was negatively correlated with oxygen concentration (χ2 = 3.90; p = 0.04; conditional R2 = 0.19; marginal R2 = 0.16; Fig. 6b). The fact that relative abundance and richness of hypoxia-tolerant fishes were greater when dissolved oxygen was lower strongly implies that dissolved oxygen gradients influenced fish distributions and thereby function as a strong environmental filter during local community assembly.

Fig. 6
figure 6

Relationships between the percentage of abundance (a) and percentage of species richness (b) of hypoxia-tolerant fish species within a location with dissolved oxygen concentration (mg L− 1) in aquatic vegetation stands

Full size image

Discussion

Habitat structural complexity has been recognized as an environmental factor that regulates species α diversity, but its relationship with β diversity is poorly explored. Our results on additive partitioning of fish species diversity indicates diversity components significantly differ from random across gradients of habitat structural complexity (Fig. 2), suggesting that deterministic mechanisms might be responsible for regulating both α and β (Chase et al. 2011). In addition, our findings demonstrate that both α and β components of fish diversity were strongly associated with a gradient of vegetation density (Figs. 1, 2, 3). Whereas α diversity tended to have highest values at intermediate levels of vegetation density, β diversity (particularly species turnover) was significant across the entire gradient (Figs. 2, 3, 4, 5). Such findings provide empirical support for the predicted hypothesis that multiple components of species diversity vary in relation to gradients of vegetation complexity in floodplain systems, and habitat structural complexity influences local community assembly.

Results for additive partitioning of diversity indicate that fish α diversity was lower than expected by chance, whereas β diversity was greater, suggesting that fish diversity across structural complexity gradients was strongly associated with species-specific responses to environmental conditions. Higher β and lower α diversity than expected by chance can indicate intraspecific aggregation within habitat patches (Crist et al. 2003; Fahr and Kalko 2011; Myers et al. 2013), resulting in clumped species turnover along environmental gradients. Our surveys focused on a gradient of vegetation density, and we infer that species turnover was strongly influenced by levels of structural complexity. Intraspecific aggregations have been reported for Pacific reef fishes and inferred to be associated with resource partitioning and restricted availability of physical habitat (Rodríguez-Zaragoza et al. 2011), mechanisms that also may apply to small fishes that inhabit structurally complex habitats in tropical floodplain systems (Arrington et al. 2005; Willis et al. 2005).

In agreement with additive partitioning of diversity, we found empirical support for species turnover along the gradient of vegetation density. The gradient of vegetation density ranges from sparse stands with relatively few submerged stems and roots and large interstitial spaces, to dense stands with high structural complexity and numerous small interstices. Large species generally require larger open spaces and would be excluded from dense stands of aquatic vegetation (Fahr and Kalko 2011; Yeager and Hovel 2017). Accordingly, our analysis of fish size revealed a significant reduction in fish size with increasing vegetation density, and only very small characids, such as Serrapinus notomelas and Serrapinnus heterodon, were captured from areas of dense vegetation (Fig. 1). Similarly, a recent study conducted in eelgrass habitat demonstrated that structural complexity regulates the size distribution of juvenile giant kelpfish, inferring that dense vegetation constrains the foraging efficiency of large fish (Yeager and Hovel 2017).

Another potential mechanism contributing to species turnover along gradients of habitat structural complexity is interspecific differences in susceptibility to predation. Predation has been shown to be a strong regulator of fish assemblage structure in freshwater systems (Jackson and Harvey 1989; Layman and Winemiller 2004), and predation mortality generally is greater when habitat structural complexity is low (Gause 1936; Huffaker 1958; Rozas and Odum 1988; Santos et al. 2009). Thus, species turnover along gradients of habitat structural complexity could indicate that predation-mediated colonization and extinction of patches is selective and species specific. For example, the characid Moenkhausia bonita has a relatively elongated body that should enhance swimming speed but reduce lateral maneuverability. In areas with sparse vegetation with large spaces of open water, this morphology likely enhances predator escape by fleeing rather than hiding. The idea that habitat structural complexity differentially influences species vulnerability to predation is consistent with Poff’s (1997) assertion that habitat structural complexity is an important environmental filter that sorts species according to functional traits and associated survival strategies (Poff 1997).

We also found that α diversity attained a maximum at intermediate levels of vegetation density, suggesting tradeoffs among mechanisms influencing species distributions along a structural complexity gradient. Theoretical models predict that α diversity of a given taxonomic group responds to environmental gradients according to a hump-shaped function, with each species being more or less normally distributed over a limited interval of the gradient (Crowder and Cooper 1979; Diehl and Kornijow 1998; Tokeshi and Arakaki 2012). This hump-shaped function can derive from the interaction of mechanisms that facilitate colonization and persistence in habitats (e.g., connectivity, availability of refuges and resources) and those that hamper them (e.g., isolation, space limitation, competition, predation). Structural complexity has been shown to reduce exposure to predators and enhance α diversity (Rozas and Odum 1988; Sánchez-Botero et al. 2007), however, excessive habitat structural complexity may limit space available for movement and foraging (Yeager and Hovel 2017). A recent study of reef fishes (Paxton et al. 2017) also found a hump-shaped relationship of α diversity along a gradient of structural complexity, suggesting that this pattern could be common in freshwater and marine systems.

Contrary to our prediction, species nestedness was not significantly associated with vegetation density, although it was significantly associated with dissolved oxygen concentration. Our fish surveys of aquatic vegetation stands included samples collected at night, when hypoxia may occur in dense patches, as well as samples taken during daytime when the water column tended to be well oxygenated. This allowed us to test for separate effects of vegetation density and oxygen concentration, two factors that have been found to influence fish abundance (Miranda and Hodges 2000). Greater assemblage nestedness and occurrence of hypoxia-tolerant fishes were associated with low dissolved oxygen, which suggests that dissolved oxygen functions as an environmental filter in concert with structural complexity of aquatic habitats (Keddy 1992; Poff 1997). Adaptation for tolerance of hypoxic conditions is relatively common among fishes inhabiting tropical floodplains. In our samples, for example, characids with auxiliary respiratory adaptations, such as Aphyocharax anisitsi (presence of dermal lip protuberances) and Hoplias spp. (anaerobic respiration), contributed to high proportional abundance and α diversity of hypoxia-tolerant species in dense vegetation. Periodic occurrence of extreme hypoxic conditions has been invoked as a major determinant of fish spatial distributions and local population dynamics in tropical and subtropical floodplains (Lowe-McConnell 1964; Winemiller 1989; Miranda and Hodges 2000; Miranda et al. 2000), and the present results support the importance of low dissolved oxygen concentration as an environmental filter structuring fish communities (De Macedo-Soares et al. 2010; Arantes et al. 2018).

In summary, we conclude that the density and associated structural complexity of aquatic vegetation stands in tropical floodplain lakes affects multiple diversity components of fish assemblages. Our findings provide empirical evidence that intermediate levels of vegetation density maximize α diversity of fish assemblages, suggesting tradeoffs among structuring mechanisms along the habitat structural complexity gradient. High β diversity along the gradient of vegetation density reveals species turnover and suggests species-specific responses to levels of structural complexity and other environmental factors. We obtained evidence that dissolved oxygen concentration is another significant environmental filter structuring these fish metacommunities, and infer that species colonization and persistence at locations along environmental gradients are likely influenced by multiple factors and mechanisms that operate on different spatiotemporal scales.