Introduction

Transitional ecosystems are complex environments with static coastal location and highly dynamic boundaries and features that support intense energy flow and exchange of species, nutrients, and other materials through pelagic systems (Sheaves & Johnston, 2009; Hazen et al., 2013). These features render biodiversity patterns a primary result of ecological processes associated with local environmental conditions and connectivity between alternative habitats (van Lier et al., 2018; Rodil et al., 2021). The spatial and temporal variation in ocean and climate regimes (e.g. temperature, precipitation, wind, currents, and tides) and freshwater input also influence strongly the physical, chemical, and biological processes that support coastal biodiversity (Garcia et al., 2017; Krueck et al., 2020). Considering the temporal dependence inherent to ecosystem processes, climate and environmental catastrophic changes quickly evolving through the current time bring severe sentences for coastal marine and estuarine life (Bellprat et al., 2019; Stewart-Sinclair et al., 2020). Therefore, disentangling environmental mechanisms supporting biodiversity patterns from small to large temporal scales can provide key information that may not be available in the near future of increasingly modified transitional ecosystems. Time may be then considered the most critical and valuable coin to negotiate actions to ameliorate the harsh future of coastal biodiversity.

Fishes are critical components of transitional ecosystems due to the plethora of shapes, sizes, behavior, life histories, habitat uses, and feeding strategies that promote large biomass production and variation within and across trophic levels (Vinagre et al., 2011; Winemiller et al., 2015). Deleterious effects of human-induced environmental changes on coastal fishes may thus strongly affect the ecological stability of transitional ecosystems. The core of anthropogenic impacts includes global warming, with higher frequency and magnitude weather extremes, shifts in fish species range and behavior, and changes in environmental drivers of coastal biodiversity (Savo et al., 2017; Nagy et al., 2019; Xu et al., 2020). Other facet of the same problem is the increasing ocean acidification affecting key biological processes, such as development, metabolism, behavior, and larvae settlement in suitable habitats (Hoegh-Guldberg et al., 2017; Rossi et al., 2018; Stewart-Sinclair et al., 2020). Biodiversity loss resulting from regional human pressures (e.g. overfishing, structural changes along the shoreline, and land-based pollution) boosts these threatening scenarios (Hilborn, 2016; Schulz et al., 2020). Micro-plastics, heavy metals and several other pollutants released into the ocean can affect both ecological and human health thousands of kilometers away (Baztan et al., 2014; Fistarol et al., 2015; Vieira et al., 2020; Savoca et al., 2021). Therefore, as a counterpoint to cumulative human impacts, it is increasingly of major relevance disentangling the complex mechanisms by which fish species and their functional roles support biodiversity in transitional ecosystems.

Coastal fish include species originating in freshwater habitats, species from groups that invaded the freshwater environment at different historical times, and a high diversity of marine species (Reis et al., 2016). Therefore, a balance between fish species with broad and restricted geographical ranges supports coastal species pools (Sheaves, 2012; Henriques et al., 2017a). The environmental variation at multiple spatial and temporal scales interacts with species traits, restricting the regional species pool and structuring local assemblages (Vilar et al., 2013; Ford & Roberts, 2020). Geographically closer sites are then prone to have higher compositional resemblance than distant sites, but different assembly processes lead different fish species to be rare or abundant over annual cycles, or spend only particular stages of their life cycles in coastal habitats (Potter et al., 2015; Camara et al., 2022). At the same time, random events, such as ecological drift (i.e. random fluctuations in species abundances), dispersal, disturbances, and speciation, may define quite surprising dynamics for the systems over different spatial and temporal scales (Chase & Myers, 2011; Ford & Roberts, 2018). Considering that taxonomic and functional perspectives of biodiversity may reveal mechanisms acting at different spatial and temporal scales (e.g. Villéger et al., 2010; Teichert et al., 2018; Ford & Roberts, 2020; Camara et al., 2021), their complementarity may represent a cornerstone to support current actions for conservation of coastal fish biodiversity.

Multiscale phenomena promoting differences between fish assemblages in relation to environmental gradients may be unraveled by assessing the variation in species composition, or beta diversity (Anderson et al., 2011). Directional measures of beta diversity express changes in species composition as a result of species replacement (turnover) from one sampling unit to another, or nestedness, whether the poorest assemblage is a subset of the richest assemblage (Baselga, 2012). Analogous processes of community assembly can be assessed by dissimilarity measures based on species abundances and functional traits (Baselga, 2013; Villéger et al., 2013). These beta diversity measures can quantify even slight differences in assembly processes by distinguishing patterns of variation in species composition and abundance (i.e. balanced replacement of individuals of different species or gradients of abundance; Baselga, 2013) and functional space filled by species (i.e. functional turnover or nestedness of functional traits; Villéger et al., 2013). Studies joining these taxonomic and functional perspectives of beta diversity over spatial and temporal scales may provide further advances in the knowledge on possible effects of environmental changes on coastal fish diversity, and primarily means to deal and/or avoid undesirable consequences.

We investigated multiple temporal scales (hereafter referred to as temporal dimensions) of environmental effects (local and landscape scales) on the taxonomic (i.e. abundance-based dissimilarity) and functional beta diversity of fishes in tropical transitional ecosystems with different degrees of marine and coastal influences (i.e. bays and coastal lagoons) during a 2-year period. We assessed directional changes at three temporal dimensions (i.e. 2-year period, per year, and per annual season). We considered that the investigated ecosystems share a high fraction of the regional species pool (Reis et al., 2016; Henriques et al., 2017a). Consequently, habitat filtering selects a wider range of functional traits to assemblages under higher environmental variation (Ford & Roberts, 2020), whereas gradual environmental changes interact with individual life cycles to promote seasonal gradients of species abundance (Castillo-Rivera et al., 2010; Lacerda et al., 2014). Altogether, such processes promote turnover and nestedness patterns from the taxonomic and functional perspectives (Camara et al., 2022). Moreover, environmental differences resulting from hierarchical habitat organization influence more the functional responses (Villéger et al., 2012; Teichert et al. 2018), whereas dispersal, demographic stochasticity, disturbances, and other spatially structured random processes influence primarily the taxonomic responses from small to large spatial and temporal scales (Blanco Gonzalez et al., 2016; Mazzei et al., 2021).

We therefore hypothesized (i) higher overall taxonomic and functional dissimilarities at larger temporal dimensions (i.e. 2-year period and per year), with (ii) higher contributions of turnover processes (i.e. balanced variation in species abundances and functional turnover) to their taxonomic and functional counterparts. Likewise, (iii) higher contributions of nestedness components (i.e. abundance gradients and functional nestedness) to their taxonomic and functional counterparts at the seasonal dimension. We also hypothesized (iv) more environmental effects on the overall beta diversity at larger temporal dimensions, with prevailing effects of local environmental variation and landscape features on the taxonomic and functional perspectives, respectively; and (v) at all temporal dimensions, functional responses to environmental gradients vary primarily between zones and types of system under different levels of marine influence, whereas unaccounted processes in individual systems influence mostly the taxonomic responses. This study jointed multiscale environmental effects to identify mechanisms supporting coastal fish diversity patterns at relevant time scales for conservation and management actions.

Materials and methods

Study area

Our study area encompassed six transitional ecosystems distributed through 150 km extent on the coast of the Rio de Janeiro State, Southeastern Brazil (Fig. 1a, b). This tropical region has the annual total rainfall (1000 to 1600 mm) primarily concentrated in the wet season (October to March) and annual mean temperature of 22 °C (Alvares et al., 2013). The transitional ecosystems include three bays (Ilha Grande, Sepetiba, and Guanabara) and three coastal lagoons (Maricá, Saquarema, and Araruama) with different degrees of connection with the ocean (Fig. 1b). Bays are larger and more complex systems characterized by higher diversity of habitats (e.g. sandy beaches, rocky shores, mangroves, and estuaries) than coastal lagoons (Camara et al., 2022). Coastal lagoons have limited connections with the ocean and reduced riverine input, culminating in pronounced salinity gradients distinguishing zones with different degrees of marine influence (Camara et al., 2021). Larger connections with the open sea support smoother salinity gradients in bays, but render these systems more influenced by tidal regimes (Azevedo et al., 2017). Bays and coastal lagoons are also pooled within distinct portions of the study area (Fig. 1b), and consequently subjected to marked differences in land use and cover (Avelar & Tokarczyk, 2014; Oliveira et al., 2019). These features support gradients of marine vs. freshwater influences primarily between the types of systems, and gradients conservation vs. human pressures primarily between individual systems and the zones in each system (Fistarol et al., 2015; Teixeira-Neves et al., 2015; Olivatto et al., 2019). We therefore considered three major zones in each system (Fig. 1b). The (1) Inner zone is more protected from the open sea and under higher land-based influences (e.g. land use and/or freshwater input), whereas a transition area less far from the open sea, but still under high land-based influences defines the (2) Middle zone. The (3) Outer zone, in turn, is closer to the open sea and under higher marine influence (e.g. ocean currents, waves, and/or tides).

Fig. 1
figure 1

Location of the a study area in Southeastern Brazil, and b sampling sites in the Inner, Middle, and Outer zones of bays (Ilha Grande, Sepetiba, and Guanabara) and coastal lagoons (Maricá, Saquarema, and Araruama)

Fish sampling

Fish sampling was performed bimonthly during 12 periods between September 2017 and July 2019 in each of the three bays (Ilha Grande, Sepetiba, and Guanabara) and three coastal lagoons (Maricá, Saquarema, and Araruama) (Fig. 1b). For each system and period, sampling was performed at three sites per zone (i.e. Inner, Middle, and Outer) (Fig. 1b). Each sample included three replicates per site. For each replicate, fishes were collected on the shore (< 1.5 m) with a beach seine net (12 × 2.5 m; 5-mm mesh size) dragging perpendicular to the shoreline (30-m long) and covering a swept area of about 300 m2. Fishes were fixed in 10% formalin, and after 48 h preserved in 70% ethanol. All fishes were identified to the species level as described in Araújo et al. (2018), and voucher specimens were deposited in the Ichthyological Collection of the Laboratório de Ecologia de Peixes of the Universidade Federal Rural do Rio de Janeiro.

A total of 648 samples were obtained during the study period (3 sites × 3 zones × 6 systems × 12 periods). For each period, whenever possible, we sampled different sets of three sites per zone (stratified random sampling) to better characterize the environmental heterogeneity in each zone and identify critical locations for different fish species. The alternative sites were always close to each other (about 200 m). We therefore considered the zones as the sampling units for analytical purposes, resulting in a total of 216 samples (3 zones × 6 systems × 12 periods). The procedures applied to calculate the environmental and species data per zone are properly described in the Environmental structures of hierarchical habitat organization, Spatial and temporal dimensions of species variation, and Data analysis subsections, respectively.

Local environmental variables

Physical and chemical parameters, substrate measures, and tidal influence were recorded at each site during fish sampling. Salinity, pH, temperature (°C), and dissolved oxygen (mg L−1) were obtained with a HANNA HI 9829 multiprobe (HANNA Instruments, São Paulo, Brazil). Depth (cm) was measured with a Speedtech model SM-5 digital probe (Speedtech Instruments, Great Falls, Virginia). Transparency (%) was measured with a Secchi disk and calculated as a percentage of Secchi depth/depth. Substrate type was classified considering the occurrence of clay + silt, fine sand, medium sand, coarse sand, gravel, and rocky bottom, estimated by visual census at three sampling points (1 m and 0.5 m depth, and at the spread washing zone) within the area of approximately 300 m2 covered by the fish sampling. The classification was based on the scale defined by the granulometric analysis described in Camara et al. (2019). Substrate type was scored from 1 (clay + silt) to 6 (rocky bottom) and calculated as the mean value per location. The tide was classified as flood, high, ebb, or low, and the tidal phase was then arbitrarily scored from 1 (flood/high tide) to 2 (ebb/low tide).

Landscape variables

Landscape metrics were obtained using a geographic information system (QGIS Development Team, 2018). The geoprocessing procedures used vectorial layers of hydrography and land use/cover (1: 25,000 scale; 2018) provided by a partnership between the Instituto Brasileiro de Geografia e Estatística (IBGE) and the Secretaria de Estado do Ambiente do Estado do Rio de Janeiro (SEA-RJ) (Portal GeoINEA, 2020).

We firstly obtained geographical coordinates for each site during field sampling with a handheld GPS Garmin eTrex 10 (Garmin International, Inc., Olathe, Kansas, USA). Metrics of land use/cover were then obtained as the areas (km2) of native forest cover, altered vegetation cover (i.e. pasture and cultivation areas), and human settlements within a 200-m radius buffer from the GPS coordinates of each site, and then calculated as a percentage of the buffer area. We also calculated the number of large industries within the buffer area. We considered 200-m buffers to assess the landscape heterogeneity primarily related to each location because the distance of 200 m separates most locations within the same zone, and previous studies revealed important effects of landscape metrics obtained within this radius on the fish assemblage structure in the studied systems (Camara et al., 2020, 2022).

We also obtained the number of nearby estuaries and mangrove cover as metrics representative of the availability of more complex alternative habitats and resources for fish refuge. For each site, nearby estuaries were considered as the total number of estuaries within a 5-km radius buffer. We aimed at emphasizing major differences in freshwater influence between zones, since the distance of 5 km corresponds to approximately the shorter distance between locations in different zones of the same ecosystem. Considering that mangroves have more restricted extent and distribution, mangrove cover was obtained as the total area (km2) within a 200-m radius buffer, and then calculated as a percentage of the buffer area. Finally, we obtained the distance from the ocean (km) as a metric representative of marine influence, calculated as the distance of each site from the open sea (i.e. the area immediately after the outer zone of the respective system; Fig. 1b).

Environmental structures of hierarchical habitat organization

We calculated the mean values per zone in each system and sampling period of all local and landscape variables obtained for each sampling site. We then performed a principal component analysis based on a correlation matrix of the centered and standardized variables to summarize prevailing environmental gradients and identify hierarchical structures resulting from the nested organization of habitats (i.e. zones, individual systems, and types of system) (Fig. S2a–c). The two first principal components were the most relevant axes to explain the environmental variance based on the broken-stick criterion (Peres-Neto et al., 2003) (Fig. S2e). Considering that the squares of all loadings for the original variables in an individual principal component sum to one, the variables that most contributed for the explained variance were considered as those with loadings larger than the hypothetical equal contribution of all variables (i.e. the square root of 1 divided by 15 variables; r = 0.26) (Fig. S2d).

Fish functional traits

For each species, we obtained information on ecological traits that influence their functional roles in fish assemblages (Table S1). Species traits were primarily representative of ecomorphological and behavioral characteristics related to feeding and life history strategies, and included body shape, body cross-section, body proportion, tail fin form, vertical distribution, mobility, trophic guild, reproductive guild, and habitat use (Winemiller et al., 2015; Villéger et al., 2017). Information was obtained primarily from FishBase online database (Froese & Pauly, 2022), complemented by other several sources (Elliott et al., 2007; Potter et al., 2015; Araújo et al., 2016; Robertson & Tassell, 2019; Andrade-Tubino et al., 2020) and our combined knowledge.

Ordinal categories indicative of degrees of functional differentiation were attributed for each trait (Table S1). Body shape: 1—filiform/slender to 4—neck curved, from skinny and straight shape to more curved body. Body cross-section: 1—highly compressed to 5—flattened, from more laterally compressed to more vertically flattened, considering a horizontal plane in water column. Body proportion: 1—extremely elongated to 4—deep, from proportionally longer to deeper body. Tail fin form, 1—absent to 14—deeply forked, from no contribution to fish movement to greater contribution to maneuvering, followed by a gradient of decreasing contribution to maneuvering and increasing contribution to acceleration. Vertical distribution: 1—benthonic to 3—pelagic, inhabiting from bottom only to water column (not on or near the bottom). Mobility, 1—mobile and 2—sedentary, from wide to small range of daily, seasonal, or even annual movement distance. Trophic guild: 1—detritivorous to 6—opportunistic, from foraging restricted to benthic sources to feeding on a wide variety of food items, whatever food becomes available. Reproductive guild: 1—larval form/ juvenile to 4—internal fertilization with internal development (viviparous), from absence of investment in reproduction to high degree of parental care. Habitat use: 1—estuarine to 6—marine straggler, from high to low dependence on estuarine habitats. Body shape, cross-section, and proportion were combined in a new variable indicative of the overall body shape (i.e. from curved and flattened to skinny, straight and compressed) by summing the values of the ordinal categories attributed to each species.

We performed a Principal Coordinates Analysis (PCoA) to summarize species trait table in a multidimensional space and assess the functional variation in fish assemblages. PCoA was based on Gower distance matrix that allows categorical traits to be properly combined. We assessed the quality of n-dimensional functional spaces resulting from the PCoA using the root of the mean squared deviation (mSD) between the initial (Gower) functional distances and standardized (Euclidean) distances in the multidimensional functional space (Maire et al., 2015). The root of the mSD ranges from 0 to 1, and the lowest value is indicative of a functional space of higher quality (i.e. the distance between each pair of species is more congruent with the initial functional distance) (Maire et al., 2015). The first two dimensions provided a good representation of the functional space, with a quality slightly lower (less than twice) than the quality of spaces formed by more dimensions (Fig. S1). Species traits were also more related to the first two PCoA axes, which explained 89% of the variation (Table S2). We therefore kept the first two PCoA axes to provide a two-dimensional representation of functional differences between fish species and calculate functional beta diversity (Fig. 3; Table S3). All analyses were performed in the R environment (version 4.1.3; R Core Team, 2022), and the quality of the functional space was assessed using the package mFD (version 1.0.3; Magneville et al., 2022).

Spatial and temporal dimensions of fish assemblage variation

Variation partitioning was used to assess the fractions of the taxonomic and functional dissimilarities between fish assemblages explained by hierarchical levels (i.e. nested organization of habitats) and temporal periods (Borcard et al., 1992). The species composition and abundance in space and time was considered as the sum of the specific abundances per zone in each system and sampling period. We calculated the abundance-based Bray–Curtis pairwise dissimilarities between all samples, and for each zone per system and sampling period, the taxonomic dissimilarity was calculated as the average of its pairwise dissimilarities (Baselga, 2013). Calculations of functional dissimilarity used a presence-absence matrix per zone in each system and sampling period, combined with functional dimensions represented by the first two PCoA axes. Species were plotted in the multidimensional functional space according to their respective functional trait values, and the Sørensen pairwise dissimilarity was calculated as differences in intersections of the volume of convex hulls occupied by species (i.e. functional richness) between each pair of samples (Villéger et al., 2013). Functional dissimilarity was then calculated as the average of the pairwise dissimilarities obtained for each zone per system and sampling.

Based on the habitat structures identified in the environmental analysis (Fig. S2a–c), the hierarchical levels included (1) types of system (i.e. bay or coastal lagoon), (2) individual systems (i.e. Ilha Grande, Sepetiba, and Guanabara bays; and Maricá, Saquarema, and Araruama lagoons), and (3) zones per systems (i.e. Inner, Middle, and Outer). Temporal periods included (1) annual cycle (i.e. year 1, September 2017 to July 2018; year 2, September 2018 to July 2019) and (2) annual season (i.e. wet season 1, September 2017 to January 2018; dry season 1, March 2018 to July 2018; wet season 2, September 2018 to January 2019; dry season 2, March 2019 to July 2019). The variation partitioning used multiple regressions applied to the taxonomic and functional dissimilarities to determine the exclusive and shared effects of hierarchical levels and periods (Borcard et al., 1992; Legendre & Gallagher, 2001). The explained variation was expressed as values of adjusted R2 to provide unbiased estimates of hierarchical and temporal effects (Peres-Neto et al., 2006). We assessed whether the testable fractions of the explained variation distinguished from random using permutational analysis of variance (999 permutations). All analyses were performed in the R environment (version 4.1.3; R Core Team, 2022) using the package vegan (version 2.6–2; Oksanen et al., 2022). Temporal structures identified in the variation partitioning were confirmed as relevant dimensions to calculate the temporal beta diversity (Fig. 2a, b). The hierarchical structures, in turn, were considered as random sources of variation in modeling procedures, including exclusive and shared effects of individual systems and types of system, and the exclusive effects of zones (Fig. 2c, d).

Fig. 2
figure 2

Venn diagrams showing variation partitioning results based on average dissimilarity in species composition and abundance (taxonomic beta diversity) and trait-based dissimilarity measured as the volume of convex hulls intersections in the two-dimensional functional space (functional beta diversity) per zone in each bay and coastal lagoon. Variation partitioning performed on exclusive and shared effects of (a, b) seasons and years, and (c, d) zones, individual systems, and type of system. Values (adjusted R2) express the explained and unexplained fractions of variation. Values in bold for testable fractions with P-values (permutation F tests) < 0.01

Taxonomic and functional perspectives of temporal beta diversity

For each temporal dimension, the temporal beta diversity expressed the variation in fish assemblage from taxonomic (i.e. abundance-based Bray–Curtis dissimilarity) and functional perspectives (i.e. Sørensen derived functional dissimilarity, based on differences in functional richness measured as the intersections of the volume of convex hulls occupied by species in the multidimensional functional space) between samples obtained per zone in each system (Baselga, 2013; Villéger et al., 2013). Temporal dimensions included the entire 2-year period (i.e. September 2017 to July 2019), and based on the results of the variation partitioning, annual cycle (i.e. year 1; year 2) and annual season (i.e. wet season 1; dry season 1; dry season 2; wet season 2). Therefore, all measures of beta diversity per zone in each system were calculated for the 2-year period, per year, and per season. Calculations of taxonomic and functional beta diversity used abundance matrix and a presence-absence matrix combined with functional space dimensions (i.e. the first two PCoA axes), respectively.

Total abundance-based dissimilarity (βBC) was partitioned in (i) balanced variation in species abundances (βBC-BAL) and (ii) abundance gradients (βBC-GRA) (Baselga, 2013, 2017). Balanced variation in species abundances is analogous to species turnover in incidence-based dissimilarity measures and expresses of the replacement of the individuals of some species by the same number of individuals of different species among multiple samples. Abundance gradients are analogous to species nestedness and expresses the loss or gain of individuals of all species among multiple samples. The total Sørensen dissimilarity (βSOR), in turn, was partitioned in (i) functional turnover component, calculated as a Simpson derived dissimilarity (βSIM), and (ii) functional nestedness component, measured as a fraction of Sørensen dissimilarity (βSNE) (Villéger et al., 2013). Species are plotted in the multidimensional functional space according to their respective functional trait values, and high dissimilarity may result from low overlap between samples (high turnover) or high nestedness, when some samples fill only a small proportion of the functional space filled by others (Villéger et al., 2013).

Each measure of taxonomic beta diversity (βBC, βBC-BAL, and βBC-GRA) and functional beta diversity (βSOR, βSIM, and βSNE) was obtained 18 times in the 2-year period (3 zones × 6 systems), 36 times in the annual cycles (3 zones × 6 systems × 2 years), and 72 times in the annual seasons (3 zones × 6 systems × 2 seasons × 2 years). All calculations were performed in the R environment (version 4.1.3; R Core Team, 2022) using the package betapart (version 1.5.6; Baselga et al., 2022).

Data analysis

We modeled the responses of each beta diversity measure (taxonomic, βBC, βBC-BAL, and βBC-GRA; and functional, βSOR, βSIM, and βSNE) to local and landscape environmental variables in the 2-year period, per year, and per season using generalized linear mixed models (GLMMs) specifying the Beta distribution for the response variables (Cribari-Neto & Zeileis, 2010; Brooks et al., 2017). Beta distribution can accommodate several shapes and is therefore flexible to model continuous variables ranging between 0 and 1, typically heteroskedastic and asymmetric (Ferrari & Cribari-Neto, 2004). GLMMs used a logit-link function to relate the expected responses to linear predictors (Gelman & Hill, 2007; Bolker et al., 2009).

Prior to models fitting, we calculated the mean values per zone in each system and period of environmental variables measured at sites. Local and landscape environmental variables were then centered and standardized to improve the parameter estimates and for fitting comparable models (Schielzeth, 2010). For each zone per system, the temporal variation of each local environmental variable was calculated by averaging the pairwise Euclidean distances between samples obtained at each temporal dimension (i.e. the 2-year period, per year, and per season) (Oksanen et al., 2022). Therefore, we obtained environmental distances for salinity, pH, temperature, dissolved oxygen, depth, transparency, substrate type, and tidal phase. A variable selection was then performed to exclude weak predictors of the taxonomic and functional measures of beta diversity. For each beta diversity measure and temporal dimension, all measures of local environmental variation (i.e. environmental distances) and landscape variables were included as fixed effects in a linear mixed model with a random effect for the hierarchical structure identified in variation partitioning (i.e. nested effect of types of system and individual systems, and exclusive effect of zones; Fig. 2c, d). We then assessed the total variance explained by each predictor using the inclusive R2, a measure of its global effect irrespective of the other variables included in the model, and retained variables with inclusive R2 > 0 (Stoffel et al., 2021; Figs. S3, S4). The inclusive R2 is calculated as the square of structure coefficients (i.e. the correlation between a predictor and fitted values) times total R2 (Stoffel et al., 2021). All selected variables were included as fixed effects in a full beta regression model and the variance inflation factor (VIF) was calculated for each predictor variable to assess multicollinearity (Zuur et al., 2010). For all cases, only environmental variables with VIF < 2 (indicative of negligible multicollinearity) were included as fixed effects in a final full GLMM (Zuur et al., 2010).

In a first step of model selection, we defined the optimum random structure for GLMMs explaining the relationships between temporal measures of beta diversity and environmental variables (local environmental variation and landscape variables) at each temporal dimension using a procedure based on the AICc (Burnham & Anderson, 2002). Random effects accounted for probable influences of the hierarchical structure (i.e. types of system, individual systems, their nested effects, or zones) (Schielzeth & Nakagawa, 2013). All candidate GLMM included all environmental variables as fixed effects and a different random effect for the hierarchical structure to account for the spatial dependence between samples (Bolker et al., 2009).

In a second step, we applied an automated model selection procedure using the full GLMMs with the best random structures as start points to obtain the submodels with the most parsimonious combinations of fixed effects (Bartoń, 2022). In both steps, the model with the lowest AICc was considered the best due to less information loss and simpler structure (Burnham & Anderson, 2002). Models were ranked according the AICc weight (wi) that represents the probability that the model is the best between the set of candidate models (Wagenmakers & Farrell, 2004). Models were then selected for interpretation based on the ΔAICc, the difference between the AICc of a given model and the AICc of the best model, which represents the probability that a given model minimizes information loss (Burnham & Anderson, 2002; Wagenmakers & Farrell, 2004). Based on this criterion, all models with ΔAICc < 2 were considered with substantial support for interpretation (Burnham & Anderson, 2002).

The goodness-of-fit of each selected model was indicated by the squared correlation between the response and the predicted value. The variation explained by fixed effects (r2 f) was the squared correlation between the response and the predicted value based only on the fixed effects included in the model. The variation explained by both fixed and random effects (r2 f + r) was the squared correlation between the response and the predicted value based on all effects included in the model.

A model averaging approach was applied when more than one model was selected to explain the temporal beta diversity measures at each temporal dimension. Inferences across the selected models were combined by calculating model-averaged parameter estimates and the associated confidence intervals (Burnham & Anderson, 2002). The model averaging approach therefore estimated the strength of environmental effects based on their contributions to the average model. A parameter was considered informative if the 95% confidence interval did not overlap zero. The relative variable importance (RVI) for the parameter estimates in the average model was calculated by summing the wi of the selected models recalculated without the other candidate models (Burnham & Anderson 2002).

Forest plots were constructed to visually compare the informative environmental effects, and associated random influences of the type of system or individual systems, on the taxonomic (i.e. βBC, βBC-BAL, and βBC-GRA) and functional (i.e. βSOR, βSIM, and βSNE) perspectives of beta diversity in the selected GLMMs for the 2-year period, per year, and per annual season.

All analyses performed in the R environment (version 4.1.3; R Core Team, 2022) with the packages vegan (version 2.6-2; Oksanen et al., 2022), partR2 (version 0.9.1; Stoffel et al., 2021), car (version 3.1-0; Fox & Weisberg, 2019), betareg (version 3.1-4; Cribari-Neto & Zeileis, 2010), glmmTMB (version 1.1.4; Brooks et al., 2017), MuMIn (version 1.46.0; Bartoń, 2022), and metafor (version 3.4-0; Viechtbauer, 2010).

Results

Environmental variation

The number of nearby estuaries and distance from the ocean were much higher in the Inner zones of bays than coastal lagoons, which had higher percentages of altered vegetation (Table 1; Fig. S2c, d). Mangrove cover was restricted to bays, but occurred only in the Inner zones of the Sepetiba and Guanabara bays (Table S4; Fig. S2a–d). Other landscape differences were supported primarily by individual systems, with higher percentages of human settlements in the Middle-Outer zones of the Guanabara bay, and to a lesser extent Saquarema lagoon, as well as the Inner-Middle zones of the Sepetiba bay and Araruama lagoon (Table S4; Fig. S2a–d). The Inner zone of the Maricá lagoon and Middle-Outer zones of the Ilha Grande bay had higher percentages of native forest cover (Table S4; Fig. S2a–d). The number of industries was quite similar between systems, but slightly higher in the nearby areas of Guanabara bay and Maricá lagoon, and lower in the Saquarema lagoon (Table S4; Fig. S2b–d).

Table 1 Local and landscape environmental variables obtained bimonthly during two annual cycles in bays and coastal lagoons

Regarding local environmental variables, differences related to types of system and zones were even slighter and primarily supported by depth, and to a lesser extent salinity, and transparency (Table 1). Altogether, depth was higher in bays than coastal lagoons, but primarily due to the higher values observed in the Guanabara and Sepetiba bays compared with Maricá lagoon (Tables 1, S4; Fig. S2b–d). Salinity was much higher in the Araruama lagoon (hyperhaline) than in the other systems, whereas intermediate values were observed in the Middle-Outer zones of the Saquarema lagoon and Guanabara bay, and quite lower values in the Inner zone of Maricá lagoon (mesohaline) (Table S4; Fig. S2a–d). Compared with the other systems, slightly higher and lower values of transparency were observed in the Saquarema and Araruama lagoons, respectively, but lower values were also observed in the Inner zones of bays (Table S4; Fig. S2a–d). The sandy substrate observed in all systems was coarser in the Araruama lagoon, and Middle-Outer zones of Saquarema lagoon and Guanabara bay compared with Ilha Grande bay and Inner zones of other systems (Tables 1, S4; Fig. S2a–d).

Other local variables formed weaker gradients and reinforced environmental differences primarily between individual systems (Table S4; Fig. S2a–d). Sepetiba bay and Maricá lagoon had lower concentrations of dissolved oxygen, regardless of the observed values had varied within a wide range comparable to the other systems (Table S4). Tidal phase was quite similar between all systems, with a slight prevalence of flood/high tide in coastal lagoons, primarily in Saquarema and Araruama lagoons, and ebb/low tides in bays, primarily in the Ilha Grande bay (Tables 1, S4). Alkaline pH and warm temperatures of about 26 °C were prevalent in both types of system (Tables 1, S4).

Hierarchical and temporal fish assemblage structures

The variation in the average taxonomic dissimilarity between samples obtained in different zones of transitional ecosystems was primarily associated with the hierarchical levels (28%) and to a much lesser extent with temporal intervals (1%) (Fig. 2). The exclusive effects of individual systems explained most of the variation (24%), with a smaller fraction shared with types of system (4%) (Fig. 2). The variation associated with temporal effects was primarily shared between seasons and annual cycles (Fig. 2). Hierarchical levels also explained most of the variation in the average functional dissimilarity (15%), primarily related to individual systems (10%) (Fig. 2). A small fraction was also shared between individual systems and types of systems, but zones explained 5% of the functional variation (Fig. 2). Temporal effects on the functional variation were exclusively related to seasons (Fig. 2).

Functional variation in fish assemblages

Gradients of functional traits allowed to identify major functional groups related to feeding, behavior, and life history of fish species (Fig. 3). Regarding body shape, species groups were primarily separately by their curved vs. straight to fusiform shapes, and then by rounded to flattened vs. compressed bodies, respectively (Fig. 3a). Tail fin form influenced this gradient separating primarily species with forked fin from species with tail fins that contribute less to fish movement (i.e. maneuvering and acceleration) (Fig. 3b). Species with curved/ rounded to flattened bodies had generally emarginate to truncated tail fins, culminating in a greater ability to maneuver in sheltered areas across the bottom (Fig. 3a, b). Species with straight to fusiform/compressed bodies, in turn, were characterized by tail fin varying from deeply to moderately forked, indicative of fast-swimming locomotion (Fig. 3a, b). Vertical distribution and mobility confirmed these functional groups, with benthonic and sedentary species vs. mobile fishes inhabiting areas over the bottom (i.e. benthopelagic and pelagic species) characterized by curved/ rounded to flattened shapes vs. straight to fusiform/compressed shapes, respectively (Fig. 3a–d). The latter group was formed primarily by planktivorous, piscivorous, and opportunistic species, whereas the former group included primarily benthophagous and hyperbenthophagous species (Fig. 3e). Considering the reproductive guild and habitat use, larval-juvenile forms represented primarily the estuarine and marine estuarine-dependent species groups, whereas viviparous species were exclusively freshwater and most species with parental care were marine straggler (Fig. 3f, g).

Fig. 3
figure 3

Principal coordinates analysis (PCoA) of the functional trait values of fish species in bays and coastal lagoons. Symbols (colored circles) correspond to individual taxa (species or larval-juvenile forms, n = 170) and different sizes (from larger to smaller) and color gradients indicate the effects of different traits on the PCoA ordinations: a overall body shape, from curved and flattened to skinny, straight and compressed; b tail fin form, a gradient of decreasing contribution to acceleration and increasing contribution to maneuvering, culminating in no contribution to fish movement; c vertical distribution, from water column to bottom; d mobility, from sedentary to mobile; e trophic guild, from feeding exclusively on benthic sources to a wide variety of available food items; f reproductive guild, from viviparous to no parental care; and g habitat use, from high to low dependence on estuarine habitats. Species separated by habitat use along the PCoA axis 2, and primarily along the PCoA axis 1 by other functional traits. Fish images obtained from Robertson & Tassell (2019)

Altogether, functional traits revealed four functional groups (Fig. 3a–g). The first group (positive scores of PCoA axes) included species more dependent on estuarine habitats (i.e. estuarine, marine estuarine-dependent, semi-anadromous and semi-catadromous species) characterized by straight, skinny or fusiform body, forked tail fin, benthopelagic or pelagic distribution, high mobility, planktivorous, piscivorous, or opportunistic feeding, and no parental care associated with the occurrence of several larval-juvenile forms. The second group (negative and positive scores of PCoA axes 1 and 2, respectively) was represented by estuarine and marine estuarine-dependent species with rounded to flattened shapes, emarginate, pointed, rounded, or truncated tail fin, sedentary habit associated with the bottom, benthophagous or hyperbenthophagous feeding, and the prevalence of no parental care. The third group (positive and negative scores of PCoA axes 1 and 2, respectively) included species less dependent on estuarine habitats (i.e. marine straggler, freshwater, and marine estuarine-opportunist species). These species were characterized by compressed shapes (deep or elongate) and forked tail fin, high mobility associated with benthopelagic or primarily pelagic distribution, a variety of feeding habits (i.e. planktivorous and benthophagous or hyperbenthophagous species, and to a lesser extent piscivorous and opportunistic species), and high degree of parental care. The fourth group (negative scores of PCoA axes) also included species less dependent on estuarine habitats, and primarily species with curved bodies, mostly moderately to highly flattened, and deep, associated with moderately forked, pointed, truncated, or absent tail fin. Species in this group were also benthonic with sedentary habit, benthophagous or hyperbenthophagous, and to a much lesser extent piscivorous or opportunistic feeding, and a slight prevalence of no parental care over parental care.

Temporal dimensions of taxonomic and functional dissimilarities

Considering the taxonomic perspective, higher overall dissimilarity (βBC) was prevailing in the 2-year period compared with years and annual seasons (Fig. 4a–c). These differences were a consequence of an increasing variation in the contributions of balanced variation in species abundances (βBC-BAL) and abundance gradients (βBC-GRA) to the taxonomic dissimilarity from the larger to the smaller temporal dimension, culminating in lower differences between individual systems (Fig. 4d–i; Table S5). Regardless of that, balanced variation in species abundances was mostly higher in bays, whereas abundance gradients contributed more to the taxonomic dissimilarity in coastal lagoons (Fig. 4d–i; Table S5). The balanced variation in species abundances had a slight decrease from the larger to the smaller temporal dimension, but differences between types of system were generally supported by individual systems (Fig. 4d–f). On the other hand, abundance gradients increased from the 2-year period to annual seasons, culminating in similar contributions in coastal lagoons and bays, and a slightly higher contribution of bays in the wet season 2 (Fig. 4g–i). Differences between zones in the contributions of different processes of community assembly to taxonomic dissimilarity were negligible at all temporal dimensions (Table S5).

Fig. 4
figure 4

Violin plots showing the temporal variation in taxonomic beta diversity and box-plots (median, lower and upper quartiles, and minimum and maximum values) of the temporal variation per individual system (black, Ilha Grande, Sepetiba, and Guanabara bays; grey, Maricá, Saquarema, and Araruama lagoons) and type of system (black, bays; grey, coastal lagoons) during a 2-year sampling period (left panel), per year (central panel), and per annual season (right panel). Beta diversity expressed as (a, d, g) total abundance-based dissimilarity (βBC), (b, e, h) balanced variation in the abundance of different species (βBC-BAL), and (g, h, i) abundance gradients (βBC-GRA) per zone in each system. Only hierarchical levels with stronger effects on beta diversity are displayed (see Table S5)

The functional perspective revealed a more marked decrease in the overall dissimilarity (βSOR) from the 2-year period to annual seasons (Fig. 5a–c). However, functional dissimilarity was mostly lower than its taxonomic counterpart due to quite similar contributions of functional turnover (βSIM) and functional nestedness (βSNE) (Fig. 5a–i). Differences in functional processes of community assembly between individual systems increased from the larger to the smaller temporal dimension, but were generally not enough to support differences between types of systems (Fig. 5a–i; Table S6). In the 2-year period, higher functional turnover and nestedness irrespective of hierarchical habitat organization supported higher overall dissimilarity between individual systems (Fig. 5a, d, g; Table S6). Functional nestedness was more influenced by individual systems per year and had slightly higher contributions to overall dissimilarity than functional turnover (Fig. 5b, e, h; Table S6). More marked differences in the functional turnover between individual systems produced slightly higher contributions to overall dissimilarity in bays than coastal lagoons at the seasonal dimension (Fig. 5f; Table S6). However, smaller differences in functional nestedness between individual systems prevented differences in overall dissimilarity between types of system (Fig. 5c–f; Table S6). At all temporal dimensions, differences in functional processes of community assembly between zones were also negligible (Table S6).

Fig. 5
figure 5

Violin plots showing the temporal variation in functional beta diversity and box-plots (median, lower and upper quartiles, and minimum and maximum values) of the temporal variation in functional beta diversity per individual system (black, Ilha Grande, Sepetiba, and Guanabara bays; grey, Maricá, Saquarema, and Araruama lagoons) and type of system (black, bays; grey, coastal lagoons) during a two-year sampling period (left panel), per year (central panel), and per annual season (right panel). Beta diversity expressed as (a, b, c) total Sørensen dissimilarity (βSOR), (d, e, f) functional turnover—Simpson derived dissimilarity (βSIM), and (g, h, i) functional nestedness—fraction of Sørensen dissimilarity (βSNE) per zone in each system. Only hierarchical levels with stronger effects on beta diversity are displayed (see Table S6)

Environmental drivers of taxonomic and functional dissimilarity

Random effects for individual systems or zones were included in full models explaining the taxonomic dissimilarity (βBC) during the 2-year period (Table 2, Fig. S2). For both cases, the hierarchical level accounted for about 20% of the variation in the responses of taxonomic dissimilarity to environmental predictors in full models, but the influence of zones was negligible in the resulting alternative models, whereas more than 30% of the variation was related to individual systems (Tables 2, S7). Models explaining the balanced variation in species abundances (βBC-BAL) and abundance gradients (βBC-GRA) included random effects only for individual systems, which influenced more than 50% of the variation in the resulting alternative models (Tables 2, S7).

Table 2 Random effects included in the full generalized linear mixed models selected for relationships between the taxonomic beta diversity (Bray–Curtis dissimilarity) and environmental variables in different temporal dimensions

Model averaging revealed that positive effects of mangrove cover on overall dissimilarity in the 2-year period were more influenced by individual systems than zones, and were not related to a specific assembly process (Fig. 6). To a lesser extent, taxonomic dissimilarity was positively associated with the number of nearby estuaries, as a primary consequence of the positive responses of abundance gradients despite the variation between individual systems (Fig. 6). The negligible effect of native forest cover on overall dissimilarity reinforced the importance of the variation between individual systems to promote taxonomic patterns (Table S8). Effects of transparency and depth, in turn, revealed negative influences of local environmental variation on the taxonomic dissimilarity in the 2-year period (Fig. 6). In models including a random effect for individual systems, the slight negative relationship between transparency variation and taxonomic dissimilarity was a result of its negative effect on the balanced variation in species abundances counterbalanced by its positive effect on abundance gradients (Fig. 6). Contrasting effects of transparency variation on taxonomic processes of community assembly differing between individual systems most likely also explains the restricted and slight effect of transparency on the overall dissimilarity in models including a random effect for zones (Fig. 6). The exclusive and slight effect of depth variation on the overall dissimilarity may thus provide further evidence for the importance of counterbalanced influences of different assembly processes irrespective of the hierarchical habitat organization at the 2-year dimension (Fig. 6).

Fig. 6
figure 6

Effects of local environmental variables (temporal variation) and landscape variables on the taxonomic beta diversity (Bray–Curtis dissimilarity) per zone in each system. Temporal beta diversity expressed as total abundance-based dissimilarity (βBC), balanced variation in abundance of different species (βBC-BAL), and abundance gradients (βBC-GRA) between samples obtained during a 2-year period, per year, and per annual season. Parameter estimates and 95% confidence intervals of model predictors based on the averages of generalized linear mixed models (GLMMs) selected (ΔAICc < 2). Results displayed only for estimates with 95% confidence intervals that do not overlap zero. Blue and orange squares represent the positive and negative fixed effects in the models, respectively. GLMMs included zones (Zo), individual systems (Sy) or types of system (TS) as random effects. For variable codes see Table 1

Higher balanced variation in species abundances was also observed under higher variations in salinity and tidal phase, whereas the percentage of altered vegetation had a slight negative effect on that turnover process (Fig. 6). Lower tidal variation and more altered vegetation, in turn, supported higher abundance gradients, reinforcing the importance of the equivalence between contrasting environmental effects on different assembly processes to produce negligible changes on the overall taxonomic dissimilarity irrespective of variations between individual systems (Fig. 6; Table S7). Regardless of negligible, contrasting effects of temperature on different assembly processes provide further evidence in this sense (Table S8).

Taxonomic dissimilarity per year was explained by models including random effects for zones, individual systems, and types of system, but the influences of the hierarchical levels were lower compared with the 2-year period (Tables 2, S9). The random influence of the zone in the resulting alternative models was quite negligible, whereas the influence of individual systems and especially types of system were stronger, as a most likely consequence of the importance of the type of system for the variation in relationships between environment and different abundance-based processes of community assembly (Tables 2, S9). Large fractions of the variations in responses of balanced variation in species abundances and abundance gradients associated with types of system (about 40%) may thus explain the negligible local and landscape environmental effects per year (Tables 2, S9 and S10; Fig. 6). These negligible effects included contrasting influences of dissolved oxygen, substrate type, and distance from the ocean on different assembly processes, which prevented their effects on the overall dissimilarity (Table S10). For taxonomic dissimilarity, averaging of alternative models revealed a primary importance of local environmental variation irrespective of the hierarchical habitat organization (Fig. 6). For all cases, taxonomic dissimilarity was higher under lower variation in transparency and depth, whereas models including types of system as a random effect also evidenced positive responses to temperature variation (Fig. 6). Mangrove cover also had a positive effect on the overall dissimilarity per year irrespective of zones, as a most likely consequence of higher balanced variation in species abundances (Tables S9 and S10; Fig. 6). In this sense, the positive relationship between mangrove cover and balanced variation in species abundances was negligible most likely due to the large variation between types of system (Tables S9, S10).

Random structures resulting from zones, individual systems, and types of system were also included in models explaining the relationships between taxonomic dissimilarity and environmental predictors per annual season (Table 2). As observed for the larger temporal dimensions, random effects for zones were quite negligible at the seasonal dimension, whereas individual systems and types of system had stronger influences on the variation in responses of taxonomic dissimilarity to environmental predictors (more than 30% and 10%, respectively) (Tables 2, S11). A similar influence of the hierarchical habitat structure was observed in models explaining the balanced variation in species abundances, whereas random hierarchical effects on responses of abundance gradients were negligible (Table S11). A higher number of environmental effects explaining taxonomic dissimilarity and abundance-based processes of turnover and nestedness in models including random effects for zones reinforced the strength of differences between individual systems and types of system in landscape effects (Table S12; Fig. S6).

According to the averaging of alternative models including zone as a random effect, variation in substrate type had a positive effect on the overall dissimilarity that was not related to a specific assembly process (Fig. 6). Taxonomic dissimilarity was also positively related to higher variation in tidal phase, primarily as a result of the positive effect of tidal variation on the balanced variation in species abundances to the detriment of its negative effect on abundance gradients (Fig. 6). Similar effects of tidal variation were observed in models including other hierarchical structures, with higher contributions of its negative effects on abundance gradients culminating in negligible changes in taxonomic dissimilarity irrespective of the variation between individual systems (Fig. 6). Other environmental effects on taxonomic dissimilarity revealed prevailing landscape influences on the balanced variation in species abundances (Fig. 6). Taxonomic turnover was negatively related to the number of industries and mangrove cover, and positively related to the number of nearby estuaries and distance from the ocean only in models including the variation related to zones (Fig. 6). These models also supported negative responses of taxonomic dissimilarity as a primary consequence of the negative effects of native forest cover, altered vegetation cover and human settlements on the balanced variation in species abundances, but similar relationships were observed in models including the random variation related to individual systems and types of system (Fig. 6).

From the functional perspective, full models selected to explain fish dissimilarity (βSOR) in the 2-year period included only zones as a random effect, which influenced about 30% of the variation in responses to environmental predictors (Table 3). Regardless of the strong random influence of zones, the averaging of the resulting selected models revealed positive effects of nearby estuaries, distance from the ocean, and primarily variation in pH on the functional dissimilarity (Fig. 7; Table S13). Responses of the functional dissimilarity to landscape predictors were not related to a specific process of community assembly, whereas models including types of system as a random effect support the positive effect of pH variation on functional dissimilarity as a most likely result of higher turnover (βSIM) (Fig. 7; Table S14). The responses of functional turnover and functional nestedness (βSNE) to environmental predictors were influenced by differences between types of system (about 30% and 20%, respectively) (Tables 3, S14). Contrasting effects of tidal variation and mangrove cover on functional turnover and nestedness prevented changes in the overall dissimilarity (Fig. 7). Higher functional turnover was related to higher variation in tidal phase and lower mangrove cover, whereas higher functional nestedness was related to lower tidal variation and higher mangrove cover (Fig. 7). Functional turnover was also negatively related to higher salinity variation (Fig. 7). Therefore, regardless of the strength of hierarchical effects, similar influences of local environmental variation and landscape variables determined functional dissimilarity at the 2-year temporal dimension (Fig. 7; Table S13).

Table 3 Random effects included in the full generalized linear mixed models selected for relationships between the functional beta diversity (Sørensen dissimilarity) and environmental variables in different temporal dimensions
Fig. 7
figure 7

Effects of local environmental variables (temporal variation) and landscape variables on the functional beta diversity per zone in each system. Temporal beta diversity expressed as total Sørensen dissimilarity (βSOR), functional turnover—Simpson derived dissimilarity (βSIM), and functional nestedness—fraction of Sørensen dissimilarity (βSNE) between samples obtained during a 2-year period, per year, and per annual season. Parameter estimates and 95% confidence intervals of model predictors based on the averages of generalized linear mixed models (GLMMs) selected (ΔAICc < 2). Results displayed only for estimates with 95% confidence intervals that do not overlap zero. Purple and golden squares represent the positive and negative fixed effects in the models, respectively. GLMMs included zones (Zo), individual systems (Sy) or types of system (TS) as random effects. For variable codes see Table 1

Random effects of the hierarchical structure on functional dissimilarity per year and per annual season were negligible, culminating quite similar environmental effects in alternative models irrespective of the random effect (Tables 3, S15, S17). Landscape variables had an increasing influence on measures of functional dissimilarity to the detriment of local environmental effects from the larger temporal dimensions to annual seasons (Fig. 7). Per year, positive and negative effects of variation in transparency and dissolved oxygen, respectively, on functional nestedness were also expressed on functional dissimilarity (Fig. 7). The high variance in estimates of the positive effect of pH variation on functional turnover, in turn, most likely prevented its effect on the overall functional dissimilarity (Fig. 7; Table S16). Positive effects of mangrove cover on functional nestedness were counterbalanced by its quite slighter negative effects on functional turnover, culminating in negligible changes in the functional dissimilarity (Fig. 7; Table S16). Likewise, higher functional dissimilarity related to higher distances from the ocean was a primary result of slightly higher functional turnover (Fig. 7; Table S16).

Regardless of stronger landscape influences on functional dissimilarity per annual seasons, quite slight positive effects of mangrove cover on functional nestedness, counterbalanced by its quite slight negative effects on functional turnover, evidenced a decrease in the importance of that alternative habitat from larger to smaller temporal dimensions (Table S18). At the seasonal dimension, human settlements and number of nearby estuaries had positive effects on functional turnover (Fig. 7). These effects were not expressed in functional dissimilarity, which was positively related to the distance from the ocean (Fig. 7). Local environmental variation also had positive influences on measures of functional dissimilarity, with higher functional turnover and nestedness positively related to variations in transparency and pH, respectively, culminating in positive effects of both measures of environmental variation on functional dissimilarity (Fig. 7).

Discussion

Our study revealed different mechanisms driving the taxonomic and functional beta diversity of coastal fish assemblages from seasonal to multiyear periods. Higher temporal dissimilarity from the taxonomic than functional perspective and the primary importance of individual systems for average dissimilarity in species composition and abundance evidenced the role of scale-dependent processes driving taxonomic beta diversity primarily at smaller spatial scales and larger temporal dimensions. Unaccounted processes such as dispersal generating random immigration/emigration, habitat fidelity, and population turnover and generation time may produce local differences in species composition and abundance through time (Di Franco et al. 2012; Lefèvre et al., 2016; Quetglas et al., 2016; Krueck et al., 2020). Likewise, larger temporal dimensions encompass wider ranges of environmental conditions and particular phases of the life cycles of more species, culminating in higher taxonomic dissimilarity (Lacerda et al., 2014; Santos & Zalmon, 2015). Environmental conditions that prevent species without the required functional traits to colonize and persist in local habitats, in turn, support a higher temporal stability in the functional structure of fish assemblages, culminating in lower functional dissimilarity (Villéger et al., 2012; McLean et al., 2019; Gomes-Gonçalves et al., 2020). In this sense, the smaller effect of the hierarchical habitat organization on environmental filtering mechanisms observed in our study is most likely a result of a primary importance of landscape differences between types of system and individual systems, and their stronger influences on functional beta diversity (Camara et al., 2020; Ford & Roberts, 2020). At the same time, the decreasing importance of landscape effects on functional dissimilarity from annual seasons to the 2-year period reinforce the importance of wider local environmental variation distinguishing zones of marine vs. freshwater influence and its consequences for ecologically distinct species over larger temporal dimensions (Santos & Zalmon, 2015; Beukhof et al., 2019).

Regardless of the differences observed from the taxonomic and functional perspectives, in both cases overall beta diversity was as expected higher at larger temporal dimensions. However, different assembly processes and environmental mechanisms supported taxonomic and functional patterns. As expected, balanced variation in species abundances (βBC-BAL) also decreased from the 2-year period to annual seasons, but the observed values were always lower in coastal lagoons than bays, most likely due to the limited marine influence and consequent extreme environmental conditions that prevent the entrance and permanence of most marine-origin fish species. The positive influences of variations in salinity and tidal phase on the balanced variation in species abundances in the 2-year period, associated with strong variations between individual systems, reinforce the importance of spatial differences in the degree of marine influence to determine taxonomic turnover at larger temporal dimensions. Decreasing degrees of connectivity with the adjacent ocean reduce temporal shifts in physical and chemical gradients, and movement of resources and consumers, producing alternative states of equilibrium that limit changes in species composition and diversity in individual systems (Netto & Fonseca, 2017; Franco et al., 2019). Camara et al. (2021) observed a higher contribution of balanced variation in species abundances to fish beta diversity in the Saquarema lagoon (euhaline) compared with Araruama and Maricá lagoons (hyperhaline and mesohaline, respectively), evidencing the importance of higher marine influence to support more suitable alternative habitats for different species (Lacerda et al., 2014; Pasquaud et al., 2015; Netto & Fonseca, 2017). Further evidence in this sense was provided by the strong influence of the type of system on higher functional turnover (βSIM) under higher stability in salinity levels and higher differences in water renewal resulting from tidal variation in the 2-year period.

Functional nestedness (βSNE) and functional turnover had general decreases from the 2-year period to annual seasons and similar contributions to functional dissimilarity (βSOR), which was always lower compared with the taxonomic perspective. Environmental filtering processes may select different functional traits under wider environmental gradients such as the observed during the 2-year period (e.g. bacterioplankton communities from river to bay to open ocean, Fortunato & Crump, 2011; sandy beach macrofauna along salinity and morphodynamic gradients, Barboza et al., 2012; and fish assemblages in estuaries worldwide, Henriques et al., 2017b). These mechanisms most likely explain higher functional turnover at larger temporal dimensions, but the similar contribution of functional nestedness, strongly associated with lower tidal variation, also evidences that loss of functional traits (by removing or limiting entrance of fish traits) is a primary mechanism supporting beta diversity patterns under lower variation in marine influence.

Higher abundance gradients (βBC-GRA) prevailing in coastal lagoons, and the strong negative relationship with tidal variation in the 2-year period reinforce the importance of lower marine influence excluding species without the appropriated suit of traits to withstand local conditions. Further evidence in this sense was provided by the positive relationship between abundance gradients and transparency variation, considering that transparency is typically lower in areas under higher influence of terrestrial landscape due to larger inputs of organic matter from watersheds (Farjalla et al., 2002; Esteves et al., 2008). Previous studies on these fish assemblages evidenced the prevalence of species with higher affinity to freshwater conditions (i.e. semi-diadromous and freshwater species) and higher contributions of abundance gradients to taxonomic beta diversity under lower marine influence (Camara et al., 2020, 2021). In addition, higher distance from the ocean supported higher functional dissimilarity under higher variation in marine influence at all temporal dimensions, revealing a likely consequence of highly counterbalanced processes of turnover and nestedness irrespective of the hierarchical habitat organization. Considering that depth and transparency supported primarily differences between individual systems, and winter temperatures are typically higher in coastal lagoons than bays (Andrade-Tubino et al., 2020), negative effects of variations in depth and transparency, and the positive effect of temperature variation on taxonomic dissimilarity also supported the importance of spatial and temporal variation in marine influence. Therefore, considering that marine-origin species constitute most of the functional space (Fig. 3g; Table S3), our results confirmed that the marine environment is the primary source of species to transitional ecosystems and environmental filtering processes acting over wide environmental gradients interact with species traits by different mechanisms to determine local assemblage structure.

Considering the aforementioned scenario, shorter environmental gradients may explain the negligible effects of local environmental variation on balanced variation in species abundances and abundance gradients per year, as well as the primary influence of landscape features on taxonomic measures at the seasonal dimension. Increasing contributions of abundance gradients from the 2-year period to annual seasons, and its negative relationship with variation in tidal phase at the smaller temporal dimension reinforce the importance of narrower environmental gradients promoting abundance gradients. From the functional perspective, the positive and negative effects of variation in transparency and dissolved oxygen, respectively, on nestedness per year, and its positive relationship with pH variation at the seasonal dimension reinforced consequences of variation in marine influence. As aforementioned, land-based organic matter reduces water transparency, and decomposition processes increases CO2 levels to the detriment of dissolved oxygen, rendering the pH of transitional ecosystems more isolated from the sea typically acid (Esteves et al., 2008). Variation in transparency, dissolved oxygen, and pH may exclude some species due to physiological limitations, promote changes in behavior and activity levels, and increase in predation risk, or be indirect indicators of changes in the availability of feeding resources for opportunistic species (Cattano et al., 2018; Andrade-Tubino et al., 2020). Therefore, even short changes in these environmental conditions may promote short-term changes in functional nestedness primarily due to loss of functional traits under lower marine influence. At the same time, regardless of the decreasing contribution of balanced variation in species abundances from larger to smaller temporal dimensions, higher values associated with higher tidal variation at the seasonal dimension evidenced the importance of the marine influence to promote taxonomic turnover even under shorter environmental variation.

Taxonomic processes of community assembly were related to a larger number of environmental effects in annual seasons. A gradual increase in the number of environmental drivers determining taxonomic dissimilarity from the 2-year period to annual seasons was a consequence of large fractions of the variance associated with individual systems and types of system that were not controlled in models including only the negligible effects of zones. Landscape features strongly associated with the hierarchical structure were then more related to taxonomic dissimilarity. At the same time, environmental effects on taxonomic dissimilarity fully overcoming differences between zones reveal the strength of environmental filtering processes operating through gradients of marine vs. freshwater influence irrespective of the temporal dimension. Models including individual systems and types of system as random effects, in turn, confirmed the expected importance of more environmental gradients supporting higher fish beta diversity at larger temporal dimensions. In this sense, negligible environmental effects on balanced variation in species abundances and abundance gradients per year at least partly also reflect discontinuous environmental influences highly associated with types of system and strong enough to support only overall taxonomic dissimilarities. However, effects of variation in transparency, temperature and depth, and mangrove cover on taxonomic dissimilarity evidenced the importance of marine vs. freshwater influences also in annual cycles. From the functional perspective, equal numbers of environmental effects driving overall dissimilarity at all temporal dimensions, despite a large fraction of the variance associated with zones in the 2-year period, reinforced the importance of environmental filtering processes acting over multiple time periods.

Temporal dimensions promoted more differences in the strength of local and landscape effects on fish beta diversity than the taxonomic and functional perspectives themselves. This scenario reinforces the importance of understanding the temporal dependence of multiple environmental effects shaping coastal fish biodiversity (Araújo et al., 2016; Chin et al., 2018; Camara et al., 2021). Other critical point is the increased role of landscape variables at smaller temporal dimensions, indicative of habitat filtering processes that restrict the species pool at the seascape scale by excluding permanently species not capable of deal with continuous environmental degradation (Henderson et al., 2020; Gomes-Gonçalves et al., 2020). Considering that landscape effects increased to the detriment of local environmental variation, it is also critical to state that local variables considered in our study are an ultimate result of climate and ocean regimes, and their interactions with coastal landscape. Rainfall promotes water runoff, increasing sediment load and input of debris from land to river and then to the ocean (Cordova & Nurhati, 2019; Ranasinghe et al., 2019), whereas tides and other ocean-and climate-related factors, such as wind and ocean currents, may transport and bring land-based pollution from far coastal areas (Baztan et al., 2014). Therefore, the temporal dependence of the balance between local and landscape effects evidences the critical consequences of jointly human-induced changes on climate and coastal landscape. Also, the interchange between major mechanisms supporting local and landscape effects on fish beta diversity evidences that even slight responses may lead to critical environmental changes irrespective to the spatial or temporal scale.

From the taxonomic perspective, the influences of mangrove cover and nearby estuaries evidenced the importance of alternative habitats to promote assembly processes from the 2-year period to annual seasons (Henderson et al., 2021; Camara et al., 2022). A variety of mechanisms, such as ontogenetic, tidal and diel feeding movements may promote connectivity between alternative habitats, which may vary strongly in heterogeneous seascape depending on environmental context (e.g. marine vs. freshwater conditions) and spatial arrangement of habitat mosaic (Martin et al., 2015; van Lier et al., 2018; Bradley et al., 2019). At larger temporal dimensions, the positive effect of mangrove cover on taxonomic dissimilarity was not shared with any component as a most likely consequence of the restricted distribution of mangroves associated with counterbalanced processes of turnover and nestedness. Indeed, the increased relevance of landscape effects at the seasonal dimension revealed higher balanced variation in species abundances associated with lower mangrove cover. Higher taxonomic dissimilarity and abundance gradients positively associated with number of nearby estuaries, in turn, were most likely a result of a large number of species that permanently inhabit estuarine habitats and render abundance-based nestedness more relevant than balanced changes in species abundances (Araújo et al., 2016; Azevedo et al., 2017). Similar relationships may be assumed for mangrove cover, considering that mangroves provide high-quality alternative habitats for estuarine species, and may influence species composition and key functional groups even in fish assemblages far from each other (Martin et al., 2015; Whitfield, 2017). This scenario evidences the critical importance of conservation actions focused on the maintenance of habitat availability, complexity, and variety to fish beta diversity.

Different mechanisms by which loss of habitat complexity affects taxonomic beta diversity were revealed in the 2-year period, based on the negative and positive effects of altered vegetation on balanced variation in species abundances and abundance gradients, respectively. More vegetated areas generally provide more complex habitats and feeding resources based on the input of wood debris and organic matter (Crook & Robertson, 1999; Whitfield, 2017). Therefore, decreasing habitat complexity and changes in water quality resulting from altered vegetation most likely promotes processes of species loss, whereas reduces balanced changes in species abundances due to decreasing habitat quality (Galaiduk et al., 2018; Camara et al., 2022). The importance of these mechanisms was reinforced at the seasonal dimension, with negative effects of altered vegetation, human settlements, and number of industries on taxonomic dissimilarity and balanced variation in species abundances. Therefore, decreasing habitat complexity may reduce the roles of alternative habitats on different phases of life cycles of several species, culminating in lower balanced changes in species abundance at all temporal dimensions (van Lier et al., 2018; Henderson et al., 2020). The negative relationship between native forest cover and balanced variation is species abundances, restricted to the seasonal dimension, in turn, was most likely influenced by the different percentages of cover observed in individual systems. Alternatively, the negative effect of forest cover on the abundance-based turnover may be a primary result of higher temporal stability in fish assemblage structure supported by high-quality habitats in more forested areas, as supported by nearby estuaries and mangrove cover.

From the functional perspective, contrary influences of mangrove cover on turnover (negative) and nestedness (positive), stronger in the 2-year period, but supported at all temporal dimensions, reinforce the critical role of mangroves in shaping fish beta diversity, regardless of its typical restricted distribution (Tables S16, S18; Henderson et al., 2021). The positive effect of mangrove cover on functional nestedness reveals that sites closer to larger extents of mangrove share a larger amount of functional traits, culminating in higher loss or gain of fish traits over the time. Otherwise, higher functional turnover at sites closer to smaller extents of mangrove evidences that local assemblages have more distinct functional structures, and turnover processes govern temporal beta diversity under lower availability of alternative habitats. The negative effect of mangrove cover on balanced variation in species abundances at the seasonal dimension, and positive influence on taxonomic dissimilarity at larger temporal dimensions provide further evidence in this sense. These relationships are even more critical considering the prevalence of functional nestedness over functional turnover per year and seasons, under an overall decrease in all dissimilarity measures, and the importance of mangroves as alternative habitats that provide safer areas for breeding and/or nursery, and sources of food and shelter for numerous species in different phases of their life cycles (Whitfield, 2017; Henderson et al., 2021).

At the seasonal dimension, positive effects of human settlements and nearby estuaries may reveal different mechanisms supporting functional turnover. A higher number of nearby estuaries promote stronger physical, chemical, and biological gradients that support environmental filtering processes selecting different suits of traits for local assemblages in space and time (Henriques et al., 2017b; Andrade-Tubino et al., 2020). A higher percentage of human settlements, in turn, reduce the extent and connectivity between alternative habitats, promoting functional replacement as a primary consequence of habitat fragmentation (Henderson et al., 2020). Therefore, whereas nearby estuaries promote environmental gradients that provide suitable habitats for fish species with different tolerances and requirements, increasing urbanization promotes the rupture of critical trophic links and population processes necessary for the maintenance of fish biodiversity (Massicotte et al., 2015; Vargas-Fonseca et al., 2016). Further evidence in this sense is provided by the higher balanced variation in species abundances associated with nearby estuaries in annual seasons, whereas negative effects of human settlements most likely reflect larger influences of types of system and systems on taxonomic responses. The primary importance of these mechanisms at the seasonal dimension highlights the critical short-term consequences of land-use changes for niche opportunities in transitional ecosystems.

We identified critical environmental drivers of fish beta diversity in transitional ecosystems at important time scales for conservation planning and restoration actions. The taxonomic and functional perspectives provided complementary information for a better understanding of environmental mechanisms supporting fish beta diversity at different temporal dimensions. As expected, the stronger influences of the hierarchical habitat structure on taxonomic beta diversity revealed the importance of stochastic processes driving species colonization and persistence in individual systems and to a lesser extent types of system primarily at larger temporal dimensions. Also, slighter influences of zones on functional dissimilarity and types of system on functional turnover and nestedness evidenced the importance of spatial differences primarily related to the degree of marine influence on functional processes of community assembly. Regardless of such random influences, there was a coincident decrease in taxonomic dissimilarity and balanced variation in species abundances from the 2-year period to the seasonal dimension, whereas decreases in both functional turnover and nestedness supported decreases in functional dissimilarity. These results are in accordance with the expected responses and evidenced the importance of abundance-based and functional turnover processes at larger temporal dimensions, regardless of a greater functional stability. Prevailing effects of local environmental variation promoted such replacement processes as a primary consequence of more environmental gradients supported by variation in marine influence.

Increasing abundance gradients from larger to smaller temporal dimensions, culminating in a negative relationship with variation in marine influence evidenced shorter environmental gradients promoting abundance-based loss of species. Coincident decreases in functional turnover and nestedness, in turn, evidenced the equivalence of such processes irrespective of the seasonal dimension as a most likely consequence of increasing landscape effects to the detriment of local environmental variation from the 2-year period to annual seasons. In this sense, the primary influence of local environmental variation on functional nestedness at the seasonal dimension reinforced the strength of assembly processes based on the loss of fish traits in shorter environmental gradients. Functional and taxonomic processes of turnover primarily associated with landscape variables at the seasonal dimension, in turn, revealed the importance of the temporal balance between local and landscape effects for beta diversity patterns. Therefore, contrary to expected, local and landscape effects varied more through temporal dimensions than between taxonomic and functional perspectives. Altogether, our results revealed that jointly human-induced changes on climate and coastal landscape may determine definitive ruptures in critical interchanges between multiscale processes driving fish beta diversity, with catastrophic consequences for coastal biodiversity. Future investigations focused on disentangling the roles of spatial and temporal variations in landscape features at larger temporal intervals may thus better clarify the mechanisms herein described and provide further support to conservation and management of fish biodiversity in transitional ecosystems.