Introduction

In recent years, indicators have gained increasing attention for short- and long-term monitoring in environmental science, biodiversity conservation, and fisheries management at the world, national and local scales. Issues include, among many other topics, the study of temporal change and ecological processes, the promotion of public environmental awareness, and the measurement of the effects of management interventions on biodiversity erosion or resource abundance (e.g., Degnbol 2005; Jennings 2005; Pomeroy et al. 2005). In developing countries where financial and technical capacities are limited, conventional environmental monitoring programs have generally been designed and conducted by research organizations, Non-Governmental Organizations (NGOs) or foreign specialists. Such approaches have often used detailed surveys that (1) are costly and time consuming, (2) need highly skilled scientists, (3) have paid insufficient attention to local needs and constraints of managers, and (4) proved barely sustainable in the long term (Sheil 2001; Danielsen et al. 2003).

Since the 1990s, participatory monitoring has received greater attention as an alternative way to involve local stakeholders in effective management and strengthen the decision process (Olsson et al. 2004; Danielsen et al. 2005a). Furthermore several studies have pointed out that valuable fishing, biological and taxonomic data could be collected by local people who lack formal higher-level education, through careful training and collaboration (Obura 2001; Tawake et al. 2001; Sheil and Lawrence 2004). The body of indigenous knowledge accumulated in traditional communities definitely shows their established ability to “see” change in their environment as a matter of survival (Spelleberg 2005). However, stakeholder willingness to contribute to monitoring programs depends on a combination of criteria which determine whether or not their investment of time and money is worthwhile. In this respect cheap and easy-to-use methods are more likely to be adapted in developing countries where logistical resources are scarce, dependency on marine resources is reduced by alternative sources of food and/or income, or where the value for fish is low (Hockley et al. 2005).

Using non-professional trained volunteers in fieldwork has been considered as another rapid and cost-effective means of collecting broad-scale information on the environment (e.g. Schmitt and Sullivan 1996). NGOs and international partnerships have proposed volunteer monitoring schemes of coral reef ecosystems that rely on students, local and non-resident volunteers, international paying volunteers or recreational divers (Hill and Wilkinson 2004; Hodgson et al. 2006; www.coralcay.org; www.icriforum.org; www.reefcheck.org; www.gcrmn.org).

Despite their use and promotion worldwide, few of the community-based or volunteer-based underwater monitoring methods have been closely examined in terms of the precision and the accuracy of the sample data. To the authors’ knowledge, no comparative study has specifically analyzed the statistical reliability of such approaches for fish populations. The available studies that compared fish observations made by non-scientific and highly qualified surveyors focused on the impact of the observers’ level of experience on species identification and counts (Williams et al. 2006; Darwall and Dulvy 1996), or provided encouraging overall results on participative surveys (Uychiaoco et al. 2005). Their ability to highlight ecological shifts in marine population dynamics remains unknown (Danielsen et al. 2005b). Confronting this issue would provide the opportunity to improve the design of such non-scientific monitoring programs. A collaborative study was carried out in Navakavu (Viti Levu Island, Fiji) in November 2006 to address this issue. The aim was to determine the strengths and the weaknesses of the ongoing biological monitoring program of the Navakavu marine area, and to improve the quality of the information that is useful to the community to assess the sustainability of its management plan. The study compared three underwater reef fish monitoring methods that used (1) community members, (2) non-resident volunteers, and (3) scientists. The three methods were tested on the same transects inside and outside the locally managed no-take zone (NTZ). The means and the variance of the scientific estimates of fish abundance were compared to the community-based and the volunteer-based data. The potential bias and the precision of the sample results were calculated because of their relevance to the statistical power of monitoring schemes. The variability generated by visual census designs can indeed make the detection of perturbation effects in the marine environment difficult, particularly in small marine reserves (Mouillot and Culioli 2002). The ability of the three methods to detect Navakavu NTZ effects was then discussed.

Methods

Study area

Under customary marine tenure the 410 Fijian customary fishing grounds (qoliqoli) delineate areas where the inhabitants of certain villages have exclusive fishing rights and a responsibility for their protection (Cooke et al. 2000). The site of Navakavu is located in Suva District close to the main urbanized zone of the country (Fig. 1). High human pressure on the reef and lagoon area due to high population density in the neighbourhoods, easy access to the capital’s fish market and heavy fishing practises has led to the depletion of marine resources (Jennings and Polunin 1996). Since 2003 Navakavu qoliqoli has been part of the Fijian Locally-Managed Marine Area (LMMA) network that aims to prevent overexploitation and increase fish stocks. A community-managed NTZ (tabu area) was implemented which encompasses various geomorphological coral zones from the sandy accumulation area to the reef crest (Fig. 1).

Fig. 1
figure 1

Map of the boundaries of the Navakavu customary fishing rights area (qoliqoli) in Viti Levu, Fiji Islands. About 20% of fishing grounds were closed in 2003 when a permanent no-take zone was implemented. Circles indicate the 59 stations that were surveyed by underwater visual census (UVC) during the present study

Full size image

Survey design

The data provided on focal taxa from three underwater fish monitoring methods were compared.

  1. 1.

    Community-based data were gathered by two members of the monitoring team of Navakavu qoliqoli. The formal educational background of these two people ranged from primary school to high school. They were elected by the community to take a leading role in biological monitoring and to report back to the qoliqoli committee the findings from their NTZ without any monetary incentive. Indeed the University of the South Pacific has trained and assisted several villagers with no formal scientific background to carry out annual fish and invertebrate surveys as part of the LMMA management plan (Cakacaka et al. 2007). It has also provided frequent follow-up and assistance with data analysis. Thus the local observers involved in this study were comfortable with UVC. Vernacular names were used because in such contexts relying on local knowledge may be more effective than training local people to learn scientific denominations. The local monitoring plan made use of five taxonomic groups that are usually targeted by local fishermen for food and sale (Table 1). These taxa referred to a monophyletic (family and single genus or species) or polyphyletic (for example groups of similar-looking species) taxonomic group of scientific species.

    Table 1 Focal taxa recorded by (a) the community-based, (b) the volunteer-based, and (c) the scientific methods
    Full size table
  2. 2.

    Four outside volunteers from different public organizations and a Fijian NGO conducted the second non-scientific survey. They had a higher educational level (Bachelor or Master of Science) than the previous team. This volunteer-based approach used the target fish list from the Reef Check methodology for the Indo-Pacific (Table 1). Selection of these species was based on their economic and ecological value, their sensitivity to overexploitation and ease of identification across the Indo-Pacific (Hodgson et al. 2006). These volunteers were already trained in UVC and identification of key species of fish.

  3. 3.

    The scientific team had the highest level of expertise. It included three scientists and trainers in fish taxonomy and UVC from two research institutes. Fish identification was undertaken at the species level across 12 families (Table 1). Data was aggregated at a higher taxonomic level for comparison to both community-based and volunteer-based schemes. All other environmental variables being equal, it was assumed that the collected data would provide the best relevant population value through greater accuracy and precision (e.g., Mumby et al. 1995; Nicholson et al. 2002). Observations by scientists were thus considered as reference values for abundance of the observed fish taxa.

Sampling protocol

Intensive spatial sampling was used to increase precision and accuracy of abundance estimations: a representative sample of 59 stations (29 UVC stations inside the harvest area and 30 stations inside the NTZ) was randomly localized within the Navakavu LMMA using a Geographic Information System (Fig. 1). Each station was surveyed once by each method in a random order at high tide during a 2-day field trip in November 2006. Permanent belt transects of 50 m length and 2.5 m width on each side were consistent across methods and were marked by ropes. Fish larger than 5 cm in length were recorded by a snorkelling census. The snorkeller speed along each transect was about 5 m.min−1. Consecutive counts on transects were separated by one to 2 h to minimize the influence of the presence of observers (Thompson and Mapstone 2002). A minimum distance of 50 m between stations was established to remove problems associated with biotic variables interdependence. The NTZ and control sites were selected to be as similar as possible in environmental conditions. The substrate of the whole area was characterized before the survey: it was composed of a patchy distribution of live massive corals, rubbles, sand, and small sea grass beds. Survey sites were restricted to the back-reef area at a depth between 1.5 and 2.5 m. It was very likely that no detectable change had occurred within the NTZ and the harvest area during the short interval between the three surveys. The current survey assumed that major differences in fish abundance between the zones were attributable to management status like a number of comparative between-site studies of NTZ effects (McClanahan et al. 2006; Denny and Babcock 2004).

Statistical analysis

The average densities, the coefficients of variation (CV) and the 95% confidence intervals of target taxa were estimated for each method. The CV was considered as a measure of the achieved precision of the estimate and thus of indicator performance (Trenkel and Rochet 2003). Statistical testing was restricted to nonparametric techniques due to right-skewed distributions of fish abundance. Rank-based methods also allow the weight of rare outlying counts to be lowered, and are not sensitive to zero values. The correlation between scientific, community-based, and volunteer-based abundance estimates was determined pair by pair over the whole area using Kendall’s coefficient of rank correlation. The possible bias of the last two methods was investigated through a two-tailed Wilcoxon matched pair signed-rank test for significant differences between scientific and non-scientific mean abundances. The precision of these estimates was compared by a log-ANOVA test of equality of variances conducted on the whole set of stations. This test was assumed to be robust to the violation of normality.

For each method, mean abundances of focal taxa were compared inside and outside the NTZ by two-sided Mann–Whitney rank sum tests. These significance tests allowed the assessment of the ability of each method to provide evidence of NTZ effects on fish abundance in Navakavu. Systat and R software were used for statistical analysis.

Results

Fish abundance

Scientists recorded 56 species belonging to 12 families. About 85% of them were found less than once per transect: Acanthuridae (mainly Acanthurus triostegus and Ctenochaetus striatus), Chaetodontidae (Chaetodon citrinellus, C. vagabundus), Labridae (Chlorurus sordidus), Lethrinidae (Lethrinus harak, L. obsoletus), Siganidae (Siganus spinus) and Scaridae (Leptoscarus vaganiensis) were the most widely distributed fish. The locally selected target taxa were present on 85% of the transects (97% inside the NTZ) whereas a large number of the taxa from the Reef Check list were very rarely or even never recorded (only Chaetodontidae and Scaridae >20 cm were observed in more than one transect out of ten). Data were aggregated at identical taxonomic levels for comparison across methods. This was not possible for five out of eight taxa from the Reef Check list either because they did not occur in the survey area or because only larger fish were recorded.

There were significant correlations of abundance estimates between the scientific and both other methods (Kendall rank correlation coefficient, P < 0.05; Table 2; Fig. 2). Results also showed that the voluntary approach led to consistent results with scientific estimates in terms of the mean and variance of fish abundance (Wilcoxon and log-ANOVA tests, P > 0.05 for all compared taxa—Table 2). However, community-collected data contrasted markedly with scientific counts (Wilcoxon test, P < 0.05 for all taxa—Table 2). The abundance of the five focal taxa was significantly overestimated in the no-take zone, as well as that of Scaridae and Chaetodontidae in the harvest area (Fig. 3a, e). The overestimation rate was systematically higher inside the NTZ: it ranged between 1.27 and 1.62 in the harvest area (except for Scaridae: 5.11) and between 1.68 and 5.35 in the NTZ. This was not significantly linked to fish number (Kendall coefficient = 0.022, P > 0.99) but there was some indication of marked overestimation rates among less abundant taxa (around three fish per transect and fewer). The locally based method also gave high variability and thus low precision of estimates. The log-ANOVA test showed that the variance of the five recorded taxa was significantly higher than the variance of scientific observations (Table 2).

Table 2 Comparison of fish counts between the scientific monitoring method and the community-based and volunteer-based methods in the whole survey area
Full size table
Fig. 2
figure 2

Mean abundance of main fish taxa estimated by the non-scientific and the scientific monitoring methods. Each taxa is presented twice (inside and outside the Navakavu no-take zone). Error bars indicate 95% confidence intervals. Open squares: community-based monitoring (n = 10, R 2 = 0.547). Solid triangles: monitoring based on non-resident volunteers (n = 6, R 2 = 0.960)

Full size image
Fig. 3
figure 3

Comparison of abundance estimates of main fish taxa (graphs a–f) between the three monitoring methods inside and outside the Navakavu no-take zone (NTZ): (1) scientific monitoring, (2) community-based monitoring, (3) monitoring based on non-resident volunteers. Statistical differences in fish abundance between the NTZ and the harvest area are given (Mann–Whitney two-sided rank sum tests). Error bars represent the 95% confidence intervals of the means. (1) Lethrinus harak, L. lentjan, L. obsoletus, L. rubrioperculatus only (vernacular name: Kabatia); (2) Scaridae larger than 20 cm only; (3) Serranidae larger than 30 cm only; (4) The number of recorded taxa differed across methods

Full size image

For each method there was a strong linear association between mean fish abundance per transect and its standard deviation: highly imprecise estimates were observed for the most abundant taxa in Navakavu (Fig. 4). The tendency for community members to overestimate the number of fish on the transect lines resulted in a large range of abundance estimates in the sample results and increased the variability of their records. Coefficients of variation ranged between 0.83 and 2.30 and between 0.76 and 1.15 outside and inside the NTZ, respectively. Scientific data showed that CVs were lowered by aggregating observations on species abundance at the family level.

Fig. 4
figure 4

Mean and standard deviation (SD) of fish abundance estimated by the three monitoring methods. All focal taxa are presented (see Table 1 and text for details). Trends in SD are explained using linear functions for scientific monitoring (circles and solid line, n = 20, R 2 = 0.687), community-based monitoring (open squares and dotted line, n = 10, R 2 = 0.880), and monitoring based on non-resident volunteers (open triangles and dotted line, n = 9, R 2 = 0.936)

Full size image

NTZ effects

The three methods consistently showed higher overall abundances of fish inside the NTZ than outside (Mann–Whitney test, P < 0.05—Fig. 3f) suggesting that the Navakavu NTZ had positively impacted on fish resources.

Scientific counts revealed strong differences for only six species including A. triostegus and Chaetodon rafflesi (P < 0.01); Chaetodon vagabundus, Centropyge flavissimus (Pomacanthidae), Epinephelus merra and Epinephelus hexagonus (Serranidae) (P < 0.05). All differences at the species level were apparent at the family level. On the contrary there was no statistical difference in abundance of any of the five species of Scaridae, whereas this family was significantly more represented in the NTZ (P < 0.05). Species richness was also much higher in the NTZ (P < 0.01) with a mean of 8.6 species per transect (against 5.4 in the harvest area).

Volunteer-based data on Chaetodontidae and Lutjanidae densities (out of nine species on the Reef Check fish list) showed that they were significantly higher in the NTZ than in the area open to fishing. The difference in the abundance of Lutjanidae was not detected by the scientists.

We also found contradictory results on NTZ effects between scientific and community-collected data on Lethrinidae (kabatia) and Mullidae (Ose) (Fig. 3b, d). Moreover the community-based conclusions on the NTZ effect on the three other LMMA focal taxa were more striking than those suggested by the scientists (Fig. 3a, c, e). Consequently the range of differences in fish density between the no-take and the harvest zones was very likely overemphasized by this method.

Discussion

The present study allowed us to compare three monitoring methods in terms of relevance, precision, and reliability. Underwater visual censuses were standardized in terms of environmental constraints, survey area dimensions, diving equipment and observer swimming speed to make it possible to compare fish records across methods.

Focal taxa

Large-scale monitoring programs based on standardized protocols and inflexible indicator species largely aim to track global threats and may not be relevant for detecting changes in specific areas (Buckland et al. 2005). This is because local environmental conditions (e.g., reef geomorphology, habitat, and depth) and human activities (e.g., fishing, urbanization) shape the dynamics and the demographic structure of fish populations (species presence/absence, abundance, size patterns, etc.,). In this study the locally based focal taxa were regularly recorded on the survey transects and were included in the Navakavu monitoring plan. On the contrary the Reef Check list for the Indo-Pacific was obviously inadequate for the environmental characteristics of the survey area (seven out of nine taxa were observed in no more than six stations). The results confirmed that focal taxa should be selected in relation to the local ecological and socio-economic context including management objectives and expected effects (Clua et al. 2005; Pelletier et al. 2005). Such taxa can be suitable for sentinel monitoring and for providing insights about cumulative changes in marine resources.

Reliability of density estimates

Scientific and volunteer-based data were consistent in terms of means and variance of density estimates. This comparison was limited to two taxa due to the above limitation and thus did not allow the quality of the data provided by the divers using the Reef Check list of fish to be fully explored. A further survey is needed in a more appropriate reef environment to be able to conduct a cross-validation that is as exhaustive as for the community-based approach.

The community-based method showed a marked lack of accuracy and precision: it returned strong overestimates and variability of abundance for each recorded taxon. The systematic overestimates drove the precision of the community-based data as the variance and the mean of non-normally distributed data tend to increase together. Moreover it varied from one transect to another, which would undermine the comparison of abundance estimates across time or space. This source of variability due to measurement error would reduce the statistical power of the Navakavu monitoring program. Some calibration is still required before general trends in abundance can realistically be depicted. Three sources of error may be considered.

Recording techniques as well as behavior and mobility of fish species strongly influence the precision and accuracy of UVC data (Samoilys and Carlos 2000; Tessier et al. 2005). For example roving and schooling species tend to be harder to count than conspicuous, immobile, and solitary fish (Cheal and Thompson 1997; Williams et al. 2006). Such patterns could partly explain the differences in density estimates that were observed between methods in our study. Indeed the error was more pronounced for Scaridae, Mullidae, and Lethrinidae than for Chaetodontidae and Serranidae.

Educational background also proved to markedly affect record quality. Double-counting of fish and records beyond the 5 m width confines of transects have previously been mentioned in participative approaches (Uychiaoco et al. 2005). The UVC design may be transformed to adapt the latter recording error: transect width can be increased according to habitat complexity and visibility. This would mathematically decrease density estimates when divers count fish outside the corridor. Stationary diver techniques may also be preferred over strip transects as they are less dependent on water clarity and fish behavior (Lang and Baldwin 1996). Fishing measurements should also be explored to evaluate the effect of the Navakavu NTZ on fish resources. Indeed local harvesters can profitably gather information on resource status through their daily catch and yield. A population-monitoring index may be directly defined from their landings and fishing effort (Moller et al. 2004) or indirectly from their perception of trends in abundance (Neis et al. 1999; Webb et al. 2004). By definition, such user-based indices could address recurrent problems of UVC such as fish detectability and identification, and count and length estimates in the field. However, the monitoring of fishing activities in Navakavu would face the recurrent limits of fishing dependent indicators (e.g., catch per unit of effort) such as gear selectivity and efficiency, effort measurements, geographical bias and catch reporting (Kleiber and Maunder 2008). New investigations would thus be needed to assess the reliability of such parameters to track changes in the resource status in the Navakavu LMMA.

Local divers participated in the survey with determination and built constructive collaboration with both scientific and voluntary partners. However, the higher overestimation rate of community-based data inside the Navakavu NTZ may come from deliberate upward bias to demonstrate NTZ performance. Indeed the people involved in the monitoring process were in a position to provide biased data that support their views on management actions. Such strategic behavior has been noticed where local people obtain opportunities (monetary incentives, recognition, proudness, etc.) by welcoming outsiders like researchers (Sheil and Wunder 2002). Our collaborative survey brought substantial financial input to the community and assessed the effectiveness of both the local monitoring plan and the NTZ, so such a risk of bias was likely to be present. Our results suggested that diver bias and the rigorousness of the recording process should be carefully measured even when simple monitoring methods are used.

Detection of NTZ effect

The concerns about focal taxa and data quality had impacts on the ability of the respective monitoring methods to assess the effectiveness of the Navakavu community-managed NTZ on fish abundance.

Aggregate metrics have been proposed for describing the fish community structure or to account for the uncertainty in species identification and the use of polyphyletic vernacular names (e.g., Obura 2001; Rochet and Trenkel 2003; Graham et al. 2005). The strengths and weaknesses of such indicators for short-term management have been discussed in terms of sensitivity to fishing impacts and ease of interpretation (Rice 2000; Nicholson and Jennings 2004). Our study pointed out that aggregate taxa were more than or as efficient as species-based indices for detecting Navakavu NTZ output. By increasing the size of the fish population and the occurrence of the focal taxa in UVC stations, family-based estimates would decrease count variability and CVs, and thus enhance the detection of NTZ effects (Maxwell and Jennings 2005). The composite taxa of the Navakavu biological monitoring protocol should therefore be appropriate. Nevertheless some polyphyletic (e.g., kabatia which groups four species of Lethrinus) and length-limited (e.g., Scaridae larger than 20 cm) taxa did not exhibit significant differences in abundance. Such indices for exploited species have often been recognized in the literature as effective criteria for NTZ monitoring but may operate on a longer temporal scale.

The three UVC methods highlighted a major difference in the overall density of fish between the protected and the non-protected areas. However, the scientific and the community-based approaches gave contradictory results on two major groups of target fish in the qoliqoli: no statistical difference was found by the scientific monitors for the Ose and Kabatia taxa. Consequently the community-based method resulted in misinterpretation of the effectiveness of the Navakavu no-take zone and overly optimistic conclusions on its impact on fish resources. The inconsistent results that were found for Lutjanidae density between volunteer-based and scientific values were more likely due to the casual occurrence of Lutjanus spp. in Navakavu rather than misestimation by divers. As mentioned above, the ability of monitoring surveys to detect a change in abundance of such rare taxa is indeed usually low.

Sampling design

UVC-based participative monitoring programs have often been conducted with low spatial replication due to financial and logistical limitations. In Navakavu qoliqoli the Fijian LMMA network has promoted the monitoring of fish in the back reef area: eight stations have been surveyed annually by snorkellers inside the permanent NTZ and the adjacent harvest zone since 2004. Our study highlighted that spatial heterogeneity due to habitat patchiness highly impacted the distribution of fish abundance both inside and outside the NTZ. Given the linkage between the sample size and the power of statistical analysis, the actual sampling rate in the Navakavu marine area appears to be insufficient to provide statistically valid information on the expected NTZ effects on the abundance of focal species. In the context of limited resources for monitoring, the sampling effort may be reduced by spatial stratification: a detailed description of the reef habitats would allow the allocation of the sample to specific habitats that characterize the NTZ and the harvest area. The use of few stations may therefore not be incompatible with the monitoring of NTZ effects if they are located in carefully selected sites.

Conclusion

The present study compared three underwater monitoring methods of reef fish in terms of data quality and the ability to highlight changes in fish abundance between the locally managed no-take zone and the fished area. We found that scientific and volunteer-based data were consistent in terms of means and variance of density estimates but that taxa used for the latter method would not be suitable for monitoring of the Navakavu marine area. On the contrary, the locally selected target taxa were appropriate but strong upward bias and imprecision were evident in the fish counts made by the community members. Calibration of community-based UVC is thus mandatory especially if the information on fish densities is used to guide management decisions. Indeed this method made the interpretation of the performance of the Navakavu NTZ confusing due to overestimation of fish abundance principally inside the protected site. The sampling design of the LMMA biological monitoring program also raised some concerns because of the spatial heterogeneity of habitats and the activity of the fish themselves. These limitations need to be weighed against the well-established importance of local participation and low cost in achieving sustainability of NTZs and their monitoring (Table 3). However, this paper shows that locally based UVC approaches need to take into account specific risks of bias due to the technique itself, the sampling effort and diver impartiality. Addressing this issue would contribute to building a form of participatory research.

Table 3 Comparison of the value of different qualitative criteria across the three monitoring methods
Full size table