Introduction

Turbidity is a physical form of water pollution that results from the presence of suspended particles that make the water appear cloudy. The level of turbidity is directly proportional to the amount of suspended particles present in the water, as higher concentrations of such particles will lead to decreased water transparency and increased turbidity. Turbidity removal is a part of wastewater treatment, carried out to improve the water quality for drinking or reuse in aquaculture systems (Nisa and Kishor 2021). Numerous industries, including aquaculture, discharge effluent that contains high levels of turbidity into receiving ecosystems without undergoing appropriate treatment, resulting in eutrophication in water bodies (Kurniawan et al. 2021a, b). Aquaculture effluent that contains high levels of turbidity typically contains significant amounts of nutrient-rich solids that can be converted into beneficial biofertilizers for use in agriculture. As a result, it is essential to employ suitable treatment methods for the removal of turbidity to promote sustainability and optimize the potential value of resources derived from aquaculture wastewater (Igwegbe et al. 2021).

An effective approach for treating aquaculture wastewater involves physical processes that incorporate aerobic/anaerobic procedures such as microalgae culture and macrophyte cultivation, as well as sand-mechanical sieving (Pfeiffer et al. 2008; Mao et al. 2021). Despite their effectiveness, the physical processes mentioned above possess certain drawbacks, such as high energy consumption, sludge generation, and the need for regular maintenance. Consequently, the coagulation–flocculation–sedimentation method is currently the most widely used approach for the treatment of aquaculture wastewater (Galloux et al. 2015). The coagulation–flocculation–sedimentation method employs coagulants to facilitate the combination of insoluble particles or dissolved organic matter into larger aggregates that can be readily removed through sedimentation (Renault et al. 2009). The commonly used chemical coagulants, such as iron (Fe) and aluminium (Al) based salts, are effective at removing various types of pollutants present in wastewater. Nevertheless, this method produces substantial amounts of non-biodegradable sludge, which is toxic to aquatic organisms and can disrupt the pH stability of water bodies (Kurniawan et al. 2021a, b). As a consequence, the utilization of natural coagulants is deemed preferable.

Numerous natural coagulants derived from various sources, such as plants, seeds, shellfish, and crustaceans, have been studied for their potential application in wastewater treatment. These coagulants are reported to be biodegradable, non-toxic, and environmentally friendly (Choudhary and Neogi 2017; Ang and Mohammad 2020). Among the extensively studied plant-based coagulants are Moringa oleifera, Hibiscus rosa-sinensis, Cicer arietinum, and Opuntia ficus-indica (Awang and Aziz 2012; Abdul Hamid et al. 2014; Wan et al. 2019; Dadebo et al. 2022). M. oleifera was found to remove 96.23% turbidity in low turbid water (Ali et al. 2010), H. rosa-sinensis leaf extract removed 86% turbidity (Nidheesh et al. 2017), while C. arietinum removed 95.89% turbidity (Asrafuzzaman et al. 2011). The wide availability of plant-based sources with coagulant properties indicates their potential to replace chemical coagulants in wastewater treatment.

Chitosan, a polycationic polymer extracted from seafood processing waste, has recently gained attention among researchers as a crustacean-based coagulant (Hu et al. 2013). Despite being underutilized, chitosan offers advantages such as availability, high compatibility, and ease of chemical modification (Chik et al. 2022). Its coagulation and flocculation properties have been demonstrated in removing turbidity (Hu et al. 2013; Soros et al. 2019; Okolo et al. 2021). However, the effectiveness of the process is dependent on variables such as initial pollutant concentration, chitosan dosage, pH, sedimentation time, and the optimization of these variables to maximize turbidity removal (Ahmad et al. 2022). The objective of this research is to investigate the impact of chitosan on the process variables and optimize the coagulation and flocculation process to reduce turbidity in aquaculture wastewater by employing RSM. Initially, an assessment of turbidity removal was conducted on kaolin clay suspension as an artificial turbid water and real aquaculture wastewater, using OVAT analysis to determine the expected range of variables for BBD.

Materials and methods

Materials

Artificial turbid water was prepared by diluting 5 g of kaolin powder into 1 L of distilled water and stirred at 300 rpm for 20 min resulting in the initial turbidity of 800 NTU (Zakaria Djibrine et al. 2018). The initial turbidity was measured using turbidity metre (Apera Instrument TN400, USA). Aquaculture wastewater was collected from an intensive shrimp pond, located at Bachok, Kelantan, Malaysia (5.94302° N, 102.24775° N) where a small-to-medium scale aquaculture is carried out for the local market. The sample was transferred from sampling location to the laboratory by using polyethylene bottle. Subsequently, the collected wastewater was cooled at ambient temperature before being stored at 4 °C. Table 1 lists the characteristics of the aquaculture wastewater.

Table 1 Characteristics of aquaculture wastewater from an intensive shrimp aquaculture pond, located at Bachok, Kelantan, Malaysia

The turbidity level of the aquaculture wastewater was found to be 72 ± 7.02 NTU, which exceeded both Standard A and B values of 50 mg/L. Similarly, the total suspended solids (TSS) in the wastewater, with a value of 62 ± 9.07 mg/L, exceeded Standard A but did not exceed Standard B, which is set at 100 mg/L. The elevated levels of turbidity and TSS observed in the aquaculture wastewater were anticipated, given the abundance of nutrients originating from the shrimp excrement and leftover food. For the purposes of this study, chitosan was utilized as a coagulant, and was extracted from the shells of Litopenaeus vannamei, with a degree of deacetylation (DDA) of 84.08 ± 1.27%. To prepare the chitosan, it was dissolved in 1% acetic acid and stirred until fully dissolved, following the method of Cheng et al. (2020).

Assessment of turbidity removal using OVAT analysis in kaolin clay suspension

Effect of the initial turbidity concentration

To evaluate the performance of chitosan coagulant on varying initial concentrations of turbidity, OVAT analysis was conducted. A total of 6 initial turbidity levels (10, 50, 200, 400, 600, and 800 NTU) were prepared by diluting a kaolin suspension stock of 800 NTU with distilled water. The process was carried out in a 1000 mL beaker with a total volume of 500 mL, operating at a rapid mixing speed of 180 rpm for 3 min, slow mixing speed of 50 rpm for 30 min, and sedimentation time of 25 min using a flocculator (JLT6 Velp Scientifica, Malaysia) (Adnan et al. 2017). A chitosan dosage of 0.5 mg/L was added without any alteration to the pH range of 6.5–7.0. The final turbidity of the supernatant was measured using a turbidity metre (Apera Instrument TN400, USA) after extracting 20 mL of the supernatant at a depth of 0.02 m for each beaker. The turbidity removal was calculated using Eq. (1) based on Tong et al. (2021):

$${\text{Turbidity}}\;{\text{removal(}}\% ) = \frac{{{\text{Initial}}\;{\text{turbidity}}\;\left( {{\text{NTU}}} \right) - {\text{Final turbidity}}\;\left( {{\text{NTU}}} \right)}}{{{\text{Initial}}\;{\text{turbidity}}\;\left( {{\text{NTU}}} \right)}} \times 100\%$$
(1)

Effect of the chitosan dosage

The assessment of turbidity removal on various chitosan dosages was conducted through OVAT. A total of 8 dosages variations (0.5, 1, 5, 10, 15, 20, 25, and 30 mg/L) with 0 mg/L as control were tested. The experimental setup involved the use of a 1000 mL beaker with a total volume of 500 mL, operated at a rapid mixing speed of 180 rpm for 3 min, slow mixing speed at 50 rpm for 30 min, and sedimentation time for 25 min, with an initial turbidity of 200 NTU at a neutral pH range of 6.5–7.0 (Bina et al. 2014; Saritha et al. 2015).

Effect of the kaolin clay suspension pH

To assess the turbidity removal at different pH levels, an OVAT was conducted using six pH variations (5, 6, 7, 8, 9, and 10). The pH was adjusted with 1 M NaOH and 1 M HCI. A 1000 mL beaker with a total volume of 500 mL was operated at a rapid mixing speed of 180 rpm for 3 min, slow mixing speed at 50 rpm for 30 min, sedimentation time for 25 min, initial turbidity of 200 NTU, and chitosan dosage of 1 mg/L (Bina et al. 2014; Saritha et al. 2015).

Assessment of turbidity removal using OVAT analysis in aquaculture wastewater

Following the assessment of turbidity removal on the kaolin clay suspension, the chitosan dosage was tested on aquaculture wastewater using 8 variations of dosage (0.5, 1, 5, 10, 15, 20, 25, and 30 mg/L) with 0 mg/L as control. A 1000 mL beaker with a total volume of 500 mL was operated with a rapid mixing speed of 180 rpm for 3 min, slow mixing speed at 50 rpm for 30 min, sedimentation time for 25 min, with the original aquaculture wastewater pH and initial turbidity (Chung 2006; Bina et al. 2014). After sedimentation, a 20 mL volume of the supernatant was extracted from each of the beakers at a depth of 0.02 m using a syringe, and the turbidity of the supernatant was measured using a turbidity metre (Apera Instrument TN400, USA). The calculation of turbidity removal was done using Eq. (1) based on Tong et al. (2021).

Statistical analysis for OVAT analysis

One-way analysis of variance (ANOVA) was used to perform correlation analysis and obtain correlation factors and their respective responses. Post hoc test using Turkey HSD was carried out to determine significant differences in the reported results. Minitab 17 software was utilized for all statistical analyses. A confidence interval of 95% (α = 0.05) was used for drawing conclusions, with p-value < 0.05 indicating significant differences in the results (Ahmad et al. 2021).

RSM for turbidity removal in aquaculture wastewater

In order to model and optimize the factors that influence the coagulation process for efficient turbidity removal in aquaculture wastewater, RSM based on BBD was used after assessing the highest turbidity removal using OVAT analysis in kaolin clay suspension and real aquaculture wastewater. The coagulation–flocculation process parameters, namely chitosan dosage (A), pH of aquaculture wastewater (B), and sedimentation time (C), were investigated. Table 2 shows the range and levels of the process variables. To minimize the effects of uncontrolled variables on responses, a randomized experimental design matrix with 15 runs (with three centre points) was generated, as shown in Table 3. The experiment design was analysed using Design of Expert 13 software, and ANOVA was performed to validate the competence of the quadratic model. The different indicators, such as f-value, p-value R2 and adjusted R2 were considered based on Inam et al. (2021).

Table 2 Independence factors and their coded value
Table 3 Design matrix and BBD experimental and predicted results for turbidity removal in aquaculture wastewater

According to Ezemagu et al. (2021) and Onoji et al. (2017), a second-order polynomial equation was developed to describe the relationships between the process independent and dependent variables. The experimental design was based on this equation, and the quadratic equation model for predicting optimal conditions can be expressed as Eq. (2).

$$Y={\beta }_{o }+{\sum\nolimits }_{i=1}^{k}{\beta }_{i}{X}_{i} + {\sum\nolimits }_{i=1}^{k}{\beta }_{ii}{X}_{i}^{2} +\sum \sum_{i<j}{\beta }_{ij }{X}_{i} {X}_{j}$$
(2)

where Y represented response variables to be modelled, \({X}_{i}\) and \({X}_{j}\) are independent variables (A, B, C), \({\beta }_{o}, {\beta }_{i}, {\beta }_{ii}\) and \({\beta }_{ij}\) are regression coefficients for intercept, linear, quadratic, and interaction coefficient, respectively.

Results and discussion

Assessment of turbidity removal using OVAT analysis in kaolin clay suspension versus aquaculture wastewater

Chitosan was assessed for its performance in removing turbidity in synthetic high turbid water by using kaolin clay suspension. Kaolinite is the substance that is commonly used to create turbidity in test waters for coagulant testing (Soros et al. 2019). Chitosan was tested in OVAT analysis for initial turbidity concentration, pH of kaolin clay suspension and chitosan dosage. Turbidity removal for each variable analysis is illustrated in Fig. 1.

Fig. 1
figure 1

Kaolin clay suspension’s turbidity removal in OVAT analysis of A initial turbidity concentration, B chitosan dosage, and C pH of kaolin clay suspension. Different letters (a-b-c-d-e) indicate significant different in turbidity removal among tested variables

The results presented in Fig. 1A indicate that chitosan exhibited a low turbidity removal (13.3 ± 2.03%) in low turbidity water, whereas turbidity removal increased as the turbidity concentration increased up to 200 NTU (69.92 ± 0.44%). However, increasing the turbidity concentration beyond 200 NTU did not significantly contribute to turbidity removal. This finding suggests that chitosan is better suited for application in high turbidity water. This result is consistent with previous studies, which have reported that biocoagulant/flocculant shows greater turbidity reduction as the initial turbidity concentration increases from low to very high levels (Asrafuzzaman et al. 2011; Abdul Hamid et al. 2014; Bina et al. 2014; Gaikwad and Munavalli 2019). The higher particle concentration in high turbidity water increases the frequency of collision between particles and coagulant, enhancing the floc formation process by a particle bridging mechanism (Asrafuzzaman et al. 2011; Kurniawan et al. 2022a, b). Based on this observation, 200 NTU was chosen as the initial turbidity for the subsequent OVAT analysis.

The OVAT analysis for chitosan dosage in kaolin clay suspension is presented in Fig. 1B, where the turbidity removal increased from 0.5 to 1 mg/L dosage. The highest turbidity removal was observed at 1 mg/L dosage. However, the use of higher dosages up to 10 mg/L caused a decrease in turbidity removal and showed no significant difference from dosages 15 to 30 mg/L. The dosage of chitosan is related to the amount of cationic charges introduced into the solution. With an increase in chitosan dosage, the cationic charges also increase, which neutralize the anionic charges of kaolin clay colloids. However, excessive cationic charge results in a destabilization effect that repels the neutralized colloids, causing an increase in turbidity (Roussy et al. 2005; Nourmoradi et al. 2015; Okaiyeto et al. 2016; Hadiyanto et al. 2021). Based on this result, a dosage of 1 mg/L was selected for the next OVAT analysis.

The OVAT analysis results for the effect of pH on kaolin clay suspension are presented in Fig. 1C. It was observed that pH values of 5, 6, and 7 resulted in higher turbidity removal with no significant difference among them. However, at alkaline conditions (pH 8, 9, and 10), the turbidity removal decreased. Consistent with previous studies, chitosan exhibited good performance in coagulation at acidic to neutral pH conditions (Roussy et al. 2005; Hassan et al. 2009). At low pH, the amine group in chitosan was predominantly protonated, which favoured charge neutralization of the anionic charges carried by kaolin clay colloids (Kurniawan et al. 2022a, b). Conversely, at alkaline conditions, the amine groups were deprotonated, and coagulation could not occur via charge neutralization (Hassan et al. 2009). Under such conditions, possible mechanisms include sweep coagulation (Ahmad et al. 2022) or bridging mechanisms with higher dosages of chitosan (Roussy et al. 2005; Chong and Kiew 2017). These results suggested that the original pH of kaolin clay suspension (pH 6.5–7) could be used directly for coagulation to achieve the highest turbidity removal.

Following the analysis of turbidity removal in kaolin clay suspension, an OVAT analysis was performed to determine the optimal chitosan dosage in real aquaculture wastewater, while maintaining its initial turbidity and pH. The initial turbidity of the wastewater was found to be 72 ± 7.02, which was lower than that of the kaolin clay suspension (200 NTU). Figure 2 depicts the results of the OVAT analysis for chitosan dosage in real aquaculture wastewater.

Fig. 2
figure 2

Aquaculture wastewater’s turbidity removal in OVAT analysis of chitosan dosage. Different letters (a–b–c–d–e) indicate significant different in turbidity removal among tested variables

The presented figure illustrates that the optimal dosage range for turbidity removal in aquaculture wastewater was found to be between 10 to 25 mg/L, while a dosage of 30 mg/L resulted in decreased turbidity removal due to overdosing, which is consistent with previous studies (Roussy et al. 2005; Nourmoradi et al. 2015; Okaiyeto et al. 2016). The results showed that a chitosan dosage of 10 mg/L was able to remove 80.41 ± 0.96% of the turbidity in aquaculture wastewater. In comparison, the highest chitosan dosage needed in kaolin clay suspension was 1 mg/L, which removed 96.64 ± 0.22% of turbidity. However, in aquaculture wastewater, the usage of 1 mg/L chitosan only removed 52.55 ± 1.2% of turbidity. These findings suggest that higher chitosan dosage is needed for the treatment of real aquaculture wastewater due to the presence of various chemical pollutants and microalgae that may interfere with the process, unlike kaolin clay suspension, which is composed solely of hydrated aluminium silicate and mineral kaolinite. Based on these results and previous studies (Momeni et al. 2018; Igwegbe et al. 2021), the range and level values for the RSM were determined.

RSM for turbidity removal in aquaculture wastewater

Model fitting and statistical analysis from BBD

Following OVAT analysis of turbidity removal in kaolin clay suspension and aquaculture wastewater, a RSM model based on BBD with 3 factors was employed to optimize turbidity removal in aquaculture wastewater. The expected value range for variables in the BBD was determined based on the OVAT analysis presented in Fig. 2, and the coded variables and their corresponding ranges are listed in Table 2. All 15 experimental runs were performed, and the results of the experimental versus predicted values are presented in Table 3.

The statistical significance of the quadratic model was evaluated by ANOVA, as tabulated in Table 4. The quadratic equation obtained from ANOVA is given in Eq. 3.

$$Y = + \,94.92\, + \,1.57A\, - \,1.38\, + \,1.96\, - \,2.34A^{2} \, - \,6.82B^{2} \, - \,3.08C^{2} \, + \,2.34AB \, - \,0.2335AC\, + \,2.37BC$$
(3)
Table 4 ANOVA for response surface quadratic model and statistical parameters for turbidity removal

Model terms were evaluated by the highest value of coefficient of determination (R2), F-test, p-value, and lack of fit. The p-value that is higher than 0.05 is not significant, while less than 0.05 is significant. To obtain a significant regression model, the insignificant terms in Eq. (3) were eliminated, as suggested by Okolo et al. (2021). Consequently, the AC terms were removed, and the resulting regression model is expressed as Eq. (4). A positive sign preceding the equations implies a synergistic effect of the factors, while a negative sign indicates an antagonistic effect of the factors, as explained by Endut et al. (2017).

$$Y= +94.92+1.57A-1.38+1.96-2.34{A}^{2}-6.82{B}^{2}-3.08{C}^{2}+2.34AB+2.37BC$$
(4)

Table 4 demonstrates the F-test value (59.96%), indicating that the developed model is significant. Furthermore, the lack of fit in the model was found to be insignificant, at 0.69. This implies that the lack of fit is not significant compared to the pure error (Mondal et al. 2017). The predicted R2 value of 0.9149 is in reasonable agreement with the adjusted R2 value of 0.9743, with a difference of less than 0.2 (Iloamaeke et al. 2021). Additionally, the adequate precision ratio, with a value of 24.973, is higher than 4, which indicates an adequate signal (Okolo et al. 2021).

Adequacy of the model

The RSM model was utilized to fit the experimental data, and the relationship between normal and residual, and actual and predicted values was analysed and presented in Fig. 3a–c. The normal plot residual of residuals in Fig. 3a showed that the data points closely followed the straight line, indicating that the model is a reliable tool for predicting response over independent input data points. Although some data points deviated from the normal distribution, the deviation was not significant. In Fig. 3b, the residual versus run number plot showed that the data points were randomly scattered within the red line boundaries and did not exhibit any trend, indicating the absence of outlier data. Additionally, the predicted versus actual plot in Fig. 3c showed that the distribution points were closely aligned along the 45° line, suggesting that the model is capable of accurately predicting turbidity removal (Endut et al. 2017; Iloamaeke et al. 2021; Okolo et al. 2021).

Fig. 3
figure 3

Plot of a normal plot of residuals, b residuals versus run number, and c predicted versus actual for turbidity removal in aquaculture wastewater using chitosan

Figure 4a–c depicts 3D response surface plots illustrating the interactive effects between response and experimental levels of each factor. The response surface plots indicate that the maximum turbidity removal is situated within the design boundary. Figure 4a exhibits the surface of the interactive effect between dosage and pH at a constant sedimentation time of 15 min. The figure demonstrates that at a lower dosage of 5 mg/L, the turbidity removal decreases as the pH approaches an alkaline condition. Conversely, as the dosage increases, the turbidity removal displays an increment. It has been reported previously that overdosage can result in higher turbidity under alkaline conditions (Roussy et al. 2005). However, the mechanism involved is not charge neutralization, but rather the entrapment of colloid particles into the chitosan network, which is also known as sweep coagulation (Chong and Kiew 2017). This finding is consistent with an earlier report that suggests that chitosan coagulation primarily follows a sweep coagulation mechanism at increased pH (Kurniawan et al. 2020).

Fig. 4
figure 4

Surface of interactive effect between a dosage and pH at sedimentation time 15 min, b dosage and sedimentation time at pH 7, and c pH and sedimentation time at dosage 15 mg/L

Figure 4a displays a surface interactive effect between dosage and sedimentation time at pH 7. The figure reveals that the turbidity removal reaches equilibrium at 15 min and increases as the dosage is raised, reaching equilibrium at 15 min of sedimentation time. However, in this study, these two terms have no significant effect. Conversely, pH and sedimentation time have a significant interactive effect. Figure 4c depicts a surface interactive effect between pH and sedimentation time at a dosage of 15 mg/L. The figure shows that at a dosage of 15 mg/L, the turbidity removal increases as the pH is raised and decreases under alkaline conditions. However, the turbidity removal increases with an increase in the sedimentation time. At pH 8, the turbidity removal is the lowest at a sedimentation time of 5 min due to the smaller size of floc at alkaline conditions, which requires a longer time to settle. Similar to a previous report, floc appears rapidly and is larger in size under acidic conditions, resulting in a faster settling time (Hassan et al. 2009).

Optimization of coagulation–flocculation process by chitosan in removing turbidity from aquaculture wastewater

Numerical optimization utilizing the BBD design was employed to obtain the optimum parameters for achieving the highest turbidity removal in this study. The goal was to maximize the turbidity removal, with the chitosan dosage, pH of aquaculture wastewater, and sedimentation time set within a defined range. The predicted highest value of turbidity removal for aquaculture wastewater treated with chitosan was 95% at a chitosan dosage of 18.25 mg/L, pH of 7, and sedimentation time of 18.1 min, with a desirability of 0.974, indicating that the response was within the targeted region. To validate the result, a confirmation coagulation–flocculation experiment was conducted in triplicate under optimum conditions, yielding a turbidity removal of 94 ± 0.61%. The experimental result was found to be in good agreement with the model prediction. The RSM model predicted that the optimum chitosan dosage and sedimentation time range were 10–25 mg/L and 15–25 min, respectively, with no significant difference in turbidity removal (p > 0.05) in OVAT analysis of aquaculture wastewater. However, the chitosan dosage needed for the optimum condition of kaolin clay suspension was lower (1 mg/L) due to the composition of only hydrated aluminium silicate and mineral kaolinite. The optimum condition determined in this study can be applied to the treatment of aquaculture wastewater for turbidity removal, although the removal efficiency may vary depending on the initial characteristics of the wastewater.

Conclusion

The effectiveness of chitosan in removing turbidity from aquaculture wastewater was evaluated by analysing the impact of initial turbidity, chitosan dosage, and pH of the kaolin clay suspension. The OVAT analysis showed that the chitosan dosage required to reduce turbidity in the kaolin clay suspension was lower (1 mg/L) and resulted in 96.64 ± 0.22% turbidity removal, compared to the dosage needed for real aquaculture wastewater, which was 10 mg/L and yielded 80.41 ± 0.96% turbidity removal. RSM was then used to optimize the process and model the interaction effects between variables. The analysis revealed that the surface interactive effect between dosage versus pH and pH versus sedimentation time significantly influenced turbidity removal. The optimal turbidity removal of 94 ± 0.61% was achieved using a chitosan dosage of 18.25 mg/L, pH 7, and sedimentation time of 18.1 min with a desirability of 0.974. The RSM method overcame the limitations of the OVAT analysis in terms of factor combination and demonstrated the optimum operating conditions. Furthermore, RSM displayed the effect of conditions interaction for turbidity removal. This study confirms that chitosan extracted from L. vannamei can serve as a sustainable substitute for chemical coagulants in coagulation/flocculation processes for removing turbidity in aquaculture wastewater treatment. Future research should investigate the point of zero charge of chitosan coagulant to gain a better understanding of the coagulation/flocculation mechanism.