1 Introduction

The arrival of new technologies and products over the past decades, promoted not only progress but several environmental problems. It is known that living organisms’ activities, urban demand, domestic consumption and industrial operations, including washing, rinsing, and cleaning equipments, generate high amounts of effluents. From the above, the industry activities are the most prominent factor of water contamination, exceeding the environment regenerative capacity and self-purification, causing imbalances in the aquatic ecosystems when an appropriate treatment is not taken into account. Thus, environmental concerns, associated with sustainable development, led to the appearance of restrictive legislation, limiting the levels of contaminants discharge in watercourses. Generally, effluents are composed by organic and inorganic substances, including nutrients and aromatic compounds, and feasible treatments are commonly required.

Aerobic systems are, nowadays, frequently established in wastewater treatment plants (WWTPs) worldwide, with conventional activated sludge (CAS) systems, a classical technology, still adopted for its convenience and simplicity (Tandoi et al. 2006). However, these systems are also prone to be affected by a multiplicity of malfunctions, as reported by Mesquita et al. (2016). In addition, the use of membrane bioreactors (MBR) has been increasingly applied in WWTP over the years, providing many advantages over CAS, such as a small footprint and high effluent quality (Van de Staey et al. 2015). On the other hand, despite extensive research efforts, the fouling process still remains one of the main problems and concerns of MBR research within the academic community (Krzeminski et al. 2012). Aerobic processes are quite dependent on the operating conditions, and, therefore, can be unstable, particularly when subjected to changes in the environment. This, in turn, leads to a negative effect on the bacterial metabolic activity and consequently on the process efficiency. On the basis of these considerations, the sequencing batch reactor (SBR) technology has been increasingly developed taking into account the early experiences gained with CAS systems. SBRs which are flexible systems where both nutrients and organic matter (OM) are removed in the same unit, have been arising with a variety of several attractive properties presented elsewhere (Tchobanoglous et al. 2003).It is well-known that in flocculent activated sludge systems, like CAS, the sludge aggregation is essential for the solid/liquid separation, and poor sludge aggregation leads to an increase in the effluent turbidity and biomass washout (Li and Yuan 2002; Bitton 2005; Jenkins et al. 2003). An attractive low-cost and low-footprint alternative to the flocculent CAS process for WWT is the aerobic granular sludge (AGS) (Lou et al. 2014). AGS systems have been developed in SBRs and have demonstrated excellent settling capabilities, due to their self-immobilization formation process, which increases their density (Adav et al. 2008). Nevertheless, several key factors have been already described as responsible for the loss of the structural long-term stability (Liu and Liu 2006; Tay et al. 2002; Zheng et al. 2006; Lemaire et al. 2008).

In the past few years, with the increase of contaminants complexity, anaerobic technology also emerged as a means to improve WWT. Indeed, anaerobic digestion (AD) is commonly used as a pre-treatment of agro-food industrial wastewaters containing high levels of biodegradable organic compounds. As Alves et al. (2009) points out, AD should be applied to concentrated effluents allowing energy production and nutrient redistribution. However, a post-treatment step is quite always necessary in order to meet the required discharge criteria in surface waters.

Monitoring biological systems is an important task for the performance enhancement of WWT processes. The increasing demand for biological processes technologies requires the development of adequate monitoring and control techniques. Several physical, chemical and operational parameters are generally monitored in WWTP for assessing wastewater and sludge quality, being of the utmost importance to meet the required discharged values. However, most traditional methodologies for determining these parameters are costly, labor intensive and time-consuming, and a few may even present environmental risks associated to end products. Therefore, a preferable alternative would be to continuously monitor the key variables within the process and to use this information to make educated decisions. Thus, a great deal of attention has been recently given to different monitoring strategies to help clarifying the behavior of WWT biological processes. Moreover, it is known that waste sludge is a by-product of the biological processes, and its treatment and disposal represents up to 50% of the running operating costs of a WWTP (Guo et al. 2014). Thus, it is quite imperative to also monitor the waste sludge post-treatment which generally includes stabilization, drying or composting stages.

Recently, high significance has been attributed to the technological evolution and advances in spectroscopic methods to investigate complex samples. In fact, over the last years, spectroscopic techniques have gained a relevant interest within the biotechnology field. With these technologies, absorption, reflectance, transmission, or vibrational properties of chemical species can be measured in order to determine the concentration or composition of a sample. Once implemented, and optimized, these methods are fast, non-destructive and user friendly, allowing rapid inference of the process state.

Furthermore, spectrometry combined with multivariate statistical analysis has been shown to be a valuable tool for monitoring physico-chemical parameters associated to water quality, being the analysis performed without the need of any special reagent or solvent, both offline and online, with potential to be applied in situ or in-line. Among the spectroscopy techniques, ultraviolet–visible (UV–Vis), infrared (IR), and fluorescence (FLC) have lately attracted substantial attention in WWT monitoring. It should be noticed, though, that other spectroscopy techniques (Raman spectroscopy, NMR spectroscopy, Terahertz spectroscopy) are distinguished by specific applications and implementations which are beyond the scope of this article.

In conclusion, this review outlines recent progresses in UV–Vis, IR and FLC spectroscopy, related to WWT monitoring. An in-depth analysis is performed to summarize the many new findings that have been obtained in the last years, and future developments for the application of spectroscopy techniques in WWT biological processes.

2 Monitoring aerobic and anaerobic systems

Traditionally, the biochemical oxygen demand (BOD5), chemical oxygen demand (COD) (including filtered and dissolved—DCOD), turbidity, total organic carbon (TOC) (including dissolved organic carbon—DOC), and volatile fatty acids (VFA) are considered key monitoring parameters and have been widely employed for assessing the wastewater quality in aerobic and anaerobic systems. In the particular case of AD systems, the monitoring of alkalinity, biochemical methane potential (BMP), gas production rate and hydrogen sulfide (H2S) production (Pontoni et al. 2015) is also essential for the evaluation of a good performance. Furthermore, BOD5, COD, and TOC have also been used to characterize various complex compositions known as OM, and namely dissolved organic matter (DOM). DOM is a heterogeneous mixture of aromatic, amino and aliphatic organic compounds containing oxygen, nitrogen and sulfur functional groups (Chen et al. 2003). However, the above parameters do not provide information on the composition of the DOM, and in addition, their analysis is rather tedious and time consuming, and sometimes requires expensive equipment and instrumentation (Janhom et al. 2009).

Additionally, in recent years, the development of new and more sensitive methods of analysis has made possible the detection of other potentially harmful contaminants, globally referred to as emerging contaminants, and present in trace amounts, in both aerobic and anaerobic systems. The most common methods include the use of gas chromatography-mass spectrometry (GC–MS) and/or liquid chromatography-mass spectrometry (LC–MS). Traditionally, these chromatography techniques, coupled with mass spectrometry, have been used for the identification and quantification of trace compounds (Aguera et al. 2006; Afonso-Olivares et al. 2012; Feng et al. 2015). However, this methodology typically requires extensive off-line sample preparation, and since the compounds of interest are generally present at trace levels, the sample preparation method requires a prior concentration step. It is known that MS always requires the use of GC and LC equipments, thus difficult to implement online (in situ or in-line) towards the monitoring of WWT. Thus, the present review will focus on the use of fast, non-destructive and user friendly spectroscopic techniques (UV–Vis, IR and FLC) without extensive sample preparation, thus not encompassing MS spectroscopy.

Total solids (TS), volatile solids (VS), suspended solids (SS), comprising total suspended solids (TSS) and volatile suspended solids (VSS) have been of utmost significance for monitoring purposes. Total nitrogen, including nitrate, nitrite, and ammonia and phosphorus concentration is also widely assessed, reflecting the extent of nitrification, denitrification, and phosphorus removal processes. The sludge quality and stability, which can be related to the extracellular polymeric substances (EPS) matrix role, is also an important factor to take in consideration, but still not implemented in real WWTP.

Globally, the wastewater quality is generally assessed using physical, chemical and microbiological tests. However, these parameters depend on expensive, labor intensive and/or time-consuming methods, offering only snapshots of moments in time, which makes them unsuitable for real-time monitoring (Carstea et al. 2016). Direct and rapid measurements of the parameters previously presented would provide close monitoring of WWTP quality and allow for a real-time process diagnosis and control. Thus, in the next sections, the most promising and recent available spectroscopic techniques for aerobic and anaerobic processes monitoring are described. Figure 1 provides brief information about the evaluated parameters in the biological processes for each spectroscopy technique discussed in this review.

Fig. 1
figure 1

Schematic representation of the spectroscopy techniques discussed in this review with the relevant parameters evaluated in each case

Full size image

3 Aerobic systems

Taking into account the advantages of the spectroscopic techniques above mentioned, it comes as no surprise that their use in WWT processes has been exponentially growing in the last decades. In this sense, the recent progresses in UV–Vis, IR and FLC spectroscopy monitoring applications regarding aerobic systems are next described.

3.1 UV–visible spectroscopy

UV–Vis light absorption has been used since the 1930s for wastewater characterization (Pons et al. 2004). There are also a great number of works dealing with natural waters including rivers and drinking waters which are outside the scope of this review. Table 1 shows the most relevant published studies in the use of solely UV or UV–Vis spectroscopy in wastewater. The first clear evidence provided by Table 1 is related to the huge application of this spectroscopy technique in full-scale WWTPs. UV–Vis refers to the interaction between samples and radiation in the 200–780 nm wavelength range (Lourenço et al. 2012), at single or multiple wavelengths to estimate a number of parameters. Several works have been already performed using UV absorbance at 254 nm (A254). In this case, right after the discharge of treated effluents, a linear regression was found between A254 and COD considering the effect of a single rainfall event (Stumwohrer et al. 2003). To account for the effects of particles (turbidity) for each type of sample in a set of rain dilutions, they considered A350 as a corrective parameter. The use of solely A254 was lately studied by Kwak et al. (2013). These authors developed a multiple linear regression model for predicting BOD5. However, they found that combining DOC and A254 provided the best model results. Wu et al. (2006) confirmed that A254 is a good surrogate parameter to monitor wastewater pollution, mainly DCOD, COD, ammonia, and turbidity when samples are collected in a short period of time in the same WWTP. However, two more WWTPs were reported with lower correlation coefficients due to higher pollutants and episodes of pollutants discharge. Also the A280 (280 nm absorbance) was used to estimate the BOD5 in wastewater resulting in: (1) a reasonable prediction in the reactor entrance and an inaccurate prediction on the final effluent for pulp and paper mill wastewaters (Muzio et al. 2001); and (2) a linear regression for raw, primary, and final effluent from full-scale WWTP (Nataraja et al. 2006).

Table 1 Main studies performed using UV–Vis spectroscopy
Full size table

UV multiwavelengths have also been used in wastewater monitoring due to the current extensive use of mathematical modeling (Thomas et al. 1996). Several authors used UV-deconvolution methods based on least square regressions mainly to: (1) estimate the DOC (Escalas et al. 2003); (2) provide information about the suspended solids (mostly on soluble and colloidal fractions) (Azema et al. 2002); and (3) estimate organic sulfur compounds such as mercaptans, in urban and industrial wastewaters (Roig et al. 2002). On the other hand, Vaillant et al. (2002) used simple methods developed for UV spectra exploitation, namely direct comparison, spectra differences and normalization to understand the UV physical response of TSS and fractions typology. The use of artificial neural networks (ANN) was also previously reported as a good strategy to estimate COD, total nitrogen, total phosphate, and TSS (Fogelman et al. 2006; Jeong et al. 2007). Influent and treated wastewater samples, collected during the week and weekend, were clearly distinguished using principal component analysis (PCA) and cluster analysis (CA) (Lourenço et al. 2006). Moreover, nitrification and denitrification processes monitoring have also been a field of interest, since most of the available methods are expensive, complicated, labor intensive, time consuming and may require large sample volumes. For instance, the work of Ferree and Shannon (2001) used a second derivative of the UV spectra to quantify total nitrogen and nitrate. Also an in situ UV spectrometer has already been used to evaluate the nitrite, nitrate, COD, DOC and TSS based on a multivariate calibration algorithm (partial least squares—PLS regression) (Rieger et al. 2004), resulting in good and acceptable precisions for nitrite, nitrate, and COD and low for DOC. The evaluation of online UV spectra was also performed using three different types of wastewater (municipal, landfill, and industrial wastewaters) with acceptable results to estimate the sum of nitrite and nitrate (Drolc and Vrtovšek 2010).

Up until quite recently, research activities were mainly focused on offline spectral acquisition (Thomas et al. 1993). A clear example of this was the monitoring of slaughterhouse wastewater biodegradation in a lab-scale SBR using offline UV–Vis spectra, relating A416 (416 nm absorbance) with COD during wastewater biodegradation (Louvet et al. 2013). Also, TOC estimation in a WWTP using offline UV–Vis spectra was performed by Lourenço et al. (2008) through the use of PLS regression models.

However, over the last years, online UV–Vis spectrophotometric data has been increasingly applied in situ to monitor WWT processes in different types of wastewaters. For instance, UV–Vis spectrometry was studied regarding its application for the evaluation of winery WWT and monitoring rain events (Carvallo et al. 2007; Maribas et al. 2008). Mathematical modelling has also been applied to UV–Vis data and PLS models have been developed to: (1) predict COD (Langergraber et al. 2003); (2) calibrate COD, filtered COD, nitrate, and TSS concentrations (Bertrand-Krajewski et al. 2007); (3) calibrate the previous parameters in paper mill wastewater and in a pilot-scale SBR with acceptable results (Langergraber et al. 2004a, b); (4) detect outliers and predict COD and TSS (Zamora and Torres 2014), and (5) predict COD, nitrate, and TSS in a lab-scale CAS (Sarraguça et al. 2009).

Furthermore, a number of other mathematical strategies, encompassing the use of PLS models, were also investigated for online prediction of several key parameters from wastewaters. Nonlinear UV–Vis deconvolution and PLS models were used during nitrophenol biodegradation (Vargas and Buitrón 2006). Different types of calibration methodologies using PLS models were developed for COD, filtered COD, nitrate, nitrite, and TSS (Rieger et al. 2006). More recently, newer modeling strategies were developed for online COD monitoring. A variable pathlength approach, combined with PLS, was developed by Chen et al. (2014), and a Boosting-Iterative Predictor Weighting-PLS model was developed to handle the noise to information unbalance (Qin et al. 2012). In the latter case, also TSS, oil and grease were monitored.

3.2 Infrared spectroscopy

The IR spectrum can be divided into three main energy (wave number) regions: the far-infrared (FIR) of less than 400 cm−1 (<25,000 nm), the mid-infrared (MIR) of 4000–400 cm−1 (2500–25,000 nm) and near-infrared (NIR) of 13,000–4000 cm−1 (750–2500 nm) (Stuart 2004). The IR spectrum can be obtained using absorbance, transmittance, and reflectance methods. These are considered simple techniques to obtain spectra from liquid, solid or gaseous samples. Transflectance combines the transmittance and reflectance measurements and can be used to acquire spectra from turbid or transparent liquids (Lourenço et al. 2012). In WWT processes the band intensity measurements could range from the three types described above depending on the sample type. Some of the works described below have been referred as using Fourier-transform infrared (FTIR) spectroscopy, mostly on the MIR spectrum region and using samples in the solid state (lyophilized or dried).

The application of IR spectroscopy to aerobic WWT processes was, until very recently, less common than UV–Vis, nevertheless, it has gain lately a greater emphasis as reported by the works presented below, namely on WWT process performance, PHA quantification, EPS characterization and excess sludge post-treatment processes.

Lab-scale experiments have been performed to evaluate the feasibility of FTIR spectroscopy to investigate WWT processes. Campos et al. (2014) performed experiments with CAS systems fed with landfill leachate/domestic wastewater and found that FTIR analyses were able to demonstrate that most of the pretreated leachate OM was removed rather than diluted. The prediction of intracellular PHA in mixed microbial cultures (MMC) from different sources has been also studied by FTIR spectroscopy, indicating that this methodology significantly reduces the analytical time needed for polyhydroxyacanoates (PHA) quantification compared to the gas chromatography (GC) analytical technique (Khardenavis et al. 2009; Arcos-Hernandez et al. 2010).

A large variety of methods for the extraction of EPS in activated sludge are now available, and a comparison of the efficiency of eight EPS extraction methods has been already conducted based on IR spectra analysis. It was found by Comte et al. (2006) that IR spectra demonstrated EPS contamination by identifying reagents, and possible products of reactions between EPS and reagents. Furthermore, EPS characterization based on FTIR spectroscopy has expanded a few years ago mainly to study: (1) the influence of antibiotics on activated sludge composition (Avella et al. 2010); (2) the bioflocculation and settling properties of the sludge using different reactors configuration (Ehlers and Turner 2011; Ehlers et al. 2012); (3) the nanoparticles effects on the physicochemical stability of activated sludge providing information about the major functional groups (Hou et al. 2015); (4) the distribution of heavy metals in microbial aggregates (Sheng et al. 2013); (5) the effect of salinity in activated sludge (Wang et al. 2013); (6) the effect of carbon/nitrogen (C/N) ratio in microbial aggregates functional groups (Wang et al. 2014a); (7) the phosphorus transfer process in the enhanced biological phosphorus removal (EBPR) system (Wang et al. 2014b); (8) nitrogen rich wastewaters from a mixed culture predominated by ammonia-oxidizing bacteria (Yin et al. 2015); (9) the importance of aromatic protein-like substances (especially tyrosine) in maintaining the stability of AGS (Zhu et al. 2012); (10) the detection of EPS proteins and polysaccharides in colloidal form in AGS (Tu et al. 2012); and (11) the EPS functional constituents on bioflocculation in activated sludge processes (Badireddy et al. 2010). Lin et al. (2010) used FTIR to study alginate-like exopolysaccharides (ALE) to improve the understanding of AGS formation and stability, concluding that a large proportion of the AGS dry weight is ALE, being the dominant exopolysaccharides in AGS. Afterwards, Lin et al. (2013) were able to investigate, by FTIR spectroscopy, differences between the gel matrix of AGS and normal aerobic flocculent sludge.

Given that the waste sludge contains a large part of biodegradable OM, an arbitrary discharge of such sludge to the environment would present a pollution problem. Hence, such WWT sludge usually undergoes a post-treatment process such as stabilization, drying or composting. Again, FTIR spectroscopy has been conducted regarding waste sludge post-treatment processes, as follows: (1) determination of different functional groups, able to react with dye molecules in aqueous solution, in dried activated sludge used for reactive dyes adsorption (Gulnaz et al. 2006); (2) study of humic acid structures change, and release of aliphatic compounds, during activated sludge composting (Amir et al. 2010); (3) evaluation of sludge post-stabilization, and identification of specific functional groups, for a number of sample sources (Smidt and Parravicini 2009); (4) characterization of lignocellulosic substances, proteins and polysaccharides in a pulp and paper mill secondary sludge (Edalatmanesh et al. 2010); (5) characterization of macromolecular OM hydrolysis, and polysaccharide-like and protein-like materials degradation, in sludge vermicomposting (Yang et al. 2014a).

It is clear from the literature survey that it is not straightforward to use FTIR as a quantitative analytical technique, due to the difficulties related to sample preparation. For instance, samples need to be dried or lyophilized, to eliminate water masking the FTIR spectrum, leading to the production of artifacts (Kunacheva and Stuckey 2014). It is also clear that FTIR data does not allow estimating key parameters (COD, BOD5, etc.) regarding the process behavior. Thus, it cannot be used for real-time and in situ monitoring due to its technical requirements and limitations (Galinha et al. 2012). Therefore, it can be concluded that, while FTIR spectroscopy is useful for predicting compound structures, it is not suitable for compound identification, which most times would be advantageous in monitoring WWT.

NIR spectroscopy has been widespread applied in the pharmaceutical industry for quality control and process monitoring (Lourenço et al. 2012). Currently, great importance is given to NIR techniques in aerobic WWT according to the works presented below. Because of overlapping bands, NIR information must be extracted by chemometric techniques, including the frequently used PLS regression to estimate the key parameters. Mathematical treatment of spectral data is a common way to extract the most relevant information. In 2002, Stephens and Walker used offline visible-NIR (VNIR) spectroscopy to develop a predictive model for rapid measurement of BOD5 in a large number of wastewater samples. The COD was investigated by Dahlbacka et al. (2014), but in this case the COD was predicted online in pulp and paper mill wastewater. Different strategies were conducted by Pan et al. (2011, 2012), Pan and Chen (2012) to estimate COD offline. For that purpose, the NIR spectra waveband selection by moving window PLS (MWPLS) method was performed. Then, the optimization of Savitzky–Golay (SG) smoothing modes was applied to optimize the model of NIR spectroscopy analysis. Finally, the authors found that the short-wave NIR (SW-NIR) region presented the best stable results in sugar refinery wastewaters. The SW-NIR region was also studied by Melendez-Pastor et al. (2013), combining SW-NIR region with the visible region (V/SW-NIR). PLS models were established for the full spectra and for the visible and NIR spectral ranges separately. The authors found good prediction abilities with the full spectra range for TSS, while for BOD5 and COD the visible spectral range provided the best overall results. To rapidly determine the pollution extent of wastewaters, calibration models were established by Yang et al. (2009) for COD and BOD determination in a WWTP using PLS and offline NIR spectroscopy. Again, offline NIR spectroscopy has been used to estimate the predictability of the contamination level in sewages and satisfactory PLS regression results were found for total phosphorous, total nitrogen, BOD, TOC, and turbidity (Inagaki et al. 2010).

Studies conducted in lab-scale CAS systems have already been performed to test the feasibility of in situ NIR spectroscopy to monitor the biological system. Dias et al. (2008) and Sarraguça et al. (2009) tested the ability of in situ NIR transflectance probe to detect changes in a CAS system where disturbances were induced. In the first case, PCA analysis revealed that NIR was able to detect system imbalances by changing the influent charge. In the second case, the authors found that just the TSS were well assessed by using PLS, whereas low regression values were obtained for COD and nitrate. In a different study, an SBR for the aerobic treatment of dairy residues was monitored by a NIR transflectance probe, and PLS regression used to calibrate parameters such as dry matter or total solids (TS), TSS, and COD (Pascoa et al. 2008). Quite recently, an SBR was inoculated with activated sludge and operated with the strategy to produce bioplastics (polyhydroxybutyrates—PHB, the main polyester compound found in PHA), fed with crude glycerol, and monitored by in-line NIR spectroscopy (Dai et al. 2015). The authors found that multivariate statistical methods could be used to predict PHB contents and to monitor substrate concentrations.

NIR spectroscopy has also been previously used as a technique to predict stability parameters (water-soluble carbon, ash content, degree of OM humification) in different composting processes (Vergnoux et al. 2009; Albrecht et al. 2011; Soriano-Disla et al. 2010). It has been found that during the humification of OM in sludge composting, NIR was able to study the compost maturity based on the prediction of humic content, mainly fluvic acids and humic acid/fluvic acid ratio (Albrecht et al. 2011). Multivariate statistical analysis was performed by Vergnoux et al. (2009), with PCA allowing for the different composting stages to be identified from the spectral data. At the same time, PLS models were applied to successively predict a number of physicochemical parameters with the obtained NIR spectra. In a different study, Soriano-Disla et al. (2010) revealed the potential of NIR to predict stability parameters in sewage sludge and derived compost by using PLS models. Table 2 shows the most relevant published studies using IR spectroscopy.

Table 2 Main studies performed using IR spectroscopy
Full size table

3.3 Fluorescence spectroscopy

FLC is the molecular absorption of light energy at one (discrete) wavelength and its nearly instantaneous reemission at another, usually longer, wavelength. FLC molecules (fluorophores) have an excitation spectrum (absorbed light wavelength and intensity) and an emission spectrum (emitted light wavelength and intensity). FLC methods are much more sensitive than absorbance methods and much less affected by the medium turbidity (Pons et al. 2004). In the last 20 years, FLC spectroscopy emerged as a widely applied tool for process control in WWTP due to several attractive advantages: fast, reagentless, highly sensitive and non-invasive (Carstea et al. 2016). In early publications, fluorescence was measured only at one excitation/emission (Ex/Em) wavelength pair, limiting the signal to one fluorophore (Lourenço et al. 2012). In WWTP, which can been considered a complex media, many natural fluorophores are present, such as amino acids (tryptophan, tyrosine and phenylalanine), vitamins, coenzymes, and aromatic OM. Simultaneously, as it was previously presented, EPS released by cells comprise molecules such as protein-like and humic-like substances which are also fluorescent. Those molecules can be detected by FLC spectroscopy whether they are intra- or extra-cellular (Galinha et al. 2011a, b, 2012). Thus, to access all the of the different fluorophore compounds, scanning a sample at many different excitation and emission wavelengths, is needed (2D FLC spectroscopy) (Galinha et al. 2011b; Pawlowski et al. 2016). 2D FLC spectroscopy results in an excitation-emission matrix (EEM) which has been applied as a substitute for some chemical analysis to identify multiple compounds and microbial products in biological samples including biofilms (Kunacheva and Stuckey 2014; Wolf et al. 2002). Also, three-dimensional excitation-emission matrix (3D-EEM) FLC spectroscopy has also been widely used to detect detailed changes and transformations of the OM in wastewater/sludge samples, by simultaneously determining three fluorescent parameters (excitation wavelength, emission wavelength, and fluorescence intensity) (Hao et al. 2012). Fluorescence peaks in EEMs are identified and quantified typically utilizing visual identification of fluorescence peaks. The Ex/Em spectra of commonly detected peaks in WWT processes are as follows: humic-like peaks at 237–260/400–500 nm, 300–370/400–500 nm, and 290–325/370–430 nm, tryptophan-like peaks at 225–237/340–381 and 270–280/330–368 nm, and tyrosine-like peaks at 225–237/309–321 and 270–275/304–312 nm. However, most of the time, the peak location may switch due to the presence of overlapping spectra or chemical compounds (Hudson et al. 2007; Henderson et al. 2009; Fellman et al. 2010, Yang et al. 2015). Thus, recently, there has been an effort to use multivariate statistical analysis techniques (PCA, PLS, and ANN) to extract the most complete set of information that can be provided by EEM spectra. Nevertheless, these methods need a large dataset to establish the training model and then to predict the unknown samples composition (Li et al. 2008). Also, the combination between EEM and parallel factor analysis (PARAFAC), another chemometric tool, has greatly advanced in recent years. PARAFAC allows the deconvolution of the overlapping EEM spectral peaks into their independent components, each of which has similar fluorophores (Hur and Cho 2012; Meng et al. 2013). Fluorescence regional integration and self-organizing maps have also been considered as alternative models to characterize compounds fractions in WWTP (Chen et al. 2003, Guo et al. 2014; Yu et al. 2014). Table 3 shows the most relevant published studies using FLC spectroscopy.

Table 3 Main studies performed using FLC spectroscopy
Full size table

The FLC spectroscopy application in WWT processes has been conducted mainly in the influent wastewater, during the biological process, or even at the end of the biological process, for wastewater quality assessment. Recently it has gained a great significance as presented below. Indeed, FLC spectroscopy has already been applied to WWT processes to estimate BOD, COD and TOC. Reynolds (2002) found that FLC data correlated well with BOD, COD and TOC and was able to differentiate biodegradable and nonbiodegradable organic material at 280 nm excitation wavelength and at 350 nm and 440 nm emission wavelengths. Bridgeman et al. (2013) found good correlations for BOD and COD with FLC intensity at excitation 280 nm and emission 350 nm, at various stages of the biological treatment. On the other hand, low correlation coefficients were found using a 280 nm excitation wavelength for COD and TOC by Vasel and Praet (2002). Samples from different sources (including sewage and industrial effluents) were analyzed by Hudson et al. (2008) to inspect the biodegradable organic material present. It was also found that the EEM analysis of the tryptophan-like peak (Ex/Em wavelength of 275/340 nm) provided an accurate indication of the biological community activity, showing a stronger correlation with BOD5. Quite recently, changes on percentages in the FLC intensities of several WWTP were investigated. Furthermore, combining EEM with PARAFAC allowed estimating TOC, DOC, BOD, and COD (Yang et al. 2014b).

Currently, extensive DOM studies have been performed using EEM to characterize: (1) paper mill wastewaters and the use of ion exchange resins in such wastewaters (Ciputra et al. 2010; Bassandeh et al. 2013); (2) textile dyeing wastewaters (Fan et al. 2014; Li et al. 2015); (3) influent and effluent samples from WWTP (Hur et al. 2011); (4) brewery wastewaters and corresponding treated wastewater (Janhom et al. 2009, 2011); (5) chlorination disinfection by-product precursors and disinfection by-product formation potential of reclaimed water samples (Hao et al. 2012); (6) dissolved extracellular and intracellular products from granular and floccular sludge (Tu et al. 2012); (7) full-scale coke WWT monitoring using a novel anoxic/aerobic/aerobic process (Ou et al. 2014); (8) determination of effluent OM fractions in full-scale MBR (Jacquin et al. 2017).

On the other hand, combining EEM with PARAFAC models has also been conducted to characterize DOM from complex samples ranging from effluent, river and lake samples, water recycling treatment plants, and domestic and industrial WWTP from different sampling sites (Chen et al. 2003; Wang et al. 2007; Murphy et al. 2011; Meng et al. 2013; Yu et al. 2013; Cohen et al. 2014; Li et al. 2014a; Yu et al. 2015). Riopel et al. (2014) assessed the changes of colored natural organic matter (CNOM) at the inlet and outlet of a WWTP, and determined if these changes were correlated with routinely measured parameters. The authors found weak but statistically significant correlations between the total FLC signals and total carbon (TC), and between protein-like FLC of the protein-like components and the effluent total Kjeldahl nitrogen (TKN) and ammonium.

The possibility of using FLC spectroscopy for real-time applications on sludge samples and biological process control was recently studied (Arunachalam et al. 2005; Galinha et al. 2011a, b, 2012). The sludge digestion process was monitored by using an online fluorometer and the protein fluorescence profiles were used to describe the volatile solids reduction (Arunachalam et al. 2005). Using an MBR for WWT, Galinha et al. (2011a, b, 2012) found that the information captured by FLC spectroscopy, coupled to multivariate statistical analysis (PARAFAC, PCA, and PLS), could replace a number of analytical and operating parameters (COD, fouling indication, nitrogen forms and phosphorus). In MBR it was also found that organic substances with fluorescence characteristics, in both external and internal foulants, were identified as protein-like substances and soluble microbial by-product-like materials by EEM technology (Zhu et al. 2011).

Sludge samples have also been analyzed through FLC spectroscopy for real-time monitoring of sludge composition, mainly for EPS characterization. Yu et al. (2010) studied the sludge dewaterability based on EEM and PARAFAC analysis, concluding that it is affected by humic acid-like and fulvic acid-like substances. Using ethylenediamine tetraacetic acid (EDTA) in the extraction method, protein-like and humic-like substances were found to be specific signatures of the EPS (Bourven et al. 2012). Furthermore, it was found that the bacterial aggregates type and source is also an important factor improving EPS sample distinction (Sheng and Yu 2006; Bhatia et al. 2013). Domínguez et al. (2010) studied four EPS extraction methods, and using EEM spectroscopy revealed that the FLC spectra showed the presence of protein-derived compounds in all extracted EPS. Furthermore, humic-like substances were found when formaldehyde was used in the EPS extraction.

Interaction studies between a number of different compounds and EPS have been conducted, regarding to their distribution in microbial aggregates, by EEM spectra analysis. Strong interactions were found to took place between EPS and (1) aluminum ions (a flocculant improving settling properties) (Ruan et al. 2013), (2) copper ions in activated sludge (a heavy metal with environmental toxicity) (Sheng et al. 2013), (3) copper ions in denitrifying phosphorus removal sludge (Wang et al. 2015), (4) antibiotics (sulfamethazine and tetracycline) (Xu et al. 2013; Song et al. 2014), and (5) salinity (Wang et al. 2013), with the functional groups changes reflecting the bacteria response to each compound.

Also lab-scale experiments in SBRs have been already conducted to evaluate the feasibility of using EEM FLC spectrometry for EPS characterization. Three main components, protein-, fulvic- and humic-like substances, were identified from the extracellular substances when PARAFAC was applied to extract the individual spectra from the overlapped spectra (Li et al. 2008). Also using an SBR and PARAFAC models, the EPS composition was studied by Ni et al. (2009), concluding that the proteins and fulvic-acid-like substances increase in the substrate uptake, but decrease in the endogenous phase. The effect of C/N ratio on EPS composition was investigated by Wang et al. (2014a) and protein-like FLC peaks were identified associated with tryptophan protein-like and aromatic protein-like substances. Furthermore, it was found by Wei et al. (2014) that the sludge EPS chemical composition changed during the granulation process in AGS.

In conclusion, the use of FLC spectroscopy, together with multivariate regression models, can provide a reliable and rapid tool for monitoring wastewaters quality, making it possible, in future, to be monitored online, without the use of the current lengthy conventional measurements (Hao et al. 2012). Also EEM spectroscopy proved to be a suitable method to characterize, and trace, the OM of bacterial origin in WWT operations (Esparza-Soto and Westerhoff 2001).

4 Anaerobic systems

Recent trends in anaerobic systems monitoring utilize IR, UV–Vis and EEM spectroscopy coupled to multivariate analysis techniques for estimating a number of AD related parameters. A summary of the most important and/or recent works published in this field will be next presented.

4.1 UV–visible spectroscopy

UV–Vis spectroscopy has been suggested, by Hur et al. (2010), as a fast and versatile monitoring tool for BOD and COD in anaerobic processes. UV absorbance is correlated with the concentration of aromatic and polyaromatic compounds and, therefore, can be used to measure TOC, COD and BOD5 online in wastewaters (Spanjers and van Lier 2006). Chaparro and Pires (2011) studied the OM removal efficiency and the relation with the decrease of toxic and mutagenic effects for anaerobic reactors treating bleaching effluents from Kraft pulp mills. These authors also reported the use of UV spectroscopy as a complementary technique for the control and analysis of typical parameters as COD, BOD5, and TOC. The chemical nature, concentration of the dissolved fraction and physical characteristics of the materials are decisive for the absorption of certain wavelengths in a sample. According to the same authors, the presence of chromophoric groups, as residual lignin or lignin derivatives, usually found in the bleaching effluents, are the responsible for the absorbance in the UV–Vis region.

Other important parameter that can be measured by UV–Vis spectroscopy is the hydrogen sulfide (H2S) produced during an AD process. The bisulfide ion (HS−) and the H2S are in equilibrium, in liquid phase, and both chemical substances absorb light directly in the UV wavelengths. The potential of UV–Vis spectroscopy for the bisulfide ion detection, and hence for H2S, constitutes an advances in real-time UV–Vis spectroscopy. H2S is generated in the wastewater aqueous phase by the sulfate bacterial reduction under anaerobic conditions. Sutherland-Stacey et al. (2008) developed a method for measuring the total dissolved sulfide in sewage, in real-time, using a field UV–Vis spectrometer and a pH probe. These authors developed a technology based on existing commercially available sensors that are normally used for the monitoring of WWT systems, without more research or development of sensor hardware. The users will be able to upgrade the existing sensor systems to employ this methodology with just a software patch.

A flow injection analysis (FIA) system, based on UV–Vis spectroscopy, was also designed, which allowed the quantification of sulfide ions in fermentation broth and hydrogen sulfide in the gas phase. Such determination is relevant in relation to biogas cleansing as well as AD processes in general. The system consisted of one detection module and two sample-modules, which allowed quantification of the above chemical species on a single analyzer. The results indicated excellent system performances, with low detection limits suitable for AD process monitoring (Madsen et al. 2011; Redondo et al. 2008).

The monitoring and quantification of VFA can also be performed by UV–Vis spectroscopy. Wolf et al. (2011) presents a novel approach for their indirect and online determination using UV–Vis spectroscopic probes, accomplished by advanced chemometric approaches. An UV–Vis spectroscopic probe was used to measure the absorption, in a range of 200–750 nm, of all the fermentation sludge of a completely fermented process. Advanced chemometric methods for pattern recognition were then used to establish the non-linear relationships between the measured absorption spectra for the determination of organic acid concentrations. Several pattern recognition methods as linear discriminant analysis (LDA), generalized discriminant analysis (GDA), support vector machines (SVM), relevance vector machines (RVM), random forest (RF) and ANN were investigated and the comparison between all the methods where performed. For that purpose, a full-scale 1.3-MW industrial biogas plant was used and online measurements have been conducted. Results showed that, despite the fact of some of the methods considered do not present satisfactory results, accurate prediction for organic acid concentration ranges can be reached with both GDA and SVM-based approaches, with a correlation higher than 87% achieved on test data.

Although a significant number of works have been published using UV–Vis methodologies for AD processes monitoring, most authors prefer IR and FLC spectroscopy techniques.

4.2 Infrared spectroscopy

IR spectroscopy has been utilized for monitoring a number of AD parameters such as VFA, alkalinity, COD, TOC, TS and VS (Madsen et al. 2011). As main advantages of using IR spectroscopy stand out the low maintenance (Vanrolleghem and Lee 2003) and ability to simultaneously measure a number of different parameters with fairly reliable results. IR spectroscopy has been applied as a non-invasive, fast and easy to operate methodology to monitor the dynamic process of AD. However, most IR spectroscopy applications imply the combination with multivariate statistical methods, including PLS regression, PCA, nonlinear kernel partial least squares (KPLS) regression, locally weighted regression (LWR) and multi-layer partial least squares (ML-PLS) regression (Dahlbacka and Lillhonga 2013), among others, to quantify the different parameters.

The feasibility of NIR spectroscopy for substrate quality online monitoring, towards an improved control of biogas plant feeding, was the aim of a study developed by Jacobi et al. (2011). These authors designed, constructed and calibrated a NIR sensor system for the continuous monitoring of several parameters including dry matter (DM), VS, crude fat, crude protein, crude fiber and maize silage nitrogen-free extracts, and concluded that NIR can be used successfully for the online supervision of a significant number of quality parameters for the process of biogas production. However, some problems have been found, namely, changes in environmental factors as temperature, humidity or others that negatively influence spectral recordings which contribute for the difficulty into achieve high accuracy. It was also found that changes on substrate concentration and quality parameters could be quickly detected allowing real-time adaption of the feeding strategy.

The IR spectroscopy, more specifically Fourier transform near infrared (FT-NIR) spectroscopy was used by Reed et al. (2011, 2013), in combination with PCA and PLS regression for the monitoring of a sewage sludge anaerobic digester process. This technique allowed distinguish between different stages of the AD process and is important to refer that the waste activated sludge (WAS) and digested sludge were distinguishable from each other using this technique. The ability of reflectance NIR spectroscopy was also employed to follow process parameters important for its stability. The predictive models obtained were able to track the relevant process parameters: VFA, bicarbonate alkalinity (BA), TS, and VS. For a more accurate prediction, variations in temperature and OLR were employed to stress the digesters. As main conclusion, these authors affirm that this technique could be used to improve the performance of sewage sludge anaerobic digesters.

Other studies have shown that NIR spectroscopy and multivariate analysis can also be applied to estimate the COD (Sousa et al. 2007), TOC, total and partial alkalinity (Steyer et al. 2002; Ward et al. 2011), VS, ammonia nitrogen (NH4–N), total inorganic carbon (TIC) (Krapf et al. 2011), VFA (Lomborg et al. 2009; Zhang et al. 2009), acetic acid and propionic acid in AD processes. In is important to highlight that an increase in the sample matrices heterogeneity do not reveal a relevant performance loss of the NIR models.

Mid-infrared (MIR) spectroscopy is also a very promising technique to characterize the OM in AD processes. One major advantage over existing NIR sensors is that process variables such as VFA, TA, TIC, NH4–N, and TS show distinctive peaks in the MIR spectrum between 1800 and 800 cm−1, which makes it easier to correlate peak intensity to actual concentrations (Eccleston et al. 2015). The results obtain by these authors indicate that MIR spectroscopy is a valid method for online measurement of critical process variables in AD processes. Indeed, the work developed by Falk et al. (2015) confirms the ability of MIR spectroscopy, in combination with suitable PLS models, to provide a satisfactory method to monitor the AD process. According to these authors, which developed the PLS models with samples from an AD process and VFA spiked samples, the absolute VFA concentration in the digestate could be well predicted for acetic and propionic acid. However, as the higher chain fatty acids only occur in a smaller concentration, the prediction of their absolute concentration value was difficult. On the other hand, the dynamics of increasing and decreasing VFA concentrations could be easily monitored by the sensor system. Furthermore, the VFA monitoring made possible to implement a feeding strategy based on the actual bioprocess status and, hence, shorten the startup time for a biogas plant. These authors further concluded that the proposed system might be used to detect bioprocess problems at an early stage, making possible timely countermeasures.

Recently, attention has been focused on the assessment of biofuel quality parameters, determination of methane production potential of WWTP biomass and the quantification of biomethane production using IR techniques, which will be next discussed.

Carchesio et al. (2014) performed the comparison of biomethane production and digestate characterization for selected agricultural substrates, using IR spectroscopy. According to the authors, the biomethane potential of a substrate could be scientifically and easily evaluated by AD tests. Furthermore, the characterization of the digestates could be correlated with the biomethane production, allowing a better evaluation of the overall bioenergy conversion process.

The use of biomass and biofuel for energy purposes can be carried out in a number of ways such as combustion, gasification, pyrolysis, AD processes, etc. According to Chadwick et al. (2014), a number of biomass quality indicators, intermediate and final biofuel products, as for example, calorific value, moisture, ash, carbon and nitrogen contents, can be predicted accurately using IR spectroscopic methods. However, the most appropriate IR spectroscopy method will depend on a number of factors, such as the biomass type, conversion process, end product use and parameters of interest. In addition, recent studies proved the usefulness of NIR spectroscopy for the determination of biogas and/or methane yields. The conventional method of measuring the potential yield, the BMP test, is based on the digestion of selected substrates using an inoculum source extracted from an active biogas reactor (Ward 2016). This traditional method for BMP estimation is labor intensive and time consuming, with digestion times in the range of 30–100 days and, according to these authors, the use of NIR to predict gas yields can reduce the time of measurement to a few minutes or even be used for online measurements.

NIR spectroscopy was also recently used for predicting the BMP of different biomasses by Triolo et al. (2014). These authors highlighted the time reduction, from around 30 days by chemical determination to some minutes by NIR spectroscopy, as the great advantage of NIR in BMP measurements. For this purpose an innovative NIR method coupled to PLS modeling was performed in a comprehensive set of plant biomass samples. BMP prediction models involve PLS modeling, and the biomass methane production ability depends on the organic fraction (proteins, lipids and carbohydrates) composition, and the concentration of non-degradable lignin, slowly degradable lignocellulosic cellulose and hemicellulose. Biomass NIR spectra of samples contain data about the vibration modes of all the molecules involved in the process, and thus include all information regarding the samples composition. An appropriated PLS model can be used to extract this information from the NIR spectra and correlate it with the biomass methane production ability. Also Doublet et al. (2013) used this technology as an alternative method to predict the BMP of a broad range of organic substrates. A total of 296 samples, including most of the substrates treated by anaerobic co-digestion, were used by these authors for calibration and validation. The NIR based predictions of the BMP values were found satisfactory with an error of prediction of 40 mL CH4 g−1 VS fed. Despite the fact that satisfactory results were obtained, the authors concluded that the integration of the entire substrate diversity in the model remained, nonetheless, difficult due to the specific OM properties of the stabilized substrates and the high level of uncertainty of the predicted BMP values. Furthermore, the future development of a model encompassing new (unstudied) substrates would allow for practical widespread use of the NIR technique to monitor and operate anaerobic co-digestion plants. Raposo et al. (2011) already reported that errors in replicates, and differences in microbial inocula, can originate accuracy problems regarding the use of the conventional reference methods for BMP assays. Therefore, it should be also stressed out that some of the relatively low quality predictions obtained may be due to these constrains.

Recently, hyperspectral imaging has gained a close attention from a number of different fields, given that this technique integrates both NIR spectroscopic and imaging technologies, and provides both spectral and spatial information simultaneously (Zhang et al. 2016). Each pixel within the hyperspectral image is scanned continuously throughout a given spectral range, ensuring a gray-scale (intensity) image for each wavelength. The advantage of hyperspectral imaging lays on the possibility to predict a given set of parameters for every pixel within the AD dynamic process acquired images. The application of this technology to AD processes allowed a more accurate monitoring and control of the parameters influencing the AD, and contributes for an higher efficient energy production. In fact, Zhang et al. (2016) already studied the application of hyperspectral imaging to monitor pH values during an AD process and concluded that this method showed great advantages over traditional pH meters and pH sensors, including large reactors, online monitoring, and intelligent and automatic control. The use of a hyperspectral imaging system was also tested by Bonifazi et al. (2014) for real time monitoring of TS, VS, COD and NH4–N contents of a number of AD plants, with satisfactory results.

4.3 Fluorescence spectroscopy

FLC spectroscopy is used essentially for the characterization and quantification of DOM in anaerobic processes further correlated with a number of other parameters (Pang et al. 2014; Zheng et al. 2014; Zhou et al. 2015; Provenzano et al. 2014; Muller et al. 2014; Kang et al. 2016). Li et al. (2014b) studied the enhancement of bio-hydrogen production from kitchen waste by a short-time hydrothermal pretreatment at different temperatures. In this study, FLC spectroscopy was used for the analysis of the DOM composition, showing a predominance of proteins-like substances peaks. Furthermore, three fluorescent components were successively separated using FLC EEM spectra coupled to a PARAFAC model. It was also found that the maximum fluorescence intensities of protein-like components, decomposed through the PARAFAC, presented a significant correlation with the raw protein concentration, and this directly impacted the hydrogen production ability. More recently, the work of Hou et al. (2017) used FLC EEM spectroscopy to characterize the intracellular and extracellular substances of anammox sludge. The authors found that humic-like peaks presented a strong correlation with nitrogen removal rates and that the two intracellular protein-like peaks were highly correlated with the mixed liquor volatile suspended solids (MLVSS) and MLVSS growth rates.

Digestates are the final products of an AD process and are characterized essentially by the high contents of recalcitrant organic molecules and nutrients and by the high biological stability. Provenzano et al. (2011) characterized the digestates obtained from different biomasses mixtures using FTIR and FLC spectroscopy. The obtained results show that the different characteristics of the biomasses provide distinct final products. It was found that the digestates FTIR and FLC spectra produced in a real co-digestion plant presented the main spectroscopic characteristics of the initial organic wastes (substracts). EPS associated to the anaerobic treatments can also be discriminated by FLC spectroscopy (Zhang et al. 2011; Dai et al. 2013). Also, Luo et al. (2013) investigated the FLC spectral characteristics of EPS and DOM, during AD processes, using 3D-EEM FLC spectroscopy to explore the destruction mechanisms of WAS enhanced by enzyme (protease and/or a-amylase) addition. The authors found this method to be a fast, sensitive and selective analytical method and concluded EEM FLC spectroscopy to be appropriate for EPS and DOM characterization in AD fed with WAS. Recently, Yu et al. (2016) tested a submerged anaerobic dynamic membrane bioreactor (AnDMBR), in order to improve AD performance. It is known that, in AD processes, sludge hydrolysis leads to cell walls rupture and EPS release, providing soluble organic substrates, such as DOM, for acidogenic microorganisms. Therefore, DOM and bound EPS contents in sludge flocs are considered as significant parameters characterizing AD processes. Indeed, these authors explored the fluorescence properties of EPS samples using EEM with fluorescence regional integration (FRI) analysis, and found this technology to be very useful for the comparison between the AnDMBR and the conventional AD system. EEM peaks were distributed by five regions that can be classified as Regions I and II that includes simple aromatic proteins such as tyrosine and tryptophan, Region III composed by fulvic acid-like substances, Region IV that are soluble microbial by-product-like materials and Region V, humic acid-like substances. Substrates from Region II and IV presented high biodegradability and substrates from Region III and V exhibit low biodegradability. Higher percentages of Region II and IV substrates, alongside lower percentages of Region III and V substrates, were observed in the DOM fraction of the AnDMBR compared to the conventional AD system, indicating that the AnDMBR provided more favorable substrates for subsequent metabolism by the anaerobic consortia.

Also the VFA, other important parameter in AD processes, can be measured by FLC spectroscopy. Palacio-Barco et al. (2010) proposed a rapid, simple and accurate online spectrofluorimetric method to measure VFA in anaerobic treatment processes. The method consists on the derivatization of VFA with N-(1-naphthyl)ethylenediamine (EDAN) and further spectrofluorimetric detection of the corresponding amide. The results, obtained for an emission wavelength of 335 nm and an excitation wavelength of 395 nm, were found to be proportional to the VFA concentration, expressed as acetic equivalent, in the range of 19–1000 mgL−1, and with a detection limit of 5.1 mgL−1. Furthermore, the obtained results showed a good selectivity for VFA, when compared to other organic and inorganic compounds usually found in sewage sludges. The authors also highlighted the possibility for online VFA monitoring during fermentation processes, with special relevance in several industrial applications, and particularly in biogas production. As referred previously, these compounds are responsible for inhibiting fermentations if present in large concentrations. According to these authors, the proposed online fluorimetric system allows for selective VFA determination with a frequency analysis of nine samples per hour. Morel et al. (2004) examined the use of multi-wavelength fluorometry for online monitoring of an AD process. A set of assays were conducted in a laboratory-scale anaerobic digester, equipped with an inline fiber optic probe installed in the reactor external recirculation loop, and further used to acquire fluorescence spectra with an interval of 5–10 min. The obtained spectra were used to develop regression models for the prediction of COD, VFA, TSS and VSS concentrations. To that effect PLS, nonlinear principal components regression (NPCR) and step-wise regression (SWR) models were studied, with the PLS model presenting the best prediction accuracy. The authors reinforced that, despite the fact that were found deviations of the fluorescence-based predictions from the real values, due to process variability, might be expected, this problem can be minimized by periodic adjustment of the regression models using dynamic PLS algorithms. The results show the capability of using multi-wavelength fluorometry for real-time monitoring AD effluent composition.

As previously stated, the BMP test is the most commonly used method for BMP determinations and is an important tool for the monitoring of the anaerobic biodegradability of sewage sludge produced during WWT. For the development of advanced dynamic models, the detailed OM characterization and intimate knowledge of the involved metabolic pathways are needed. Jimenez et al. (2014) investigated a methodology able to characterize municipal sludge (in particular the two OM key parameters), the biodegradability and the bioaccessibility. The methodology is simple and is based on coupling sequential chemical extractions with 3D FLC spectroscopy. The increasingly use of different and stronger solvents conducted to distinct levels of OM accessibility, whereas the spectroscopy analysis was able to properly characterize the OM. The authors tested 52 municipal sludge samples and the results showed that the method developed can predicted with successful the two key parameters. The time required for the determination is about 5 days, while for the BMP tests the time needed is in the range of 30–60 days. Furthermore, experimental data obtained from two independent laboratory scale reactors was used to validate the advanced dynamic model, The PLS results combined with the advanced dynamic model allowed an efficient methane production prediction and better prediction of reactor performances. In 2016, Bellaton et al. (2016) presented a rapid method using a fluorescent redox indicator and the aim is to estimate the biodegradability of sewage sludge in AD processes. For that, the biodegradability and BMP of nineteen municipal sludge samples were analyzed and it was concluded that the proposed FLC analysis allowed a fast parameters estimation in only 2 days. The comparison with Automatic Methane Potential Test System II (AMPTS) confirmed the BMP prediction to a maximum admissible uncertainty limit of 25%. As main conclusion, the authors highlighted the potential of this rapid assay for predicting AD sewage sludge biodegradability.

It is known that EEM FLC spectroscopy provides an overflow of information, which by itself, can be very difficult to interpret. In the past, the techniques for characterizing EEM have generally relied on visual peaks identification and fluorescence ratios in different regions of the spectrum, but now multivariate data analysis techniques have been applied to the study of complex fluorescence signals data (Stedmon and Bro 2008). Zhou et al. (2013) refer PARAFAC and FRI as two commonly used methods for such endeavor. According to these authors, PARAFAC is a multivariate modeling technique able to decompose the DOM fluorescence signature into individual components and estimate the relative contribution of each component in the total DOM fluorescence. On the other hand, FRI is a quantitative technique that integrates the volumes beneath operationally defined EEM regions, with the normalized region-specific EEM volumes representing the relative abundance of DOM fractions in a water sample. Despite the fact that the above two methods are the most used in the FLC spectra analysis (Li et al. 2014b; Zhang et al. 2011; Dai et al. 2013; Zhou et al. 2015; Luo et al. 2013; Muller et al. 2014; Yu et al. 2016), other works have been published using PLS (Jimenez et al. 2014; Morel et al. 2004), PCA (Aemig et al. 2016) and SWR models (Morel et al. 2004). Table 4 shows the most relevant published studies using IR and FLC spectroscopy.

Table 4 Main studies performed using IR and FLC spectroscopy
Full size table

5 Concluding remarks and future prespectives

This review has shown that considerable efforts are being undertaken to develop spectroscopic methods for monitoring and quantifying key parameters involved in WWT processes. Hence, conventional methods, usually labor and time consuming could be replaced by low sample handling and faster methods. Nevertheless, some limitations have been also found, mainly regarding offline and online applicability. Also, spectroscopy methods are strongly influenced by temperature and other environmental factors, controlled solely by using offline methodologies, and the majority of online measuring techniques is still under development and mainly applied to lab-scale systems. Future studies will need to explore the applicability of these techniques to online full-scale systems for monitoring purposes.

Literature already described the availability of compact and low-cost UV–Vis spectrophotometers allowing for several applications of UV–Vis spectroscopy in biological WWT processes. However, the number of online studies remains relatively limited due to a number of drawbacks. Molecules such as saturated hydrocarbons and sugars are not detected by this methodology and UV–Vis spectra of aqueous samples are highly affected by the presence of suspended particles due to light scattering effects. Moreover, the visual inspection of spectral data is often not appropriate to extract significant information on sample composition, given the obtained broad and unspecific bands. Thus, it should be pointed out that the use of chemometric techniques is quite mandatory.

Regarding IR spectral data, it was found that the prediction of a number of compounds seems to be highly dependent on the sample composition. Thus, caution needs to be taken during the interpretation of the data and the extrapolation to other sample matrices, since indirect calibration models need to be developed in this methodology. A large number of variables must be considered before, during and after the development of a specific application, in order to develop robust IR spectroscopy applications. A significant challenge in the use of fluorescence spectroscopy as a monitoring technique is to overcome the matrix effects changing the fluorescence signature. Such effects could mask small differences in parameters concentration. Furthermore, the implementation of fluorescence instrumentation for online monitoring is relatively slow due to several factors, such as high quantities of suspended solids and fouling, among others. In order to counteract these issues, sample dilution is usually recommended. However, regarding real-time wastewater samples, the concentration and composition are highly variable and, therefore, the use of a single dilution factor may not be feasible. In addition, post-measurement mathematical correction may be necessary to reduce the impact produced by external factors.

Overall, to overcome the matrix effects provided by biological WWT processes is important to know the physics and chemical characteristics of the samples, as well as other key parameters such as particle size, moisture content, etc. Calibration models are commonly established with the knowledge of the conventional methods used as well as the standard error and the limits of detection. Careful interpretation of these calibrations regarding the measured parameter is also a critical point that should be taken into account.