1 Introduction

Sensitivity analysis (SA) is a fundamental tool for supporting mathematical models development and use (Tarantola and Saltelli 2003) because of its capability of explaining the variability in the outputs of the models themselves (Cariboni et al. 2007). Although SA was traditionally used to identify the parameters with the highest impact on model outputs and therefore those on which concentrate the calibration activities, it is increasingly used to analyse model structure and behaviour (Confalonieri et al. 2010a). In this context, SA was recently recommended as a tool to be used iteratively during the process of model development (Jakeman et al. 2006), to assure coherence in mathematical formalizations, to avoid over-parameterizations by driving simplification processes (Ratto et al. 2001; Tarantola and Saltelli 2003) and to support the development of balanced models (Confalonieri 2010). These features favoured the introduction of SA in different typologies of documents defining guidelines for model development (e.g. European Commission 2005; Jakeman et al. 2006).

Advanced SA techniques are increasingly used also in the field of agrometeorological modelling. Van Griensven et al. (2006) applied a novel sampling strategy to identify the most relevant parameters in the Soil and Water Assessment Tool catchment model for water flow, concluding that hydrologic parameters had the greatest impact on water quality. Ravalico et al. (2005) proposed new criteria for using SA within integrated models for environmental management and decision making. They applied different SA methods and found out that the Fourier Amplitude Sensitivity Test was the most suitable according to the proposed criteria. In the context of crop growth modelling, Richter et al. (2010) used the Morris SA method (Morris 1991) to identify the parameters of a complex crop model with the highest impact on Durum wheat yield formation at two locations, identifying the parameters involved with development and early light interception as the most relevant. Confalonieri et al. (2010b) applied the Morris and Sobol’ (1993) methods to a model for rice growth and development, comparing the SA results obtained for five European countries and, within each country, for 3 years characterized by different degree of continentality.

In spite of the interest of the modellers community in analysing the effect of weather variability on agrometeorological models behaviour (e.g. Wolf et al. 1996), most of the studies on SA carried out on agrometeorological models refer to works aiming at identifying the most relevant parameters of the models. In particular, according to the author, it is impossible to find in the literature studies where standard SA methods were used to analyse the impact of weather variables on cropping system model outputs. The reason is that SA methods sample values from a multi-dimensional hyperspace defined by an array of inputs; thus, within a SA experiment, a single value is assigned to an input for each simulation and that value is related to the simulation output(s). Regardless to the sampling strategy used, this leads SA methods to analyse a couple of two-dimensional matrixes: one with a certain number of parameters combinations and a second one with a list of outputs for each combination of parameters. In practice, when the relevance of weather variables on model outputs has to be analysed, it is not possible to use standard sampling techniques to derive coherent series of weather data where each value cannot be sampled independently from those of the neighbouring days and from the other variables in the same day.

The aims of this paper were:

  • To propose a procedure for analysing the sensitivity of agrometeorological models to weather variables via the analysis of the relevance of the parameters of a weather generator (WG), considered as synthetic representations of series of weather variables

  • To test the procedure using the CropSyst model (Stöckle et al. 2003) for spring barley simulations in northern Italy

2 Materials and methods

2.1 Methodology for estimating crop model sensitivity to weather variables

The methodology used for estimating crop model sensitivity to weather variables is shown in Fig. 1. In order to apply a standard SA method to quantify the relevance of weather variables (array of daily values for each variable) on the outputs of an agrometeorological model (state of variables in a specific moment), a sample of the possible combinations of WG parameters was created. This led to a 1:1 correspondence between each single combination of input factors (i.e. WG parameters) and the corresponding array of model outputs, thus allowing the use of a standard method for SA.

Fig. 1
figure 1

Flow chart of the methodology used to estimating crop model sensitivity to weather variables

Full size image

For each of the WG parameters (Table 1), a distribution was derived by (a) taking a 30-year weather series, (b) creating 30 29-year series by eliminating each time a single year and (c) estimating the WG parameters for the 30 29-year series. The Shapiro–Wilk test was then applied to verify the normality of the parameters distributions, and in case of deviation from normality, log-normal or triangular distributions were tested using the Kolmogorov–Smirnov test (Fig. 1).

Table 1 Parameters of the weather generator and statistical settings used for the sensitivity analysis
Full size table

The absence of significant auto-correlations among the WG parameters allowed avoiding distortions in the generated weather series. After generating the sample of WG parameter combinations using the SA method, a 30-year series of weather data was generated for each combination using the WG. Thirty years were generated to reduce the weight of the stochastic component of the generator in affecting crop model outputs. After the generation of the weather series from the sampled combinations of parameters, the input files for the crop model were prepared (one set for each combination) and the 30-year simulations carried out. Relevant model outputs at physiological maturity were then averaged for the 30 years and used to calculate the sensitivity indices for each of the parameters of the WG.

In order to discriminate between the impact of the variation in WG parameters on the outputs of the crop model and on those of the WG itself, (a) climatic indicators were calculated to synthesize the generated weather series, (b) they were used to calculate a second series of sensitivity indices for the generator parameters and (c) the two series of sensitivity indices (calculated on crop model outputs and on the climatic indicators) were compared. The synthetic climatic indicators used were (Table 2): accumulated degree days (ADD), synthetic agrometeorological indicator (SAM; Confalonieri et al. 2010c), Budyko aridity index (Bud; Budyko 1974) and cumulated radiation (CumRad).

Table 2 Synthetic agroclimatic indicators used to estimate the relevance of the weather generator parameters on the output of the generator itself
Full size table

2.2 The weather generator

Climak (Danuso 2002) is the WG engine of CLIMATICA (http://www.dpvta.uniud.it/~Danuso/docs/Climatica/Climatica_Home.html), a software for exploring, validating, rebuilding and managing climatic data. Climak proved its reliability under a variety of conditions, also in comparative studies (e.g. Acutis et al. 1998).

The occurrence of rain events is estimated stochastically by using a first-order Markov chain, with the state of each day (rainy or not) obtained from the state of the previous day and from the monthly based transition probabilities dry to dry, rainy to dry, dry to rainy and rainy to rainy. Precipitation amount is generated by sampling the values from a two-parameter (alpha, beta) gamma distribution. alpha and beta are estimated for each month by using the method of the moments. Maximum and minimum air daily temperatures are estimated separately, using different parameters for rainy and dry days. Sinusoidal trends are calculated using a second-order Fourier series for average daily minimum temperature for the dry (Tnd) and rainy (Tnr) days and for average daily maximum temperature (Txd for dry days, Txr for rainy ones). For Tn, the parameter of the Fourier series are annual average minimum temperature of the dry days (A; degree Celsius), semi-amplitude of the first and second terms (B and D, respectively; degree Celsius) and phase shift for the first and second terms (C and E, respectively; days; constants for a location). A, B and D are sampled from the normal distributions N(μ X , σ X ), N(μ X , σ X ), N(μ X , σ X ), with X representing A, B and D, to derive the trend for each year. The same procedure, with specific values for A, B, C, D and E is carried out for Tnr, Txd and Txr. Once the four trends are calculated, random residuals—sampled from bivariate normal distributions—are calculated on a monthly base for maximum and minimum temperatures and added to the trends. Maximum daily radiation is calculated as a linear function of daily astronomical photoperiod, with the latter derived from latitude and day of the year according to Keisling (1982). Daily global solar radiation is then derived from the ratio between daily and maximum radiation, calculated from the daily thermal excursion using a beta probability density function. Daily reference evapotranspiration is obtained as a linear function of daily solar radiation (Doorembos and Pruitt 1977). If radiation is not available, reference evapotranspiration is derived as a linear function of TP (maximum air temperature multiplied by the square of photoperiod and divided by 1,000). In both cases, reference evapotranspiration values are then adjusted by additive residuals sampled from a normal distribution.

2.3 The crop model

CropSyst (Stöckle et al. 2003) is a cropping systems simulation model, implementing a generic approach for crop growth. It has been successfully used worldwide for many crops (e.g. maize, Jara and Stöckle 1999; cotton, Sommer et al. 2008) under a variety of agroclimatic conditions.

Crop development is simulated as a function of thermal time accumulated between base and maximum temperature, optionally corrected to account for photoperiod, vernalization and water stress. Daily biomass accumulation is simulated using a net photosynthesis approach, with potential accumulation calculated both on intercepted photosynthetically active radiation (radiation use efficiency (RUE) approach) and on potentially transpired water (vapour pressure deficit-corrected transpiration use efficiency). Each day, the minimum between the two daily biomass rates is selected. Thermal limitation is explicitly accounted for only in the RUE-based approach. Water and nitrogen limitations are then applied to get actual daily biomass accumulation. Leaf area development is calculated from daily accumulated biomass using a constant specific leaf area and an empiric parameter. Crop yield is derived by multiplying the total biomass at harvest by a harvest index (i.e. yield to biomass ratio, HI), with the latter varying according to the specific sensitivity to water stress of the simulated crop or variety. Root depth is simulated as a function of leaf area development and was set to reach its maximum at flowering in this study. Leaf senescence is calculated by subtracting the dead leaf area index to the total one, with each daily emitted green leaf unit dying once a threshold amount of degree days is accumulated. Soil water redistribution can be simulated using a numerical solution (finite difference) of the Richards soil flow equation, or with a simple cascading approach.

The algorithms of the CropSyst model (version 3.02.23, last release of CropSyst 3) were implemented in a high-granularity, component-based environment (http://agsys.cra-cin.it/tools/bioma/help/). This allowed (a) to use the CropSyst algorithms for potential and water limited crop growth and development and for soil hydrology and (b) to feed the model with exogenous variables, like reference evapotranspiration (generated by Climak).

2.4 The sensitivity analysis method

The high computational requirements due to the high number of parameters of the WG (i.e. 141) suggested to carry out the SA using the Morris method (Morris 1991). The Morris method calculates elementary effects due to each input by calculating an array of incremental ratios (∆output/∆parameter) in different points of the inputs hyperspace, explored over different trajectories (r), composed by individual one-factor-at-a-time experiments. Assuming each input x i belonging to the k-dimensional vector X = (x 1,…, x k ) of the model inputs and after having rescaled all the variables in the 0–1 range, x i is forced to assume only p discrete values (i.e. levels) in the set {0, 1/(p − 1), 1/(p − 2),…,1}. The inputs hyperspace is thus sampled through a k-dimensional p-level grid. Average (μ) and standard deviation (σ) of the incremental ratios distribution are then calculated, with μ representing the overall influence (total effect) of the parameter and σ identifying (when it assumes high values) nonlinearities in model response or interactions with other parameters. In this study, the evolution of the Morris method proposed by Campolongo et al. (2007) was used, in light of its improved sampling strategy (better scan of the inputs hyperspace without increasing the number of model evaluation required). The Campolongo et al. (2007) approach also allows using μ* (instead of μ), representing the mean of the distribution of the absolute values of the elementary effects, thus preventing effects of opposite signs (with effects cancelling each other out) which occur when the model is non-monotonic. In spite of the low number of simulations required, i.e. r·(k + 1), the Morris method proved its reliability in ranking the parameters according to their relevance in different studies where it was compared with other methods more demanding in terms of model executions (e.g. Confalonieri et al. 2010a; Yang 2011).

2.5 The simulation experiment

The scenario identified for the simulations refers to spring barley sown on March 2 in the 45,057 50 × 50-km cell of the EC-Joint Research Centre MARS database (latitude 45°04′ N; longitude 8°41′ E, altitude 100 m a.s.l.; data refer to cell centroid). The climate of the area belongs to the mesoclimate of the Po Valley, with a discrete level of continentality mitigated by the relative closeness of the Mediterranean. Precipitation (about 850 mm year−1) is relatively well distributed with two maxima in autumn and spring. Mean annual temperature is about 13.5°C, with thermal maximum and minimum usually in July–August and January–February, respectively. The soil chosen is clay loam (well represented in the study area), with average values for sand and clay of 38.8% and 36.8%, respectively, in the first 100 cm.

Barley is not irrigated in the region, and no nitrogen effect on crop growth was simulated. The simulations were therefore carried out only under water-limiting conditions. The CropSyst parameters for spring barley proposed by Donatelli et al. (1997) were used, and soil water redistribution was simulated using the approach based on the Richards’ equation. Local experience suggested a slight/moderate HI sensitivity to water stress for spring barley: Related model parameters were thus set to 0.3 (like in Donatelli et al. 1997), with 0 and 1 corresponding to no sensitivity and maximum sensitivity, respectively. Output variables considered are physiological maturity date (MatDate; day of the year), yield (tons per hectare), cumulated drainage (CumDrain; millimetres) and actual evapotranspiration (CumActET; millimetres).

The Morris method was parameterized with seven trajectories and six levels, since these values allowed obtaining stable results in previous studies performed with similar models (Confalonieri et al. 2010a) and in preliminary tests carried out within this study (data not shown). This led to a total of 994 30-year simulations (seven trajectories multiplied by the number of Climak parameters plus one).

3 Results and discussion

3.1 Distributions of the weather generator parameters

For each of the WG parameters, the obtained distribution (see Section 2.1) is shown in Table 1. The parameters resulted normally distributed in 84 out of 141 cases, whereas log-normal distributions were adopted for 49 parameters; triangular distributions were used for the remaining ones. For parameters involved with rainfall generation, transition probabilities followed almost always a normal distribution, whereas other distributions were generally adopted for the alpha and beta parameters of the gamma distribution used for rainfall amount. Concerning the parameters directly involved with temperature data generation, non-normality was detected for the standard deviation of the parameters of the Fourier series (A, B, D) and for about half of the parameters involved with temperature residuals. Also the radiation and evapotranspiration parameters resulted non-normally distributed in about half of the cases.

3.2 Sensitivity analysis results

The CropSyst output variables simulated for the 994 combinations of WG parameters were averaged for the 30 years simulated for each combination. Mean simulated yield and maturity date were 4.57 t ha−1 and June 24, in both cases coherent with what is usually observed for fully fertilized spring barley in the region. Simulated maturity dates presented a low variability (standard deviation = 0.4 days), whereas yield data were more spread (with a coefficient of variation (CV) of 3.1%), like CumActET ones (CV = 2.3%), although the highest variability was observed for CumDrain (CV = 13.9%). In any case, all the output variables resulted to come from normal distributions, with p values of 0.117, 0.452, 0.669 and 0.507 for MatDate, yield, CumDrain and CumActET, respectively.

Looking at the overall influence of parameters (μ*), crop development (Fig. 2a) was, to a large extent, influenced by b1 (slope of the linear regression equation to derive maximum radiation from photoperiod) and PRD Feb (rainy–dry transition probability for February), with a secondary, although important, role played by three temperature-related parameters (Tnd mean A, Tnr mean C, Txd mean D, all involved with the Fourier series used to calculate the yearly sinusoidal trends) and by a another radiation-related parameter (alpha2). The importance of temperature-related parameters is explained by the driving role of this variable for the accumulation of thermal time, whereas the relevance of PRD Feb is probably related to the availability of sufficient water to avoid water stress in case of dry springs. Water stress, indeed, accelerates development because of the assumption that low transpiration leads to a warmer plant (Stöckle et al. 2003), thus increasing the temperature perceived by the plant. The relevance of b1 and alpha2 is due to their effect on evapotranspiration, in turn involved in the computation of water stress too. The effect of b1 and PRD Feb in modulating the plant response to temperature (via the impact on water stress) was confirmed by the high values they achieved for Morris σ, suggesting interactions with other parameters (i.e. those related with temperature generation).

Fig. 2
figure 2

Sensitivity analysis results (Morris μ* and σ) for different output variables of the CropSyst model: a maturity date, b yield, c cumulated drainage and d cumulated actual evapotranspiration. The different symbols indicate parameters involved with rain (white circles), temperature (crosses), radiation (black triangle) and reference evapotranspiration (grey squares). Labels indicate the names of the most relevant parameters

Full size image

Figure 2b shows that the nine top-ranked parameters according to their relevance on yield were all related with temperature. In six out of nine cases, they were involved with the Fourier series for sinusoidal trends, with a prevalence of parameters related with the generation of temperature during dry days. The importance of temperature in influencing yield is explained by its effect both on thermal limitation to photosynthesis and on development, with the latter in turn influencing the simulation of processes involved with biomass accumulation like, e.g. leaf area evolution. Four out of six parameters involved with the generation of radiation data were in the 17% of the top-ranked parameters according to μ*. The lower importance of radiation-related parameters compared with that of temperature-related ones is explained by the fact that radiation-dependant biomass accumulation is used by CropSyst only in the days when vapour pressure deficit (VPD) is low; otherwise, the VPD-corrected transpiration use efficiency approach (Tanner and Sinclair 1983) to biomass accumulation is used. This consideration and the relatively low year-to-year variability in cumulated radiation during the spring barley growing period in the study area explain the overall highest relevance of temperature-related parameters, although radiation-related ones resulted anyhow having a great importance in influencing yields, as discussed by other authors (e.g. Rivington et al. 2002; Weiss et al. 2001; White et al. 2011). Seven parameters related with rainfall data generation were ranked among the 30 most relevant, with a slight prevalence of those influencing winter rainfall, probably because of their role in refilling the soil profile before the spring, when rainfall in the region is normally able to guarantee a sufficient amount of water to the crop.

SA results calculated on CumDrain are shown in Fig. 2c. Although parameters involved with the generation of temperature data were in the first six positions according to the value of μ* and they represented the 80% of the 20 top-ranked parameter, all the radiation-related parameters were among the 27% most relevant. Besides the impact of radiation on crop growth and therefore on the plant capability to uptake water from soil, the relevance of the parameters involved with radiation generation is due to the direct effect of this variable in driving evapotranspiration. The second effect (on evapotranspiration) is probably more important than that on crop water uptake because of the relatively low values of σ for the radiation-related parameters, thus suggesting low interactions with other parameters. On the contrary, the effect due to crop water uptake would have led to high interaction with others (e.g. temperature-related ones). Four rainfall-related parameters were classified in the 30 top-ranked, with a clear prevalence of parameters related with the generation of rainfall in winter months, when the crop is not present in the field (no crop water uptake).

Compared with the situation discussed for the other output variables, results of SA calculated on CumActET (Fig. 2d) show that, although temperature-related parameters were again the most relevant, the parameters involved with generation of radiation data presented a higher importance. alpha2 and b1 were ranked 4th and 5th, respectively, and other two radiation-related parameters were ranked among the 13% most relevant. This can be explained considering also the impact of radiation data on crop growth, therefore on how CropSyst calculates transpiration. The relevance of Setr and a0 (ranked 3rd and 16th, respectively) is explained by their effect in determining the atmospheric evapotranspiration demand.

Although the methodology proposed (Fig. 1) is generic, without site/crop/model-specific elements, the obtained results obviously depend on the explored conditions and cannot be generalized to spring barley grown under different agroenvironmental conditions and/or simulated by other models. For all the output variables analysed, the μ* values achieved by the most relevant inputs (those marked with Climak parameters names in Fig. 2) passed the significance test proposed by Morris (1991), based on the comparison between μ* values and the double of the standard error of the mean.

3.3 Analysis of the relationships between the sensitivity indices calculated on the outputs of the crop model and on those of the weather generator

Figure 3 presents the relationships between (a) the ranks—according to μ*—of the ten most relevant WG parameters resulting from the SA on crop model outputs and (b) the ranks obtained by the same parameters according to the SA carried out on the synthetic climatic indicators. Black bars indicate the rank obtained by each parameter for the four crop model outputs analysed. A value of zero indicates that the rank of the parameter (according to the SA carried out on the climatic indicators) is higher than 10. With the exception of MatDate (Fig. 3a), for which a certain coherence between the rankings calculated for the crop model output and for the accumulated degree days (base 0°C) was expected, the rankings derived from model outputs were always clearly different from those calculated on the climatic indicators. For example, the three top-ranked parameters for yield (RRnx Oct, SRx Feb, Txd st.dev. A) were out of the first ten for all the climatic indicators, like the 5th and those from the 7th to the 9th (Fig. 3b), whereas SRx Dec, which was ranked 4th for yield was ranked 2nd for Budyko aridity index and 6th for SAM. Another proof of the incoherence between the rankings obtained from crop model outputs and synthetic climatic indicators is that a single parameter for CumRad (alpha2), ADD (Tnr mean C) and Bud (SRn Dec) was present among the top ten parameters for yield (Fig. 3b), CumDrain (Fig. 3c) and CumActET (Fig. 3d), respectively. These considerations demonstrate that the sensitivity of the crop model outputs to the WG parameters is not affected by the sensitivity of the WG to the same parameters.

Fig. 3
figure 3

Comparison of sensitivity analysis results calculated on the outputs of the crop model and on climatic indicators synthesizing the outputs of the weather generator. Ranks of the 10 most relevant weather generator parameters according to μ* for maturity date (MatDate; a), yield (b), cumulated drainage (CumDrain; c) and cumulated actual evapotranspiration (CumActET; d) are compared with the ranks obtained by the same parameters for the climatic indicators accumulated degree days (ADD), synthetic agrometeorological indicator (SAM), Budyko aridity index (Bud) and cumulated radiation (CumRad)

Full size image

4 Conclusions

According to the author, this is the first time the sensitivity of a cropping system model to weather variables is analysed using standard SA techniques. The proposed procedure proved to be feasible and able to identify quantitatively the most relevant driving forces during the crop cycle. For rainfall, and minimum and maximum temperature, it was even able to discriminate the relevance of the variable in the different moments during the season.

In general, under the explored conditions, parameters involved with the generation of temperature were the most relevant, achieving the highest values for Morris μ* when the output analysed was yield, cumulated drainage and cumulated actual evapotranspiration. For maturity date, they represented 16 out of the 22 top-ranked parameters. For yield, and cumulated drainage and actual evapotranspiration, temperatures in the winter months achieved some of the highest relevance values. Radiation-related parameters played an important role in the simulation of cumulated drainage and actual evapotranspiration. Parameters involved with rainfall generation were generally high-ranked, although usually not achieving the highest values for none of the Morris metrics.

The procedure proposed was carried out under two assumptions. The first is that crop model sensitivity to weather variables can be analysed by quantifying the outputs variability in response to variations in the parameters of a WG, in turn assumed as synthetic representations of the weather variables themselves. The second assumption derives from the fact that the impact of parameters variation was estimated on the outputs of a ‘chain’ of models, constituted by the WG (in a certain way a model itself) and by a crop model. This leads to realize that part of the effect of the variation in the WG parameters surely affected the behaviour of both the generator itself and, in a second moment, that of the crop model. The procedure used to verify that the SA results calculated on crop model outputs were not compromised by the effect due to the WG led to conclude that, for all the outputs analysed, there was no coherence between the ranking of the most relevant parameters calculated on the outputs of CropSyst and on those of the WG. This allows concluding that the impact of WG parameter variations on the crop model is in a certain way more relevant than that on WG outputs, since the crop model proved to react to the parameters variations in such a way to hide the direct impact of parameters variation on WG outputs.

The proposed procedure increases our knowledge on the behaviour of crop and environmental models and could be used to support their development. Moreover, it could find application in refining the process of development of agroclimatic databases, e.g. by identifying the variables most impacting on operational models and thus the variables on which to concentrate the efforts for improving the approaches used for measuring, estimating, interpolating, downscaling etc. the variables themselves. Estimating the sensitivity of impact models to weather variables in different moments during the crop cycle could also support the definition of adaptation strategies, e.g. by supporting the identification of suitable sowing periods under climate change scenarios.