1 Introduction

According to the Intergovernmental Panel on Climate Change (IPCC), climate change is “a long-term, typically decades or longer, change in the state of the climate that can be identified by changes in the mean and/or the variability of its properties”. It may be due to natural internal processes, external forcing, or to persistent anthropogenic changes in the composition of the atmosphere or land (IPCC 2007). Based upon significant effect of climate change on water resources (Özdoğan 2011; Grossi et al. 2013; Shen et al. 2014) and flood events (Kundzewicz et al. 2013), general circulation models (GCMs) are increasingly used as main sources of information required to hydrological impact studies specifically in the regions vulnerable to climate change.

GCMs are main tools to generate climate change projections. Progress in GCMs during the recent decades is vast. Their validation with observations for the twentieth century has indicated that they are able to simulate the basic features of the climate of the Earth (Şen 2013). However, owing to the inherent uncertainty of GCMs (Foley 2010), they still suffer from spatial and temporal differences in their climate outputs, particularly in precipitation (Varis et al. 2004; Kundzewicz et al. 2008; Şen 2013). In addition, GCMs commonly have relatively low spatial resolution and therefore unable to resolve significant catchment-scale features, including topography and land use, as needed in hydrologic modelling and impact assessment (Moradkhani et al. 2010; Najafi et al. 2011a; Bozkurt et al. 2012). One way to make informed decision associated with a specific region is to obtain fine-scale regional information (i.e. downscaling) using one or more regional climate models (RCMs) that takes boundary conditions from different GCMs. Then either a multi-model approach (e.g. Van der Linden and Mitchell 2009; Najafi et al. 2011b) or an intercomparison strategy (e.g. Vaze et al. 2011; Bozkurt et al. 2012) can be used to find more appropriate combination(s). In both, a catchment-scale performance evaluation of GCM-driven RCM outputs at a certain period of time, commonly referred as reference period, would be crucial to realize inevitable biases of the ensemble or the best GCM-RCM combination.

In addition to intercomparison studies (e.g. Hofstadter and Bidegain 1997; Errasti et al. 2011; Barfus and Bernhofer 2014; Anandhi and Nanjundiah 2015), there are number of researches about the performance of GCM-RCM combinations (e.g. Jones and Reid 2001; Xuejie et al. 2002; Harvey and Wigley 2003; Jacob et al. 2007; Smith and Chandler 2010; Bozkurt et al. 2012; Roosmalen et al. 2010; Fu et al. 2013; Deidda et al. 2013). Xuejie et al. (2002), using a GCM-RCM combination model, CSIRO-driven RegCM2, evaluated the performance of the combination for present-day reproducing and future change of different measures of extreme rainfall and temperature events in China. The authors reported that the CSIRO-RegCM2 combination can reproduce extreme events well in the country. Nieto and Rodríguez-Puebla (2006) compared the observational and simulated precipitation variations over the Iberian Peninsula by using empirical orthogonal function and spectral analyses. Their results pointed out that the Geophysical Fluid Dynamics Laboratory Coupled Model (version 2) resulted in the best correlation with the observed annual rainfall cycle. Perkins et al. (2007) evaluated the performance of the IPCC’s fourth assessment report models over Australia in terms of simulating daily maximum temperature, daily minimum temperature, and rainfall probability density functions (PDF). The MIROC-M, CSIRO, and ECHO-G models were recommended as more skilful ones for use in climate change impact assessments over the country. In a similar study, Maxino et al. (2008) reported that the CSIRO, IPSL, and MIROC-M are relatively well models to capture the PDFs of the maximum and minimum temperatures and rainfall over the Murray-Darling Basin in Australia. Fu et al. (2013) developed a score-based multi-criteria method to assess and rank the regional performance of 25 GCMs for the southeastern Australia. The results indicated that two GCMs including CGCM3.1-T47 developed by the Canadian Centre for Climatic Modelling and Analysis and INM:CM30 developed by the Institute for Numerical Mathematics in Russia are the two best models, whilst the Beijing Climatic Centre CM1, Bjerknes Centre for Climatic Research BCM20, and Atmospheric Research CSIRO:MK35 resulted in relatively crude representations of monthly rainfall climate.

There are also some regional climate change impact assessments and intercomparison studies in Turkey (e.g. Önol and Semazzi 2009; Ozkul 2009; Önol et al. 2014; Bozkurt and Sen 2013). To the best of our knowledge, the only performance evaluation study for the country (so far) has been carried out by Bozkurt et al. (2012). Comparing three GCM-RCM combinations (i.e. ECHAM5-RegCM3, CCSM3-RegCM3, and HadCM3-RegCM3) with those of different sets of observations at eastern Mediterranean–Black Sea region for the period 1960–1990, the authors recommended that all of the combinations can be considered for climate change impact assessment in the study region.

Considering spatial resolution degree of current GCMs or even GCM-RCM combinations, none of the abovementioned results obtained from either intercomparison or performance evaluation studies are directly applicable for catchment-scale floodplain engineering works. Additionally, most of the studies compare climate model outputs with reanalysis data that might not be reliable enough for performance evaluation at catchment scale, specifically in the ungauged or poorly gauged catchments. The reason behind this is that not all reanalysis data are created by observations of the catchment of interest. Some data types, such as precipitation (depending on the reanalysis) are obtained by running (presumably newer) GCM or numerical weather prediction models. Addressing these issues, an applied grid-based GCM-RCM performance evaluation method as well as an intercomparison study between two GCM-RCM combinations are presented in this study. The results will be used for selecting the more reliable climate model outputs as boundary conditions for an ongoing catchment-scale climate change impact assessment project FRA-PFC Rize: “Flood Risk Assessment: Present and Future Conditions in Rize Province” in Turkey. In addition, in the present study, we provide a grid-based overall performance analysis, whereas most of the previous intercomparison studies did not discuss the overall performance of climate models. Although some catchment-scale hydrological impact studies have been carried out in recent years (e.g. Graham et al. 2007a, 2007b; Bastola et al. 2011; Mondal and Mujumdar 2012; Halmstad et al. 2013; Demirel et al. 2013), to the best of our knowledge, there is only a few studies in the relevant literature focused on the performance evaluation of GCM-RCM combinations in catchment scale (e.g. van Vliet et al. 2012 and Deidda et al. 2013). Since such studies make use of small areas to compare the performance of several GCM-RCMs for their ability to reproduce hydrological variables (such as rainfall, temperature, and others), there is still need for more studies to establish a reference for hydrological applications in general. Information about the implemented models, methods, and observational data together with the obtained results has been described in the following sections.

2 Models description

Our analysis is restricted to the two GCMs (i.e. ECHAM5 and CCSM3) integrated with an RCM namely RegCM3 due to the fact that they were only available models for the study region providing daily outputs at the time of analysis, and therefore, the ongoing climate change impact assessment project on the study area is based on projections obtained by these combinations. Since the results are planned to be used as a basis for (i) selecting more reliable GCM-RCM combination, (ii) implementing/developing an appropriate bias correction method, and (iii) subsequent projected extreme rainfall modelling (Danandeh and Kahya, submitted), five rainfall-borne statistics (are explained in Section 5), derived from GCM-RCM outputs have been considered for performance evaluation in this study.

RegCM3, an upgraded version of RegCM2 developed by the International Centre for Theoretical Physics in Italy, is an RCM with sigma-pressure vertical coordinate that comprises a radiative transfer package, a non-local boundary layer scheme, an ocean surface flux parameterization, an explicit moisture scheme, and a large-scale cloud and precipitation scheme with several options for cumulus convection scheme for the model simulations (Bozkurt et al. 2012). Detailed descriptions of physical parameterizations of RegCM3 can be found in Pal et al. (2007).

ECHAM5 is a coupled atmosphere-ocean GCM developed at the Max Planck Institute for Meteorology, Germany. The model is used in a number of configurations which differ in the vertical extent of the atmosphere as well as the relevant processes. ECHAM5 is described in detail in Roeckner et al. (2003). CCSM3 is the third generation of the Community Climate System Models that has recently been released to the climate community. CCSM3 is a coupled GCM with components representing the atmosphere, ocean, sea ice, and land surface connected by a flux coupler. The model was designed to produce realistic simulations over a wide range of spatial resolutions, enabling inexpensive simulations lasting several millennia or detailed studies of continental-scale dynamics, variability, and climate change (Collins et al. 2006). The code, documentation, input datasets, and model simulations are freely available from the model website. Detailed descriptions of numerical parameterizations and physics of CCSM3 can be found in Collins et al. (2006).

In order to downscale the outputs of ECHAM5 and CCSM3 over Turkey, the relevant simulations were performed continuously from 1 January 1960 to 31 December 1990 at Istanbul Technical University through the project of the United Nations Development Program entitled “Enhancing the Capacity of Turkey to Adapt to Climate Change”. Detailed information about the project can be found in Bozkurt et al. (2012) and Önol et al. (2014). The reference period of GCM-RCM outputs used in this study has been selected identical to the present-day performance evaluation period (i.e. 1960–1990) implemented in the project. Considering the adjustment period (spin-up), we removed the first year outputs from the simulations’ archive before computing climate statistics.

3 Rainfall climatology of the study area

The study area is Rize Province, located in eastern Black Sea region between the Pontic Mountains and the Black Sea in Turkey, which is considered as the “wettest” corner of the country (Fig. 1). The province with a total area approximately around 3900 km2 encompasses ten small to mid-size catchments. According to Koppen-Geiger climate classification, it has a borderline oceanic/humid subtropical climate similar to most of the eastern Black Sea region, with warm summers and cool winters. Annual precipitation averaging in the province is around 2500 mm with a maximum rate commonly in late autumn (October to December). Rainfall climatology (i.e. long-term mean) of the study area at reference period 1961–1990 is investigated in two portions as inland and coastland areas by mainly considering the elevation differences from sea level that depict two primary precipitation regimes. With respect to the well-documented significant effect of coastline on rainfall distribution over the coastal area of the eastern Black Sea region (Eris and Agiralioglu 2009), such a separation provides more accurate climatology for each portion due to elimination of averaging error. A digital elevation model including the borders of the study region and utilized rain gauges locations to derive the province climatology is presented in Fig. 1.

Fig. 1
figure 1

Locations of the study area, Rize Province, and rain gauges used in the study

Full size image

According to the figure, coastland is classified as the portion where the elevation is below 1500 m and inland corresponds to the areas where the elevation is greater than 1500 m, mainly constituting the headwaters of the catchments. As shown in the figure, precipitation data were obtained from ten rain gauges. Half of them are operated by the Turkish State Meteorological Service (TSMS) and the remaining stations are operated by the Turkish State Hydraulic Works known as DSI in Turkey. DSI provides only monthly records. Thus, we used the relevant data only for estimating the monthly rainfall climatology of the study area. Homogeneity of the annual rainfall records in the eastern Black Sea region for the period of 1960–2005 has been already proved by Eris and Agiralioglu (2009). Rainfall climatology of each gauge for the reference period is presented in Table 1 that indicates notable annual rainfall difference between gauges located in the coastland and inland portions. In contrast to the generally accepted idea of the increase of precipitation with height, the orographic effects cannot be seen for the study area. It means that mean annual precipitation decrease with distance from sea. Such a pattern was already shown for entire eastern Black Sea region by Eris and Agiralioglu (2009). The authors investigated the precipitation records of 33 rain gauge stations (46-year period) across the entire eastern Black Sea region and demonstrated that mean annual precipitation decreases with elevation as well as distance from the sea.

Table 1 Monthly and annual rainfall climatology (mm) of utilized rain gauges
Full size table

In order to estimate areal distribution of rainfall climatology, two different averaging methods including arithmetic mean and isohyet lines have been used in this study, and the results have been presented in Table 2. We utilized Kriging optimal interpolation method (Kitanidis 1997) to generate isohyet maps. For instance, isohyets produced for autumn season have been presented in Fig. 2. Five selected RegCM3 grids, providing good spatial coverage of the province, having the resolution of 27 km are also shown in the figure. In this study, the results of isohyet line method have been considered to compare with those of climate model outputs at each grid (i.e. grid-based performance evaluation). In addition, with respect to the total area of grids overlaid on each portion, the area weighted average method has also been applied to portion-based performance evaluation.

Table 2 Climatology of Rize Province (total monthly precipitation, mm)
Full size table
Fig. 2
figure 2

Examples of long-term mean of total monthly rainfall over Rize Province: a September, b October, and c November

Full size image

4 Evaluation criteria

Our review in the literature demonstrated that there is no agreement in the scientific community on how to determine the best-performing climate models. It is up to the researchers which performance measures are used. In the present study, precipitation realization performance of the GCM-RCM combinations is analysed based on five measures comprising (i) scatterplots of simulation/observation, (ii) bias (simulation–observation), (iii) mean absolute bias (MAB), (iv) root mean squared error (RMSE), and (v) model performance index (MPI). The first four measures are applied only to individual climatic statistics whereas the last one (i.e. MPI) is applied to represent the overall performance of the model. Concepts and mathematical expressions of the scatter diagram, bias, MAB, and RMSE are well documented in a number of model performance studies (e.g. Danandeh Mehr et al. 2013; Liu et al. 2014; Demirel and Moradkhani 2015). In order to avoid repetition, only the MPI measure is therefore described in this section. The MPI, firstly introduced by Reichler and Kim (2008), is an overall performance evaluation index, which measures the reliability of a model based on the scaled-normalized error variance (I 2) of a broad range of climate variables (v).

$$ {I}_i^2={e}_{vi}^2/\overline{e_{vi}^2} $$
(1)
$$ {e}_{vi}^2={\displaystyle \sum_{j=1}^J\left({w}_i\left({\overline{M}}_{vij}-{\overline{O}}_{vj}\right)/{\sigma}_{vj}^2\right)} $$
(2)

where \( {e}_{vi}^2 \) is normalized error variance of ith model; \( \overline{e_{vi}^2} \) is the average error found in a reference ensemble of models; J is the number of grid cells in the catchment; M vij and O vj are the modelled and corresponding observed climatology, respectively; σ 2 is the inter-annual variance from the observations; w i is proper weight needed for area and mass averaging model; subscripts i and j are the indices for each model and grid points, respectively; and the overbar indicates averaging.

To rank climate models with respect to overall climatic statistics, the overall MPI is derived as the mean over the I 2 of all climate variables. This is based on the assumption that a reliable model should represent several components of the climate system of whole catchment.

$$ MPI=\overline{I_i^2} $$
(3)

The MPI varies around one with values greater than one for underperforming models and values less than one for more accurate models. The expression \( \overline{M}-\overline{O} \) in Eq. (2) indicates the error between simulation and observation. The smaller the error, the smaller the index and the better the model. More details about MPI can be found in Reichler and Kim (2008).

5 Results and discussion

In this section, ability of ECHAM5-RegCM3 and CCSM3-RegCM3 to reproduce five climatic statistics at each RCM grid as well as two different lands across Rize Province are investigated. The climatic statistics include the following: (i) total monthly rainfall (i.e. sum of long-term mean of daily rainfall at each month; TMR), (ii) monthly maximum rainfall (i.e. the maximum value among long-term mean of daily rainfalls at each month; MMR), (iii) total annual rainfall (i.e. sum of long-term mean of daily rainfall; TAR), (iv) annual maximum rainfall (i.e. the maximum of daily rainfalls at each year; AMR), and (v) number of rainy days (NRD). The choice of each grid as performance evaluation zone is dictated by the type of observational data sets available at each grid. It is worth mentioning that simulation results at the reference period 1961–1990 composed of 10,957 days (365 × 30 + 7 leap days) at each grid.

5.1 Monthly performance

It is conventional to evaluate simulation/prediction models first by scatterplots of simulation/observation, so that we will be able to realize the uncertainty features of simulation results. The scatterplots of the observed and modelled TMR and MMR data for the two portions and the entire province are provided in Fig. 3. Significant differences between the results of different GCM-RCM combinations are clear in the figure. According to the Fig. 3 (left), generally, ECHAM5 tends to overestimate the TMR for the entire province whilst CCSM3 yields slightly dryer condition. In terms of MMR reproduction associated with each portion, it can be indicated that ECHAM5 resulted in higher performance than CCSM3, particularly for the inland portion (see Fig. 3 (right)). To evaluate the efficiency of the GCM-RCM models more thoroughly, variations of the observed and modelled TMR and MMR values at each month for each portion are depicted in Fig. 4. Differing largely from each other, both GCMs show the same annual trends, which are more or less consistent with the observed climate trend. In contrast with ECHAM5, the CCSM3 underestimates the TMR amount in seasons other than winter and early spring. With respect to the MMR simulation results (see Fig. 4 (left)), both GCM-RCM combinations reproduce substantial biases in the coastland portion. Relatively good agreement with reference climatology is only provided by ECHAM5 in the inland portion. The coastland area in eastern Black Sea frequently receive more rainfall in the second half-year (July–November), whilst both ECHAM5 and CCSM3 simulated the opposite pattern of approximately first half-year rainfall (December–May) greater than the second half-year. Apart from February, March, and April, ECHAM5 outperforms CCSM3 at whole year. It can be concluded that the major uncertainty in maximum monthly simulation for the province belongs to the coastland portion, which may originate from the local effect of the Black Sea that is not well simulated by the models.

Fig. 3
figure 3

Scatterplots of total monthly rainfall (left) and monthly maximum rainfall (right) at each portion and the entire Rize Province for the period 1961–1990

Full size image
Fig. 4
figure 4

Total monthly rainfall (top) and monthly maximum rainfall (bottom) climatology of Rize Province for the period 1961–1990

Full size image

In order to provide profound discussion on simulation results, Table 3 presents quantitative errors of climate models for the both TMR and MMR statistics at each portion. Considering TMR, the highest bias value comes from ECHAM5 reproduction at the inland portion, whereas it interestingly reproduces the lowest bias for the coastland estimation. The spatial inconsistency of a climate model in fine resolution reveals the fact that GSM-RCM intercomparison studies should be carried out in catchment scale if the results are expected to be applied in hydrological design. The RMSE values of ECHAM5 and CCSM3 for the TMR estimation vary from 67 to 188 and 70 to 93 mm, respectively. According to the mean observed monthly rainfall for the entire province being equals to 122 mm, the results indicated up to 154 and 76 % uncertainty for ECHAM5 and CCSM3, respectively. Regarding the mean of observed MMR (equal to 9.0 and 13 mm at the inland and coastland portions, respectively, see Fig. 4), the best simulation performance for reproducing MMR over the province belongs to ECHAM5 with 32 and 47 % uncertainty at inland and coastland portions, respectively. Corresponding uncertainty values for CCSM3 results are 52 and 71 %, respectively.

Table 3 Portion-based bias, MAB, and RMSE of GCMs driven RCM simulations (mm)
Full size table

5.2 Annual performance

In order to identify uncertainty features of annual rainfall reproductions, three different statistics including TAR, AMR, and NRD are considered in this study. Owing to types of the data sets available in TSMS and DSI rain gauges (see Section 3), the annual performance is examined in grid-based level where at least one TSMS rain gauge is located in the grid of interest. In grid-based analysing, expected results are more reliable for different part of the province; however, one can develop portion-based deduction using the mean of performance of grids laying at each portion. In the present study, we derived overall result based upon both arithmetic mean and MPI calculated at grid level. Areal distribution of TSMS and DSI rain gauges across the province as well as grids identification number adopted for the study have been displayed in Fig. 5.

Fig. 5
figure 5

Distribution of rain gauges Turkish State Meteorological Service (TSMS) and Turkish State Hydraulic Works (DSI) and the location of five RegCM3 grid boxes surrounding the Rize Province

Full size image

Thirty-year scatterplots of the TAR and AMR for grids 2 and 4 as well as 23-year scatterplot for grids 3 (due to 7-year recording gap in Kaptanpasa station) have been presented in Fig. 6. Because of the lack of daily precipitation records at grids 1 and 5, the comparison was only performed for grids 2 through 4. Highly scattered simulations of CCSM3-RegCM3 particularly in grids 2 and 4, which roughly represent the coastland portion, indicate the inability of the combination to simulate reliable TAR and AMR specifically in coastland portion. According to the figure, ECHAM5-RegCM3 provides wetter condition than CCSM3-RegCM3 in all grids. Dense scatter of ECHAM5-RegCM3 outputs around the perfect model diagonal line in all grids implies superiority of ECHAM5 to CCSM3 for both total annual and annual maximum rainfall simulation at Rize Province. Thus, projected AMR series produced by ECHAM5-RegCM3 may be more reliable to consider in the FRA-PFC Rize project.

Fig. 6
figure 6

Scatterplots of modelled and observational total annual rainfall (top) and annual maximum rainfall (bottom). Plots were given for the RegCM3 grid possessing at least one TSMS rain gauge

Full size image

Owing to the importance of projected AMR series on extreme events, efficiency of the climate models is considered to supplementary evaluation using boxplot of the observed and modelled AMR series at reference period (see Fig. 7). The horizontal line inside the boxes represents the median and the upper and lower vertical lines, whiskers, indicate the 75 and 25th percentiles, respectively. The figure depicts a positive spatial trend in both minimum and maximum values over the province, which indicates an increasing rainfall pattern from the southwest to northeast of the study region. Both ECHAM5 and CCSM3 more or less follow this pattern in all grids, except in grid 1. The figure shows considerable differences between modelled and observational extremes. Only in grid 2, CCSM3 reproduced maximum value of AMR series (equals to 139 mm) and its event year (i.e. 1979) identical to those of observational data series interestingly. Considering the medians, the figure also implies that ECHAM5 reproduces closer values to the observed median than those of CCSM3.

Fig. 7
figure 7

Box and whisker plots of modelled and observational annual maximum rainfall values at the RegCM3 grids for the period 1961–1990

Full size image

As another annual climatic statistic for grid-based intercomparison, Fig. 8 compares the observed and simulated long-term average of NRD, defined here as days of rainfall in excess of 1 mm. The figure demonstrates that the annual NRD in the entire province falls in the reasonably consistent range 129–135 days. With respect to ECHAM5 and CCSM3 reproductions, the rainy days are within the wide range 158–196 and 78–115 days, respectively. The figure also indicates that ECHAM5 tends to overestimate the observational NRD for the entire province in the range 20–52 %. By contrast, CCSM3 underestimates in the range of 15–42 %. It is obvious from the figure that the lowest percentage error (−15.4 %) of rainy days estimation belongs to CCSM3 and the highest one (51.9 %) belongs to ECHAM5.

Fig. 8
figure 8

Number of rainy days at each RegCM3 grids (left) and percentage of simulation error (right). The percentage errors were given for the grid possessing at least one TSMS rain gauge

Full size image

The annual simulation results at each grid have been numerically compared in Table 4. As qualitatively discussed, the tabulated results verify the superiority of ECHAM5 to CCSM3 in terms of simulating both TAR and AMR. The averaged RMSE values of ECHAM5 and CCSM3 for TAR estimation are 578 and 1081 mm, respectively. Considering mean observed annual rainfall at grids 1–3 equals to 1800 mm (not given in the table), the results indicated up to 32 and 60 % uncertainty for ECHAM5 and CCSM3, respectively. In terms of simulating AMR, uncertainty value of ECHAM5 increases up to 42 % whereas corresponding value at CCSM3 remains constant around 60 %. With respect to the averaged error measures computed for rainy days simulation, it can be pointed out that CCSM3 provides more agreement than ECHAM5 with observed rainy days climatology.

Table 4 Grid-based MAB and RMSE of GCMs driven RCM simulations
Full size table

In addition to the implementation of simple arithmetic mean of grid errors (Table 4, column 7), the MPI measure was also used in this study (Table 5) to investigate the overall performance of each model regarding the annual statistics (i.e. TAR and AMR). Here, we scaled the normalized error variance by the average error found in ensemble of GCM-RCM combinations used in this study. According to the relative difference between MPI values given in the table, ECHAM5-RegCM3 is 33 % more accurate than CCSM3-RegCM3 for annual rainfall reproduction. Referring to the average errors given in Table 4, one might claim that the superiority of the ECHAM5 implied from Table 5 is perhaps not so surprising. For this purpose, it should be mentioned that the averaging of grid-based errors to judge about the entire province is limited to statistics with same dimension whereas the MPI does not suffer from this drawback.

Table 5 Overall comparison of the climate models
Full size table

6 Conclusions

Catchment-scale performance evaluation of GCM-RCM combinations is required for selecting a proper set of GCM-RCM combinations to run hydrological models in small catchments. This important issue was addressed in this study to investigate precipitation realization performance of two GCM-RCM combinations, ECHAM5-RegCM3 and CCSM3-RegCM3, over ten small to mid-size catchments situated in Rize Province, northeastern part of Turkey. To this end, different daily, monthly, and annual precipitation statistics as well as two performance evaluation frameworks, grid- and portion-based, are implemented, which are of interest to a wide audience of hydrologists and practitioners.

The results show that both ECHAM5-RegCM3 and CCSM3-RegCM3 reproduced the monthly rainfall cycle rather weak. In general, the former tends to overestimate total monthly rainfall whilst the latter yields slightly dryer condition in the entire province. Consequently, CCSM3-RegCM3 underestimates observational total annual rainfall. For monthly and annual maximum rainfall statistics, ECHAM5-RegCM3 resulted in higher performance than CCSM3-RegCM3, particularly in the inland portion. Relating to the number of rainy days in different parts of the study area, ECHAM5-RegCM3 tends to overestimate the relatively constant rainy days of the entire province. By contrast, CCSM3-RegCM3 underestimates the statistic in the range 20–50 days. Since the major uncertainty of both models appeared in the coastland portion, implementation of model projections in this portion evidently requires special cares such as a bias correction procedure.

The evaluation performed here showed that no GCM-RCM combination reproduces satisfactory results for the entire province. When a combination performs well for a portion/grid of study region, it is not necessarily a well combination for the other portions/grids. Similarly, the evaluation for different variables showed that no combination was best/worst in all variables even though all of the variables are from a same family (i.e. rainfall-born statistics). In other words, if a combination performs well for reproducing a specific statistic, it is not necessarily a well combination to reproduce the other statistics. These conclusions are consistent with those of reported by Kjellström et al. (2010) at regional-scale evaluation of GCM-RCM combinations. Therefore, it is suggested that when the interest is impact assessment at even relatively small areas, as it is the case of FRA-PFC Rize project, validation of GCM-RCM outputs should be conducted at a single-grid level, rather than at regional scales. In this case, it is also necessary to check models’ performance in reproducing different prescribed observations in order to make a fair judgment among models available for the catchment of interests.