1 Introduction

The accurate prediction of seasonal rainfall over Asia is one of the most challenging problems for any numerical model especially over South and Southeast Asia. It is a well-known fact that the large parts of Asia receive major proportion (75 %) of their annual rainfall during summer monsoon season (June–August (JJA)). The climatology of the observed and MME seasonal summer monsoon rainfall (SMR) over Asia from 1983 to 2003 (21 years) has been well depicted in panels a and b of Fig. 1, respectively. Figure 1a shows that MME well captured the belts of high rainfall along the head Bay of Bengal, Northeast India, adjoining Bangladesh, west coast of India and over the equatorial Indian Ocean extending up to the west Pacific sector. In contrast, low rainfall can be seen over Northwest India, south-east peninsular India, Malaysia, Indonesia and north-east and north-west parts of the China. Figure 1a also shows a large spatial variability in seasonal rainfall over the South Asia. This variability in rainfall has significant impacts on agricultural fortunes of the farmers and, hence, the economy of South Asian countries. The importance of climate forecast to crop yield, risk management of rain-fed farming, conservation of available water resources and other related impacts has been well documented in Krishna Kumar et al. (2004), Challinor et al. (2005), Sivakumar (2006), Pattanaik and Kumar (2010), etc.

Fig. 1
figure 1

a Observed (CMAP) and b multi-model ensemble (MME) Climatology of Asian summer monsoon rainfall (ASMR) (millimetres per day) from 1983–2003)

Full size image

Numerical weather prediction (NWP) models are not adequate to address satisfactorily in detail the aspect of the Asian monsoon. This may be due to the large temporal and spatial variability in rainfall and some inherent limitation of NWP models. These models are built on the foundation of deterministic modeling which starts with some initial conditions. The inherent limitation of NWP models is that they generally neglect small-scale climate variations and cannot approximate complicated physical processes and interactions. The models loose skill because of the growth of inevitable uncertainty in the initial conditions. In order to overcome these shortcomings, a new approach known as ensemble forecasting was introduced in the 1990s (Molteni et al. 1996; Zhang and Krishnamurti 1997; etc.). In this method, forecasts are made either with different models or different initial conditions or both and combined into a single forecast to take into account the uncertainty in the model formulation and initial conditions.

During the past two decades, climate scientists have made groundbreaking progress in the dynamical seasonal prediction model that is used as an intermediate complexity coupled ocean-atmosphere model (Cane et al. 1986). Recent studies (Kumar et al. 2005; Wang et al. 2005; Rajeevan et al. 2012) have found that simulated ASMR with the coupled atmosphere-ocean general circulation models (CGCMs) performed better as compared to the atmospheric-oceanic GCMs over Asia. Atmospheric chaotic dynamics may cause seasonal forecast errors, inherently limiting seasonal climate predictability. Since seasonal predictability does not depend on the initial atmospheric conditions, an ensemble forecast with different atmospheric initial conditions was developed (Krishnamurti et al. 1999, 2000) to reduce the errors arising from atmospheric chaotic dynamics. Another considerable source of errors in seasonal forecast arises from the uncertainties in model parameterizations of unresolved sub-grid scale of processes. In an individual model, stochastic physical schemes were developed to alleviate the uncertainty arising from sub-grid scales (Bowler et al. 2008), which are now operationally used in many centres for the forecasting. Meanwhile, a more effective way, the MME approach, was designed for quantifying the forecast uncertainties due to model formulation (Krishnamurti et al. 1999, 2000; Shukla et al. 2000). The idea behind MME is that if the model parameterization schemes are independent of each other, the model error associated with the model parameterization schemes may be random in nature; thus, an average approach may cancel the model errors contained in individual models. The MME, in general, is superior to the predictions made by any single model component and exhibits better performance than most of the component models and predict the large-scale features quite reasonably (Kharin and Zwiers 2002; Peng et al. 2002; Palmer et al. 2004; Min et al. 2009). The MME is defined as averaged simulation results from multiple models. The main reason to focus on MME is that the averages across structurally different models empirically show better agreement with the observations. The application of MME can also be seen in other modeling applications to produce the simulated climate features that have improved the simulations over single models (Christensen et al. 2007; Meehl et al. 2007). This method is now frequently used in model evaluation/estimation of the climatology from coupled atmosphere-ocean GCMs (AOGCMs) (McAvaney et al. 2001), projection of climate change from AOGCMs (Cubasch et al. 2001; Giorgi and Mearns 2002) and climate change detection from AOGCMs (Gillett et al. 2002). A detailed discussion on merit and demerit of MME for the simulation of Asian-Australian monsoon precipitation using 10 different CGCMs can be seen in the work of Wang et al. (2008). Gadgil and Sajani (1998), Kang et al. (2002) and Wang and An (2002) have suggested that there are several problems in simulating the mean monsoon climate and its spatial and temporal variations. Using 20 atmospheric GCMs under the Atmospheric Model Inter-comparison Project (AMIP), Gadgil and Sajani (1998) have shown that the atmospheric models did not evolve to a stage where they can simulate the inter-annual variability of the Indian summer monsoon realistically. One of the main problems in the prediction using atmospheric models is surface boundary conditions like SST which has to be prescribed for the time length of prediction. AMIP simulations were made with the SST, specified from the observations and are therefore expected to have better skill than the prediction made with predicted SST. Using 20 models products, Gadgil and Sajani (1998) have shown that out of 20 models, only one model could predict the rainfall deficiency of 1979. Wang et al. (2005) have suggested that the state-of-the-art AGCMs when forced by the observed SST are unable to simulate the Asian-Pacific summer monsoon rainfall. Their analysis also illustrated that over the parts of the western Pacific, the relationship between SST and rainfall was negative while it was insignificant over the Indian regions. All the GCMs show a direct correlation between SST and seasonal summer monsoon rainfall. Their study suggested that the coupled ocean atmosphere processes are crucial in the monsoon regions where the atmospheric feedback on SST is critical. The above study suggests that the unprecedented levels of evaluation of the climate models have been done over the last decade in the form of multi-model ensemble inter-comparison project. Preethi et al. (2009) have tested seven models in simulating the summer monsoon rainfall over India and have found low skill of simulation for inter-annual rainfall variability. Rajeevan et al. (2012) have well described the main problem in simulating the inter-annual rainfall variability in coupled atmosphere-ocean general circulation models. The exercises were done by analyzing the model simulated forecast from the ‘Development of European Multimodel Ensemble System for seasonal to inTERannual prediction’ (DEMETER) project.

The present study is therefore aimed to asses the performance of all the 11 sets of global climate models (coupled and uncoupled both) of the Asia-Pacific Economic Cooperation Climate Center (APCC) in simulating the Asian summer monsoon rainfall variability (Indian and East Asian land masses). The APCC can provide a good opportunity to examine the simulation characteristics of the Asian summer monsoon from the outputs of available APCC set of global climate models. In the present study, hind-cast data of 11 models has been used to examine and understand the variability in simulated summer monsoon rainfall. Data and methodology are given in Section 2, and evaluation of simulation skills is described in Section 3. Section 4 discussed the main results of the study.

2 Data and methodology

The APCC is successfully maintaining and continuously improving an international science and technology network mainly for reducing and mitigating the adverse impact of climate and extreme weather events. The Asia-Pacific Economic Cooperation (APEC) provides operational 3-month lead dynamical seasonal predictions through the MME technique. MME predictions are facilitated through the multi-institutional co-operation within the APEC countries. Eighteen dynamical seasonal forecasts are available to APCC from 15 national hydro-meteorological centers/research institutes of eight APEC member economies. Currently, APCC operates the world's largest and most extensive operational MME dynamical seasonal prediction system. Organizations and institution participating in joint real-time MME operational forecasts are NASA, National Centers for Environmental Prediction (NCEP), IRI and COLA (USA); HMC and MGO (Russia); KMA, METRI and SNU (Korea), Japan Meteorological Agency (JMA); CWB of Chinese Taipei; IAP and BCC of China; Canadian Meteorological Center and the Australian Bureau of Meteorology.

The present study uses 21 years (1983–2003) of rainfall data of hind-cast experiments conducted in a group of 11 operational climate prediction models, which is used by APCC in the MME prediction system. These models are CWB, GCPS, JMA, GDAPS_F, NCEP, Predictive Ocean Atmosphere Model for Australia (POAMA), MSC_GEM, MSC_GM2, MSC_GM3, MSC_SEF and National Institute of Meteorological Reaserch (NIMR). Among these models POAMA, NCEP and JMA are coupled models and the remaining models are uncoupled (atmospheric/oceanic) ones. Detailed descriptions of the group of APCC global climate models are presented in Table 1. For each model, the data comprised 3-month-long hind-cast performed 12 times in a year with the first day of each 12 months as the starting date and ensemble members were varied between 10 and 20. In the present study, we have confined our analysis to the hind-cast which started with the initial conditions of 1 May which includes the Asian summer monsoon periods from June to August (all the uncoupled models have 1 May as an initial condition with 3-month time lead and all the coupled models have 1 May initial conditions with 3-month lead as well as 1 February initial condition with 6-month time lead). Model simulated monthly precipitation (millimetres per day) has been used in the present study. Observed monthly precipitation (millimetres per day) from the CMAP at a resolution of 2.5° × 2.5° available for the entire globe (Xie and Arkin 1997) was used for model validation. NOAA Extended Reconstructed Sea Surface Temperature at 2.5° × 2.5° resolutions has been also used for correlation analysis.

Table 1 Model description of APCC participating global models
Full size table

In order to understand the skill of each model in simulating SMR over Asia (15°S–40°N and 40°E–150°E), the mean spatial pattern of the seasonal rainfall for each model and MME have been studied and results are compared with the observed (CMAP) rainfall. The skill of the forecast models has been assessed as per World Meteorological Organization (WMO) standard forecast verification methods (WMO 2010). Various statistical measures like long-term mean climatology, biases, map-to-map correlation, Gerrity skill score (WMO 2010), coefficient of variation (CV), correlation coefficients (CCs), standardized rainfall anomalies, spatial resemblance of the simulated and observed patterns (Taylor diagram), etc. have been studied in detail using model simulated and observed rainfall for the period of 1983–2003 (21 years).

3 Evaluation of model simulation skill over Asia

3.1 Spatial rainfall patterns

The spatial distribution of long-term mean rainfall (millimetres per day) climatology derived from the CMAP (observed) and MME data set for a period of 21 years (1983–2003) have been shown in panels a and b of Fig. 1, respectively. Summer monsoon rainfall biases for individual global climate models along with MME are presented in Fig. 2. Comparison of Fig. 1a with the mean rainfall simulated with different models (figure not shown here) shows that most of the global climate models well captured the mean rainfall characteristics like zones of rainfall maxima, minima mainly over the land areas, high and low rainfall zones. etc. including MME. However, a significant difference in simulated rainfall of different ensemble members of the individual model is also noticed and shown in Fig. 2 in terms of model biases. Most of the models show a high variability in spatial rainfall like an underestimation of rainfall over high-rainfall regions such as over the west coast of India, Northeast India and the equatorial regions. Larger fluctuations (highly under estimated rainfall) in simulated rainfall can be seen in MSC_GEM, MSC_GM2, GDAPS_F, MSC_SEF and CWB models, while smaller fluctuations (comparatively low amount of underestimated rainfall) can be seen in POAMA, NCEP and JMA. Figure 2 also shows that GCPS, GDAPS_F and CWB models simulated a high rainfall amount over the land regions compared to that observed. The low and high rainfall amount can be associated with the model resolution and parameterizations schemes used in individual models. As far as South Asia (mainly India) is concerned, Fig. 2 clearly shows the characteristics of high rain belt over the Western Ghats and Northeast India and low rain belt mainly over Northwest India. Figure 2 shows that the models well captured the characteristics of high and low rainfall but amount of rainfall has shown large variations. Figure 2 clearly shows that NCEP and POAMA (coupled model) simulated rainfall is closer to the observed rainfall compared to the other model simulated rainfall but NCEP has shown to be the best among them. This suggests that the seasonal mean rainfall could be sensitive to the initial conditions and individual model behaves differently over the same regions of Asia like an under/overestimation of rainfall over the same regions.

Fig. 2
figure 2

Biases in ensembles mean ASMR (millimetres per day) simulated in APCC set of global climate models and MME (1983–2003)

Full size image

To quantify the models’ skill in reproducing the spatial rainfall patterns, the CCs between the simulated and observed (CMAP) rainfall were computed and shown in Fig. 3. Figure 3 shows different spatial pattern of CCs; however, most of the models have a significant and high correlation over the Southeast Asian and west Pacific sectors. The spatial distribution in rainfall biases already showed a large variation from model to model models (Fig. 2); hence, pattern (map-map) correlations were computed for regions (1) over Asia and (2) over India separately (tables not shown). Pattern correlation depicts that some of the models like JMA, NCEP, GCPS and MME show better skill (CCs exceeding 0.70) in reproducing spatial mean rainfall climatology over Asia and skills decreased over the Indian continent (except POAMA). This illustrates that most of the models show a better spatial pattern of rainfall climatology only over Asia as a whole compared to South Asia (Indian region). Inter-comparison of the spatial pattern of rainfall climatology of different models was also carried out and CCs were calculated. The minor changes in the patterns of CCs for MSC_GEM, JMA and CWB while an increase in CCs were found in MSC_GEM, MSC_SEF and POAMA models from the Asian to South Asian regions. The Meteorological Service of Canada (MSC) set of models (MSC_GEM, MSC_SEF, MSC_GM2 and MSC_GM3) has shown a large variation in spatial rainfall pattern over Asia as a whole and over South Asia. MSC_GM2 models show significant CCs of 0.54 over the Asian regions, while weak CCs (CC = 0.16) were found over the South Asian regions. MME and JMA have the highest CCs over both regions (Asia and South Asia).

Fig. 3
figure 3

Spatial map of correlation between observed and model simulated rainfall

Full size image

Rainfall is highly discontinuous in space and time; its distribution is positively skewed and characterized by the presence of many zero values. Spatial maps are very noisy and often contain large outliers. These characteristics make verification of rainfall amount difficult, and a new intensity-scale method for verifying the spatial precipitation forecasts is introduced. This technique provides a way of evaluating the forecast skill as a function of rainfall rate intensity and spatial scale of the error. For this purpose, a detailed model skill analysis has been performed using the Gerrity skill score (GSS) which uses three-by-three contingency table to test the skill of the model in terms of simulating below-normal, normal and above-normal rainfall. GSS was calculated for each grid cell in the domain from 1983 to 2003. Figure 4 illustrates the GSS map for the 11 APCC sets of climate models including MME. Most of the models show a positive skill score over the oceanic area, mainly over the west Pacific and Indian Ocean. Almost all the models show a better forecast skill over the oceanic sector than the land. Positive skills can be seen over the west Pacific and Indian Ocean with mixed patches of weak skill. Most of the models have almost no skills to forecast SMR over west China. The MME shows slightly a better and positive forecast skill mainly over the oceanic sectors.

Fig. 4
figure 4

Spatial map of Gerrity skill score for APCC sets of climate models.

Full size image

The present study has also investigated the atmosphere ocean feedback during the summer monsoon season by computing summer monsoon seasonal contemporaneous correlation (CCs) between simulated mean rainfall (over the land area only) by different models and observed SST along with MME from 1983 to 2003. The results are presented in Fig. 5. A significant positive (direct association) simultaneous CC provides the influence of SST on the atmosphere while a negative CC indicates the influence of atmosphere on SST. Figure 5 shows a direct association over the larger parts of west and east Indian Ocean and an inverse one over the south China and north-west Pacific Ocean in CMAP (observed). Coupled models (NCEP, JMA and POAMA) show an almost similar trend but weaker CCs. Comparison of different models simulated and MME CC trend with CMAP indicates that MME CC trend is slightly closer compared to individual model simulated rainfall. The MME shows an inverse pattern of CCs over the northern Indian Ocean and positive CCs slightly southward compared to other models. The positive and negative CCs over the west and East Indian basin suggest that SST strongly forces the atmosphere over the eastern Indian Ocean which in turn has positive feedback on SSTs. The Bjerkness feedback mechanism relating the thermocline and equatorial upwelling is an important thermodynamics for an air-sea feedback mechanism, and it is well described in the work of Vinayachandran et al. (2009). The above-said mechanism in the Indian Ocean basin operates when the equatorial trade wind is weak easterly and the wind over the Indonesian coast favours upwelling (Wang et al. 2004; Vinayachandran et al. 2009). The high and significant CCs of the three models suggest that the coupled models are characterized by a more realistic and excessive oceanic forcing on the atmosphere over the equatorial Indian Ocean. Rajeevan and Nanjundiah (2009) have also found a similar type of strong relationship between SST rainfall coupling over the Indian Ocean in the IPCC models.

Fig. 5
figure 5

Correlation between SST and rainfall (observed and APCC sets of climate models

Full size image

Overall, MME, POAMA, NCEP and JMA simulated rainfall patterns are slightly closer to the verification (CMAP) rainfall over the Asian regions. The MSC set of models has almost the same experimental design but simulating dissimilar systematic biases, even though the models are developed in the same center. The SST and rainfall correlation also suggests that NCEP, JMA and POAMA performed better compared to other models.

3.2 Coefficient of variation and correlation coefficient

The CV represents the ratio of the standard deviation to the mean, and it is frequently used in comparing the degree of variation from one data set to another, even if the means are drastically different from each other. The analysis found that the CV in SMR is about 8 and 6.2 % and long-term mean is about 650 and 420 mm over the Indian and East Asian land masses, respectively. For a detailed study of inter-annual variability in rainfall, various statistical analyses have been done over the Indian (5°N–35°N, 68°E–90°E) and East Asian (20°N–50°N, 90°E–150°E) land masses separately.

Over Indian land mass

Figure 6a–d shows the inter-annual variability over the Indian land mass (excluding the oceanic regions). The analysis found that the mean SMR varies from 500 mm (JMA) to 910 mm (CWB) while the mean SMR of MME is 700 mm. The CV varies between 3.2 % (CWB) and 12.2 % (MSC_SEF), and for MME, the CV is 5.0 %. The analysis indicates that the CV is low for the ensemble mean compared to its ensemble members and dispersion among the ensemble members was also high. Comparison of simulated CV with the observed CV shows that the ensemble members of POAMA model are slightly closer to those observed but simulated mean rainfall over India with POAMA model was overestimated.

Fig. 6
figure 6

a Scatter plot of SMR (millimetres) and CV (percent) from observed and APCC set of global climate model, b CCs between observed and model simulated SMR, c Taylor diagram showing standard deviation (centimetres) and CCs between observed and model simulated rainfall and d inter-annual variability of SMR anomalies for observed and MME over Indian land masses

Full size image

To get a clear idea of inter-annual variability of SMR and the spread among the ensemble members, CCs has been computed between the observed and models simulated SMR (only over the Indian land mass) and shown in Fig. 6b. Figure 6b shows that most of the model ensembles have inverse relationships with the observed rainfall except NCEP and POAMA where ensemble mean CCs lie between 0.2 and 0.3. The observed rainfall shows positive CCs with NCEP and POAMA, while the highest negative CCs (CC = −0.48) are found with GCPS and MME (CC = −0.3).

To examine the spatial resemblance of model simulated and observed patterns, a Taylor diagram was analysed and shown in Fig. 6c. A Taylor diagram (Taylor 2001) is used to provide the graphical summary of how closely a set of patterns matches with the observations. The association between the two patterns is quantified in terms of their correlations, their centred root-mean-square error and the amplitude of their variations (represented by their standard deviations). Figure 6c represents the Taylor diagram analysis for the precipitation over the Indian land mass. In the Taylor diagram, the radial distance from the origin represents the standard deviation ratio of the model simulated pattern with the reference (CMAP) pattern. The pattern CCs between two variables are given by the azimuthal position and the normalized centred root-mean-square error (RMSE) of the simulated pattern is given by the distance from the reference point of observations. Out of the 11 selected models (details are provided in Table 1), NCEP seems to be the best selection for true representation of spatial patterns which has maximum patterns of CCs, minimum RMSE and low bias. The spatial rainfall distribution based on CMAP and other selected models as well as their MME is already described in Section 3.1. MME compared to other selected models reasonably well captures the spatial characteristics of seasonal monsoon over the Indian land masses. Hence, MME has been considered for further analysis. Standardized MME and CMAP (observed) SMR anomalies are shown in Fig. 6d over the Indian land mass, which only show that the model well captured the excess monsoon rainfall in 1988 over India.

Over East Asian land masses

The inter-annual rainfall variability over the East Asian land mass is shown in Fig. 7a–d. Figure 7a shows that the mean SMR varies from 280 mm (MSC_GEM) to 560 mm (CWB) while the mean SMR for MME is 455 mm. The CV varies between 1.5 % (NIMR) and 6.5 % (MSC_GEM), and for MME, the CV is 1.5 %. The present analysis clearly indicates that the CV is low for the ensemble mean compared to its ensemble members and dispersion among the ensemble members was high. Comparison of observed CV to simulated CV shows that the ensemble members of MSC_GEM are slightly closer to those observed but the MSC_GEM model overestimated the mean rainfall over the East Asian land mass.

Fig. 7
figure 7

Same as in Fig. 6 except for SMR over East Asian land masses

Full size image

For a detailed study of inter-annual variability of SMR and the spread among the ensemble members, CCs were computed between the observed and model simulated SMRs over the East Asian land mass and shown in Fig. 7b. Figure 7b shows that approximately all the model ensembles have a direct association with the observed rainfall while MSC_SEF ensemble mean CCs are negative (inverse relationship). The observed rainfall shows the highest CCs with GCPS and NIMR (CCs above 0.4) and CC for MME is 0.38. Figure 7c presents the Taylor diagram analysis for the East Asian rainfall. Figure 7c shows that out of the 11 considered models, GCPS and NIMR seem to be the best selection for true representation of spatial patterns which have maximum patterns of CCs, minimum RMSE and low bias. Standardized MME and CMAP (observed) SMR anomalies are shown in Fig. 7d over the East Asian land mass and MME well captured low rainfall of 1989 over East Asia.

The above analysis of observed rainfall shows random fluctuations without any long-term clear trend over the Indian and East Asian land masses. The spread in SMR with the different models and their ensemble members suggests the diverse nature of the model and their dependency on the initial conditions. The MSC set of climate models (MSC_GEM, MSC_SEF, MSC_GM2 and MSC_GM3) has shown a large variation in mean rainfall pattern over both land masses. The analysis of CCs between ensemble SMR and observed (CMAP) rainfall shows that the skills in reproducing the inter-annual variability varies from model to model and from region to region over Asia. The analysis also indicates that among the models in APCC, MME has some deficiency in simulating mean and inter-annual variability of SMR.

Overall, rainfall variability over two different regional domains (Indian and East Asian land masses) almost shows an opposite CC pattern nearly in all the models. The mean value computed with MME shows some improvement compared to the individual models, indicating that MME is not affected by the performance of individual models and the uncertainty in the initial condition.

4 Conclusions

Simulation skills of 11 APCC global climate models in predicting mean seasonal rainfall over Indian and East Asian land masses were studied by applying various statistical techniques. The results indicate that majority of the models have negative bias over some parts of the Asian land mass and over equatorial zones (Indian and west Pacific Oceans). Large variability in simulated ASMR was found from model to model and from region to region in uncoupled models compared to couple models. Most of the models underestimated the rainfall over high rainfall belts. Atmospheric chaotic dynamics uncertainties in the representation of unresolved sub-grid scales in the models may cause large bias in the models. Hence, a method of ensemble forecast was used for minimizing the errors. The study also found large spreads in individual members of the model, and these spreads were as large as the spread of ensemble means of different models, suggesting uncertainty due to errors in initial conditions. The uncertainties in the model largely depend on model formulations.

The analysis of CCs between SST and simulated ASMR found a large variation in producing inter-annual variability especially over Indian and west Pacific Oceans mainly in uncoupled model. It may be due to poor representation of air-sea interaction. The skill in reproducing the inter-annual variability varies again from model to model and from region to region. The prediction skill of models was found slightly better over the oceanic area compared to land masses. The exact amount of simulated SMR in MME over Asia is not fully captured; rather, simulated SMR with MME was slightly better than the any individual model. Among the set of APCC models, NCEP, POAMA, GCPS and NIMR can provide slightly a better skill in simulating inter-annual variability of SMR over Indian and East Asian land masses based on the analysis of skill of the individual models and MME. The present study suggests that there is a need to fully understand the physical processes used for improving the individual model skills rather than the methods of MME for producing better seasonal rainfall prediction systems.