Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

During October 2013 Bay of Bengal (BOB) tropical cyclone (TC) ‘Phailin’ hit east coast of India. This was the most intense cyclone that made landfall over India after the Odisha Super Cyclone (29 October 1999). This TC originated from a remnant cyclonic circulation from the South China Sea. It intensified into a cyclonic storm on the 9 October 2013 and moved northwestwards. It further intensified into a very severe cyclonic storm on 10 October 2013 over east central BOB. It crossed Odisha coast near Gopalpur around 2230 h IST of 12 October 2013 with a sustained maximum surface wind speed of 200–210 kmph gusting to 220 kmph. Some of its unique features included the rapid intensification of the system from 10 October to 11 October 2013 resulting in an increase of wind speed from 83 to 215 kmph. Also, at the time of landfall on 12 October, maximum sustained surface wind speed in association with the cyclone was about 215 kmph and estimated central pressure was 940 hPa with pressure drop of 66 hPa at the center compared to surroundings (RSMC, New Delhi, 2014).

This TC caused heavy rainfall over Odisha leading to floods and storm surge leading to coastal inundation in the state of Odisha. Based on post-cyclone survey report, maximum rainfall occurred over northeast sector of the system centre at the time of landfall. Banki in Cuttack district of Orissa received a maximum 24 h cumulative rainfall of 38 cm.

The verification of TC track predicted by NCMRWF’s Unified model (NCUM) and NCMRWF’s Global Forecast System (NGFS) in terms of track error with respect to observed track based on Joint Typhoon Warning Center (JTWC) shows that the forecast track error is lowest in NCUM forecasts (Ashrit et al. 2013). Here in the present study, an attempt has been made to evaluate the skill of NCUM and NGFS in terms of rainfall after the landfall of the very severe cyclonic storm (VSCS) ‘Phailin’ which hit the Orissa coast near Gopalpur on the evening of 12 October 2013 with a sustained maximum surface wind speed of 215 kmph. Available at: http://www.imd.gov.in/section/nhac/dynamic/phailin.pdf.

This manuscript is divided in to following sections: Section 2 deals with the details and description of Numerical Weather Prediction (NWP) models used in the present study. Section 3 covers the data and methodology used for rainfall verification. The tracks produced by NCUM and NGFS along with the observed track have been presented in Sect. 4. Section 5 describes the results obtained from the present study. Finally, in Sect. 6 conclusions based on the current study are presented.

2 NWP Models at NCMRWF

In this section, we briefly discuss about some noticeable differences between the formulations of the two models. Details about the deterministic models operational at NCMRWF can be found at (Rajagopal et al. 2007; Prasad et al. 2011) for NGFS and (Rajagopal et al. 2012) for the Unified Model (NCUM). Table 1 gives a brief overview of the main features of the two models. The differences in the formulations of two models arise due to several factors including: horizontal and vertical resolutions, physical parameterisations, different time integration methods, as well as data assimilation schemes etc. (Table 1). The most important among these are the different data assimilation schemes. NGFS utilises the Grid point Statistical Interpolation (GSI; Wu et al. 2002) which is based on the three dimensional variational data assimilation (3D-VAR) system whereas NCUM uses four dimensional variational data assimilation (4D VAR) system (Rawlins et al. 2007) for data assimilation. 4D-VAR is a simple generalisation of 3D-VAR and it takes into account the temporal evolution processes which lead to improved representation of synoptic systems in the initial conditions. Extensive studies have been conducted, at various meteorological organisations (UKMet Office, Meteorological Service of Canada, National Centre for Environmental Prediction (NCEP), USA etc.), for comparing the respective skills of 4D-VAR and 3D-VAR in assimilating data and forming the initial conditions for different models. All these studies have helped in forming a consensus that the 4D-VAR performs better than the 3D-VAR scheme (Lorenc and Rawlins 2005; Laroche et al. 2005). This is reflected in the better estimation of observed synoptic systems in the initial conditions (analysis) of the models using 4D-VAR for data assimilation. Additional details about the models’ configuration and forecast products are summarised in Table 1.

Table 1 Details and description about NCUM and NGFS model
Full size table

3 Data and Methodology

Rainfall forecasts produced by NGFS and NCUM model has been verified against IMD-NCMRWF merged satellite gauge (NMSG) data (Mitra et al. 2009, 2013). This rainfall data is a merged product of satellite estimates (Tropical Rainfall measuring mission (TRMM)) and IMD rain gauge observations. The rainfall data from NMSG and from the two models are 24 h accumulated valid at 03 UTC. The model forecast rainfall, originally at a higher grid resolution, is interpolated at 0.5° grid resolution to match with the grid and resolution at which NMSG data is available. The landfall occurred on 12 October evening 1700 UTC followed by the movement in north-northeast direction on 13 and 14 October which led to heavy rain and flooding. So here in the present study, analysis has been performed for 14 October (rainfall received between 03 UTC of 13 October and 03 UTC of 14 October) and 15 October 2013 (rainfall received between 03 UTC of 14 October and 03 UTC of 15 October).

Spatial verification of the rainfall forecasts after the landfall of the cyclone in the present study is carried out using the contiguous rain area (CRA) method. This method was developed for estimating the systematic errors in the rainfall forecasts (Ebert and McBride 2000; Stefano and Marco 2008; Ebert and Gallus 2009). It was one of the first methods to measure errors in predicted location and to separate the total error into components due to errors in location, volume and pattern. The steps involved in the CRA technique are described in Ebert and McBride (2000). The CRA method is an object-oriented verification procedure suitable for gridded quantitative precipitation forecasts (QPFs). In the CRA framework a weather system is defined as a region bounded by a user specified isohyet (entity) of precipitation in the union of the forecast and observed rain field. This technique is then simply based on a pattern matching of two contiguous areas (entities), defined as the observed and forecast precipitation areas delimited by the chosen isohyet. The forecast and observed entities need not overlap, but they must be associated with each other, which means that they should be close to each other. The best match between the two entities can be determined either: (a) by maximising the correlation coefficient, (b) by minimising the total mean squared error, (c) by maximising the overlap of the two entities, or (d) by overlaying the centers of gravity of the two entities. For a good forecast, all the methods should give very similar location errors. In the present study, the best match is determined by maximising the correlation.

For each entity that can be identified in forecast and observations, the CRA method determines the location, volume and pattern errors, which are then combined in the form of a total mean squared error (MSE). To estimate the location error, the forecast field is horizontally translated over the observed field until the best match is obtained. The location error is then simply the vector displacement of the forecast. MSE and its decomposition (location error, volume error and pattern error) are shown below:

$$ {\mathrm{MSE}}_{\mathrm{Total}}={\mathrm{MSE}}_{\mathrm{Displacement}}=+{\mathrm{MSE}}_{\mathrm{Volume}}+{\mathrm{MSE}}_{\mathrm{Pattern}} $$
(1)

where the component errors are estimated as

$$ \begin{array}{l}{\mathrm{MSE}}_{\mathrm{Displacement}}=2{S}_{\mathrm{F}}{S}_{\mathrm{O}}\left({r}_{\mathrm{O}\mathrm{PT}}-r\right)\\ {}{\mathrm{MSE}}_{\mathrm{Volume}}={\left({F}^{\prime }-{O}^{\prime}\right)}^2,\\ {}{\mathrm{MSE}}_{\mathrm{Pattern}}=2{S}_{\mathrm{F}}{\mathrm{S}}_{\mathrm{O}}\left(1-{r}_{\mathrm{O}\mathrm{PT}}\right)+{\left({S}_{\mathrm{F}}-{S}_{\mathrm{O}}\right)}^2\end{array} $$
(2)

In the above expressions F′ and O′ are the mean forecast and observed precipitation values after shifting the forecast to obtain the best match, s F and s O are the standard deviations of the forecast and observed precipitation, respectively, before shifting. The spatial correlation between the original forecast and observed features (r) increases to an optimum value (r OPT) in the process of correcting the location via pattern matching.

Displacement and pattern errors are associated with errors in dynamics (predicted flow) while volume error is associated with errors in physics (moisture) treatment. These components provide guidance for model developers when the statistics of error components are studied for large sample of cases.

4 Observed and Forecast Track

TC ‘Phailin’ originated from a remnant cyclonic circulation from the South China Sea. It intensified into a cyclonic storm on 9 October 2013 and moved northwestwards. It further intensified into a very severe cyclonic storm on 10 October 2013 over east central Bay of Bengal (BOB).

The tracks predicted, with different initial conditions, by deterministic models, NCUM and NGFS along with observed track based on Joint Typhoon Warning Center (JTWC) are shown in Fig. 1a–d. Forecast positions based on NGFS are shown at 6-h interval while the forecast positions based on NCUM is shown at 24-h interval. Figure 1a–d shows the observed and forecasts tracks based on 00 UTC of 09, 10, 11 and 12 October 2013, respectively. The forecasts indicate landfall over Andhra Pradesh and Odisha border. On one hand, Fig. 1a, b shows that the tracks produced, based on initial conditions of 9 and 10 October, by two models, are close to observed track at the time of landfall. On the other hand, tracks produced, based on initial conditions of 00 UTC of 11 and 12 October (Fig. 1c, d), by NCUM show the movement of the cyclone in north-northeast direction while NGFS predicted tracks move in north-northwest direction (Fig. 1c, d).

Fig. 1
figure 1

(a–d). Tracks of Tropical Cyclone ‘Phailin’ predicted by NCUM and NGFS based on Initial Conditions of 00 UTC of 9 (a), 10 (b), 11 (c) and 12 (d), October 2013 along with observed track-based Joint Typhoon Warning Center (JTWC)

Full size image

5 Results and Discussion

First, qualitative inter-comparison of spatial pattern of wind and rainfall produced by NGFS and NCUM will be discussed. Further, to quantify the forecast biases in both the models, verification of quantitative precipitation forecast (QPF) using the CRA has been used.

5.1 Wind and Rainfall After the Landfall of TC

TC ‘Phailin’ hit Orissa coast near Gopalpur around 2230 h IST of 12 October 2013. So, the rainfall associated with this cyclone over the land experienced on 14 October 2013 (rainfall received during 03 UTC of 13 October and 03 UTC of 14 October 2013) as well as 15 October 2013 (rainfall received during 03 UTC of 14 October and 03 UTC of 15 October 2013) are verified. Figures 2 and 3 show the observed and predicted rainfall along with 850 hPa wind (wind vectors are valid at 00 UTC) over eastern part of India valid on 14 and 15 October produced by NGFS. Day-1 forecast valid for 14 October 2013 produced by NGFS captures the rainfall band with peak rainfall much to the southwest with respect to observations while Day-3 and Day-5 rainfall forecast with peak of the rainfall band can be seen much to the south as compared with observation. Wind circulation at 850 hPa shows the movement of the cyclone is slower as compared to observation in all Day-1, Day-3 and Day-5 forecasts valid on 14 October 2013. Rainfall in Day-1 forecast shown by NGFS valid for 15 October 2013 matches well with the peak rainfall and is slightly to the north compared to observations. Day-3 and Day-5 rainfall forecasts also capture the rainfall band very well with slight underestimation in the peak rainfall amounts valid for 15 October 2013. Similar plots valid for 14 and 15 October 2013 produced by NCUM are presented in Figs. 4 and 5. NCUM predicts rainfall band and circulation in Day-1 and Day-3 forecast close to observations. However, peak rainfall amount and associated circulation is located much to the south with respect to observations in Day-5 forecast valid for 14 October 2013. Day-1 and Day-3 forecasts valid on 15 October 2013 match well as compared with observation with slight overestimation in peak amount of rainfall in Day-1 forecast. However, Day-5 forecast predicts the rainfall band slightly to the south of the observed locations. Thus, the rainfall and circulation predicted by NCUM are close to observation while NGFS prediction is slightly in the south with respect to observations on 14 October. However, rainfall and wind predicted by both models in Day-1 and Day-5 forecast valid on 15 October are close to observations while NCUM are much near to observations in Day-3 relatively to NGFS.

Fig. 2
figure 2

Observed and NGFS Model Predicted rainfall in cm/day (horizontal colour bar) and 850 hPa circulation in m/s (vertical colour bar) over Indian region. valid for 03Z/14 October 2013

Full size image
Fig. 3
figure 3

Observed and NGFS Model Predicted rainfall in cm/day (horizontal colour bar) and 850 hPa circulation in m/s (vertical colour bar) over Indian region valid for 03Z/15 October 2013

Full size image
Fig. 4
figure 4

Observed and NCUM Model Predicted rainfall in cm/day (horizontal colour bar) and 850 hPa circulation in m/s (vertical colour bar) over Indian region valid for 03Z/14 October 2013

Full size image
Fig. 5
figure 5

Observed and NCUM Model Predicted rainfall in cm/day (horizontal colour bar) and 850 hPa circulation in m/s (vertical colour bar) over Indian region valid for 03Z/15 October 2013

Full size image

5.2 Verification of Rainfall Forecasts During 14 and 15 October Using CRA Method

Since TC ‘Phailin’ crossed Orissa coast as a very severe cyclonic storm, it produced extremely heavy rainfall after landfall. So, CRAs defined by higher thresholds of 10, 20, 40 and 80 mm/day are used to isolate the events corresponding to the high rainfall region due to ‘Phailin’. As a next’ step, a pattern matching technique is used for estimating the location error. In this case, the best match between the forecast and observed entities is done by maximising the correlation coefficient (Pearson correlation) between the forecast and observed fields. Figures 6 and 7 show the CRA verification isohyets for Day-1, Day-3 and Day-5 forecasts from NCUM and NGFS valid for 14 and 15 October.

Fig. 6
figure 6

Isohytes in mm over eastern coast of India (a) Analysis, (b) Day-1,(c) Day-3, (d) Day-5, from NCUM and (e) Day-1,(f) Day-3, (g) Day-5, from NGFS. Scatter plots show the number of matching grid points between model forecast and observation (analysis) valid on 14 October 2013. Plots (h–m) represents Day-1, Day-3 and Day-5 of NCUM and NGFS, respectively

Full size image
Fig. 7
figure 7

Isohytes in mm over eastern coast of India (a) Analysis, (b) Day-1, (c) Day-3, (d) Day-5 from NCUM and (e) Day-1, (f) Day-3, (g) Day-5 from NGFS. Scatter plots show the number of matching grid points between model forecast and observation (analysis) valid on 15 October 2013. Plots (h–m) represents Day-1, Day-3 and Day-5 of NCUM and NGFS, respectively

Full size image

Table 2 shows MSE and root mean square error (RMSE) (before and after the CRA procedure) obtained from CRA verification for Day-1–Day-5 forecasts valid on 14 and 15 October, respectively. The comparison of MSE between NCUM and NGFS reflect that NGFS has larger error than NCUM which implies the performance of NCUM is better than NGFS in terms of matching the displacement, volume and pattern of the forecast precipitation entities valid on 14 October. However, NCUM shows slightly larger MSE only in Day-1 and Day-5 forecast as compared to NGFS valid on 15 October. RMSE is also shown in the table for the original and shifted rainfall (i.e., before and after the CRA method). It is known as a direct indicator of the forecast accuracy. When we compare RMSE before and after applying CRA procedure, we find NCUM show lower value than NGFS which leads to better forecast than NGFS valid on 14 October 2013. However, Day-1 forecast valid on 15 October 2013 produced by NGFS has 23 % lower RMSE as compared with NCUM while in Day-5 forecast errors are close to each other before and after the shift.

Table 2 MSE, initial and shifted RMSE for Day-1–Day-5 forecasts obtained from the CRA verification for rainfall forecast from NCUM and NGFS valid on 14 and 15 October 2013
Full size table

5.3 Verification of QPF Statistics

Quantitative verification of rainfall associated with TC Phailin after the landfall is also made using CRA method. The rainfall forecast after the landfall has been categorised as ‘hits’, ‘misses’ depending upon whether the position and intensity were well predicted.

We have evaluated the CRA statistics for a number of rainfall thresholds ranging from 10 to 80 mm/day. Based on these thresholds, the components of contingency table (number of hits, number of misses, false alarms and number of correct negatives) have been calculated during CRA analysis. Frequency of number of hits and misses valid for 14 and 15 October is shown in Fig. 8a–d for different rainfall thresholds (<1, 1–10, 10–20, 20–40 and 40–80 mm/day). The rainfall forecast from Day-1 to Day-5 predicted by NCUM shows higher number of hits and lower number of misses as compared with NGFS for all rainfall thresholds valid for 14 October. But the number of hits is nearer in Day-1 forecast at all thresholds for the rainfall valid for 15 October. Further, we have computed equitable threat score (ETS), probability of detection (POD) and Hanssen & Kuipers (HK) score by using the components of the contingency table. The scores are shown in Figs. 9 and 10 in the form of bar graph from Day-1 to Day-5 for two rainfall thresholds (10–20 mm/day: left panel and 20–40 mm/day: right panel) for both the models valid for 14 and 15 October.

Fig. 8
figure 8

Number of hits and misses based on the event verification during CRA procedure. Upper Panel correspond to number of hits and misses for Day-1, Day-3 and Day-5 forecasts valid on 14 October and Lower panel corresponds to the hits and misses for Day-1, Day-3 and Day-5 forecasts valid on 15 October 2013. These are based on <1, 1–10, 10–20, 20–40 and 40–80 mm/day rainfall thresholds

Full size image
Fig. 9
figure 9

ETS, POD, and HK score for NCUM and NGFS from Day-1 to Day-5 forecasts valid for 14 October 2013 based on 10–20 mm/day (left panel) and 20–40 mm/day (right panel)

Full size image
Fig. 10
figure 10

ETS, POD, and HK score for NCUM and NGFS from Day-1 to Day-5 forecasts valid for 15 October 2013 based on 10–20 mm/day (left panel) and 20–40 mm/day (right panel)

Full size image

ETS measures the fraction of events that are correctly predicted accounting for hits by random chance. High ETS would imply that there is a large number of correctly predicted forecast entities near to the location of the matching observed entities (hits) and lesser number of forecast entities far away from the observations (misses and false alarms). The top panel of Figs. 9 and 10 show ETS from Day-1 to Day-5 forecast for two rainfall thresholds (10–20 mm/day: left panel and 20–40 mm/day: right panel) valid for 14 October and 15 October, respectively. NCUM shows higher ETS from Day-1 to Day-4 for both the rainfall threshold (10–20 and 20–40 mm/day) for 14 October. For 15 October, NGFS shows slightly higher ETS only in Day-1 for both the thresholds. However, for Day-2–Day-5, NCUM shows higher ETS than NGFS for both the thresholds.

HK score, also known as the True Skill Score (TSS), is defined as the difference between the hit rate and the false alarm rate (Hanssen and Kuipers 1965). A high HK score indicates more hits relative to false alarms. The bottom panel of Figs. 9 and 10 show HK score from Day-1 to Day-5 forecast for two rainfall thresholds (10–20 mm/day: left panel and 20–40 mm/day: right panel) valid for 14 and 15 October, respectively. Here in this present case of TC ‘Phailin’, NCUM shows higher HK score than NGFS till Day-4 while in Day-5, score is slightly higher NGFS for 14 October for both rainfall thresholds. For 15 October, NGFS and NCUM are close to each other in Day-1 forecast for both the threshold. However, NCUM shows higher HK score from Day-2 to Day-5 as compared with NGFS for both the thresholds.

POD is defined as the fraction of observed events that are correctly predicted. Therefore a high POD indicates good forecast skill of a model. In the current case, a high POD would imply that many forecast entities with intensities approximately matching the observations are lying close enough to the observed entities hence having a higher number of hits (Fig. 9). From the middle panels of Figs. 9 and 10, it is seen that POD is consistently higher for NCUM (Day-1–Day-4) for the forecast but in Day-5 NGFS shows higher POD than NCUM valid for 14 October for both thresholds. On one hand, rainfall forecast valid on 15 October, NGFS and NCUM shows comparable skill in Day-1 forecast similar to HK score and ETS. On the other hand, NCUM shows higher POD for Day-2–Day-5 forecast as compared to NGFS for both rainfall thresholds from 10–20 to 20–40 mm/day. Thus, based on the above statistics, we can say that NCUM has higher skill in predicting the rainfall due to TC ‘Phailin’ as compared to NGFS.

6 Summary

In this paper, the comparison of skills of NCUM and NGFS in predicting the rainfall associated with the TC ‘Phailin’ after the landfall has been carried out. After hitting the coast, it caused heavy to extremely heavy rainfall over Odisha, Coastal Andhra Pradesh, Jharkhand, Bihar, Sub-Himalyan West Bengal and some parts of Sikkim on 14 and 15 October 2013. The findings of the paper are as follows:

  • The forecast tracks produced by both models indicate landfall over Andhra Pradesh and Odisha border. However, after the landfall, NCUM-predicted track shows the movement of the cyclone in north-northeast direction while NGFS track drifted the system in north-northwest direction.

  • On one hand, the forecast produced by NCUM was successfully able to capture the intensity of circulation and rainfall. On the other hand, NGFS shows the circulation and associated movement of rainfall band slower and representing rainfall much to the south with respect to observations.

  • The verification of spatial pattern of rainfall indicates the underes­timation of average rain rate and the rain volume produced by NGFS as compared to NCUM which shows lower percentage errors and total MSE and RMSE.

  • Event verification indicates NGFS has lower number of hits and higher number of misses than NCUM. Also, a comparison of statistical scores (POD, HK score and ETS) between the two models again confirms that NCUM performs better than NGFS in predicting the rainfall associated with this cyclone specifically after the landfall.