1 Introduction

Main components of tropical cyclone (TC) forecasting are the routine preparation of best tracks parameters and detailed forecast performance statistics at the end of each season. While this is a time-consuming task, which uses valuable resources, it provides the best possible storm archive, together with invaluable information on overall forecast performance and the relative performance of individual techniques. Such information is essential for:

  • Developing new techniques and monitoring forecast improvements resulting from new techniques and observing systems;

  • Evaluation of value addition by forecasters to guidance received from the objective aids;

  • Providing a basis for objectively estimating forecast uncertainty, which is used in deciding the timing and extent of warnings and watches, especially for critical regions;

  • Providing feedback on the relative accuracy of available forecast techniques, perhaps including stratification into different synoptic, latitudinal, or seasonal types;

  • Providing feedback to external organisations, such as the global and regional modelling centres for comparison of their performance in the region with the operational forecast.

All the above are essential and one of the most important parameters in TC prediction is storm’s intensity or maximum sustained surface wind (MSW). Improved short range (1–3 day) intensity forecasts lead to fewer unnecessary warnings and evacuations. Additionally, a better intensity forecast in association with a better track forecast will yield improved forecasts of other metrics such as wind speed, storm surge, and precipitation.

India Meteorological Department (IMD), as one of the six Regional Specialised Meteorological Centres (RSMCs), is responsible for TC monitoring and prediction over the north Indian Ocean (NIO). A cyclonic disturbance is considered as a TC over the NIO, when the associated MSW is 34 knots or more as per the classification adopted by IMD. It corresponds to the definition of tropical storms over other Ocean basins like Pacific and Atlantic Oceans. Detailed classification of TCs into cyclonic storm (CS), severe cyclonic storm (SCS), very severe cyclonic storm (VSCS) and super cyclonic storm (SuCS) are given in cyclone manual published by IMD (2003).

IMD introduced the objective TC intensity forecast valid for next 24 h over the NIO in 2003. It further extended the validity period up to 72 h in 2009. It may be mentioned that the RSMC, Tokyo introduced 24-h objective forecast in 1982, 48-h forecast in 1988, 72-h forecast in 1997 and 96- and 120-h forecast in 2009 for northwest Pacific Ocean (RSMC, Tokyo 2011). Similarly, National Hurricane Centre (NHC) and RSMC, Miami, USA, introduced 24-h objective forecast in 1954, 48-h forecast in 1961, 72-h forecast in 1972 and 96- and 120-h forecast in 2001 for Atlantic Ocean (NHC 2012). Prior to 2003, the TC forecasts issued by IMD for the NIO were subjective and textual in form without mentioning expected MSW of the TC in 12- and 24-h forecast period. Of course, it contained the expected wind speed along the coastal belt only at the time of landfall. Before landfall, the intensity forecast was mentioned in terms of predicted T number for +24 h in the TC advisory bulletins issued by IMD, which indirectly indicates the MSW, as T number is associated with MSW (Dvorak 1984).

Most of the TC forecasting centres including NHC, USA and RSMC, Tokyo verify their TC intensity forecasts regularly on yearly basis. The parameters verified include MSW in most of the TC centres. However, in some centres, the forecast central pressure of TC is also verified. IMD introduced the objective TC forecast verification in 2009 (RSMS, New Delhi 2010) based on MSW only, as it does not issue any forecast of central pressure. The TC forecast errors can be analysed from several perspectives. The most common is the absolute error (AE) and root mean square error (RMSE). While NHC, USA calculates AE of MSW, RSMC, Tokyo calculates RMSE of MSW and central pressure of TC.

In order to compare forecasts made in extremely variable conditions and evaluate their quality by including these forecasts’ degree of difficulty (in particular with the aim of detecting forecasts’ trends with time), there are several possible options, the first being the measurement of gain (or loss) in skill in forecast in relation to a reference model. In other words, an essential component of the error analysis is to normalise all forecast errors by a standard technique. Climatology and persistence (CLIPER) model is a statistical forecast technique which is used as a standard in NHC, USA as the standard technique to calculate the skill in track forecast (NHC 2012). In this study, an attempt is made to evaluate the TC intensity forecast issued by IMD during 2005–2011 (7 years) based on the calculation of AE, RMSE and skill in intensity (MSW) forecast. Here, the skill has been calculated considering persistence forecast as the reference, as there is no CLIPER model for intensity forecast over the NIO. Also, the skill has been calculated with reference to random forecast following the methodology suggested by Aberson (2008).

Apart from the utilities as mentioned in the first paragraph, this study can be utilised to interpret the intensity forecast properly and decide effective cyclone management response actions by the disaster managers. Further, though the TC forecasts have been steadily improving for several decades, some uncertainty still remains. A part of this uncertainty is due to an inherent predictability limit that future improvements in numerical models and in forecasting techniques may not be able to overcome. End users of TC forecasts, such as risk managers and public agencies, need both reliable intensity forecasts and an estimation of the forecast uncertainty. To determine the forecast uncertainty, it is essential to know the intensity forecast errors and skill in the past years.

2 Data and methodology

Tropical cyclone intensity forecast error is defined as the difference between the forecast and the best track intensities for the same time. The best track and intensity parameters are prepared by IMD after the reanalysis of the track and intensity parameters after the two cyclone seasons (April–June and October–December). This reanalysed track and intensity parameters are reviewed and finalised by a committee of experts in the annual cyclone review meeting usually held in the month of January of the following year. The above-mentioned definition of intensity forecast error has been used in this study to verify the intensity forecast issued by IMD for the TCs over the NIO. The data sample for this purpose is composed of all TCs with different intensities (CS and above) over the NIO (north of equator and between longitude 45°E and 100°E), over the period of 7 years (2005–2011). The intensity forecast has been issued by IMD from deep depression stage onwards since 2009 for 12-, 24-, 36-, 48-, 60- and 72-h forecast periods in terms of MSW. It has been issued during CS stage onwards during 2003–2008 and for 12- and 24-h forecast periods only. The TC forecast based on 00, 06, 12 and 18 UTC observations has been considered for verification. The forecasts are issued about three hours after the above-mentioned observation time. The IMD forecast intensity results from a manually analysed forecasting process, which relies on the output from several numerical weather prediction (NWP) models (RSMC 2010) including IMD Global Forecast System (GFS), European Centre for Medium range Weather forecast (ECMWF), the UK Meteorological Office, Japan Meteorological Agency (JMA), ARP-Meteo-France, National Centre for Environmental Prediction (NCEP) GFS and a dynamical statistical model (Kotal et al. 2008). Consensus forecasts that gather all or part of the numerical forecast intensities and uses synoptic and statistical guidance are utilised to issue operational forecasts.

A total of 24 TCs during 2005–2011 (Fig. 1) have been considered in the study including 18 over the Bay of Bengal (BOB) and 6 over the Arabian Sea (AS). Detailed characteristics of the TCs considered in the study are given in Table 1. The total number of forecasts verified for all forecast lengths (12, 24, 36, 48, 60 and 72 h) are 671 including 215, 171, 71, 54, 38 and 27 for 12-, 24-, 36-, 48-, 60- and 72-h forecasts, respectively (Table 2). The number of such forecasts decreases with increase in forecast length, as some of the cyclonic disturbances (CDs) have less life period and some weaken below the intensity of depression within 72 h of their genesis (Fig. 1).

Fig. 1
figure 1

Tracks of tropical cyclones (TCs) during 2005–2011 under consideration in the study

Full size image
Table 1 Details of the tropical cyclones (TCs) under consideration
Full size table
Table 2 Statistics of operational six hourly track forecasts verified during 2005–2011
Full size table

All the TC MSW forecasts issued by IMD have been verified against the observed MSW based on best track parameters published by IMD. In this study, the AE, RMSE and skill of MSW forecast have been calculated for the TCs during 2005–2011 for 12- and 24-h forecasts and during 2009–2011 for 36-, 48-, 60- and 72-h forecasts.

In order to measure the skill in MSW forecast, the persistence method has been used as a reference model. The persistence forecast calculates the past 12 h trend in MSW of the current TC and assumes that TC will intensify/decay with the same trend in MSW. Since persistence method is based entirely on current and past 12 h MSW, techniques that do not improve on this have no real skill. This normalisation also allows a ready comparison between “easy” and “difficult” forecast years and different ocean basins. The skill has been calculated for both AE and RMSE in MSW of TCs.

The gain in skill in relation to persistence method (PER) is quantified in percentage terms by the following:

$$ {\text{Gain (loss) in skill (\% ) in terms of AE}} = \frac{{\left( {{\text{PER\; AE}} - {\text{AE}}} \right)}}{\text{PER AE}} \times 1 0 0 $$
$$ {\text{Similarly}},{\text{gain (loss) in skill (\% ) in terms of RMSE}} = \frac{{ \left( {{\text{PER\; RMSE}} - {\text{RMSE}}} \right)}}{\text{PER RMSE}} \times 1 0 0 $$

An alternative verification method to calculate the skill as proposed by Aberson (2008) has also been applied. For this purpose, the best track MSW and operational forecast MSW have been rounded up to the nearest 5 knots. At each forecast time, a matrix (or contingency table) is filled with the count of each forecast verification pair. A sample is shown in Table 3. The first row and the column represent the forecast and best track intensities, respectively. Each subsequent row and column represents increase in intensity in 5-knot increments. Perfect forecasts are along the contingency table’s diagonal. The farther each forecast from the diagonal, the larger the forecast error is. A skill score is defined as

$$ S = \left( {C - E} \right)/\left( {T - E} \right), $$

where C is the number of correct forecast (sum of diagonal elements in the matrix), T is the total number of forecasts and E is the number of forecasts expected to be correct. It can be calculated for each contingency table (Panofsky and Brier 1958). The skill score will be unity if all cases are correctly predicted (T = C) and less than or equal to zero for no skill (C ≈ 0). The expected number of correct forecast can be calculated using any standard forecast, such as chance or CLIPER forecast. For chance forecast, the expected number of correct forecast is given by

$$ E = \Upsigma \left( {R_{\text{i}} C_{\text{i}} } \right)/T, $$

where R i and C i are the total number of cases in the ith row and ith column, respectively. As there is no CLIPER for intensity forecast over NIO, only chance forecast has been applied to calculate E in this study.

Table 3 Contingency table for 12 h operational forecast of IMD
Full size table

Since, the skill score does not consider the distance from the diagonal in the contingency table, error probability distributions are also examined for different range of AE.

The accuracy of TC intensity forecast has been further analysed with respect to basin of formation (BOB, AS and NIO as whole), season of formation (pre-monsoon and post-monsoon seasons), intensity of TCs (CS and SCS or higher intensities) and type of track of TCs (climatological/straight moving and recurving/looping type) by calculating AE and RMSE separately. The data set includes 13 CS and 11 SCS or higher intensity (Table 2). The TCs over the NIO show bi-modal behaviour in their genesis with primary maxima in post-monsoon season (October–December) and secondary maxima in pre-monsoon season (March–May). The performance of TC intensity forecast has been analysed during these two seasons separately, as their track, genesis and intensification characteristics are different in these two seasons. There have been three TCs during monsoon months (one in June and two in September) and another in winter season. Since the system in the month of June during onset phase of monsoon has pre-monsoon characteristics, it has been considered as the TC in pre-monsoon season (March–May) for analysis purpose. Similarly, the TCs in September have been considered in the category of the post-monsoon season (October–December), as they occurred during withdrawal phase of monsoon. The only TC in winter season (January) has also been considered in the post-monsoon season category, as it occurred in the northeast monsoon phase in January 2005. As such, there are 09 and 15 TCs during pre-monsoon and post-monsoon seasons, respectively. Climatologically, the TCs move northwestwards over the BOB and AS. Some of the TCs move northwards or northeastwards continuously without changing the direction of motion. These two categories of TCs have been grouped together as climatological/straight moving TCs. Some TCs change the direction of movement towards northeast/southwest from the original northwestwards movement. Even sometimes, a few TCs continuously recurve leading to a looping track. These two categories of TCs have been grouped together as recurving/looping TCs for analysis of variation of intensity forecast error with respect to nature of track.

The results are analysed and presented in Sect. 3. The broad conclusions are presented in Sect. 4.

3 Results and discussion

The average error and skill in operational TC intensity forecasts over the NIO is presented in Sect. 3.1. Their interannual variations are presented in Sect. 3.2. The probability of error in different ranges in TC intensity forecasts is analysed and discussed in Sect. 3.3. The average errors with respect to basin of formation (NIO as a whole, BOB and AS) of TCs, type of track, intensity of TCs (CS and SCS or higher intensities) and season of formation (pre-monsoon and post-monsoon) are presented and discussed in Sects. 3.4–3.7.

3.1 Average error and skill in TC intensity forecast

3.1.1 Average error

The statistics of average AE and RMSE of TC intensity forecast over the NIO by IMD during 2005–2011 are shown in Fig. 2a and b, respectively. It is found that the average AE during 2005–2011 is about 7.7 and 12.6 knots for 12- and 24-h forecasts, respectively. For the period 2009–2011, when there has been significant improvement in monitoring and prediction of TCs over the NIO due to modernisation programme of IMD, the AE is about 6.7 and 10.6 knots for 12- and 24-h forecasts, indicating that there has been slight improvement in intensity forecasts in recent years (Table 4). The AE is about 14 and 20 knots for 48- and 72-h forecast period. The standard deviation (SD) in AE is about 11, 14 and 17 knots for 24-, 48- and 72-h forecasts, respectively. The higher value of SD, which is almost equal to average AE, indicates low reliability of the average for any application. The average AE of NHC, USA for the TCs over the northern Atlantic Ocean during 2009–2011 is similar to that over the NIO (NHC 2012) as it is about 11, 16 and 18 knots for 24-, 48- and 72-h forecasts over the north Atlantic Ocean.

Fig. 2
figure 2

a Average absolute error (knots) and b RMS error (knots) of IMD for TCs over the north Indian Ocean during 2005–11

Full size image
Table 4 Average AE (knots), RMSE (knots) and skill (%) of intensity forecast of IMD for TCs over the NIO during 2009–2011
Full size table

The RMSE of MSW forecast by IMD is about 14, 19 and 26 knots for 24-, 48- and 72-h forecasts, respectively, during 2009–2011 (Table 4). It is found that the RMSE over the NIO is slightly higher than that over the northwest Pacific Ocean as it is about 11, 14 and 17 knots for 24-, 48- and 72-h forecasts, respectively, during 2009–2010 (RSMC, Tokyo 2011). Considering the fact that the NIO is a data-sparse region compared to northwest Pacific and north Atlantic Oceans, as there is no aircraft probing facility in the NIO at present unlike the northwest Pacific and north Atlantic Oceans and the in situ data from the meteorological buoys are significantly less over the NIO (Mohapatra et al. 2012), the MSW forecasts are rather good and there is more scope for further improvement in intensity forecast over the NIO with expected increase in observational network over NIO in the coming years. Scarcity of observational data leads to poor initial condition in NWP models leading to increase in forecast error of NWP models, which are the backbone for operational forecast of TCs, especially in 36–72-h forecast range (RSMC, New Delhi 2010).

Considering the bias in intensity forecast, it is found that cases of over-prediction of MSW is higher than the cases of under-prediction by about 15–20 % (not shown) for all forecast times (12–72 h).

3.1.2 Average skill

The skill of intensity forecast is about 44 % for 24-h forecast and 60 % each for 48- and 72-h forecasts during 2009–2011 with respect to AE (Table 4). Similarly, it is about 48, 58 and 65 % for 24-, 48- and 72-h forecasts during the same period with respect to RMSE in TC MSW forecast. The skill in 24-h forecasts is less than that in 48- and 72-h forecasts like that over other Ocean basins. It may be due to the fact that the skill is calculated by comparing the operational forecast error with the forecast error by persistence method. The forecast error by persistence method significantly increases with increase in lead time of forecast. Hence, the skill compared to persistence method increases with increase in lead time of forecast. The average skill of NHC, USA for the TCs over the northern Atlantic Ocean with respect to reduction in AE during 2009–2011 is about 13 % each for 24- and 48-h forecasts and 18 % for 72-h forecasts (NHC 2012). It may be mentioned that the skill of TC intensity forecast by NHC, USA is calculated by comparing the operational forecast error with that based on CLIPER method (Decay-Statistical Hurricane Intensity Forecast, D-SHIFOR), which is considered as standard reference method. Hence, it shows lower skill compared to that over the NIO.

Skill in intensity (MSW) forecast has also been calculated with respect to chance forecast. It is found that skill in intensity (MSW) forecast is about 0.18, 0.06, 0.01, 0.11, 0.41 and −0.03 (not shown) for 12-, 24-, 36-, 48-, 60- and 72-h forecasts, respectively. Comparing the skills in track and intensity forecast, skill is comparable in 12-h forecast. But the skill in 24- and 48-h track forecasts are 3–4 times higher than that for the intensity forecast. DeMaria et al. (2007) have also shown the similar result (track forecast skill is 2–5 times higher than intensity forecast skill) over the north Atlantic and east and west Pacific Oceans.

3.2 Interannual variation in error and skill of intensity forecast

Verification of TC intensity forecast is important to assess the year to year improvement (or otherwise). Hence, the interannual variation in TC intensity forecast has been analysed based on trend in AE, RMSE and the skill in intensity forecast. The results are shown in Figs. 2 and 3. It is found that the AE and RMSE have decreased gradually and the skill has increased during the period of study as evident in the trend lines for 12- and 24-h forecasts. It is difficult to draw any conclusion with respect to 36–72-h forecasts due to limited period of data (3 years). The improvement in intensity forecast error (reduction in AE and RMSE) is about 0.34 knots and 0.26 knots per year for 12-h forecast, respectively, and about 1 knot each for 24-h forecasts during 2005–2011 (Table 5). However, the trend in improvement is not statistically significant according to one-tailed Students’ t test.

Fig. 3
figure 3

Average skill (%) of IMD in maximum sustained wind (MSW) forecast a based on absolute error (AE) compared to persistence forecast error, b based on RMSE compared to persistence forecast error and c compared to chance forecast for TCs over the north Indian Ocean during 2005–2011

Full size image
Table 5 Average annual improvement per year (IMP), variance explained (Var), statistical significance (Sig) of trend lines shown in Fig. 2 for AE and RMSE
Full size table

Considering the long-term trend based on data of 30 years of NHC and Joint Typhoon Warning Centre, USA, DeMaria et al. (2007) have also shown that there has been some marginal improvement in the mean AE at 24- and 48-h forecasts for the Atlantic and at 72-h forecasts for east and west Pacific Ocean. It indicates that the intensity forecasting is almost reaching the saturation stage in 24-h forecast range over the Pacific and Atlantic Oceans. However, there is still scope for further reduction in forecast error over the NIO in view of improved data availability in coming years based on the latest technology including aircraft reconnaissance, deployment of buoys, assimilation of more observational data from satellite and Doppler weather radars, etc. in the NWP models.

Considering the improvement in skill in intensity forecast compared to persistence method (Table 6), it has improved by about 6 and 9 % per year, respectively, for 12- and 24-h forecasts in the case of AE and by about 10 and 8 %, respectively, in the case of RMSE during 2005–2011. These trends are statistically significant. Considering the skill with respect to chance forecast (Fig. 3c), there is similar increasing trend in 12- and 24-h intensity forecast skill (about 0.9 and 1.7 % per year, respectively). However, the trend is not statistically significant (Table 7). The lower rate of improvement in skill in intensity forecast compared to chance forecast may be due to more rigorous nature of the methodology followed in calculation of skill as mentioned in Sect. 2. The skill in operational intensity forecast with reference to chance forecast over the north Atlantic Ocean by the NHC, USA shows also insignificant rising trend (0.23 % per year) in recent years (Aberson 2008).

Table 6 Average annual improvement per year (IMP), variance explained (Var), statistical significance (Sig) of trend lines shown in Fig. 3a, b for skill in terms of AE and RMSE
Full size table
Table 7 Average annual improvement per year (IMP), variance explained (Var), statistical significance (Sig) of trend lines shown in Fig. 3c for skill with respect to chance forecast
Full size table

3.3 TC intensity error probability

The mean percentage of TC intensity forecasts with AE ≤ 10 knots and >30 knots are shown in Fig. 4 for the period of 2005–2011. It indicates that the probabilities of the operational forecast intensity lying within an error limit of 10 knots or less are significantly higher than the probabilities of the forecast with an error of more than 30 knots. The majority of the 12–36-h forecasts are currently within 10 knots or less error category. Less than 10 % of total forecasts are associated with error of more than 30 knots in 12–60-h forecast periods and less than 20 % of total forecasts are associated with error of more than 30 knots in 72-h forecast period. Kaplan and DeMaria (2003) have shown that the approximate ratio of all over-water cases that undergo intensity changes of 35 knots or more within 24 h is about 10 %. Hence, the increase in percentage frequency of cases with 30 knots error (Fig. 4) with increase in lead period of forecast (from about 1 % in 12 h to 18 % in 72-h forecast) may be attributed to the cases of rapid intensification of the TCs.

Fig. 4
figure 4

Percentage of total operational forecasts a with absolute error (AE) of 10 knots or less and b with AE of more than 30 knots for TCs over the north Indian Ocean during 2005–2011

Full size image

The operational forecasts with error of more than 30 knots have been further examined to find out whether the forecast overestimates or underestimates the intensity of TCs. It is found that the operational forecasts overestimate the intensity of TCs in 80 % cases and underestimate in 20 % cases for different forecast lead times taken together. The ratio of overestimation to underestimation of intensity in the operational forecast leading to error of more than 30 knots increases with increase in forecast lead time from 12 to 72 h.

Considering the annual trend (Fig. 5), the percentage of forecasts with error of ≤10 knots are increasing during 2005–2011 for both 12- and 24-h forecasts. Similarly, the percentage of forecasts with error of >30 knots has decreased over the years. However, the trend is statistically significant only in case of percentage of 24-h forecasts with error of 10 knots or less (Table 8). Comparing the error probability with that over the north Atlantic and northwest Pacific Oceans, the probability distribution and the trends are almost similar (Aberson 2008; Chen et al. 2011). However, the percentage of 48-h forecast with error of more than 30 knots is about 6 % over the NIO compared to about 20 % over north Atlantic Ocean (Aberson 2008).

Fig. 5
figure 5

Percentage of total annual operational forecasts a with absolute error (AE) of 10 knots or less in 12-h forecast, b with AE of 10 knots or less in 24-h forecast and c with AE of more than 30 knots in 24-h forecast of IMD for TCs over the north Indian Ocean

Full size image
Table 8 Average annual improvement per year (IMP), variance explained (Var), statistical significance (Sig) of trend lines shown in Fig. 5 for percentage of forecasts with error (a) less than or equal to 10 knots and (b) >30 knots
Full size table

3.4 Average error in intensity forecast with respect to basin of formation of TCs

The statistics of AE and RMSE in intensity forecasts are shown in Fig. 6 for the TCs over BOB and AS. There are 405 and 171 cases of track forecasts for different forecast lengths (Table 2) for TCs over the BOB and AS, respectively. It is found that the AE is less over the BOB than over the AS for all forecast times by about 4–5 knots. The difference in AE is significant at least at 90 % level of confidence according to Student’s t test for 12-, 24-, 36- and 60-h forecasts and not significant for 48 and 72-h forecasts. However, the AE increases as the forecast time increases over both BOB and AS like that for the NIO as a whole. Considering the RMSE in intensity forecast (Fig. 6b), the RMSE is also generally higher over the AS for all forecast times under consideration except 24-h forecast. The difference in RMSE is about 05 knots for 24-h forecasts and about 10 knots for 48- and 72-h forecasts. The higher accuracy over the BOB may be attributed to the fact that the open sea area, which is a data-sparse region, is relatively less in case of BOB than in AS. While there is a dense observatory network along the coast surrounding the BOB due to dense coastal observatories and cyclone detection radars, the coastal area surrounding AS does not have such a network. The enhanced observations, available on real time, help in more accurate determination of location and intensity over BOB than in AS (Mohapatra et al. 2012). The higher initial error in location and intensity estimation of CDs over the AS results in higher intensity forecast error in the NWP models.

Fig. 6
figure 6

a Absolute error (AE) and b root mean square error (RMSE) in TC intensity forecast issued by IMD for the TCs over the Bay of Bengal (BOB), Arabian Sea (AS) and north Indian Ocean (NIO) as a whole

Full size image

3.5 Average error in intensity forecast with respect to season of formation of TCs

There were nine and fifteen TCs in pre-monsoon and post-monsoon seasons, respectively, during the period considered in the study (Fig. 1; Table 1). The numbers of TC intensity forecasts issued for different forecast times (Table 2) were 281 and 295 in pre-monsoon and post-monsoon seasons, respectively. The statistics of AE and RMSE in intensity forecast for these TCs are shown in Fig. 7. It is found that the AE and RMSE are less in case of TCs during post-monsoon season for all forecast times except for 12 h. They are almost equal to each other in case of 12-h forecasts. The AE varies from 8(8) knots for 12-h forecast to 19(21) knots for 72-h forecast during post-monsoon (pre-monsoon) season. The difference in AE is statistically significant at least at 90 % level for 24- and 48-h forecasts. Considering the RMSE in intensity forecast (Fig. 7b), it varies from 11(11) knots for 12-h forecast to 21(29) knots for 72-h forecast during post-monsoon (pre-monsoon) season. The difference in RMSE and AE increases with increase in forecast times. The higher accuracy during post-monsoon season may be attributed to the fact that the percentage of severe TCs is less in post-monsoon than in pre-monsoon season (Fig. 1; Table 1). Out of 9 TCs, six (67 %) are severe TCs in pre-monsoon season, whereas 5 out of 15(33 %) TCs are severe in post-monsoon season. As discussed in the next section, the accuracy in intensity forecast is less in case of severe TCs.

Fig. 7
figure 7

a Absolute error (AE) and b root mean square error (RMSE) in TC intensity forecast issued by IMD for the TCs during pre-monsoon season and post-monsoon season

Full size image

3.6 Average error in intensity forecast with respect to category of TCs

As the TCs intensify gradually, their location and intensity estimation become more accurate with well-developed characteristic features of TCs such as eye, central dense overcast (CDO) and curved band features (IMD 2003). It results in more accurate location and intensity estimation over the open seas using satellite technique. Hence, to verify the impact of intensity of TCs on the intensity forecast, the TCs have been classified into two categories, viz., (1) CS (maximum surface wind speed of 34–47 knots) and (2) SCS and higher intensity (48 knots or more). The definition of the TCs analysed here is same as that adopted by IMD (2003). There were thirteen and eleven TCs with intensity of CS and SCS or higher, respectively, during 2005–2011 (Fig. 1) and 166 and 410 cases of intensity forecasts for different forecast times (Table 2) for TCs with intensity of CS and SCS or higher, respectively. The statistics of AE and RMSE for intensity forecasts are shown in Fig. 8. It is found that the AE is significantly higher in case of SCS or higher intensity TCs for all forecast lengths at least at 95 % level of confidence according to one-tailed Student’s t test. Similarly, the RMSE is also higher in case of SCS. It may be due to the fact that with the availability of sophisticated NWP models though some progress has been made in TC track prediction, the skill of intensity prediction still remains a difficult task to forecasters (Elsberry et al. 2007). The relatively low skill of intensity forecasts is primarily due to the complexity of the TC intensification process, which involves scale interactions between the environment, the storm and convection. Intensity forecasting in the NIO is also made more difficult by the limited number of guidance models specifically designed for that task, relatively poor data assimilation from NIO region in NWP models and the inability for current NWP models to be run at the resolutions needed to explicitly resolve convection in a real-time and operational manner.

Fig. 8
figure 8

a Absolute error (AE) and b root mean square error (RMSE) in TC intensity forecast issued by IMD for cyclonic storms (CS) and severe cyclonic storms (SCS) or higher intensity

Full size image

3.7 Average error in intensity forecast with respect to nature of the track of TCs

As evident from Fig. 1, the TCs exhibit different track characteristics. To find out the impact of nature of track on TC intensity forecast, the TCs over the NIO have been further categorised as climatological/straight moving and recurving/looping TCs and the corresponding intensity forecast errors have been analysed and discussed. There were 13 and 11 such TCs of climatological/straight moving and recurving/looping type, respectively. The numbers of TC intensity forecasts issued for different forecast lengths (Table 2) were 228 and 348 for climatological/straight moving and recurving/looping TCs, respectively. The statistics of AE and RMSE in intensity forecast are shown in Fig. 9 for the climatological/straight moving and recurving/looping TCs. It is found that the AE is less in case of recurving TCs, being about 11, 13 and 20 knots for 24-, 48- and 72-h forecasts, respectively, compared to 15, 16 and 21 knots in case of straight moving/climatologically moving TCs. The difference is statistically significant at least at 90 % level for 12-, 24- and 48-h forecasts. The RMSE is higher in case of recurving TCs for 48–72 h forecasts and less for 12–24-h forecasts and almost equal for 36-h forecasts with that for straight moving/climatologically moving TCs. The AE measures the average magnitude of the errors in a set of forecasts, without considering their direction. The AE is a linear score which means that all the individual differences are weighted equally in the average. The RMSE is a quadratic scoring rule which measures the average magnitude of the error. Since the errors are squared before they are averaged, the RMSE gives a relatively high weight to large errors. The AE and the RMSE can be used together to diagnose the variation in the errors in a set of forecasts. The RMSE will always be larger or equal to the AE; the greater the difference between them, the greater the variance in the individual errors in the sample. Hence, the higher RMSE in case of recurving/looping TCs for 48–72-h forecasts indicates higher variance in individual errors in this category of TCs.

Fig. 9
figure 9

a Absolute error (AE) and b root mean square error (RMSE) in TC intensity forecast issued by IMD for the recurving/looping TCs and straight/climatologically moving TCs

Full size image

4 Conclusions

The operational TC intensity (MSW) forecast issued by IMD during 2005–2011 (7 years) has been evaluated by calculating the AE, RMSE and skill in intensity forecast. While AE and RMSE in TC intensity forecast have been calculated against the observed intensity, the skill in operational intensity forecast has been calculated by comparing it with both forecasts by persistence method and chance forecast. The accuracy of TC intensity forecast has been analysed with respect to basin of formation (BOB, AS and NIO as whole), season of formation (pre-monsoon and post-monsoon seasons), intensity of TCs (CS and SCS or higher intensities) and type of track of TCs (climatological/straight moving and recurving/looping type). The interannual variations in the AE, RMSE and skill have also been analysed by calculating the linear trend coefficients. The results of the study have been compared with those over northwest Pacific and north Atlantic Oceans. The broad conclusions of the study are given below.

The average AE (RMSE) in operational intensity (MSW) forecast is about 11(14), 14(19) and 20(26) knots, respectively, for 24-, 48- and 72-h forecasts over the NIO as a whole during 2009–2011. The SD in AE is about 11, 14 and 17 knots for 24-, 48- and 72-h forecasts, respectively. The higher value of SD, which is almost equal to average AE, indicates low reliability of the average. Considering the bias in intensity forecast, the cases of over-prediction of MSW are higher than the cases of under-prediction by about 15–20 % for all forecast times (12–72 h).

Considering the skill in terms of AE of operational intensity forecast compared to persistence method, it is about 44 % for 24-h forecast and 60 % each for 48- and 72-h forecasts during 2009–2011. Similarly, it is about 48, 58 and 65 % for 24-, 48- and 72-h forecasts during the same period in the case of RMSE in MSW forecast. With respect to chance/random forecast, the skill is about 0.06, 0.11 and −0.03 for 24-, 48- and 72-h forecasts, respectively. It indicates that the operational forecast is skilful compared to chance forecast up to 48-h forecast and has no skill in 72-h forecast.

There is no significant improvement in terms of reduction in AE and RMSE of MSW forecast over the NIO like that over the northwest Pacific and northern Atlantic Oceans. However, there is still scope for further reduction in forecast error over the NIO, in view of improved data availability in coming years based on the latest technology including aircraft reconnaissance, deployment of buoys, assimilation of more observational data from satellite and Doppler weather radars in the NWP models.

The improvement in skill in operational intensity forecast compared to persistence method has significantly improved by about 6 and 9 % per year, respectively, for 12- and 24-h forecasts in terms of trend in AE and by about 10 and 8 % in terms of trend in RMSE during 2005–2011. Considering the skill with respect to chance forecast, there is insignificant increasing trend in 12- and 24-h intensity forecast skill (about 0.9 and 1.7 % per year, respectively).

The majority of the 12–36-h forecasts are currently within 10 knots or less error category. Less than 10 % of total forecasts are associated with error of more than 30 knots in 12–60-h forecast periods, and less than 20 % of total forecasts are associated with error of more than 30 knots in 72-h forecast. There is significant increasing trend in percentage of 24-h forecasts with error of 10 knots or less during 2005–2011.

AE and RMSE are significantly higher in case of SCS or higher intensity TCs for all forecast times. The AE and RMSE are less in post-monsoon than in pre-monsoon season and less over the BOB than over the AS for all forecast times.

Though the analysis is based on data of limited period, especially in the forecast range of 48–72 h period, the results are very interesting and in agreement with the global trend during the same period.