How Accurate Are Risk Models During COVID-19 Pandemic Period?

Abstract

During times of a financial crisis, knowledge of modern risk management approaches is required. In this sense, only financial risk managers with the required expertise to measure and understand risk could avoid crisis. Value at Risk (VaR) has been considered for a long time as a significant risk management tool. This measure has become one of the most popular indicators of financial market risk since JP Morgan published its RiskMetrics system in 1994. In this chapter, we study the relative performance of Value-at-Risk models prior and after the recent financial crisis referring to the COVID-19 pandemic period. Using the NGARCH model, which considers the leverage effect, we model the conditional volatility of each series. We compared the accuracy of five VaR estimates using backtest methods. The result suggests that the conditional EVT remarkably achieve reliable VaR forecasts and then is more relevant and the best performing model. In terms of VaR forecasting, given that this model obviously beats other competitive models, we encourage the use of this model when controlling market risk.

1 Introduction

Managing risk has always been an important part of financial institutions such as banks, insurance industries and investment funds. Recently, “risk management” has become a popular buzzword – the phrase appeared more than 450,000 times in web of science search engine in 2021. For many years risk managers have long been searching for a “good” risk measure. The growing complexity of financial products, particularly derivatives, has made assessing and measuring the risks faced by financial institutions more challenging. Value at Risk (VaR) emerged as the favored method for measuring risk. It has become a very popular risk management method for many multinational companies in the last two decades. The VaR measures how much the value of a portfolio could decline over a certain time horizon because of changes in market prices. In this sense, it summarizes the risk of the entire portfolio in a single number that non-specialists can understand quickly and easily.

The VaR concept has been applied since 1994, when J.P. Morgan provided the first set of standardized assumptions called RiskMetrics (RM). The most used VaR models assume that the probability distribution of the daily financial asset return is normal. However, many of firm’s returns show significant levels of skewness and kurtosis. In this context, most empirical studies focused on market risk and estimates VaR for stocks index.

This VaR estimation seems to be a difficult task during periods of financial turmoil. In fact, the main concern in the estimation of market risk with the VaR method is the choice of the appropriate model. As a result, many studies have used several methods that allow to better assess the market risk. For example, Bao, Lee, and Saltoglu (2006) compared different VaR models and support the dominance of filtered models over unfiltered and Riskmetrics models. Dimitrakopoulos et (2010), using historical simulation and extreme value theory, assesses the precision and effectiveness of different VaR approaches in 16 emerging and 4 developed stock markets. Echaust and Just (2020) compared the accuracy of VaR forecasts between conditional and unconditional models. They found that the choice between the two models depend on the period of the analysis and the type of risk measure. Abad et al. (2014), In a review paper, found that the best methods for forecasting are approaches based on Extreme Value Theory and Filtered Historical Simulation. In a more recent paper, Ghorbel and Trabelsi (2014), studied the performance of VaR estimates from three energy commodity markets. They find that GARCH-t, conditional EVT and FIGARCH extreme value copula methods produce acceptable estimates of risk both for standard and more extreme VaR quantiles. In the same line, Aloui and Mabrouk (2010), computed the VaR for three ARCH/GARCH-type models including FIGARCH, FIAPARCH and HYGARCH. They conclude that asymmetry, fat-tails and long-range memory are common facts on energy markets volatility. Hence studies provide different results based on the data used and the period studied.

The right way of assessing risk has become the de facto industry standard. Overreliance on value at risk, however, might give risk managers a feeling of security or mislead them into complacency. While VaR has become a standard risk management tool, the methods for calculating it have improved significantly since 1994. For practitioners and regulators, generating precise VaR estimates for specific applications has become a challenging task. Therefore, multiple methods of VaR are necessary to be investigated.

In this chapter we evaluate the performance of several common VaR estimation in the context of the recent COVID-19 financial crisis. We analyze many methodologies developed to estimate VaR, ranging from standard models to recently proposed ones. We used models that best matching with our economic question (how risks are best managed by models?). Besides, COVID-19 has become an important area of investigation, as suggested by Goodell (2020). This coronavirus disease 2019 (COVID-19) which first occurred in Wuhai, China in December 2019 has spread rapidly to other countries. From both theoretical and practical viewpoints, we will expose the relative significance of these methodologies in Tunisia until March 2020. Tunisia was chosen because, it has a high daily death rate, hence placing its health-care system under severe stress and depleting oxygen supplies. Second, it received one of the highest financial assistances (600 million) from European union. Moreover, in developed and emerging markets, financial assets behave differently and are subject to frequent and substantial shocks. The volatility distributions are fat-tailed and difficult to model analytically. As a result, VaR estimation in emerging markets is a difficult task.

We are trying to find the best risk model that might be applied during the period of COVID-19. The goal is to provide the financial risk researchers with models which better capture risk in Tunisia during COVID-19 and therefore bringing them to the limits of this field of knowledge. By giving a more precise understanding of risks, an accurate VaR estimation technique would substantially benefit practitioners and regulators.

The paper is structured as follows. The next section provides a brief literature review. In Sect. 12.3, a complete range of methodologies developed to estimate VaR from filtered historical simulation to extreme value theory are reviewed. Section 12.4 presents empirical results, and Sect. 12.5 concludes.

2 Related Literature

Since the financial crisis of 2008, economists and practitioners have paid more attention to risk measures, which are at the center of risk management practices. In this context, the so-called Value-at-Risk (VaR) is an important new idea in managing risk. The pioneering work on VaR was Morgan (1996), Jorion (2007), Duffiee and Pan (1997). Essentially, VaR introduces the question: “How much money might we lose over the next period of time?”. Accordingly, a common measuring unit, a temporal horizon and a probability must be chosen in advance to calculate VaR. This measure is usually employed by risk managers and market regulators to define the maximum loss with a given probability level (α). In the literature, several approaches for calculating VaR have been proposed and can be classified into two categories. The non-parametric methods and the parametric (and semi parametric) models based on an econometric model for volatility dynamics and the extreme value theory approach which models only the tails of the return distribution.

Parametric VaR (also known as the analytical or correlation method) is based on the estimate of the variance-covariance matrix of asset returns, using historical time series of asset returns to calculate their standard deviations and correlations. This means that the variance-covariance matrix completely describes the distribution. However, the main assumption of the parametric VaR is that the distributions of asset returns are normal. This type of VaR is called VaR with historical simulation (HS) or stochastic simulation also known as Monte Carlo (MCS).

Historical Simulation is the most popular procedure to forecast VaR. For instance, According to Pérignon and Smith (2010) nearly three-quarters of banks who disclose their VaR approach use historical simulation. The success of HS can be attributed to its ease of use and smoothness. Inspired by this method and by including Exponential Weighted Moving Average into the VaR forecast, Žiković and Aktan (2011) propose forecasting the VaR using Weighted Historical Simulation (WHS). Also, Vlaar (2000) used the historical simulation (HS) approach and found that the HS-estimates outperformed those produced from Gaussian approaches. Yi-Hou Huang and Tseng (2009) found that the HS is slightly more accurate than the MCS VaR. The HS estimations’ increased accuracy can be due to better matching of tail probability. However, the best-known parametric VAR model is J.P. Morgan’s RiskMetrics.

Since the introduction of the Risk Metrics method (RM), academics and practitioners have questioned the best method for calculating VaR. From a theoretical point of view, many studies have examined into the effectiveness of HS-VaR model. According to some authors (Daníelsson and de Vries 2000; Angelidis et al. 2004) HS is inaccurate, with high significant standard errors particularly in rare events (in tail). As a consequence, HS estimates are difficult to verify.

Alternatively, several authors propose nonparametric returns distribution estimators that avoid the effects of possible misspecification. These nonparametric methods are more computationally difficult, but they can yield inferential improvements when the parametric models’ assumptions are incorrect. For example, extreme Value Theory (EVT), which models the tails of the distribution of returns without making any precise assumptions about the distribution’s center, can be used to estimate the quantile of the distribution. This EVT approach was first employed by Koedijk (1990) in which he tried to evaluate how heavy-tailed are bilateral European Monetary System (EMS) foreign exchange rates. Likewise, Neftci (2000) compared the EVT approach to VaR calculation to the standard one based on the normal distribution. He concluded that the statistical theory of extremes and implied tail estimation are indeed effective for VaR calculations. Some other authors like Ben Ameur et al. (2020) used high-frequency energy data and applied different extreme risk measures to capture the intraday dynamic dependence between oil and gas prices. For a detailed and highly useful survey on EVT in finance, see Rocco (2014).

Since the 1996 Market Risk Amendment in the Basle accord, U.S. and international banking authorities adopted VaR models for determining market risk capital requirements for large banks. VaR has become a standard measure of financial market risk and, as a result, it has increasingly been adopted by other non-financial firms.

3 Methodologies

3.1 Specifying Volatility

Like GARCH model developed by Bollerslev (1986), We used the modified GARCH models (NGARCH) so the weight given at the return will depends on the sign of the return. In fact, assuming that the negative return increases variance by more than a positive return of the same magnitude, we can take into account the leverage effect. This model is expressed as follow:

$$ {\sigma}_{t+1}^2=w+\alpha {\left({R}_t-\theta {\sigma}_t\right)}^2+\beta {\sigma}_t^2=w+\alpha {\sigma}_t^2\left({Z}_t-{\theta}^2\right)+\beta {\sigma}_t^2 $$

(12.1)

A positive piece of news, Z _t > 0, (rather than raw return R _t) will has less impact on Variance than a negative piece of news, if θ > 0.

3.2 VaR with the Filtered Historical Simulation Approach

Since a negative return increases Variance by more than a positive return of the same magnitude the equity value will drop and the company becomes more risky and highly levered (assuming the level of debt stays constant). In this case, we can use NGARCH model as the weight given to the return depends on whether the return is positive or negative. By allowing a dynamic Variance model, we can write this:

$$ {R}_{TUIX,t+1}={\sigma}_{TUIX,t+1}{z}_{t+1} $$

(12.2)

where σ _{TUIX, t + 1} is the vaiance simulated by the GARCH /NGARCH model. Hence

$$ {VaR}_{t+1}^p=-{\sigma}_{TUIX,t+1}\mathrm{Percentile}\left\{{\left\{{\hat{z}}_{t+1-\tau}\right\}}_{\tau =1}^m,100p\right\} $$

(12.3)

where

$$ {\hat{z}}_{t+1-\tau }=\frac{R_{TUIX,t+1-\tau }}{\sigma_{TUIX,t+1-\tau }}\mathrm{for}\ \tau =1,2,\dots, m $$

(12.4)

3.3 VaR with the Cornish-Fisher Approximation

The Cornish–Fisher is used to determine the percentiles of the distribution that are non-normal. The VaR with coverage rate p can then be calculated as:

$$ Va{R}_{t+1}^p=-{\sigma}_{pf,t+1}C{F}_p^{-1} $$

(12.5)

where

$$ {\displaystyle \begin{array}{c}C{F}_p^{-1}={\phi}_p^{-1}+\frac{\xi_1}{6}\left[{\left({\phi}_p^{-1}\right)}^2-1\right]\\ {}+\frac{\xi_2}{24}\left[{\left({\phi}_p^{-1}\right)}^3-3{\phi}_p^{-1}\right]\\ {}-\frac{\xi_1^2}{36}\left[2{\left({\phi}_p^{-1}\right)}^3-5{\phi}_p^{-1}\right]\end{array}} $$

(12.6)

ξ ₁ is the skewness and ξ ₂ is the excess kurtosis of the standardized returns, z _t, and ${Z}_{t+1}=\frac{R_{TUIX,t+1}}{\sigma_{TUIX,t+1}}{}_{\sim }{}^{iid}\ D\left(0,1\right)$, D(0, 1) represents a distribution with mean equal to 0 and Variance equal to 1.

3.4 VaR with the Standardized T-Distribution

The standardized $\overset{\sim }{t}(d)$ distribution is defined by:

$$ {f}_{\overset{\sim }{t}(d)}\left(z,d\right)=c(d){\left(\frac{1+{z}^2}{\left(d-2\right)}\right)}^{-\left(1+d\right)/2}\ \mathrm{for}\ d>2 $$

Where $c(d)=\frac{\varGamma \left(\left(\mathrm{d}+1\right)/2\right)}{\varGamma \left(\left.d\right|2\right)\sqrt{\pi \left(d-2\right)}}$

$$ z=\frac{x-E(x)}{\sqrt{VaR(x)}}=\frac{x}{\sqrt{d/\left(d-2\right)}} $$

(12.7)

By combining a powerful NGARCH model and a standard distribution t, we can specify the model portfolio return as:

$$ {R}_{TUIX}={r}_{TUIX}{z}_t.\mathrm{With}\ {z}_t{\displaystyle \begin{array}{c} iid\\ {}\sim \end{array}}\overset{\sim }{t}(d) $$

(12.8)

The parameter d will be estimated using maximum likelihood by selecting d that maximizes:

$$ {\displaystyle \begin{array}{c}\ln {L}_1=\sum_{t=1}^T\ln \left({f}_{\overset{\sim }{t}(d)}\left({z}_t;d\right)\right)\\ {}=T\left\{\ln \left(\varGamma \left(\right(d+1\right)/2\right)\left)-\ln \right(\varGamma \left(d|2\right)-\ln \left(\pi \right)/2-\ln \left(d-2\right)/2\Big\}\\ {}-\frac{1}{2}\sum_{t=1}^T\left(1+d\right)\ln \left(1+{\left({R}_{pf,t}/{\sigma}_{pf,t}\right)}^2/\left(d-2\right)\right)\end{array}} $$

The VaR is defined by

$$ Va{R}_{t+1}^p=-{\sigma}_{PF,t+1}\sqrt{\frac{d-2}{d}{t}_p^{-1}(d)} $$

(12.9)

3.5 VaR with the Extreme Value Theory

Gnedenko (1943) proved the celebrated EVT theorem, which specifies the shape of the cumulative distribution function (cdf) for the value of x beyond a cut off point u. The main result of extreme value theory is that when the threshold u increases, the distribution of observations beyond the threshold converges to the generalized Pareto distribution.

$$ GPD\left(y;\xi, \beta \right)=\left\{\begin{array}{c}1-{\left(1+\xi y|\beta \right)}^{-\frac{1}{\xi }}\ if\ \xi >0\ \\ {}1-\mathit{\exp}\left(-y|\beta \right)\ if\ \xi =0\end{array}\right. $$

(12.10)

With β > 0 and y ≥ u and the tail index parameter ξ controls the shape of the distribution tail and especially how fast the tail goes to zero when the extreme y goes to infinity.

The tail index can be estimated non-parametrically with the Hill estimator (Hill 1975) and VaR will be estimated by

$$ Va{R}_{t+1}^{\mathrm{p}}={\sigma}_{TUIX,t+1}{F}_{1-p}^{-1} \vspace*{-12pt} $$

$$ Va{R}_{t+1}^p={\sigma}_{TUIX,t+1}u{\left[\frac{p}{\raisebox{1ex}{${T}_u$}\!\left/ \!\raisebox{-1ex}{$T$}\right.}\right]}^{-\xi } $$

(12.11)

where T _u is the number of observations y larger than u.

4 Empirical Results

4.1 Descriptive Statistics

We used the return of Tunisia stock index TUNINDEX defined by:

$$ {R}_{TUIX}= Ln\left(\frac{TUIX_t}{TUIX_{t-1}}\right) $$

(12.12)

where TUIX_tis the daily closing value of the TUNINDEX on day t

The daily price is shown on panel (a) of Fig. 12.1 and the related volatility of NGARCH is shown on panel (b). We observe a significant high volatility after the first appearance of COVID-19 in 02/03/2020. This increase encourages market professionals to manage risk through conventional methods. The volatility in March is four times (0.004) more important compared to other months (0.00005) on average.

Fig. 12.1

Dynamics of TUNINDEX stock markets

Note: This figure presents data for TUNINDEX stock index from January 2000 to May 2020; panel (a) presents the stock market price of TUNINDEX stock index; panel (b) presents stock market volatility of TUNINDEX stock index

Table 12.1 reports descriptive statistics for the daily returns before and after 01/03/2020. The mean daily return is positive (negative) before (after) March 2020 justifying a downward movement of stock prices after 02/03/2020.

For the two subsamples, the sample skewness is negative, which means that the negative shocks are more common than the positive ones. The estimation of kurtosis is very strong before the first appearance of the Corona, indicating that the return distributions are leptokurtic, with heavier tails than the normal distribution. On the contrary, Jarque-Berra statistics shows that during COVID-19 sub-period the distribution is not normal with a significant level of 1%. This result is confirmed for the full sample.

Table 12.1 Descriptive statistics of TUNINDEX and VaR at 99% confidence interval

The first step is to suit the model that enables the leverage effect. Depending on whether the return is positive or negative we can model volatility by Nonlinear GARCH model. In the context of COVID-19, firms become more highly levered and riskier. In panel 2 of Table 12.1, we present descriptive statistics of VaR’s for the different models used. On average, The VaR is 1% for the extreme value estimation but higher for other models. The value is likely to increase steadily for the Cornish Fisher approximation. This is also shown in Fig. 12.2. The VaR with extreme values have the lowest value and the VaR with Cornish Fisher have the highest value, for the entire period. It is clear that all these VaR’s estimation reach high values by March 2020, the date of the first case of COVID-19 in Tunisia. The comparison of VaR against TUNINDEX return show how well all measures of the risk can detect the abnormal fluctuation of the market. However, in order to demonstrate which model is most effective, we conducted a backtest procedure.

Fig. 12.2

time Varying VaR for TUNINDEX index

Note: This figure presents VaR’s forecasts using filtered historical simulation, Cornish Fisher, Standardized t-Distribution and Extreme Value Theory from January 2019 to May 2020

4.2 The Performance of the VaR’s Models

Every model’s performance is measured in terms of the number of violations. if the return on the following day is greater than today’s VaR, a violation is occurred. This framework (back-testing) consists of three tests method developed by Christoffersen (1998). The unconditional coverage test checks whether the total number of violations is statistically acceptable or not. The independence test aims to check, over time, potential clustering of violations. Conditional coverage test checks in which respect the time series of VaR violations does not satisfy the correct conditional coverage. These tests are distributed asymptotically as a chi-squared distribution (Table 12.2).

Table 12.2 Backtesting results of COVID-19 period (Test of Unconditional Coverage, Independence and Conditional Coverage)

The back-test study carried out according to the parameter L _uc, L _ind, L _cc shows that all methods are favorable and allow to control the market risk. It shows that because the probability ratio test is always equal to zero, the difference between theoretical and empirical violation ratios is statistically significant. In other words, no negative performance of the TUNINDEX index exceeded the VaR’s limit. However, it appeared that Conditional extreme value theory is the only model that can be very close to the TUNINDEX return. Figure 12.3 illustrated this result and indicates that risk in Tunisia can be best controlled by the extreme value theory. All models passed all violation tests, but the extreme value Theory is preferable. Filtered historical simulation however rejected the model at 1% level. According to Žiković and Aktan (2011) and Adesi et al. (2014) historical simulation yields significantly lower forecasts than other alternative procedures and contains very little information about future volatility. In the context of exchange rates, Wang et al. (2010) also find that EVT is more appropriate to forecast the VaR of the Yuan in comparison to Historical Simulation and other methods. Out of 4781 cases, the filtered historical simulation fails 67 times however no fails were registered in the other method.

Fig. 12.3

VaR’s forecasts superimposed on TUNINDEX return

Note: This figure presents VaR’s forecasts superimposed on TUNINDEX return from January 2019 to May 2020; (a) this panel presents VaR’s forecasts using Cornish Fisher approximation; (b) this panel presents VaR’s forecasts using Extreme Value Theory; (c) this panel presents VaR’s forecasts using Filtered Historical Simulation; (d) this panel presents VaR’s forecasts using Standardized t Distribution

Backtest examination and visualizations for the full sample shows that the Conditional EVT, followed by the Filtered Historical Simulation, the standardized t-distribution and finally the Cornish Fisher approximation, is the best performing model. This result is also valid under each model for the same range of quantiles. Dimitrakopoulos et al. (2010) evaluated and compared several models of VaR during normal, crises, and post-crises periods. Overall, they find that the among the most successful VaR models for both emerging and developed markets is the VaR with EVT. Hence, the Conditional EVT model consistently performs the best in estimating and forecasting VaR for the entire period. Even though, it is the best model, the asymptotic properties of EVT are based on the assumption of iid returns which is usually not satisfied in practice. Finally, because intradaily Ultra-High Frequency Data (UHFD) is becoming more widely available, a growing literature recommends estimating VaR using daily volatility based on these types of observations.

5 Conclusion

In this study, we investigate a comparative predictability of VaR estimates from various estimation techniques in COVID-19 pandemic period. The main emphasis has been to evaluate how well the VaR’s estimation models performs efficiently well in modeling market risk during financial turmoil. It has presented the direct evidence on the performance of Value-at-Risk models during Covid-19 period. In order to investigate this, daily TUNINDEX returns has been considered and from an econometric point of view, we employ several approaches to calculate VaR.

The preliminary analysis of the data shows high volatility which occasionally happened after March 2020 when COVID-19 appeared for the first time. To compare the accuracy of each model, we conducted a backtest estimations for each model. The result shows that the conditional EVT is the best performing model, as it gets closer to TUNINDEX returns. Therefore, it is important to take into account the implications of the estimation of VaR model during exogenous crisis, like the recent COVID-19 crisis. These findings might have useful policy implications for investors, regulators, and hedge fund managers in case they are looking to manage market risk. Results offer important implications regarding the recent financial turmoil with respect to the estimation of VaR. It is clear that in our study, we joint past results were VaR models face difficulties in estimating moderate loss quantiles in non-parametric models compared to parametric models. In some countries which experience unstable financial market we should consider similar recommendations.