Estimation of Volatility on the Small Sample with Generalized Maximum Entropy

Abstract

Generalized autoregressive conditional heteroscedasticity (GARCH) provides useful techniques for modeling the dynamic volatility model. Several estimation techniques have been developed over the years, for examples Maximum likelihood, Bayesian, and Entropy. Among these, entropy can be considered an efficient tool for estimating GARCH model since it does not require any distribution assumptions which must be given in Maximum likelihood and Bayesian estimators. Moreover, we address the problem of estimating GARCH model characterized by ill-posed features. We introduce a GARCH framework based on the Generalized Maximum Entropy (GME) estimation method. Finally, in order to better highlight some characteristics of the proposed method, we perform a Monte Carlo experiment and we analyze a real case study. The results show that entropy estimator is successful in estimating the parameters in GARCH model and the estimated parameters are close to the true values.

1 Introduction

Estimation of volatility is very important in financial economics, because volatility is a measure of uncertainty on observed time series of financial data such as stock price or stock index. Many studies realized that the volatility of financial data should not be constant overtime but invariably varying through time. The popular model to estimate the time-varying volatility is Autoregressive conditional heteroskedasticity model (ARCH) proposed by Engel [1] in 1982 and was extended by Bollerslev [2]. This paper introduces a new volatility model called Generalized autoregressive conditional heteroskedasticity model (GARCH(p,q)) in the following form

$$\begin{aligned} \varepsilon _t=\sigma _t\nu _t,~,\nu _t\sim N(0,1), t=1,2\cdots ,T, \end{aligned}$$

where

$$\sigma _t^2=\omega _0+\sum \limits _{i=1}^p\omega _{1i}\sigma _{t-i}^2+\sum \limits _{j=1}^q\omega _{2j}\varepsilon _{t-j}^2,~\omega _0,\omega _{1i},\omega _j>0 ~\forall i=1,\cdots ,p,~j=1,\cdots , q.$$

In this study, we consider GARCH(1,1) model because this is an extension of ARCH model and relies only on past observation and on past volatility. In general, GARCH parameters have been estimated by using Maximum Likelihood (MLE) approach which assumes normality. However, the assumption of conditional normality is not always appropriate. Maximum Entropy (ML) modeling which has a flexible functional form to use with many distributions has been applied in financial field. Park and Bera [3] applied two separate maximum entropy densites in ARCH model (MEARCH model) where moment functions are selected based on the sample to estimate NYSE stock returns. They show the MEARCH model was useful to capture the behavior of the sample.

From assumption on long-term stability, GARCH model will give a wrong answer if data is not stability trend. Financial time series data generally are characterized with a large sample size and structural change. To be consistent with this stability assumption, we suggest small sample size for estimating GARCH(1,1) model. Hwang and Valls Pereira [4] investigated ML estimation with small sample in GARCH model with non-negative Bollerslev’s condition that guarantees positive conditional volatility and they showed GARCH models with small sample problems from their results that the estimated parameters are negatively biased. They also suggested the minimum size of sample needed for GARCH(1,1) model to be 500 observations.

When the data has a heavy-tailed distribution, the analysis of GARCH time series data by using quasi maximum likelihood estimation (QMLE) can lead to inconsistency in parameter noted by Lee et al. [5]. Who applied maximum entropy to estimate GARCH(1,1) model for 503 observations of S&P 500 index.

Generalized Maximum Entropy (GME) method for ARCH model can be found in the book by Golan et al. [6]. In this paper, we use GME to estimate GARCH(1,1) parameters because this method does not require the large observation and assumption about distribution function for innovation. This method has only independent assumption for all random variables and support space for each random variable. We show the result by using simulation and applying the model to estimate volatility on stock price returns with small number of observations.

2 Methodology

Let $\varepsilon _t,~t=1,\cdots , T$ is sequence random variable from time series data with mean zero. The GARCH(1,1) model is defined by

$$\begin{aligned} \varepsilon _t&=\sigma _t\cdot \nu _t,~t=1,2,\cdots ,T,\end{aligned}$$

(1)

$$\begin{aligned} \sigma _t^2&=\omega _0+\omega _1\sigma _{t-1}^2+\omega _2\varepsilon _{t-1}^2,~ t=1,2,\cdots ,T, \end{aligned}$$

(2)

where $\omega _0>0,\omega _1,\omega _2\in (0,1)$ and stationary condition is $\omega _1+\omega _2<1$ and $\nu _t\sim F(\cdot )$ is sequence of independent random variable or innovation. In this study, the parameters $\omega _0,\omega _1,\omega _2$ are estimated using generalized maximum entropy (GME). The basic idea of this estimator is that the entropy, which refers to an amount of the uncertainty, is maximized subject to model and data constraints. Here, we consider the Shannon’s entropy measure proposed by Shannon [7]. This Shannon’s entropy is represented by the amount of the uncertainty of a discrete probability distribution and the sum of all outcomes probability equals to one. The constraint primal problem for ARCH model can be written as follows:

$$\begin{aligned} \max H(P_0,P_1,P_2,W_1,W_2,\cdots ,W_T)=H(P_0)+H(P_1)+H(P_2)+\sum _{t=1}^T H(W_t), \end{aligned}$$

(3)

subject to

$$\begin{aligned} \sqrt{\left( \sum _{i=1}^k z_{0i}p_{0i}\right) } \left( \sum _{i=1}^k s_{i}w_{1i} \right)&=\varepsilon _1,\\ \sqrt{\sum _{i=1}^k z_{0i}p_{0i}+\left( \sum _{i=1}^k z_{1i}p_{1i}\right) \left( \dfrac{\varepsilon _1}{\sum \limits _{i=1}^k s_{i}w_{1i} } \right) ^2 +\left( \sum _{i=1}^k z_{2i}p_{2i}\right) \cdot \varepsilon _1^2}\left( \sum _{i=1}^k s_{i}w_{2i} \right)&=\varepsilon _2,\\ \sqrt{\sum _{i=1}^k z_{0i}p_{0i}+\left( \sum _{i=1}^k z_{1i}p_{1i}\right) \left( \dfrac{\varepsilon _2}{\sum \limits _{i=1}^k s_{i}w_{2i} } \right) ^2 +\left( \sum _{i=1}^k z_{2i}p_{2i}\right) \cdot \varepsilon _2^2}\left( \sum _{i=1}^k s_{i}w_{2i} \right)&=\varepsilon _3,\\&\vdots \\ \sqrt{\sum _{i=1}^k z_{0i}p_{0i}{+}\left( \sum _{i=1}^k z_{1i}p_{1i}\right) \left( \dfrac{\varepsilon _{T-1}}{\sum \limits _{i=1}^k s_{i}w_{(T-1)i} } \right) ^2 {+}\left( \sum _{i=1}^k z_{2i}p_{2i}\right) \cdot \varepsilon _{T-1}^2}\left( \sum _{i=1}^k s_{i}w_{(T-1)i} \right)&=\varepsilon _T, \end{aligned}$$

$$\begin{aligned} \dfrac{\sum \limits _{i=1}^k z_{0i}p_{0i}}{1-\sum \limits _{i=1}^k z_{1i}p_{1i}-\sum \limits _{i=1}^k z_{1i}p_{0i}}=var(\varepsilon _t),\\ \sum _{i=1}^k p_{0i},\sum _{i=1}^k p_{1i},\sum _{i=1}^k p_{2i},\sum _{i=1}^k w_{1i},\cdots =\sum _{i=1}^k w_{Ti}=1,\\ p_{ji},w_{ti}\in (0,1) ~\forall j=0,1,2,~i=1,\cdots ,k,~t=1,\cdots ,T, \end{aligned}$$

where $H(P_j)=-\sum \limits _{i=1}^k p_{ji}\log (p_{ji}),j=0,1,2$ and $H(W_j)=-\sum \limits _{i=1}^k w_{ti}\log (w_{ti}),~t=1,\cdots ,T,$ $z_{0i},z_{1i},z_{2i},s_{i}$ are the discrete support space, and $var(\varepsilon _t)$ is the sample variance. After optimizing this function, we can estimate $\widehat{\omega _0},\widehat{\omega _1},\widehat{\omega _2},\widehat{\nu _t}$ by

$$\begin{aligned} \widehat{\omega }_0&=\sum _{i=1}^k z_{0i}p_{0i},~\widehat{\omega }_1=\sum _{i=1}^k z_{1i}p_{1i},~\widehat{\omega }_2=\sum _{i=1}^k z_{2i}p_{2i}, ~\widehat{\nu }_t=\sum \limits _{i=1}^k s_{i}w_{ti}, ~t=1,2,\cdots ,T. \end{aligned}$$

The standard deviation of parameters $\widehat{\omega _0},\widehat{\omega _1}$ and $\widehat{\omega _2}$ is to be estimated by

$$\begin{aligned} \text { std. of } \widehat{\omega }_j =\sqrt{\sum _{i=1}^k(z_{ji}-\widehat{\omega }_j)^2 p_{ji}}, ~j=0,1,2. \end{aligned}$$

Table 1. Case 1: $\omega _0=\omega _1=\omega _2=0.2$

3 Simulation Study

In this section, a simulation study was conducted to evaluate performance and accuracy of GME estimation in GARCH(1,1) model with small observation $T=\lbrace 50,100\rbrace$. For every support space, we define 5 points support for $\omega _0,\omega _1,\omega _2$ with $z_0=z_1=z_2=[0, 0.25, 0.50, 0.75, 1]$ and support space for innovation $\nu _t$ is $[-10, -5,0,5,10]$ for all $t=1,2,\cdots ,T.$ We simulated the data from the GARCH(1,1) model with 1,000 paths, where the innovation is assumed to have standard normal distribution N(0, 1). We consider $\omega _0,\omega _1,\omega _2$ for 5 cases.

Fig. 1.

Histogram of parameter estimates from simulation $T=50$(left), $T=100$(right)

Table 2. Case 2: $\omega _0=.7,~\omega _1=.5,~\omega _2=0.1$

Table 3. Case 3: $\omega _0=.2,~\omega _1=.2,~\omega _2=0.7$

Fig. 2.

Histogram of parameter estimates from simulation $T=50$(left), $T=100$(right)

Fig. 3.

Histogram of parameter estimates from $T=50$(left), $T=100$(right)

Table 4. Case 4: $\omega _0=.5,~\omega _1=.4,~\omega _2=0.4$

Fig. 4.

Histogram of parameter estimates from $T=50$(left), $T=100$(right)

Table 5. Case 5: $\omega _0=.6,~\omega _1=.7,~\omega _2=0.2$

Fig. 5.

Histogram of parameter estimates from $T=50$(left), $T=100$(right)

The simulation results are provided in Tables 1, 2, 3, 4 and 5 and the similar results are obtained. According to Table 1, by the simulation we observe that the estimation mean of $\omega _0$ is underestimated but those for $\omega _1,\omega _2$ are overestimated for both T = 50 and 100. From Table 2, we see that the estimation mean of $\omega _0$ and $\omega _1$ are underestimated while $\omega _2$ value is overestimated for both T = 50 and 100. From Table 3, the different results are obtained, the means of $\omega _0$ and $\omega _1$ are overestimated but that for $\omega _2$ is underestimated for both T = 50 and 100. From Table 4, we see that the means of $\omega _0,\omega _1$ and $\omega _2$ are overestimated for both T = 50 and 100; however, they are close to the true values. Finally, from Table 5, the estimation mean of $\omega _0$ is underestimated and overestimated for T = 50 and T = 100 respectively; the estimation mean of $\omega _1$ is underestimated for both T = 50 and T = 100; and the mean of $\omega _2$ is overestimated for both T = 50 and T = 100.

The overall results of the simulation are likely to perform well for all case studies as the estimated mean parameters are not far away from the true values. Moreover, we also plot the histogram of estimated parameters from 1,000 paths. We present all case studies and plot in Figs. 1, 2, 3, 4 and 5. We can observe that most estimated parameters are close to the true values and that the standard deviations are rather small. We may thus conclude that the proposed method estimates the GARCH(1,1) parameters quite well.

4 Real Data Application

In this section, we compare the performance of the entropy GARCH(1,1) using the stock index data. We consider closing stock price and daily return from Advanced Micro Devices, Inc. (AMD) from March 1, 2017 to July 28, 2017. The data is obtained from Thomson Reuters DataStream. We plot the daily closing price and return of AMD in Fig. 6. The summary statistics is shown in Table 6.

In this application study, we consider the simple GARCH(1,1) to estimate the volatility of the AMD return. The support spaces of the GARCH parameters are specified just like in the simulation study section. The estimated results are shown in Table 7. We can see that our estimated standard errors of $\omega _0,\omega _1,\omega _2$ in Table 7 are large and all parameters are insignificant. We suspect that our support spaces are perhaps too large and thereby leading a high standard error. Therefore, we try to estimate again but using a narrower range of support spaces for $\omega _0,\omega _1,\omega _2$. We define a new support space as $z_0=z_1=z_2=[0, 0.125, 0.25,0.375, 0.5]$ and $s_i$ as $[-10, -5, 0, 5, 10]$. The results are shown in Table 8, and when compared to Table 7, values of parameter estimates change and standard errors decrease; however, we can still obtain the significant results. We also plot the conditional variance and innovation error from our GARCH(1,1) (Fig. 7).

Fig. 6.

Closing stock price (up) and return of AMD (down)

Table 6. Summary statistics of AMD

Table 7. Estimation of GARCH(1,1) parameter by GME

Fig. 7.

Estimation of conditional variance (upper) and innovation (lower)

Table 8. Parameters estimated from GME with the support space

5 Conclusions and Future Research

It is not easy to get the big data with certain economic situation or stable environment. Thus many estimations face the ill-posed problem. In this study, GME estimator is proposed to estimate the unknown parameters in GARCH model. The simulation results show that the GME is workable well on some values of parameters since either underestimated or overestimated results are obtained in some parameters. However, the results are still acceptable in this study. From the real data analysis, we cannot find the significant result of the GARCH parameters. The problem of GME estimation in GARCH(1,1) is that the value of parameter estimates depends on the support space. We find that the narrow support space will lead a smaller standard error of the parameters.

Therefore in future research, we should find the new method to estimate a GARCH model or other volatility models that can handle the small observation problem. GME method should also be extended to GARCH(p,q) and other stochastic volatility (IGRACH, GJR-GRACH, NGARCH etc.) models.