Path Properties of a Generalized Fractional Brownian Motion

Abstract

The generalized fractional Brownian motion is a Gaussian self-similar process whose increments are not necessarily stationary. It appears in applications as the scaling limit of a shot noise process with a power-law shape function and non-stationary noises with a power-law variance function. In this paper, we study sample path properties of the generalized fractional Brownian motion, including Hölder continuity, path differentiability/non-differentiability, and functional and local law of the iterated logarithms.

1 Introduction

We consider the generalized fractional Brownian motion (GFBM) $X:=\{X(t): t\in {{\mathbb {R}}}_+\}$ defined via the following (time-domain) integral representation:

$$\begin{aligned} \{X(t)\}_{t\in {{\mathbb {R}}}} {\mathop {=}\limits ^\mathrm{d}}\left\{ c \int _{{\mathbb {R}}}\left( (t-u)_+^{\alpha } - (-u)_+^{\alpha } \right) |u|^{-\gamma /2} B( {\mathrm d} u) \right\} _{t\in {{\mathbb {R}}}}, \end{aligned}$$

(1.1)

where

$$\begin{aligned} \gamma \in [0,1), \quad \alpha \in \Big (-\frac{1}{2}+ \frac{\gamma }{2}, \ \frac{1}{2}+ \frac{\gamma }{2}\Big ), \end{aligned}$$

(1.2)

and $c= c(\alpha , \gamma ) \in {{\mathbb {R}}}_+$ is the normalization constant. Here, $B( {\mathrm d} u)$ is a Gaussian random measure on ${{\mathbb {R}}}$ with the Lebesgue control measure du. It is shown in [19, Proposition 5.1] that the process X is a continuous mean-zero Gaussian process with $X(0)=0$, and has the self-similarity property with Hurst parameter

$$\begin{aligned} H=\alpha - \frac{\gamma }{2} + \frac{1}{2} \in (0,1). \end{aligned}$$

(1.3)

This process arises as the scaling limit of the so-called power-law non-stationary shot noise processes which have the shot shape function of power-law with parameter $\alpha$ and the non-stationary noise distributions with a power-law variance function of parameter $\gamma$. This is established in Pang and Taqqu [19]. With i.i.d. (stationary) noises, the scaled power-law shot noise processes converge to the standard FBM, see, e.g., [20, Chapter 3.4] and [14]. Note that the power-law in the shot shape function captures the long range dependence while the power-law in the non-stationary noises captures the dispersions of their variabilities, and thus does not contribute to the long range dependence.

The GFBM X in (1.1) is a natural generalization of the standard FBM, since it preserves the same long range dependence structure as FBM, while the power-law perturbation of the (Brownian) Gaussian random measure not only introduces non-stationarity (in the increments) but also preserves the important self-similarity property.

There are three parameters, H, $\alpha$ and $\gamma$ and two relations (1.2)–(1.3). Eliminating $\alpha$ yields the following representation of the self-similar process X with Hurst parameter $H \in (0,1)$ and (scale/shift) parameter $\gamma \in (0, 1)$:

$$\begin{aligned} \{X(t)\}_{t\in {{\mathbb {R}}}} {\mathop {=}\limits ^\mathrm{d}}\left\{ c \int _{{\mathbb {R}}}\left( (t-u)_+^{H- \frac{1}{2} +\frac{ \gamma }{2} } - (-u)_+^{H- \frac{1}{2} +\frac{ \gamma }{2} } \right) |u|^{-\gamma /2} B( {\mathrm d} u) \right\} _{t\in {{\mathbb {R}}}}. \end{aligned}$$

(1.4)

Evidently when $\gamma =0$, this becomes the standard FBM $B^H$:

$$\begin{aligned} \{B^H(t)\}_{t\in {{\mathbb {R}}}} {\mathop {=}\limits ^\mathrm{d}}\left\{ c \int _{{\mathbb {R}}}\left( (t-u)_+^{H-\frac{1}{2}} - (-u)_+^{H-\frac{1}{2}} \right) B( {\mathrm d} u) \right\} _{t\in {{\mathbb {R}}}}. \end{aligned}$$

(1.5)

Although one may think of $|u|^{-\gamma /2}$ as a time change of the Brownian motion which introduces non-stationarity increments, we observe from the representation in (1.4) that for a given Hurst parameter value $H \in (0,1)$, the parameter $\gamma$ also shifts the exponents in $(t-u)_+^{H-\frac{1}{2}} - (-u)_+^{H-\frac{1}{2}}$ by the positive amount $\frac{\gamma }{2} \in (0,\frac{1}{2})$. For instance, for $H=\frac{1}{4}$, the exponent in the FBM $B^H$ is $H-\frac{1}{2}=-\frac{1}{4}$, but with $\gamma = \frac{3}{4}$, that exponent in the process X becomes $H- \frac{1}{2} +\frac{ \gamma }{2} =\frac{1}{8}$. For another instance, for $H=\frac{3}{4}$, the exponent in the FBM $B^H$ is $H-\frac{1}{2}=\frac{1}{4}$, but with $\gamma = \frac{3}{4}$, the exponent in the process X becomes $H- \frac{1}{2} +\frac{ \gamma }{2} =\frac{5}{8}$. We see that the positive shift in the exponent makes the function $(t-u)_+^{H- \frac{1}{2} +\frac{ \gamma }{2} } - (-u)_+^{H- \frac{1}{2} +\frac{ \gamma }{2} }$ smoother than $(t-u)_+^{H-\frac{1}{2}} - (-u)_+^{H-\frac{1}{2}}$. On the other hand, the function $|u|^{-\gamma /2}$ has the opposite effect, making the paths “rougher.” It is then interesting to ask how the parameter $\gamma$ affects the path properties of the GFBM.

To answer this question, we focus on the sample path properties of the GFBM X, Hölder continuity, path differentiability/non-differentiability, and functional and local law of iterated logarithms (FLIL and LLIL, respectively). In Theorem 3.1, we prove that the paths of the process X are Hölder continuous with parameter $H-\epsilon$ for $\epsilon >0$. In Theorems 5.1, 6.1 and 6.2, we prove the functional Law of Iterated Logarithm (FLIL) and LLIL as well as an LLIL for the composition of the GFBM X with itself, which again, depends only on the Hurst parameter H. These are somewhat surprising results, indicating that the nice path properties of Hölder continuity, FLIL and LLIL are preserved by the construction of the GFBM X in (1.1) and (1.4), and are not being affected by the parameter $\gamma$.

On the other hand, the differentiability of the paths of the GFBM X is affected by the parameter $\gamma$. It is well known that the FBM $B^H$ is non-differentiable for $H\in (0,1)$. In Theorem 4.1, we show that if the parameters $(\alpha ,\gamma )$ are in the region $\{ \alpha \in (1/2,1/2+\gamma /2), \gamma \in (0,1)\}$, leading to $H\in (1/2,1)$, the paths of X are differentiable, while in the rest of parameter ranges, the paths of X are non-differentiable. It is interesting to observe that for $H\in (1/2,1)$, there are distinct path differentiability properties in the two regions distinguished by $\alpha >1/2$ (differentiable) and $\alpha \le 1/2$ (non-differentiable). In addition, we show that when $\alpha >1/2$, the paths of the GFBM X is once continuously differentiable but not twice (with probability one), and we derive the first-order derivative. These results are distinct from the non-differentiability property of the FBM $B^H$.

It is worth mentioning that all these properties of the standard FBM $B^H$ rely critically on the stationary increment property, i.e., the familiar elegant covariance function and the second moment of its increment [see (2.1) and (2.2)]. The proofs of these properties are relatively straightforward, and have become standard textbook materials [20]. However, for the GFBM X in (1.1), non-stationary increments result in a rather complicated covariance function [see (2.3)]. For the proof of the Hölder continuity, we provide a useful decomposition of the increment of the GFBM X, and then evaluate their increments separately. This decomposition may turn out to be useful in other purposes. For the other properties, we draw upon some important results that were established for general Gaussian processes (some with self-similarity properties), for example, the (non)differentiability property by Yeh [25], FLIL by Taqqu [22], and local LIL and compositions of certain Gaussian processes with itself by Arcones [1]. For the GFBM X in (1.1), due to its non-stationary increment property and the particular structure of its covariance function, it is challenging to verify some of the technical conditions imposed in these results. The proofs of the non-differentiability and FLIL rely critically upon the Hölder continuity property we establish.

We also remark that the GFBM X in (1.1) is different from the so-called Brownian semi-stationary (BSS) processes introduced by Barndoff-Nielsen and Schmiegel [5], which was used to study volatility/intermittency inference problems in financial markets. The process was introduced to circumvent the non-semimartingale issues on the inference problems concerning the underlying volatility process based on realized quadratic variation (see the multi-power variation for BSS process in [4]). However, their assumptions on the spot intermittency process exclude functions of the type $|u|^{-\gamma /2}$ as we assume (see, for example, equation (4.7) in [4]).

FBMs have been recently used to study “rough” volatility [8, 12, 16]. On the other hand, non-stationary increments have been well recognized in various financial data, see, e.g., [7, 18]. The GFBM X in (1.1) and the path properties studied in this paper may be useful in the study of “rough” volatility.

We start in the next section with some preliminary results on basic properties of the GFBM X. The Hölder continuity, differentiability/non-differentiability, FLIL and LLIL results are stated and proved in Sects. 3–6, respectively.

2 Some Preliminaries

A striking distinction from the standard FBM is the non-stationary increment property. Recall that the standard FBM $B^H$ with the Hurst index H has the covariance function: for $s, t\in {{\mathbb {R}}}_+$,

$$\begin{aligned} E\bigl [B^H(s) B^H(t)\bigr ] = \frac{1}{2}c^2 \left( t^{2H} + s^{2H} - |t-s|^{2H}\right) , \end{aligned}$$

(2.1)

and the second moment of its increment:

$$\begin{aligned} E\bigl [(B^H(s) - B^H(t))^2\bigr ] = c^2 |t-s|^{2H}. \end{aligned}$$

(2.2)

This stationary increment property plays the fundamental role in proving many properties of FBM and the associated processes, for example, stochastic integrals with respect to FBM.

For the GFBM X in (1.1), the covariance function $\Psi$ between X(s) and X(t) and the second moment function $\Phi$ of its increment $X(s) - X(t)$ are given, respectively, by

$$\begin{aligned} \Psi (s,t)&:= \mathrm {Cov}(X(s), X(t)) = E[X(s)X(t)] \nonumber \\&= c^2 \int _{{\mathbb {R}}}\Big ( \left( (t-u)_+^{\alpha } - (-u)_+^{\alpha } \right) \left( (s-u)_+^{\alpha } - (-u)_+^{\alpha } \right) \Big ) |u|^{-\gamma } \mathrm{d}u, \nonumber \\&= c^2 \int _0^s (t-u)^{\alpha } (s-u)^{\alpha } u^{-\gamma } \mathrm{d}u \nonumber \\&\quad + c^2 \int _0^{\infty } ((t+u)^{\alpha } - u^{\alpha }) ((s+u)^{\alpha } - u^{\alpha })u^{-\gamma }\mathrm{d}u, \end{aligned}$$

(2.3)

$$\begin{aligned} \Phi (s,t)&:=E\bigl [(X(s) - X(t))^2\bigr ] = c^2 \int _{{\mathbb {R}}}\Big ( (t-u)_+^{\alpha } - (s-u)_+^{\alpha } \Big )^2 |u|^{-\gamma } \mathrm{d}u \nonumber \\&= c^2 \int _s^t (t-u)^{2\alpha } u^{-\gamma } \mathrm{d}u\nonumber \\&\qquad + c^2\int _0^s ( (t-u)^{\alpha } - (s-u)^{\alpha } )^2 u^{-\gamma } \mathrm{d}u \nonumber \\&\qquad + c^{2}\int _0^\infty ((t+u)^{\alpha } -(s+u)^\alpha )^2 u^{-\gamma } \mathrm{d} u, \quad 0\le s \le t. \end{aligned}$$

(2.4)

When $\gamma =0$, the GFBM X in (1.1) becomes the standard FBM with the covariance function (2.1) and stationary second moments (2.2) of increments.

For standard FBM $B^H$, we usually distinguish two cases: $H \in (0, \frac{1}{2} )$ and $H \in ( \frac{1}{2}, 1)$, which corresponds to the exponents in (1.5) being negative and positive, respectively. However, for the GFBM X, we distinguish the following two cases:

$$\begin{aligned} H \in \left( 0, \frac{1-\gamma }{2} \right) \quad \text{ and }\quad H \in \left( \frac{1-\gamma }{2},1 \right) , \quad \text{ for }\quad \gamma \in [0,1), \end{aligned}$$

which correspond to the exponents in (1.4) being negative or positive. Note that $\gamma$ can be very close 1, in which case the interval $\big (\frac{1-\gamma }{2},1 \big )$ in the second scenario becomes very close to (0, 1), the whole range of the Hurst parameter H. The two cases can be also written in terms of $\alpha$ and $\gamma$:

$$\begin{aligned} \alpha \in \left( -\frac{1-\gamma }{2}, 0 \right) \quad \text{ and }\quad \alpha \in \left( 0, \frac{1+\gamma }{2} \right) , \quad \text{ for }\quad \gamma \in [0,1). \end{aligned}$$

Remark 2.1

(The role of $\gamma$) We highlight the following on the role of the parameter $\gamma$:

• (i) When $\gamma \in (0, 1)$, the increment is not second-order stationary, that is, $\Phi (s,t)$ is not a function of $|s-t|$.

• (ii) $\mathrm {Var}(c^{-1}X(t)) = c^{-2} \Psi (t,t) = t^{2H}$ is decreasing in $\gamma$ and increasing $\alpha$ (where $c = c(\alpha , \gamma ) \in {{\mathbb {R}}}_+$ is the normalization parameter in Lemma 2.1).

• (iii) Flexibility for Hurst parameter H: Fig. 1 illustrates the range of $\alpha$ and $\gamma$ for the Hurst parameter $H \in (0,1)$. The middle dotted line corresponds to the value $H=0.5$ (including the special cases $(\alpha =0, \gamma =0)$ and $(\alpha \approx 0.5, \gamma \approx 1)$), instead of the single value $H=0.5$ in the case of BM; see further discussions in Remark 2.2. For $\alpha \approx 0$ and $\gamma \approx 1$, the Hurst parameter H can be arbitrarily close to zero, while for $\alpha \approx -0.5$ and $\gamma \approx 0$ (which is close to the FBM case), the same is also true.

• (iv) Roughness of paths: Hölder continuity, FLIL and LLIL hold with the Hurst parameter H as for the standard FBM $B^H$. However, the GFBM X is differentiable when $\alpha \in (1/2, 1/2+\gamma /2)$ and $\gamma \in (0,1)$ (resulting $H \in (1/2,1)$), while it is non-differentiable when $\alpha \in (-1/2+\gamma /2, 1/2]$ and $\gamma \in (0,1)$ (resulting $H\in (0,1)$).

Fig. 1

The set of parameters $\,(\gamma , \alpha )\,$ given in (1.2) is shown in the shaded area. The boundary points are not included in (1.2). The dotted lines corresponding to the Hurst parameters $\,H = 0, 0.5, 1\,$ are plotted, respectively. The thick line segment corresponds to the FBM with $H \in (0, 1)$ and $\gamma = 0$. In the neighborhood of the point $\,\gamma = 1\,$, $\, \alpha \, =\, 0 \,$, the Hurst index is close to $\,0\,$

Remark 2.2

Although the FBM $B^H$ becomes a standard BM when $H=1/2$, this is not the case for the GFBM X. The values of $\alpha$ and $\gamma$ corresponding to the case $H=1/2$ lie in the line $\alpha - \frac{\gamma }{2} =0$ for $\gamma \in [0,1)$. The GFBM X only becomes a standard BM in the special case $\gamma =0$. This is due to the fact that the process X does not have stationary increments if $\gamma >0$. For $\gamma \in (0,1)$ and $\alpha = \frac{\gamma }{2}$, the process X provides an example of a H-self-similar Gaussian process with Hurst parameter $H=1/2$ which is not a BM. We remark that there are H-self-similar processes with $H=1/2$, that may not be Gaussian, see [3]. It is clear that when $H=1/2$ and $2 \alpha = \gamma > 0$, the process X is not a martingale with respect to the filtration ${\mathcal {F}}^{B}(s) := { \sigma } \{B(u), u \le s \}$, $s \in {\mathbb {R}}$ generated by the BM B, because for every $s < t$,

$$\begin{aligned} \begin{aligned} E [ X(t) - X(s) \, \vert \, {\mathcal {F}}^{B}(s) ] \,&=\, E \Big [ \int _{{\mathbb {R}}} ((t-u)^{\alpha }_{+} - (s-u)^{\alpha }_{+} ) |u |^{-\alpha } B({\mathrm d}u) \, \Big \vert \, {\mathcal {F}}^{B}(s) \Big ] \\&= \, \int ^{s}_{-\infty } ((t-u)^{\alpha } - (s-u)^{\alpha } ) |u |^{-\alpha } B({\mathrm d}u) \ne 0 . \end{aligned} \end{aligned}$$

It is worth mentioning the work on “fake” Brownian motions constructed from martingales in [11, 13].

Remark 2.3

In [19, Sections 5.1 and 5.2], generalized FBMs are stated in a more general form with the additional terms involving $(t-u)_{-}^\alpha - (-u)_{-}^\alpha$ in the integrands. This can be treated similarly with additional terms, and so we focus on the representations of X in (1.1).

2.1 The Normalization Constant $c=c(\alpha ,\gamma )$

With the increments

$$\begin{aligned} {\widetilde{B}}(t) \, :=\, B(- t) - B(0) \,, \quad t \ge 0, \end{aligned}$$

(2.5)

independent of the increments $\, B(t) - B(0) \,$, $\,t \ge 0 \,$, we obtain

$$\begin{aligned} \begin{aligned} c^{-1} X(t) \, =&\, \int _{{\mathbb {R}}} \big ( ( t-u)_{+}^{\alpha } - ( - u )_{+}^{\alpha } \big ) |u|^{-\gamma /2 } B({\mathrm d} u ) \\ \, =&\, \int ^{t}_{0} ( t-u)^{\alpha } |u |^{-\gamma /2} B({\mathrm d} u) + \int ^{\infty }_{0} [ ( t+v)^{\alpha } v^{-\gamma /2}- v^{\alpha -\gamma /2}] {\widetilde{B}}({\mathrm d} v) \\ \, {\mathop {=}\limits ^{(d)}}&\, \bigg ( \int ^{1}_{0} ( 1-v)^{\alpha } v^{-\gamma /2} B({\mathrm d} v) + \int ^{\infty }_{0} [ ( 1+v)^{\alpha } v^{-\gamma /2} - v^{\alpha -\gamma /2}] {\widetilde{B}}({\mathrm d} v) \bigg )\\&\cdot t^{\alpha - \frac{\,\gamma \,}{\,2\,} + \frac{\,1\,}{\,2\,}} \, , \end{aligned}\nonumber \\ \end{aligned}$$

(2.6)

where the last equality is a distributional identity from the scaling property of Brownian motion, and ${\mathop {=}\limits ^{(d)}}$ denotes “equal in distribution.” Thus, one can express the constant c in terms of Beta and Gamma functions as follows.

Lemma 2.1

With

$$\begin{aligned} c= c(\alpha , \gamma ):=\kappa (\alpha , \gamma )^{-1/2}, \end{aligned}$$

(2.7)

and

$$\begin{aligned} \kappa (\alpha , \gamma ):= & {} \text {Beta} ( 1 - \gamma , 2 \alpha + 1 )\nonumber \\&+ \Big ( \frac{\,\Gamma ( 1 - \gamma ) \,}{\,\Gamma (-2\alpha ) \,} - \frac{\,2 \Gamma ( 1+ \alpha - \gamma ) \,}{\,\Gamma (-\alpha ) \,} \Big ) \Gamma ( - 1 - 2\alpha + \gamma ), \end{aligned}$$

(2.8)

the GFBM X(t) in (1.1) is then normalized:

$$\begin{aligned} \mathrm {Var}(X(t)) = t^{2\alpha -\gamma +1} = t^{2H}, \quad t\ge 0. \end{aligned}$$

(2.9)

Proof

By the distributional identity (2.6) and the Itô isometry of stochastic integral, we obtain

$$\begin{aligned} \begin{aligned}&\text {Var} ( c^{-1} X(t)) \, =\, \, {\mathbb {E}} [ c^{-2} X^{2}(t) ] \\&\quad = t^{2\alpha - \gamma + 1} \cdot \bigg ( \int ^{1}_{0} ( 1-v)^{2\alpha } v^{-\gamma } {\mathrm d} v + \int ^{\infty }_{0} [ ( 1+v)^{\alpha } v^{-\gamma /2} - v^{\alpha -\gamma /2}] ^{2} {\mathrm d} v \bigg ) \\&\quad = \kappa \, t^{2H} , \end{aligned}\nonumber \\ \end{aligned}$$

(2.10)

where the last equality can be verified by Mathematica. $\square$

Remark 2.4

(Integrability) In (2.10), the indefinite integral over the infinite interval $(0, \infty )$ appears. Its integrability is verified under the condition (1.2) by direct calculation. As discussed above, we consider the two cases $\, 0< \alpha < (1+\gamma )/2\,$ and $\, - (1-\gamma )/ 2< \alpha < 0 \,$ for a gamma $\gamma \in (0,1)$.

Indeed, if $\, 0< \alpha < (1+\gamma )/2\,$, then $\,u \mapsto (1+u)^{\alpha } - u^{\alpha } \,$ is a decreasing function in $(0, \infty )$ and by Taylor expansion, $\,(1+u)^{\alpha } - u^{\alpha }\, \le \, \alpha u^{\alpha -1}\,$ for every $\, u > 0 \,$, and hence, the indefinite integral in (2.10) is integrable:

$$\begin{aligned} \begin{aligned}&\int ^{\infty }_{0} [ (1 + u)^{\alpha } - u^{\alpha }]^{2} u^{-\gamma }{\mathrm d} u \\&\quad =\, \Big (\int ^{1}_{0} + \int ^{\infty }_{1}\Big ) [ (1 + u)^{\alpha } - u^{\alpha }]^{2} u^{-\gamma }{\mathrm d} u \, \\&\quad \le \int ^{1}_{0} u^{-\gamma } {\mathrm d} u + \int ^{\infty }_{1} \alpha ^{2} u^{2(\alpha - 1) - \gamma } {\mathrm d} u \,\\&\quad =\, \frac{1}{\, 1 - \gamma \, } + \frac{\,\alpha ^{2}\,}{\,1 + \gamma - 2 \alpha \,}\, =:\, c_{1} < \infty . \, \end{aligned} \end{aligned}$$

(2.11)

Similarly, if $\, - (1-\gamma )/ 2< \alpha < 0 \,$, then set $\, {\widetilde{\alpha }} := - \alpha \in (0, 1/2) \,$, ${\widetilde{\gamma }} := - 2 \alpha + \gamma \in (0, 1)$. Rewriting the indefinite integral in (2.10) in terms of ${\widetilde{\alpha }}$ and ${\widetilde{\gamma }}$, we obtain

$$\begin{aligned} \begin{aligned}&\int ^{\infty }_{0} [ (1 + u)^{\alpha } - u^{\alpha }]^{2} u^{-\gamma }{\mathrm d} u \\&\quad =\, \int ^{\infty }_{0} \frac{\,[ (1 + u)^{-\alpha } - u^{-\alpha }]^{2} \,}{\,u^{-2\alpha } ( 1 + u)^{-2\alpha } \,} u^{-\gamma }{\mathrm d} u \, \\&\quad \le \, \int ^{\infty }_{0} {\,[ (1 + u)^{ {\widetilde{\alpha }}} - u^{ {\widetilde{\alpha }}}]^{2} \,} u^{- {\widetilde{\gamma }}}{\mathrm d} u\\&\quad \le \frac{1}{\, 1 - {\widetilde{\gamma }}\, } + \frac{\, {\widetilde{\alpha }}^{2}\,}{\,1 + {\widetilde{\gamma }} - 2 {\widetilde{\alpha }}\,} < \infty \, , \end{aligned} \end{aligned}$$

(2.12)

where we used (2.11) with $\alpha , \gamma$ being replaced by ${\widetilde{\alpha }}, {\widetilde{\gamma }}$ in the second inequality. If $\, \alpha \, =\, 0 \,$, the indefinite integral in (2.10) is 0. Thus, under the condition (1.2), the GFBM X is well defined.

For the standard FBM $B^H$, we have $\mathrm {Var}(c^{-1}B^H(t)) = t^{2H}$, which is increasing in H for each $t>0$, and is also increasing in t for each H. It is clear that the same properties hold for the process X. In addition, we observe that $\mathrm {Var}(X(t))$ is decreasing in $\gamma$ for each $t>0$.

Remark 2.5

(Standard FBM) By the recursion formula of the gamma function, when $\gamma = 0$ and $\alpha \in (-1/2, 1/2)$, it is the standard FBM $B^H$ and the constant $\kappa$ in (2.8) is reduced to $\kappa = \Gamma ( 1 + \alpha ) \Gamma ( 1 - 2 \alpha ) / [(1+2\alpha ) \cdot \Gamma ( 1 - \alpha ) ]$.

2.2 Generalized Riemann–Liouville (R–L) FBM

A special model is the generalized Riemann–Liouville (R–L) FBM, introduced in Remark 5.1 in [19]. It is defined by

$$\begin{aligned} X(t) = c \int _0^t (t-u)^\alpha u^{-\gamma /2} B( {\mathrm d} u), \quad t\ge 0, \end{aligned}$$

(2.13)

where $B( {\mathrm d} u)$ is a Gaussian random measure on ${{\mathbb {R}}}$ with the Lebesgue control measure du and

$$\begin{aligned} c \in {{\mathbb {R}}}, \quad \gamma \in [0,1), \quad \alpha \in \Big (-\frac{1}{2}+ \frac{\gamma }{2}, \ \frac{1}{2}+ \frac{\gamma }{2}\Big ). \end{aligned}$$

Such a process is also a continuous self-similar Gaussian process with Hurst parameter $H = \alpha - \frac{\gamma }{2}+ \frac{1}{2} \in (0,1)$. Equivalently, given the Hurst parameter $H \in (0,1)$ and a parameter $\gamma \in [0,1)$, the process X in (2.13) can be represented as

$$\begin{aligned} X(t) = c \int _0^t (t-u)^{H- \frac{1}{2} +\frac{\gamma }{2} } u^{-\frac{\gamma }{2}} B( {\mathrm d} u), \quad t\ge 0. \end{aligned}$$

(2.14)

When $\gamma =0$, this becomes the standard R–L FBM:

$$\begin{aligned} B^H(t) = c \int _0^t (t-u)^{H- \frac{1}{2} } B( {\mathrm d} u), \quad t\ge 0. \end{aligned}$$

(2.15)

This process was introduced by Lévy ([15], see also Chapter 6 in [20]; modulo some constant scaling). When $H=1/2$, i.e., $\alpha =\frac{\gamma }{2}$ for $\gamma \in [0,1)$,

$$\begin{aligned} X(t) = c\int _0^t (t/u -1)^{\gamma /2} B( {\mathrm d}u) = c \int ^{t}_{0} ( t / u -1)^{\alpha } B( {\mathrm d} u ), \quad t \ge 0. \end{aligned}$$

(2.16)

If $\gamma =0$, then $X(t) =c B(t)$ is a Brownian motion, but if $\gamma \, =\, 2 \alpha \, \in (0,1)$, the increment can be rewritten as

$$\begin{aligned} X(t) - X(s)= & {} c \int ^{s}_{0} ((t-u)^{\alpha } - (s-u)^{\alpha }) u^{-\alpha } B({\mathrm d} u) \\&+ \int ^{t}_{s} ( t - u)^{\alpha } u^{-\alpha } B ({\mathrm d} u ) \, , \quad 0 \le s < t \, , \end{aligned}$$

indicating the non-stationarity of its increments.

The process X in (2.13) has the covariance function

$$\begin{aligned} \Psi (s,t) = E[X(s)X(t)] = c^2 \int _0^s (s-u)^{\alpha } (t-u)^{\alpha } u^{-\gamma } \mathrm{d}u, \end{aligned}$$

(2.17)

and the second moment of its increment

$$\begin{aligned} \Phi (s,t) = E[(X(t) - X(s))^2]&= c^2 \int _s^t (t-u)^{2\alpha } u^{-\gamma } \mathrm{d}u \nonumber \\&\quad + c^2\int _0^s |(t-u)^{\alpha } - (s-u)^{\alpha } |^2 u^{-\gamma } \mathrm{d}u, \end{aligned}$$

(2.18)

for $0 \le s\le t$.

We also have the variance

$$\begin{aligned} \mathrm {Var}(c^{-1}X(t))&= \int _0^t (t-u)^{2\alpha } u^{-\gamma } \mathrm{d}u \\&= t^{2\alpha - \gamma +1} \int _0^1 (1-v)^{2\alpha } v^{-\gamma } \mathrm{d}v = t^{2\alpha - \gamma +1}\text {Beta} (1-\gamma , 2\alpha +1). \end{aligned}$$

Thus, with normalization constant $c=c(\alpha , \gamma ) = \big (\text {Beta} (1-\gamma , 2\alpha +1)\big )^{-1/2}$, we have $\mathrm {Var}(X(t)) = t^{2\alpha -\gamma +1} = t^{2H}$ for $t\ge 0$. It is clear that with this normalization factor, we have $\mathrm {Var}(X(t))$ increasing in $\alpha$ and decreasing in $\gamma$ for each fixed $t>0$.

3 Hölder Continuity

In this section, we prove the Hölder continuity property of the GFBM X in (1.1). For convenience, we assume from now on that the GFBM X is normalized with $c=c(\alpha , \gamma )$ given in (2.7).

Recall that for FBM $B^H$ with Hurst parameter $H \in (0,1)$, by self-similarity, we have

$$\begin{aligned} E\big [ |B^H(t) - B^H(s)|^p \big ] = |t-s|^{pH} E\big [ |B_H(1)|^p \big ] \end{aligned}$$

for any $p>0$. Then, the Hölder continuity property follows from applying the Kolmogorov–Chentsov continuity criterion. Namely, the FBM $B^H$ admits a version whose sample paths are almost surely Hölder continuous of order strictly less than H.

Due to the lack of stationary increments, the proof of the Hölder continuity property of the GFBM X requires a delicate study of the second moment of the increment.

Theorem 3.1

For every $\, T > 0 \,$, there exists a positive constant $\,C_{T}\,$ such that the covariance function $\,\Phi \,$ in (2.4) satisfies $\, \Phi (s, t) \le C_{T} |s - t |^{2H} \,$ for $0< s < t \le T\,$, and hence, by Kolmogorov–Chentsov continuity criterion, the sample path $\, t \mapsto X(t) \,$ of the Gaussian process $\, X \,$ in (1.1) is almost surely $\,\alpha _{0}\,$-Hölder continuous for $\, 0 \le t \le T\,$ with $\, 0< \alpha _{0} < H = (2 \alpha - \gamma +1)/2\,$.

Proof

When $\, \gamma \, =\, 0 \,$, it is the case of FBM with Hurst index $\, \alpha + 1/2 \, \in (1/2, 1)\,$, and the result of Hölder continuity is known. Thus, let us consider the case with $c = 1$ and $\gamma \ne 0$. First, let us decompose $\, X(t)- X(s) \,$ from (2.6) into three independent components ${\mathcal {C}}_{1}, {\mathcal {C}}_{2}, {\mathcal {C}}_{3}$:

$$\begin{aligned} \begin{aligned} X(t)-X(s)&\, =\, \int _{{\mathbb {R}}} \big ( ( t-u)_{+}^{\alpha } - ( - u )_{+}^{\alpha } \big ) |u|^{-\gamma /2 } B({\mathrm d} u )\\&\quad - \int _{{\mathbb {R}}}\big ( ( s-u)_{+}^{\alpha } - ( - u )_{+}^{\alpha } \big ) |u|^{-\gamma /2 } B({\mathrm d} u )\\&\, =:\, {\mathcal {C}}_{1} + {\mathcal {C}}_{2} + {\mathcal {C}}_{3} \, , \end{aligned} \end{aligned}$$

(3.1)

where

$$\begin{aligned} \begin{aligned} {\mathcal {C}}_{1}&\, :=\,\int ^{s}_{0} [( t-u)^{\alpha } - (s-u)^{\alpha } ] u^{-\gamma /2} B({\mathrm d}u) \, , \\ {\mathcal {C}}_{2}&\, := \, \int ^{t}_{s} ( t-u)^{\alpha } |u |^{-\gamma /2} B({\mathrm d} u)\,, \\ \quad {\mathcal {C}}_{3}&\, := \, \int _{-\infty }^0 \big ( ( t-u)_{+}^{\alpha } - ( - u )_{+}^{\alpha } \big ) |u|^{-\gamma /2 } B({\mathrm d} u ) \\&\,\,\,=\, \int ^{\infty }_{0} [ ( t+u)^{\alpha } - (s+u)^{\alpha }] u^{-\gamma /2} {\widetilde{B}}({\mathrm d} u) \, \end{aligned} \end{aligned}$$

(3.2)

with ${\widetilde{B}}$ being given in (2.5). Thus, we have

$$\begin{aligned} \,\Phi (s, t) \, =\, E \big [ (X(t)-X(s))^{2}\big ] \, =\, E \big [ {\mathcal {C}}_{1}^{2} + {\mathcal {C}}_{2}^{2} + {\mathcal {C}}_{3}^{2}\big ] \,, \end{aligned}$$

and hence, we shall evaluate squared expectation of these three terms separately. It is worth noting that all the three summands are basically of the same order (all are less than or equal to $C|t-s|^{2H}$ for some $C>0$).

• (Evaluation of $\, {\mathcal {C}}_{1}\,$). By the change of variables with $\, u \, =\, s - (t-s)v \,$ and $\,v = x w \,$, $\,x = s/ (t-s)\,$ for every s, t with $0< s < t \le T$, we have

  $$\begin{aligned} \begin{aligned} E [ {{\mathcal {C}}}_{1}^{2} ] \,&=\, \int ^{s}_{0} |( t - u)^{\alpha } - (s- u)^{\alpha } |^{2} u^{-\gamma } {\mathrm d} u \\&= \int ^{s/(t-s)}_{0} (t-s)^{2\alpha + 1 - \gamma } \Big ( \frac{\,s\,}{\,t-s\,} - v\Big )^{-\gamma } ( ( 1 + v)^{\alpha } - v^{\alpha })^{2} {\mathrm d} v \\&= (t-s)^{2H} \Big ( \int ^{1}_{0} ( 1 - w)^{-\gamma } (( 1 + x w)^{\alpha }- (xw)^{\alpha })^{2} x^{1-\gamma } \, {\mathrm d} w \Big ) \Big \vert _{ x = s/(t-s) } \,. \end{aligned} \end{aligned}$$

  (3.3)

  When $\, 0< \alpha < (1+\gamma )/2\,$, we have

  $$\begin{aligned} \begin{aligned}&\int ^{1}_{0} ( 1 - w)^{-\gamma } (( 1 + x w)^{\alpha }- (xw)^{\alpha })^{2} x^{1-\gamma } \, {\mathrm d} w \\&\quad = \int ^{1}_{0} ( 1 - w)^{-\gamma } w^{\gamma -1} (( 1 + x w)^{\alpha }- (xw)^{\alpha })^{2} (xw)^{1-\gamma } \, {\mathrm d} w \\&\quad \le \sup _{y > 0 } \big \{ ( (1 +y)^{\alpha } - y^{\alpha })^{2} y^{1-\gamma } \big \} \cdot \int ^{1}_{0} ( 1 - w)^{-\gamma } w^{\gamma -1} {\mathrm d} w \\&\quad \le 4 \text {Beta} ( 1 - \gamma , \gamma ) < \infty \, , \end{aligned} \end{aligned}$$

  (3.4)

  where we used the inequality

  $$\begin{aligned} \sup _{y > 0 } ( (1 +y)^{\alpha } - y^{\alpha })^{2} y^{1-\gamma }\le & {} \max \Big \{ \sup _{y \ge 1 } ( (1 +y)^{\alpha } - y^{\alpha })^{2} y^{1-\gamma } ,\nonumber \\&\sup _{0 < y \le 1 } ( (1 +y)^{\alpha } - y^{\alpha })^{2} y^{1-\gamma } \Big \} \le 4 \, . \end{aligned}$$

  (3.5)

  To verify (3.5), firstly we use $\, (1 + y)^{\alpha } - y^{\alpha } \le \alpha \, y^{\alpha - 1} \,$, $\, y > 0 \,$, $\, \alpha > 0 \,$ to obtain

  $$\begin{aligned} \,( (1 +y)^{\alpha } - y^{\alpha })^{2} y^{1-\gamma } \le \alpha ^{2} y^{2(H-1)} \le \alpha ^{2} < 1\, \quad \text {for} \quad \, y \ge 1 \,, \end{aligned}$$

  and secondly, we evaluate

  $$\begin{aligned} \,( (1 +y)^{\alpha } - y^{\alpha })^{2} y^{1-\gamma } \le (1 +y)^{2\alpha } y^{1-\gamma } \le 4^{\alpha } \le 4 \, \quad \text {for} \quad \, 0 < y \le 1\,, \end{aligned}$$

  and then combine the inequalities. Similarly, when $\, -(1-\gamma )/2< \alpha < 0 \,$, considering ${\widetilde{\alpha }} = - \alpha$, ${\widetilde{\gamma }} = - 2 \alpha + \gamma < 1$, as we derived in (2.12), and also using a similar inequality to (3.5) (but now with $\,{\widetilde{\alpha }}\,$ and $\,{\widetilde{\gamma }}\,$, instead of $\, \alpha \,$, $\, \gamma \,$), we obtain the upper bound for every $\,x = s/ (t-s) > 0 \,$,

  $$\begin{aligned}&\int ^{1}_{0} ( 1 - w)^{-\gamma } (( 1 + xw)^{\alpha } - (xw)^{\alpha })^{2} x^{1 - \gamma } {\mathrm d} w \nonumber \\&\quad =\, \int ^{1}_{0} ( 1 - w)^{-\gamma } \frac{(( 1 + xw)^{ {\widetilde{\alpha }}} - (xw)^{ {\widetilde{\alpha }}})^{2}}{( 1 + xw)^{2 {\widetilde{\alpha }}} (xw)^{2 {\widetilde{\alpha }}} } x^{1 - \gamma } {\mathrm d} w \nonumber \\&\quad \le \int ^{1}_{0} ( 1 - w)^{-\gamma }w^{-2 {\widetilde{\alpha }} } {(( 1 + xw)^{ {\widetilde{\alpha }}} - (xw)^{ {\widetilde{\alpha }}})^{2}} x^{1 - {\widetilde{\gamma }} } {\mathrm d} w \nonumber \\&\quad = \int ^{1}_{0} ( 1 - w)^{-\gamma }w^{\gamma -1} {(( 1 + xw)^{ {\widetilde{\alpha }}} - (xw)^{ {\widetilde{\alpha }}})^{2}} (xw)^{1 - {\widetilde{\gamma }} } {\mathrm d} w \nonumber \\&\quad \le \sup _{y> 0 } \big \{ ((1+y)^{ {\widetilde{\alpha }}} - y^{ {\widetilde{\alpha }}})^{2} y^{1- {\widetilde{\gamma }}} \big \} \cdot \int ^{1}_{0} ( 1 - w)^{- \gamma }w^{\gamma -1 } {\mathrm d} w \, \nonumber \\&\quad \le \, 4 \, \text {Beta} ( 1-\gamma , \gamma ) < \infty \, . \end{aligned}$$

  (3.6)

  Thus, combining (3.4)–(3.6) with (3.3), we claim that there exists a positive constant $\,c_{3}\,$ such that

  $$\begin{aligned} E [ {{\mathcal {C}}}_{1}^{2} ] \le c_{3} (t-s)^{2H}\,. \end{aligned}$$

  (3.7)

• (Evaluation of $\, {\mathcal {C}}_{2}\,$). Similarly, for the second term in (3.1), by the change of variables with $\,u \, =\, (t-s)v + s\,$ and $\, H\, =\, 2\alpha - \gamma + 1 \,$, we obtain for $\, 0 \le s< t < \infty \,$,

  $$\begin{aligned} \begin{aligned} E [ {\mathcal {C}}_{2}^{2} ]&\,=\,\int ^{t}_{s} ( t- u)^{2\alpha } u ^{-\gamma } {\mathrm d} u \, \le \, ( t- s)^{2\alpha - \gamma +1}\\&\quad \int ^{1}_{0} (1-v)^{2\alpha } v^{-\gamma } {\mathrm d} v \, =\, c_{4} |t - s|^{2H} \, , \end{aligned} \end{aligned}$$

  (3.8)

  where $\, c_{4} \, :=\, \text {Beta} ( 1 + 2\alpha , 1 - \gamma ) \,$. This holds for every $\, \alpha > -1/2\,$.

• (Evaluation of $\, {\mathcal {C}}_{3}\,$). For the third term $\, {{\mathcal {C}}}_{3}\,$ in (3.1), when $\,0< \alpha < (1+\gamma )/2\,$, because of (2.11) in Remark 2.4, we have with the constant $\, c_{1}\,$ in (2.11), for $\, s < t \,$, with $\,x \, :=\, s / (t-s) > 0 \,$,

  $$\begin{aligned}&\int ^{\infty }_{x} ( (1+v)^{\alpha } - v^{\alpha } ) ^{2} ( v - x )^{-\gamma } {\mathrm d} v \nonumber \\&\quad \le \int ^{\infty }_{x} (( 1+v - x ) ^{\alpha } - ( v - x) ^{\alpha }) ^{2} ( v - x)^{-\gamma } {\mathrm d} v \nonumber \\&\quad \le \int ^{\infty }_{0} ( (1 + u)^{\alpha } - u^{\alpha })^{2} u^{-\gamma }{\mathrm d} u \, \le \, c_{1} . \end{aligned}$$

  (3.9)

  Similarly, when $\, - (1- \gamma )/2< \alpha < 0 \,$, again with $\, {\widetilde{\alpha }} \, =\, - \alpha \,$ and $\, {\widetilde{\gamma }} \, =\, - 2 \alpha + \gamma < 1\,$, for every s, t with $0< s < t$ and $\,x := s / (t-s) > 0 \,$, we have

  $$\begin{aligned}&\int ^{\infty }_{x} ((1+v)^{\alpha } - v^{\alpha } ) ^{2} ( v - x )^{-\gamma } {\mathrm d} v \nonumber \\&\quad = \int ^{\infty }_{x} \frac{ (1+v)^{ {\widetilde{\alpha }}} - v^{ {\widetilde{\alpha }}} ) ^{2}}{( 1+v)^{2 {\widetilde{\alpha }}} v^{ -2 {\alpha }}} \cdot ( v - x )^{-\gamma } {\mathrm d} v \nonumber \\&\quad \le \int ^{\infty }_{x} {( (1+v)^{ {\widetilde{\alpha }}} - v^{ {\widetilde{\alpha }}} ) ^{2}} \cdot ( v - x)^{2\alpha -\gamma } {\mathrm d} v \nonumber \\&\quad \le \int ^{\infty }_{0} ( ( 1 + u)^{ {\widetilde{\alpha }}} - u ^{ {\widetilde{\alpha }}})^{2} u^{- {\widetilde{\gamma }}} {\mathrm d} u < \infty \, , \end{aligned}$$

  (3.10)

  where we used (2.12) in the last part of inequalities. Then, for $0< s < t$, by changing the variables with $\, u = (t-s)v - s \,$ and then using (3.9)–(3.10) separately, we claim that there exists a positive constant $\,c_{5}\,$ such that

  $$\begin{aligned} \begin{aligned} E \big [ {\mathcal {C}}_{3}^{2}\big ] \,&=\, \int ^{\infty }_{0} [ (t + u)^{\alpha } - (s+u)^{\alpha }] ^{2} u^{-\gamma } {\mathrm d} u \\&=\, (t-s)^{2H} \int ^{\infty }_{s/(t-s)} [ (1+v)^{\alpha } - v^{\alpha } ] ^{2} \Big ( v - \frac{\,s\,}{\,t-s\,} \Big )^{-\gamma } {\mathrm d} v\\&\le c_{5} (t-s)^{2H}. \end{aligned} \end{aligned}$$

  (3.11)

  Combining these inequalities (3.7), (3.8), (3.11) with (3.1), we obtain the desired inequality, because the second moments $E [ |X(t)|^{2}]$ are finite as it is given in (2.9) and for $\, 0< s< t < T$,

  $$\begin{aligned} \, \Phi (t, s) \, \le \, (c_{3} + c_{4} + c_{5}) \cdot |t - s |^{2\alpha - \gamma + 1} \, . \end{aligned}$$

  (3.12)

  Since $\, X \,$ is a zero-mean Gaussian process, $\, X(t) - X(s) \,$ is a Gaussian random variable with mean $\, 0 \,$ and variance $\, \Phi (s, t)\,$, and hence, its $\,2p\,$-th moment ($\, p \ge 1 \,$) can be evaluated by

  $$\begin{aligned} E [ |X (t) - X(s) |^{2p}] \le c_{p} [\Phi (t, s) ]^{p} \le c_{p} C^{p} |t - s |^{(2\alpha - \gamma +1)p} \, \end{aligned}$$

  for some positive constant $\,c_{p}\,$ which depends on $\,p\,$. Then, applying the Kolmogorov–Chentsov continuity criterion (e.g., Theorem 1.2.1 of Revez and Marc [21]), we conclude that the sample paths of the GFBM $\, X\,$ in (1.1) is $\,\alpha _{0}\,$-Hölder continuous on every finite interval $\,[0, T]\,$ with probability one for $\, 0< \alpha _{0} < H= (2 \alpha - \gamma +1)/2\,$.$\square$

Remark 3.1

When $\gamma$ is close to 1 and $\alpha > 0$, the Hurst parameter H can be chosen with $H < 1/2$. Thus, Theorem 3.1 covers the whole range of $\, H \in (0, 1)\,$.

For the generalized R–L FBM X in (2.13), the same Hölder continuity property holds.

Remark 3.2

Consider $\, X(t) \, =\, \int ^{t}_{0} \kappa (t, u ) B({\mathrm d} u) \,$, $\, t \ge 0 \,$ with a Volterra kernel $\,\kappa (t,u) \, =\, (t-u)^{\alpha } u^{-\gamma /2} \,$. This process or similar processes have been recently studied by Yazigi [24]. By Theorem 2.1 in [24], the Volterra kernel $\kappa$ can be written as

$$\begin{aligned} k(t, u) \, =\, t^{H - 1/2} F(u/t) \, ; \quad t \ge 0 \, , 0 \le u \le t , \end{aligned}$$

where $H=\alpha - \gamma /2+1/2$ is the Hurst parameter, and $F(v) = (1-v)^\alpha v^{-\gamma /2}$ and $F(v)\equiv 0$ for $v>1$ (it is clear that $F \in L^2({{\mathbb {R}}}_+, du)$, i.e., $\int _{{{\mathbb {R}}}} |F(u)|^2 \mathrm{d}u < \infty$.) In the related work [2], necessary and sufficient conditions are derived for Hölder continuity of such self-similar processes. The condition involves the function $\Phi (s,t)$, and is closely related to the Fernique’s theorem on the continuity of Gaussian processes. By Theorem 1 in [2], we obtain that there exist constant $c_\epsilon$ such that the function $\Phi (s,t)$ in (2.18) satisfies

$$\begin{aligned} \Phi (s,t)^{1/2} \le c_{\epsilon } |t-s|^{H-\epsilon }, \quad \text {for all} \quad \epsilon >0. \end{aligned}$$

On the other hand, the proof of Theorem 1 in [2] uses the Garsia–Rodemich–Rumsey inequality (see Lemma 2 in [2]), which unfortunately only holds on the finite time interval [0, T]. We are unable to prove the Hölder continuity property with that approach for the GFBM X(t) in (1.1).

Remark 3.3

For standard FBM $B^H$, it is shown in Theorem 1.6.1 in [10] that the local Law of Iterated Logarithm holds:

$$\begin{aligned} \limsup _{t\rightarrow 0^+} \frac{|B^H(t)|}{t^H\sqrt{\log \log t^{-1}}} = c_H \end{aligned}$$

(3.13)

with probability one for some appropriate constant $c_H>0$. This implies that $B^H$ cannot have sample paths with Hölder continuity of order greater than $H = \alpha + 1/2$. For the GFBM X in (1.1), we establish the local law of iterated logarithm in Sect. 6, see Theorem 6.1. That result will imply that the process X cannot have sample paths with Hölder continuity of order greater than $H= \alpha - \gamma /2 + 1/2$.

4 Path Differentiability

We prove the following differentiability/non-differentiability property of the sample paths of X. For FBM $B^H$ with $\,H \in (0, 1)\,$, the path non-differentiability property was established in [17]; see also [10, Proposition 1.7.1]. The proof of (4.1) for FBM $B^H$ uses its self-similarity and stationary-increment properties, in particular, for $t>s$, the law of the ratio ${(B^H(t) - B^H(s))}\, /\, {(t-s)}$ is the same as $(t-s)^{H-1} B^H(1)$, and the probability of $\{|B^H(1)|>a t_n^{1-H}\}$ converges to zero where $a>0$ is any positive constant and $\{t_n\}$ is any sequence decreasing to zero as $n\rightarrow \infty$. Distinct from the standard FBM $B^H$, the GFBM X has a region of parameters $\,1/2<\alpha < 1/2+\gamma /2\,$ and $\gamma \in (0,1)$, which gives $H\in (1/2,1)$, in which its paths are differentiable, while in the rest of the parameter regions of $(\alpha ,\gamma )$, resulting $H \in (0,1)$, its paths are non-differentiable. Recalling Remark 2.2, in the case of $H=1/2$, we remark that the paths of X are non-differentiable regardless of whether X is a Brownian motion ($\gamma =0$) or not ($\gamma \in (0,1)$).

Theorem 4.1

The following differentiability properties hold:

• (i) If $\,-1/2+\gamma /2<\alpha \le 1/2\,$ and $\gamma \in (0,1)$ (resulting in $H\in (0,1)$), the GFBM X in (1.1) is not mean square differentiable and does not have differentiable sample paths; In particular, for every $s\in {{\mathbb {R}}}_+$,

  $$\begin{aligned} \limsup _{t\rightarrow s} \left| \frac{X(t) - X(s)}{t-s}\right| = + \infty \end{aligned}$$

  (4.1)

  with probability one.

• (ii) If $\,1/2<\alpha < 1/2+\gamma /2\,$ and $\gamma \in (0,1)$ (resulting in $H\in (1/2,1)$), the sample path of X in (1.1) is continuously differentiable once with derivative

  $$\begin{aligned} \frac{\,{\mathrm d} X (t) \,}{\,{\mathrm d} t \,} \, =\, c \int ^{t}_{-\infty } \alpha ( t - s)^{\alpha - 1} |s |^{-\gamma / 2} {\mathrm d} B(s) \,, \quad t \ge 0 \,, \end{aligned}$$

  (4.2)

  but not twice with probability one.

Proof

We firs prove part (i). We apply Theorem of Yeh [25] (see also its correction [26]). It says that if a separable Gaussian process $\xi =\{\xi (t): t\in [0,T]\}$ has mean zero, and satisfies the Kolmogorov’s continuity condition, and the lower bound: for some $\alpha , a>0$,

$$\begin{aligned} E[|\xi (t)- \xi (s)|^2] \ge a |t-s|^\alpha , \quad t,s \in [0,T], \end{aligned}$$

then for $\lambda >\alpha /2$ and for any t,

$$\begin{aligned} \lim _{s\rightarrow 0} \sup \frac{|\xi ( t\pm s) -\xi (t)|}{s^\lambda } =+\infty . \end{aligned}$$

For the differentiability of sample paths of (1.1), we calculate a lower bound of $\,\Phi (s, t)$ in (2.4).

When $\, \gamma \in (0, 1) \,$, $\, 0< \alpha < 1/2 \,$, it follows from the calculation of Hölder continuity for $\, 0< s < t \,$,

$$\begin{aligned} \Phi (s, t)&\ge c^{2}\int ^{t}_{s} (t-u)^{2\alpha } u^{-\gamma } {\mathrm d} u \\&\ge c^{2 }\int ^{(t+s)/2}_{s} (t-u)^{2\alpha } u^{-\gamma } {\mathrm d} u\\&\ge \frac{c^{2}}{2 \alpha + 1 } \Big ( \frac{\,t+s\,}{\,2\,} \Big )^{-\gamma } \cdot \Big ( \frac{\,t-s\,}{\,2\,}\Big )^{2\alpha + 1} \,. \end{aligned}$$

Thus, if $\, \alpha < 1/2\,$ with $\, 2 \alpha + 1 < 2\,$, by the Theorem of Yeh [25], the sample paths are almost nowhere differentiable in $\, [0, \infty ) \,$.

Similarly, when $\, \alpha \, =\, 1/2 \,$ and $\, \gamma \in (0, 1) \,$, we may compute directly for $\,0< s< t < T \,$

$$\begin{aligned} \begin{aligned} \Phi (s, t)&\ge c^{2} \int ^{t}_{s} (t - u ) u^{-\gamma } {\mathrm d} u \\&\,=\, c^2 \bigg ( \frac{\,t (t^{1-\gamma } - s^{1-\gamma }) \,}{\, 1 - \gamma \,} - \frac{\,t^{2-\gamma } - s^{2-\gamma }\,}{\,2-\gamma \,} \bigg ) \\&\ge \frac{\,c^{2}\,}{\,(1-\gamma ) (2-\gamma ) \,} \cdot (t - s)^{2 - \gamma } \, , \end{aligned} \end{aligned}$$

since for every $\, t < T\,$, the function

$$\begin{aligned} {\mathfrak {h}}(s) \, :=\, \frac{\,t (t^{1-\gamma } - s^{1-\gamma }) \,}{\, 1 - \gamma \,} - \frac{\,t^{2-\gamma } - s^{2-\gamma }\,}{\,2-\gamma \,} - \frac{\,(t - s)^{2 - \gamma } \,}{\,(1-\gamma ) (2-\gamma ) \,} \,, \quad 0 \le s \le t \, \end{aligned}$$

of $\, s\,$ is nonnegative. Indeed, it has the first derivative

$$\begin{aligned} {\mathfrak {h}}^{\prime }(s) \, =\, \frac{\,t-s\,}{\,1 - \gamma \,} \Big ( (t-s)^{-\gamma } - (1 - \gamma ) s^{-\gamma }\Big ) \end{aligned}$$

and $\,{\mathfrak {h}}\,$ has a maximum at $\, s_{0} \, :=\, (1 + ( 1- \gamma )^{-1/\gamma })^{-1}\, t \,$ with minima at $\,{\mathfrak {h}}(0) \, =\, 0 \, =\, {\mathfrak {h}}(t) \,$. Thus, if $\, \alpha = 1/2\,$, $\,\gamma \in (0, 1) \,$ with $\, 2 -\gamma < 2\,$, again by the Theorem of Yeh [25], the sample paths are almost nowhere differentiable in $\, [0, \infty ) \,$. Therefore, we conclude that the GFBM $\,X\,$ is almost nowhere differentiable in the parameter set $\,\alpha \le 1/2 \,$, $\, \gamma \in (0, 1) \,$.

Next we prove part (ii). If $\, \alpha > 1/2\,$, then the process $\, X\,$ is a semimartingale of finite variation with the following representation

$$\begin{aligned} X(t) \, =\, c \int ^{t}_{0} \Bigg ( \int ^{t}_{-\infty } \alpha ( r - s)^{\alpha -1}_{+} |s |^{-\gamma /2} {\mathrm d} B(s) \Bigg ) {\mathrm d} r, \quad t \ge 0 . \end{aligned}$$

This follows from a stochastic version of Fubini theorem (Theorem 4.6 of [6]), because for $0< r < t$, the stochastic integral

$$\begin{aligned} h_{r,t}\, :=\, \int ^{t}_{-\infty }\alpha ( r - s)^{\alpha -1}_{+} |s |^{-\gamma /2}{\mathrm d} B(s) \, =\, \int ^{r}_{-\infty }\alpha ( r - s)^{\alpha -1} |s |^{-\gamma /2}{\mathrm d} B(s) \end{aligned}$$

is well defined. Indeed, we have

$$\begin{aligned}&\int ^{t}_{-\infty } (r-s)_{+}^{2(\alpha - 1) } |s |^{-\gamma } {\mathrm d}s =\, \int ^{r}_{-\infty } (r-s)_{+}^{2(\alpha - 1) } |s |^{-\gamma } {\mathrm d}s \\&\quad =\, \int ^{r}_{0} ( r - s)^{2(\alpha - 1) } s^{-\gamma } {\mathrm d} s + \int ^{0}_{-\infty } ( r - s)^{2(\alpha - 1) } |s |^{-\gamma } {\mathrm d} s \\&\quad =\, r^{2H} \Bigg ( \int ^{1}_{0} (1-u)^{2(\alpha - 1) } u^{-\gamma } {\mathrm d} u + \int ^{\infty }_{0} ( 1 + u)^{2(\alpha -1) } u^{-\gamma } {\mathrm d} u \Bigg ) < \infty \, . \end{aligned}$$

Since $\, (r, t) \mapsto h_{r, t}\,$ does not depend on $\,t\,$, we claim the sample path of $\, X \,$ is differentiable with its derivative $\,{\mathrm d} X(t) / {\mathrm d} t \, =\, c h_{t, t}\,$, $\, t \ge 0 \,$ almost surely. However, $\, t \mapsto h_{t,t}\,$ is not differentiable with probability one, because of a similar reasoning. Indeed, for $s<t$,

$$\begin{aligned} \int ^{t}_{s}(t-u)^{2(\alpha -1)}_{+} u^{-\gamma } {\mathrm d} u \, \ge \, \Big ( \frac{\,t+s\,}{\,2\,} \Big )^{-\gamma } \cdot \Big ( \frac{\,t-s\,}{\,2\,}\Big )^{2(\alpha -1) + 1} \,, \end{aligned}$$

and hence, by applying the result from [25] again, we see $\, 2 (\alpha - 1)+1 < 2\,$ or equivalently, $\, \alpha < 3/2 \,$, the sample path of $\,h\,$ is not differentiable with probability one.

Consequently, the sample paths of $\,X\,$ are continuously differentiable once but not twice almost surely for the fixed parameter $\, \alpha \in (1/2, (1+\gamma )/2)\,$. $\square$

5 Functional Law of Iterated Logarithm

In this section, we establish the functional Law of Iterated Logarithm (FLIL) of the GFBM X in (1.1). We refer to [22, 23] for the FLIL of FBM. We apply Theorem A1 in [22] to prove the FLIL for the process X. We first introduce some notation and terminology.

Let ${{\mathbb {C}}}[0,T]$ be the space of continuous functions. Recall the covariance function $\Psi$ in (2.3), which is shown to be continuous in [19]. Let $\mathcal {H}(\Psi )$ be the reproducing kernel Hilbert space (RKHS) with $\Psi$ as the reproducing kernel. It is defined as the completion of the vector space spanned by the functions $\Psi (s, \cdot )$, $s \in [0,T]$, and endowed with the scalar product

$$\begin{aligned} \Bigg \langle \sum _i c_i \Psi (s_i, \cdot ), \sum _j c'_j \Psi (s'_j,\cdot ) \Bigg \rangle = \sum _i \sum _j c_i c'_j \Psi (s_i, s'_j). \end{aligned}$$

Let $K:= \big \{h\in \mathcal {H}(\Psi ): \langle h, h \rangle ^{1/2} \le 1\big \}$ be the unit ball of $\mathcal {H}(\Psi )$. The FLIL in general states that

• (a) a certain sequence of functions $z_n$ of ${{\mathbb {C}}}[0,T]$ is contained in an $\epsilon$-neighborhood of K when n is large (another way to say this, is that this sequence is relatively compact as $n\rightarrow \infty$, namely that $\{z_n\}$ contains a converging subsequence to an element of K), and

• (b) the functions that are limiting points of the sequence $\{z_n\}$ fill up the set K.

Let $d(\cdot ,\cdot )$ be the sup-norm distance in ${{\mathbb {C}}}[0,T]$, and ${{\mathbb {C}}}\{z_n\}$ represents the cluster set (the set of the limit points) of the sequence $\{z_n\}$.

The same properties in (5.1) and (5.2) below hold for the FBM $B^H$. They are stated in Corollary A1 in [22], applying Theorem A1 with the reproducing kernel $\Gamma _H(s,t) = E\bigl [B^H(s) B^H(t)\bigr ]$ in (2.1), and K equal to the unit ball of $\mathcal {H}(\Gamma _H)$.

Theorem 5.1

Let K be the unit ball of the RKHS $\mathcal {H}(\Psi )$ with the covariance kernel $\Psi$ in (2.3). The GFBM X in (1.1) satisfies

$$\begin{aligned} \lim _{n\rightarrow \infty } d \left( \frac{X(nt)}{ (2n^{2H} \log \log n)^{1/2}}, K \right) = 0, \quad a.s. \end{aligned}$$

(5.1)

and

$$\begin{aligned} {{\mathbb {C}}}\left\{ \frac{X(nt)}{ (2n^{2H} \log \log n)^{1/2}}\right\} = K, \quad a.s. \end{aligned}$$

(5.2)

where $H= \alpha - \gamma /2+1/2$.

Proof

It is clear that $\Psi (t,t)$ is strictly increasing in t. We check the three conditions (C-1), (C-2) and (C-3) in Theorem A1 in [22], that is,

• (C-1)
  $$\begin{aligned} \lim _{r\rightarrow \infty }\sup _{0\le s, t \le T} \left| \frac{E[X(rs)X(rt)] }{r^{2H}L(r)} - \Psi (s,t) \right| = 0. \end{aligned}$$

  (5.3)

• (C-2) There is a nonnegative, strictly increasing and continuous function $\phi$ on ${{\mathbb {R}}}_+$ satisfying $\int _1^\infty \phi (e^{-u^2})du <\infty$ such that

  $$\begin{aligned} E\big [(X(rs) - X(rt))^2\big ] \le \phi ^2(|s-t|) r^{2H} L(r), \quad s, t \ge 0,\, r \ge 0. \end{aligned}$$

  (5.4)
• (C-3)
  $$\begin{aligned} \lim _{n\rightarrow \infty , m/n\rightarrow \infty } E \left[ \frac{X(ms)}{m^HL^{1/2}(m)}\frac{X(ns)}{n^HL^{1/2}(n)} \right] = 0 \end{aligned}$$

  (5.5)

We take $L(\cdot ) \equiv 1$.

For (C-1), for $r>0$ and $s<t$, we have

$$\begin{aligned} E[X(rs)X(rt)] = \Psi (rs, rt)= r ^{2\alpha - \gamma +1} \Psi (s,t), \end{aligned}$$

(5.6)

which immediately implies that

$$\begin{aligned} \sup _{0\le s, t \le T} \left| \frac{E[X(rs)X(rt)] }{r^{2H}} - \Psi (s,t) \right| = 0. \end{aligned}$$

Thus, (5.3) holds.

For (C-2), as shown in Theorem 3.1 for the Hölder continuity property, for $r>0$ and $s<t$, we have

$$\begin{aligned} E\big [(X(rs)-X(rt))^2\big ] \le C_T |s-t|^{2H} r^{2H}, \end{aligned}$$

for some constant $C_T>0$. Let $\phi (t) = C_T t^{2H}$ for $t \in [0,T]$. It is clear that $\phi (\cdot )$ is a nonnegative, strictly increasing and continuous function satisfying $\int _1^\infty \phi (e^{-u^2}) {\mathrm d} u = C_T \int _1^\infty e^{- 2H u^2} <\infty$ since $H \in (0,1)$. This verifies the condition (C-2).

For (C-3), for $s<t$, and for $m>n>0$ satisfying $m/n\rightarrow \infty$ (noting that $mt>ns$), we have

$$\begin{aligned} \Psi (ns, mt)&= c^2 \int _0^{ns} (mt-v)^{\alpha } (ns-v)^{\alpha } v^{-\gamma } \mathrm {d}v \nonumber \\&\qquad + c^2 \int _0^{\infty } ((mt+u)^{\alpha } - u^{\alpha }) ((ns+u)^{\alpha } - u^{\alpha })u^{-\gamma }\mathrm {d}u. \end{aligned}$$

(5.7)

The first integral term is equal to

$$\begin{aligned}&\int _0^s (mt -nu)^\alpha (ns-nu)^\alpha (nu)^{-\gamma } n \mathrm {d}u\\&\quad = \int _0^s m^{\alpha } \left( t- \frac{n}{m} u \right) ^{\alpha } n^{\alpha +1 -\gamma } (s-u)^\alpha u^{-\gamma } \mathrm {d}u. \end{aligned}$$

Dividing by $m^{H}n^{H}$, we obtain

$$\begin{aligned} \left( \frac{n}{m}\right) ^{(1-\gamma )/2} \int _0^s \left( t- \frac{n}{m} u \right) ^{\alpha } (s-u)^\alpha u^{-\gamma } \mathrm {d}u \rightarrow 0 \end{aligned}$$

as $n, m \rightarrow \infty$ and $n/m\rightarrow 0$.

The second integral term in (5.7), we have

$$\begin{aligned}&m^{\alpha } n^{\alpha } \int _0^{\infty } ((t+u/m)^{\alpha } - (u/m)^{\alpha }) ((s+u/n)^{\alpha } - (u/n)^{\alpha })u^{-\gamma }\mathrm {d} u \\&\quad = m^{\alpha } n^{\alpha } \int _0^{\infty } \left[ \left( t+ \frac{n}{m} r\right) ^\alpha - \left( \frac{n}{m}r\right) ^\alpha \right] \left[ (s+r)^{\alpha } - r^{\alpha } \right] n^{-\gamma }r^{-\gamma }n \mathrm {d} r. \end{aligned}$$

Dividing by $m^{H}n^{H}$, we obtain

$$\begin{aligned} \left( \frac{n}{m}\right) ^{(1-\gamma )/2} \int _0^{\infty } \left[ \left( t+ \frac{n}{m} r\right) ^\alpha - \left( \frac{n}{m}r\right) ^\alpha \right] \left[ (s+r)^{\alpha } - r^{\alpha } \right] r^{-\gamma }\mathrm {d} r \rightarrow 0 \end{aligned}$$

as $n, m \rightarrow \infty$ and $n/m\rightarrow 0$.

Thus, for $s<t$,

$$\begin{aligned} \lim _{n\rightarrow \infty , m/n\rightarrow \infty }E\left[ \frac{X(ns)}{n^H} \frac{X(mt)}{m^H} \right]&= \lim _{n\rightarrow \infty , m/n\rightarrow \infty } \frac{1}{n^H m^H} \Psi (ns, mt) =0. \end{aligned}$$

For the case $s>t$, we can switch s and t above in (5.7), and note that we can let $m>>n$ such that $mt>ns$. Then, the same argument will follow. Therefore, we have verified (5.5) in condition (C-3). This completes the proof. $\square$

6 Local Law of the Iterated Logarithms

For FBM $B^H$, the local Law of Iterated Logarithm states that with probability one,

$$\begin{aligned} \limsup _{u\rightarrow 0^+} \frac{|B^H(ut)|}{u^H\sqrt{\log \log u^{-1}}} = c_H \end{aligned}$$

for all $t \in (0,T]$, as in [1]. See the equivalent expression in (3.13). For Gaussian processes, Arcones [1] has established a useful criterion to prove the local law of the iterated logarithm in Theorem 4.1. We apply that to prove the following for the GFBM X in (1.1).

Theorem 6.1

For the GFBM X in (1.1), if $\,\alpha > 0 \,$, with probability one,

$$\begin{aligned} \limsup _{u\rightarrow 0^+} \frac{|X(u t)|}{u^H\sqrt{\log \log u^{-1}}} \end{aligned}$$

exists for all $t \in (0,T]$.

Proof

We check the nine conditions in [1, Theorem 4.1]. Here, we consider the interval [0, T] and use natural pseudometric $\rho (s,t) = \sqrt{E[(X(s) - X(t))^2]} = \sqrt{\Phi (s,t)}$. Let $\tau (u) =u$ and $w(u) = u^{H}$. Condition (i) is evident and condition (ix) is clear since $\Phi (s,t)$ is continuous. For (v), $([0,T], \rho )$ is totally bounded, since $\rho (0,T) < \infty$. It is clear that the conditions in (vii) and (viii) holds since these functions are continuous. Condition (ix) holds since $\Phi (s,t)$ is continuous.

For (ii), by the scaling identity of the covariance in (5.6), we obtain

$$\begin{aligned} E \left[ \frac{X(\tau (u) s) X(\tau (u) t)}{w(u)^2} \right] = E \left[ \frac{X(u s) X(u t)}{u^{2H}} \right] = \Psi (s,t). \end{aligned}$$

For (iii), we shall show that for each $m\ge 1$, each $\epsilon >0$, each $t_1,\dots ,t_m\in (0,T]$ and each $\lambda _1, \dots , \lambda _m \in {{\mathbb {R}}}$,

$$\begin{aligned} \lim _{r\rightarrow 1-} \limsup _{u \rightarrow 0+} \sup _{v: u e^{-(\log u^{-1})^r} \le v \le u e^{-(\log u^{-1})^\epsilon }} \sum _{j,k=1}^m \lambda _j \lambda _k E \left[ \frac{X(ut_j) X(v t_k)}{w(u)w(v)} \right] \le 0. \end{aligned}$$

(6.1)

For $r > 0$, let $\varphi (u) := u e^{-(\log u^{-1})^r}<u$. Note that $\varphi (u)/u=e^{-(\log u^{-1})^r} \rightarrow 0$ as $u \rightarrow 0+$ and for $u > 0$, $\varphi (u)/u=e^{-(\log u^{-1})^r} \rightarrow u$ as $r \rightarrow 1-$. Consider for $t>s$,

$$\begin{aligned}&E \left[ \frac{X(ut) X(\varphi (u)s)}{u^H \varphi (u)^H} \right] = \frac{1}{u^H \varphi (u)^H} \Psi (ut, \varphi (u) s), \end{aligned}$$

where $\Psi (\cdot , \cdot )$ has two terms as in (2.3). By the change of variables from v to $\theta \varphi (u) s$, we have

$$\begin{aligned}&\frac{1}{u^H \varphi (u)^H}\int _0^{\varphi (u)s} (ut-v)^{\alpha } (\varphi (u)s-v)^{\alpha } v^{-\gamma } {\mathrm d} v \nonumber \\&\quad \le \frac{1}{u^H \varphi (u)^H}\int _0^{\varphi (u)s} (ut)^{\alpha } (\varphi (u)s-v)^{\alpha } v^{-\gamma } {\mathrm d}v \nonumber \\&\quad = \frac{(ut)^\alpha (\varphi (u)s)^{\alpha -\gamma +1}}{u^H \varphi (u)^H} \int _0^1 (1-\theta )^{\alpha }\theta ^{-\gamma } {\mathrm d} \theta \nonumber \\&\quad = (ts)^{H}\left( \frac{\,s\,}{\,t\,} \cdot \frac{\varphi (u)}{u}\right) ^{(1-\gamma )/2} \cdot \nonumber \\&\qquad \text {Beta} (\alpha + 1 , 1 - \gamma ) \rightarrow 0 \quad \text{ as }\quad u \rightarrow 0+. \end{aligned}$$

(6.2)

This corresponds to the first term of $\Psi (\cdot , \cdot )$ in (2.3). Similarly, by $\alpha$-Hölder continuity of function $\,x \rightarrow x^{\alpha }\,$, $\,\alpha > 0 \,$, we have $(u t+v)^{\alpha } - v^{\alpha } \le (ut)^{\alpha }$ and hence, again by the change of variables, we have

$$\begin{aligned}&\frac{1}{u^H \varphi (u)^H} \int _0^{\infty } ((u t+v)^{\alpha } - v^{\alpha }) ((\varphi (u) s+v)^{\alpha } - v^{\alpha })v^{-\gamma } {\mathrm d} v \nonumber \\&\quad \le \frac{1}{u^H \varphi (u)^H} \int ^{\infty }_{0} (ut)^{\alpha } (\varphi (u) s)^{\alpha - \gamma + 1} ( ( 1 + \theta )^{\alpha } - \theta ^{\alpha } ) \theta ^{-\gamma }{\mathrm d} \theta \nonumber \\&\quad = (ts)^{H}\left( \frac{\,s\,}{\,t\,} \cdot \frac{\varphi (u)}{u}\right) ^{(1-\gamma )/2} \cdot \nonumber \\&\qquad \int ^{\infty }_{0} ( ( 1 + \theta )^{\alpha } - \theta ^{\alpha } ) \theta ^{-\gamma }{\mathrm d} \theta \rightarrow 0 \quad \text{ as }\quad u \rightarrow 0+. \end{aligned}$$

(6.3)

where the integral is finite as in (2.11). This corresponds to the second term of $\Psi (\cdot , \cdot )$ in (2.3). Thus, we have shown that condition (iii), i.e., (6.1) holds with equality to zero.

For (iv), we show that in probability,

$$\begin{aligned} \sup _{t \in (0,T]} \frac{|X(ut)|}{u^H (2 \log \log u^{-1})^{1/2}} \rightarrow 0 \quad \text {as}\quad u \rightarrow 0+. \end{aligned}$$

By the Fernique inequality (see, e.g., [9]), we obtain for each u, there exists $q>0$ such that

$$\begin{aligned}&P \left( \sup _{t \in (0,T]} \frac{|X(ut)|}{u^H (2 \log \log u^{-1})^{1/2}}> \epsilon \right) \\&\le C P\left( Z>\frac{\epsilon }{\sup _{t \in (0,T]} (u^H (2 \log \log u^{-1})^{1/2})^{-1}\Psi (ut,ut)^{1/2} + q} \right) \\&= C P\left( Z>\frac{\epsilon }{ (2 \log \log u^{-1})^{1/2})^{-1}\sup _{t \in (0,T]} ( \Psi (t,t)^{1/2} + q} \right) \\&\rightarrow 0 \quad \text{ as }\quad u \rightarrow 0+, \end{aligned}$$

for all $\epsilon \ge \epsilon _0$ for some $\epsilon _0>0$ and some constant $C>0$, and $Z\sim N(0,1)$. This proves that condition (iv) holds.

For (vi), we show that for each $\eta >0$, there exists a $\delta >0$ such that

$$\begin{aligned} \limsup _{\theta \rightarrow 1-} \sum _{n=1}^\infty \exp \left( \frac{-\eta (\theta ^n)^{2H} \log n}{ \sup _{s,t \in [0,T], \Phi (s,t)\le \delta ^2} \Phi (\theta ^n s,\theta ^n t) } \right) <\infty . \end{aligned}$$

Observe that, similar to (5.6), for $r>0$,

$$\begin{aligned} \Phi (rs,rt)= r^{2H} \Phi (s,t). \end{aligned}$$

Thus, it becomes to show that for each $\eta >0$, there exists a $\delta >0$ such that

$$\begin{aligned} \sum _{n=1}^\infty \exp \left( \frac{-\eta \log n}{ \delta ^2 } \right) = \sum _{n=1}^\infty n^{-\eta /\delta ^2} <\infty , \end{aligned}$$

which holds for any $\delta < \sqrt{\eta }$. This completes the proof. $\square$

6.1 Compositions

We consider the composition $X(|X(\cdot )|)$ of $X(\cdot )$ itself. In Example 4.1 of [1], by applying its Corollary 4.2, it is shown that for FBM $B^H$ with $1/2\le H <1$, given $b>0$, with probability one,

$$\begin{aligned} \left\{ \frac{B^H(|B^H(ub)|)}{ u^{H^2} (2 \log \log u^{-1})^{(H+1)/2}} \right\} \end{aligned}$$

is relatively compact as $u\rightarrow 0+$ and its limit set is $[-\sigma , \sigma ]$ where $\sigma = b^{H^2} H^{H/2} (H+1)^{-(H+1)/2}$.

We apply [1, Corollary 4.2] to the GFBM X in (1.1). We remark that the following result requires $\alpha >0$, since we need to use the $\alpha$-Hölder continuity of the function $t\rightarrow t^\alpha$ in the proof. Note that when $\gamma =0$, the condition $\alpha >0$ is equivalent to $H>1/2$ in the case of FBM $B^H$. However, for the GFBM X, as shown in Fig. 1, in the region of $\alpha >0$ and $\gamma \in (0,1)$, the range of the Hurst parameter H can cover the entire interval $H \in (0,1)$ (observing that when the $\alpha \approx 0$ and $\gamma \approx 1$, H can be very close to 0).

Theorem 6.2

If $\,\alpha > 0 \,$, with probability one, the set

$$\begin{aligned} \left\{ \frac{X(|X(ub)|)}{ u^{H^2} (2 \log \log u^{-1})^{(H+1)/2}} , \, \, u > 0 \right\} \end{aligned}$$

is relatively compact, as $u\rightarrow 0+$, and its limit set is $[-\sigma , \sigma ]$ where

$$\begin{aligned} \sigma = \sup _{0 \le r \le \sqrt{\Psi (b,b)}} \sqrt{\Psi (r,r)} (1-r^2/ \Psi (b,b))^{1/2}, \quad a.s. \end{aligned}$$

Proof

We verify the four conditions (i)–(iv) in [1, Corollary 4.2]. Condition (i) requires $E[X(ut) X(us)] = u^{2H} E[X(t)X(s)]$, which holds by (5.6). Condition (ii) requires that $\sup _{0 \le t \le T}|X(t)|<\infty$ a.s. Condition (iii) requires that

$$\begin{aligned} \lim _{u\rightarrow 1-} E[X(ut) X(t) ] = E\big [X(t)^2\big ] \quad \text {for each} \quad t\ge 0. \end{aligned}$$

This follows from the continuity of $\Psi (s,t)$ in (2.3).

For condition (iv), we show for each $s,t \ge 0$,

$$\begin{aligned} \lim _{u \rightarrow 0+} u^{-H} E[X(s) X(ut)] =0. \end{aligned}$$

By (2.3), for u small enough such that $s>ut$, we have

$$\begin{aligned} u^{-H} E[X(s) X(ut)]&= c^2 u^{-H} \int _0^{ut} (s-v)^{\alpha } (ut-v)^{\alpha } v^{-\gamma } \mathrm{d}v \\&\qquad + c^2 u^{-H} \int _0^{\infty } ((s+v)^{\alpha } - v^{\alpha }) ((ut+v)^{\alpha } \\&\qquad - v^{\alpha })v^{-\gamma }\mathrm{d}v. \end{aligned}$$

For the first term, by change of variables, it is equal to

$$\begin{aligned}&c^2 u^{H} \int _0^{t} (s/u-v)^{\alpha } (t-v)^{\alpha } v^{-\gamma }\mathrm{d}v\\&\quad \le c^2 u^{H} \int _0^{t} (s/u)^{\alpha } (t-v)^{\alpha } v^{-\gamma } \mathrm{d}v \\&\quad = c^2 u^{H-\alpha } \int _0^{t} s^{\alpha } (t-v)^{\alpha } v^{-\gamma } \mathrm{d}v \rightarrow 0 \quad \text{ as }\quad u \rightarrow 0+ \end{aligned}$$

since $H- \alpha = 1/2- \gamma /2>0$. For the second term,

$$\begin{aligned}&u^{-H} \int _0^{\infty } ((s+v)^{\alpha } - v^{\alpha }) ((ut+v)^{\alpha } - v^{\alpha })v^{-\gamma }\mathrm{d}v \\&\quad \le u^{-H} \int _0^\infty s^\alpha ((ut+v)^{\alpha } - v^{\alpha })v^{-\gamma }\mathrm{d}v \\&\quad = u^{-H} \int _0^\infty s^\alpha (ut)^{\alpha -\gamma +1} ((1+\theta )^\alpha - \theta ^\alpha ) \theta ^{-\gamma } \mathrm{d}\theta \\&\quad = u^{(1-\gamma )/2} s^\alpha t^{\alpha -\gamma +1} \int _0^\infty ((1+\theta )^\alpha - \theta ^\alpha ) \theta ^{-\gamma } \mathrm{d}\theta \\&\quad \rightarrow 0 \quad \text{ as }\quad u \rightarrow 0+, \end{aligned}$$

where the integral is finite as in (2.11). Thus, we have verified condition (iv). This completes the proof. $\square$