Summability Methods for Random Variables

Abstract

Let (X _k ) be a sequence of independent, identically distributed (i.i.d.) random variables with E | X _k | < ∞ and EX _k = μ, k = 1, 2, …. Let A = (a _nk ) be a Toeplitz matrix, i.e., the conditions (1.3.1)–(1.3.3) of Theorem 1.3.3 are satisfied by the matrix A = (a _nk ). Since

$$\displaystyle\begin{array}{rcl} E\sum _{k=1}^{\infty }\vert a_{ nk}X_{k}\vert = E\vert X_{k}\vert \sum _{k=1}^{\infty }\vert a_{ nk}\vert \leq ME\vert X_{k}\vert,& & {}\\ \end{array}$$

the series $\sum _{k=0}^{\infty }a_{nk}X_{k}$ converges absolutely with probability one.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

5.1 Introduction

Let (X _k ) be a sequence of independent, identically distributed (i.i.d.) random variables with E | X _k  |  < ∞ and EX _k  = μ, k = 1, 2, …. Let A = (a _nk ) be a Toeplitz matrix, i.e., the conditions (1.3.1)–(1.3.3) of Theorem 1.3.3 are satisfied by the matrix A = (a _nk ). Since

$$\displaystyle\begin{array}{rcl} E\sum _{k=1}^{\infty }\vert a_{ nk}X_{k}\vert = E\vert X_{k}\vert \sum _{k=1}^{\infty }\vert a_{ nk}\vert \leq ME\vert X_{k}\vert,& & {}\\ \end{array}$$

the series $\sum _{k=0}^{\infty }a_{nk}X_{k}$ converges absolutely with probability one.

There is a vast literature on the application of summability to Probability Theory. Here, we study only few applications of summability methods in summing sequences of random variables and strong law of large numbers (c.f. [86]).

5.2 Definitions and Notations

In this section, we give some required definitions.

Definition 5.2.1 (Random variables).

A function X whose range is a set of real numbers, whose domain is the sample space (set of all possible outcomes) S of an experiment, and for which the set of all s in S, for which X(s) ≤ x is an event if x is any real number. It is understood that a probability function is given that specifies the probability X has certain values (or values in certain sets). In fact, one might define a random variable to be simply a probability function P on suitable subsets of a set T, the point of T being “elementary events” and each set in the domain of P an event.

Definition 5.2.2 (Independent random variables).

Random variables X and Y such that whenever A and B are events associated with X and Y, respectively, the probability P(A and B) of both is equal to P(A) × P(B).

Definition 5.2.3 (Distribution).

A random variable together with its probability density function, probability function, or distribution function is known as distribution .

Definition 5.2.4 (Distribution function).

A real-valued function G(x) on $R = [-\infty,\infty ]$ is called distribution function (abbreviated d.f.) if G has the following properties:

• (a) G is nondecreasing;

• (b) G is left continuous, i.e., lim _{y → x, y < x} G(y) = G(x),   all  x ∈ R; 

• (c) $G(-\infty ) =\lim _{x\rightarrow -\infty }G(x) = 0,$ $G(\infty ) =\lim _{x\rightarrow \infty }G(x) = 1$.

Definition 5.2.5 (Independent, identically distributed random variable).

A sequence $(X_{n})_{n\geq 1}$ (or the random variables comprising this sequence) is called independent, identically distributed (abbreviated i.i.d.) if X _n , n ≥ 1, are independent and their distribution functions are identical.

Definition 5.2.6 (σ-field).

A class of sets F satisfying the following conditions is called a σ-field :

• (a) if E _i  ∈ F (i = 1, 2, 3, …), then $\cup _{i=1}^{n}E_{i} \in F$;

• (b) if E ∈ F, then E ^c ∈ F.

Definition 5.2.7 (Probability Space).

Let F be a σ-field of subsets of $\Omega$, i.e., nonempty class of subsets of $\Omega$ which contains $\Omega$ and is closed under countable union and complementation. Let $P$ be a measure defined on $F$ satisfying $P(\Omega ) = 1$. Then the triple $(\Omega,F,P)$ is called probability space.

Definition 5.2.8 (Expectation).

Let f be the relative frequency function (probability density function) of the variable x. Then

$$\displaystyle\begin{array}{rcl} E(x) =\int _{ a}^{b}xf(x)dx& & {}\\ \end{array}$$

is the expectation of variable x over the range a to b, or more usually, −∞ to ∞.

Definition 5.2.9 (Almost Everywhere).

A property of points x is said to hold almost everywhere , a.e., or for almost all points, if it holds for all points except those of a set of measure zero.

The concept of almost sure (a.s.) convergence in probability theory is identical with the concept of almost everywhere (a.e.) convergence in measure theory.

Definition 5.2.10 (Almost Sure).

The sequence of random variables (X _n ) is said to converge almost sure , in short a.s. to the random variable X if and only if there exists a set E ∈ F with P(E) = 0, such that, for every w ∈ E ^c, | X _n (w) − X(w) | → 0, as n → ∞. In this case, we write $X_{n}\stackrel{a.s.}{\rightarrow }X$.

Definition 5.2.11 (Median).

For any random variable X a real number m(X) is called a median of X if P{X ≤ m(X)} ≥ (1∕2) ≤ P{X ≥ m(X)}.

Definition 5.2.12 (Levy’s inequalities).

If {X _j ;  1 ≤ j ≤ n} are independent random variables and if $S_{j} =\sum _{ i=1}^{j}X_{i}$, and m(Y ) denotes a median of Y, then, for any ε > 0,

• (i) $P\{\max _{1\leq j\leq n}[S_{j} - m(S_{j} - S_{n})] \geq \epsilon \}\leq 2P\{\vert S_{n}\vert \geq \epsilon \}$;

• (ii) $P\{\max _{1\leq j\leq n}\vert S_{j} - m(S_{j} - S_{n})\vert \geq \epsilon \}\leq 2P\{S_{n} \geq \epsilon \}$.

Definition 5.2.13 (Chebyshev’s inequality).

In probability theory, Chebyshev’s inequality (also spelled as Tchebysheff’s inequality) guarantees that in any probability distribution, “nearly all” values are close to the mean—the precise statement being that no more than 1∕k ² of the distribution’s values can be more than k standard deviations away from the mean.

Let X be a random variable with finite expected value μ and finite nonzero variance σ ². Then for any real number k > 0,

$$\displaystyle\begin{array}{rcl} P\{\vert X -\mu \vert \geq k\sigma \} \leq \frac{1} {{k}^{2}}.& & {}\\ \end{array}$$

Definition 5.2.14 (Markov’s inequality).

In probability theory, Markov’s inequality gives an upper bound for the probability that a nonnegative function of a random variable is greater than or equal to some positive constant. It is named after the Russian mathematician Andrey Markov.

If X is any nonnegative random variable and any a in (0, ∞), then

$$\displaystyle\begin{array}{rcl} P\{X \geq a\} \leq \frac{1} {a}EX.& & {}\\ \end{array}$$

Definition 5.2.15 (Infinitely often (I.O.)).

Let $(A_{n})_{n\geq 1}$ be a sequence of events. Then $\lim _{n\rightarrow \infty }A_{n} =\{ w: w \in A_{n}\ \ \mbox{ for infinitely many }\ \ n\},$ or $\lim _{n\rightarrow \infty }A_{n} =\{ w: w \in A_{n},\mbox{ I.O.}\}.$ Moreover, $\lim _{n\rightarrow \infty }A_{n} = \cap _{n=1}^{\infty }\cap _{k=n}^{\infty }A_{k}.$

Lemma 5.2.16 (Borel-Cantelli Lemma).

If $(A_{n})_{n\geq 1}$ is a sequence of events for which $\sum _{n=1}^{\infty }P\{A_{n}\} < \infty$ , then P{A _n , I.O.} = 0.

5.3 A-Summability of a Sequence of Random Variables

Let F be the common distribution function of X _k s and X, a random variable having this distribution. It is also convenient to adopt the convention that $a_{nk} = 0,\vert a_{nk}{\vert }^{-1} = +\infty$. In the next theorem, we study the convergence properties of the sequence

$$\displaystyle\begin{array}{rcl} Y _{n} =\sum _{ k=0}^{\infty }a_{ nk}X_{k},\ \ \mbox{ as}\ \ n \rightarrow \infty.& & {}\\ \end{array}$$

Theorem 5.3.1.

A necessary and sufficient condition that Y _n →μ in probability is that $\max _{k\in \mathbb{N}}\vert a_{nk}\vert \rightarrow 0,\mbox{ as}\ \ n \rightarrow \infty.$

Proof.

The proof of the sufficiency is very similar to the corresponding argument in [48], but it will be given here for the sake of completeness. First, we have that

$$\displaystyle\begin{array}{rcl} \lim _{T\rightarrow \infty }TP[\vert X\vert \geq T] = 0& &{}\end{array}$$

(5.3.1)

since E | X |  < ∞. Let X _nk be a _nk X _k truncated at one and $Z_{n} =\sum _{ k=0}^{\infty }X_{nk}$. Now for all n sufficiently large, since $\max _{k\in \mathbb{N}}\vert a_{nk}\vert \rightarrow 0$, it follows from (5.3.1) that

$$\displaystyle\begin{array}{rcl} P[Z_{n}\neq Y _{n}] \leq \sum _{k=0}^{\infty }P[X_{ nk}\neq a_{nk}X_{k}] =\sum _{ k=0}^{\infty }P[\vert X\vert \geq \frac{1} {\vert a_{nk}\vert }] \leq \epsilon \sum _{k=0}^{\infty }\vert a_{ nk}\vert \leq \epsilon M.& & {}\\ \end{array}$$

It will therefore suffice to show that Z _n  → μ in probability. Note that

$$\displaystyle\begin{array}{rcl} \lim _{n\rightarrow \infty }[EZ_{n}-\mu ] =\lim _{n\rightarrow \infty }\left [\sum _{k=0}^{\infty }a_{ nk}\left (\int _{\vert x\vert <\vert a_{nk}{\vert }^{-1}}xdF-\mu \right )+\,\mu \left (\sum _{k=0}^{\infty }a_{ nk}-1\right )\right ]=\,0.& & {}\\ \end{array}$$

Since

$$\displaystyle\begin{array}{rcl} \frac{1} {T}\int _{\vert x\vert <T}{x}^{2}dF = \frac{1} {T}\left \{-{T}^{2}P[\vert x\vert \geq T] + 2\int _{ 0}^{T}xP[\vert x\vert \geq x]dx\right \} \rightarrow 0,& & {}\\ \end{array}$$

it follows that for all n sufficiently large

$$\displaystyle\begin{array}{rcl} \sum _{k=0}^{\infty }\mbox{ Var}\ X_{ nk} \leq \sum \vert a_{nk}{\vert }^{2}\int _{ \vert x\vert <\vert a_{nk}{\vert }^{-1}}{x}^{2}dF \leq \epsilon \sum _{ k=0}^{\infty }\vert a_{ nk}\vert \leq \epsilon M.& &{}\end{array}$$

(5.3.2)

But $E{(\sum _{k=0}^{\infty }\vert X_{nk}\vert )}^{2}$ is easily seen to be finite so that $\mbox{ Var}\ Z_{n} =\sum _{ k=0}^{\infty }\ \mbox{ Var}\ X_{nk}$ which tends to zero by (5.3.2). An application of Chebyshev’s inequality completes the proof of sufficiency. For necessity, let $U_{k} = X_{k}-\mu$, $T_{n} =\sum _{ k=0}^{\infty }a_{nk}U_{k}$ so that T _n  → 0 in probability and hence in law. Let $g(u) = E{e}^{iuU_{k}}$ be the characteristic function of U _k . We have that $\prod _{k=1}^{\infty }g(a_{nk}u) \rightarrow 1\ \mbox{ as}\ n \rightarrow \infty.$ But

$$\displaystyle\begin{array}{rcl} \left \vert \prod _{k=1}^{\infty }g(a_{ nk}u)\right \vert \leq \vert g(a_{nm}u)\vert \leq 1& & {}\\ \end{array}$$

for any m, so that for any sequence k _n ,

$$\displaystyle\begin{array}{rcl} \lim _{n\rightarrow \infty }\vert g(a_{n,k_{n}}u)\vert = 1.& &{}\end{array}$$

(5.3.3)

Since U _k is nondegenerate, there is a u ₀ such that | g(u) |  < 1 for 0 <  | u |  < u ₀ [57, p. 202]. Letting $u = u_{0}/2M$, it follows that $\vert a_{n,k_{n}}u\vert \leq Mu = u_{0}/2$ and then $a_{n,k_{n}}u \rightarrow 0$, as $n \rightarrow \infty$, by (5.3.3). Choosing $k_{n}$ to satisfy $\vert a_{n,k_{n}}\vert =\max _{k\in \mathbb{N}}\vert a_{nk}\vert$.

This completes the proof of Theorem 5.3.1. □ 

In Theorem 5.3.1 excluding the trivial case when X _k is almost surely equal to μ, it has been shown that Y _n  → μ in probability if and only if $\max _{k\in \mathbb{N}}\vert a_{nk}\vert \rightarrow 0.$ This condition is not enough, however, to guarantee almost sure (a.s.) convergence. To obtain this the main result is proved in the following theorem [56].

Theorem 5.3.2.

If $\max _{k\in \mathbb{N}}\vert a_{nk}\vert = O({n}^{-\gamma }),\ \gamma > 0,$ then $E\vert X_{k}{\vert }^{1+\frac{1} {\gamma } } < \infty$ implies that $Y _{n} \rightarrow \mu \text{ a.s.}$

For the proof of Theorem 5.3.2, we need the following lemmas.

Lemma 5.3.3 ([81, Lemma 1]).

If $E\vert X{\vert }^{1+\frac{1} {\gamma } } < \infty$ and $\max _{k\in \mathbb{N}}\vert a_{nk}\vert \leq B{n}^{-\gamma }$ , then for every ε > 0,

$$\displaystyle\begin{array}{rcl} \sum _{n=0}^{\infty }P[\vert a_{ nk}X_{k}\vert \geq \epsilon,\ \mbox{ for some k}] < \infty & & {}\\ \end{array}$$

Proof.

It suffices to consider B = 1 and ε = 1 for both the matrix A and the random variables X _k may be multiplied by a positive constant if necessary. (Assumption (1.3.2) is not used in this proof). Let

$$\displaystyle\begin{array}{rcl} N_{n}(x) =\sum _{[k:\vert a_{nk}{\vert }^{-1}\leq x]}\vert a_{nk}\vert.& & {}\\ \end{array}$$

Notice that N _n (x) = 0, for x < n ^γ, and $\int _{0}^{\infty }dN_{n}(x) =\sum _{ k=0}^{\infty }\vert a_{nk}\vert \leq M.$ If G(x) = P[ | x | ≥ x], limTG(t) = 0, as T → ∞ since E | X |  < ∞, and thus

$$\displaystyle\begin{array}{rcl} \sum _{k=0}^{\infty }P[\vert a_{ nk}X_{k}\vert \geq 1]& =& \sum _{k=0}^{\infty }G(\vert a_{ nk}{\vert }^{-1}) \\ & =& \int _{0}^{\infty }XG(x)dN_{ n}(x) \\ & =& \lim _{T\rightarrow \infty }TG(T)N_{n}(T) -\int _{0}^{\infty }N_{ n}(\bar{x})d[xG(x)] \\ & \leq & M\int _{{n}^{\gamma }}^{\infty }d\vert XG(x)\vert. {}\end{array}$$

(5.3.4)

To estimate the last integral, observe that, for z < y, 

$$\displaystyle\begin{array}{rcl} yG(y) - zG(z) = (y - z)G(z) + y[G(y) - G(z)],& & {}\\ \end{array}$$

so that

$$\displaystyle\begin{array}{rcl} \int _{{n}^{\gamma }}^{\infty }d\vert xG(x)\vert & =& \sum _{ j=n}^{\infty }\int _{{ j}^{\gamma }}^{{(j+1)}^{\gamma }}d\vert xG(x)\vert {}\\ &\leq & \sum _{j=n}^{\infty }[{(j + 1)}^{\gamma }-{j}^{\gamma }]G({j}^{\gamma })\,+\sum _{ j=n}^{\infty }{(j + 1)}^{\gamma }[G({j}^{\gamma })-\,G({(j\,+\,1)}^{\gamma })]. {}\\ \end{array}$$

Summing the first of the final series by parts and using the existence of E | X | , we see that it is dominated by the second series, and thus

$$\displaystyle\begin{array}{rcl} \int _{{n}^{\gamma }}^{\infty }d\vert xG(x)\vert \leq 2\sum _{ j=n}^{\infty }{(j + 1)}^{\gamma }[G({j}^{\gamma }) - G({(j + 1)}^{\gamma })].& &{}\end{array}$$

(5.3.5)

Finally, by (5.3.4) and (5.3.5),

$$\displaystyle\begin{array}{rcl} \sum _{n=1}^{\infty }P[\vert a_{ nk}X_{k}\vert \geq 1\ \mbox{ for }k]& \leq & \sum _{n=1}^{\infty }\sum _{ k=1}^{\infty }P[\vert a_{ nk}X_{k}\vert \geq 1] {}\\ & \leq & 2M\sum _{n=1}^{\infty }\sum _{ j=n}^{\infty }{(j + 1)}^{\gamma }[G({j}^{\gamma }) - G({(j + 1)}^{\gamma })] {}\\ & =& 2M\sum _{j=1}^{\infty }j{(j + 1)}^{\gamma }[G({j}^{\gamma }) - G({(j + 1)}^{\gamma })] {}\\ & \leq & {2}^{\gamma +1}M\int \vert x{\vert }^{1+\frac{1} {\gamma } }dF(x) < \infty. {}\\ \end{array}$$

This completes the proof of Lemma 5.3.3. □ 

Lemma 5.3.4 ([81, Lemma 2]).

If $E\vert X{\vert }^{1+\frac{1} {\gamma } } < \infty$ and $\max _{k\in \mathbb{N}}\vert a_{nk}\vert \leq B{n}^{-\gamma }$ , then, for $\alpha <\gamma /2(\gamma +1)$ ,

$$\displaystyle\begin{array}{rcl} \sum _{n=0}^{\infty }P[\vert a_{ nk}X_{k}\vert \geq {n}^{-\alpha },\ \text{for at least two values of k}] < \infty.& & {}\\ \end{array}$$

Proof.

By the Markov’s inequality,

$$\displaystyle\begin{array}{rcl} \sum _{n=0}^{\infty }P[\vert a_{ nk}X_{k}\vert \geq {n}^{-\alpha }] \leq \vert a_{ nk}{\vert }^{1+\frac{1} {\gamma } }E\vert X{\vert }^{1+\frac{1} {\gamma } }{n}^{\alpha (1+\frac{1} {\gamma } )},& & {}\\ \end{array}$$

so that

$$\displaystyle\begin{array}{rcl} P[\vert a_{nk}X_{k}\vert & \geq & {n}^{-\alpha }\ \mbox{ for at least two $k$}] {}\\ & \leq & \sum _{j\neq k}P[\vert a_{nj}X_{j}\vert \geq {n}^{-\alpha },\ \vert a_{ nk}X_{k}\vert \geq {n}^{-\alpha }] {}\\ & \leq & {(E\vert X{\vert }^{1+\frac{1} {\gamma } })}^{2}{n}^{2\alpha (1+\frac{1} {\gamma } )}\sum _{j\neq k}\vert a_{nj}{\vert }^{1+\frac{1} {\gamma } }\vert a_{nk}{\vert }^{1+\frac{1} {\gamma } } {}\\ & \leq & {(E\vert X{\vert }^{1+\frac{1} {\gamma } })}^{2}{B}^{2/\gamma }{M}^{2}{n}^{2[-1+\alpha (1+\frac{1} {\gamma } )]}, {}\\ \end{array}$$

and the final estimate will converge when summed on n provided that α < γ∕ 2(γ + 1). 

This completes the proof of Lemma 5.3.4. □ 

Lemma 5.3.5 ([81, Lemma 3]).

If $\mu = 0,\ E\vert X{\vert }^{1+\frac{1} {\gamma } } < \infty$ , and $\max _{k\in \mathbb{N}}\vert a_{nk}\vert \leq B{n}^{-\gamma }$ , then for every ε > 0,

$$\displaystyle\begin{array}{rcl} \sum _{n=0}^{\infty }P\left [\left \vert {\sum _{ k}}^{{\prime}}a_{ nk}X_{k}\right \vert \geq \epsilon \right ] < \infty,& & {}\\ \end{array}$$

where

$$\displaystyle\begin{array}{rcl}{ \sum _{k}}^{{\prime}}a_{ nk}X_{k} =\sum _{[k:\vert a_{nk}X_{k}\vert <{n}^{-\alpha }]}a_{nk}X_{k},& & {}\\ \end{array}$$

and 0 < α < γ.

Proof.

Let $X_{nk} = \left \{\begin{array}{ccl} X_{k}&,&\vert a_{nk}X_{k}\vert < {n}^{-\alpha }, \\ 0 &,&\text{otherwise} \end{array} \right.$ and $\beta _{nk} = EX_{nk}$. If a _nk  = 0 then $\beta _{nk} =\mu = 0$, while if a _nk ≠ 0, then

$$\displaystyle\begin{array}{rcl} \vert \beta _{nk}\vert = \left \vert \mu -\int _{\vert x\vert \geq {n}^{-\alpha }\vert a_{nk}{\vert }^{-1}}xdF\right \vert \leq \int _{\vert x\vert \geq {n}^{-\alpha }{B}^{-1}{n}^{\gamma }}\vert x\vert dF.& & {}\\ \end{array}$$

Therefore β _nk  → 0, uniformly in k and $\sum _{k=0}^{\infty }a_{nk}\beta _{nk} \rightarrow 0.$

Let $Z_{nk} = X_{nk} -\beta _{nk}$, so that E | Z _nk  |  = 0; $E\vert Z_{nk}{\vert }^{1+\frac{1} {\gamma } } \leq C$, for some C, and $\vert a_{nk}Z_{nk}\vert \leq 2{n}^{-\alpha }$. Now

$$\displaystyle\begin{array}{rcl}{ \sum _{k}}^{{\prime}}a_{ nk}X_{k} =\sum _{ k=0}^{\infty }a_{ nk}X_{nk} =\sum _{ k=0}^{\infty }a_{ nk}Z_{nk} +\sum _{k}a_{nk}\beta _{nk}& & {}\\ \end{array}$$

and so for n sufficiently large,

$$\displaystyle\begin{array}{rcl} \left (\left \vert {\sum _{k}}^{{\prime}}a_{ nk}X_{k}\right \vert \geq \epsilon \right ) \subseteq \left (\left \vert \sum _{k=0}^{\infty }a_{ nk}Z_{nk}\right \vert \geq \frac{\epsilon } {2}\right ).& & {}\\ \end{array}$$

It will suffice, therefore, to show that

$$\displaystyle\begin{array}{rcl} \sum _{n=0}^{\infty }P\left (\left \vert \sum _{ k=0}^{\infty }a_{ nk}Z_{nk}\right \vert \geq \epsilon \right ) < \infty.& &{}\end{array}$$

(5.3.6)

Let ν be the least integer greater than 1∕γ. The necessary estimate will be obtained by computing $E{(\sum _{k=0}^{\infty }\vert a_{nk}Z_{nk}\vert )}^{2\nu }$ which is finite so that

$$\displaystyle\begin{array}{rcl} E{\left (\sum _{k=0}^{\infty }\vert a_{ nk}Z_{nk}\vert \right )}^{2\nu } =\sum _{ k_{1}\cdots k_{2\nu }}E\prod _{j=1}^{2\nu }a_{ n,k_{j}}Z_{n,k_{j}}.& & {}\\ \end{array}$$

There is no contribution to the sum on the right so long as there is a j with $k_{j}\neq k_{i}$, for all i ≠ j, since the Z _nk are independent and EZ _nk  = 0. The general term to be considered then will have

$$\displaystyle\begin{array}{rcl} q_{1}\ \mbox{ of the}\ {k}^{{\prime}}s =\xi _{ 1},\ldots,q_{m}\ \mbox{ of the}\ {k}^{{\prime}}s =\xi _{ m},& & {}\\ r_{1}\ \mbox{ of the}\ {k}^{{\prime}}s =\eta _{ 1},\ldots,r_{p}\ \mbox{ of the}\ {k}^{{\prime}}s =\eta _{ p},& & {}\\ \end{array}$$

where $2 \leq q_{i} \leq 1 + \frac{1} {\gamma },\ r_{j} > 1 + \frac{1} {\gamma }$, and

$$\displaystyle\begin{array}{rcl} \sum _{i=1}^{m}q_{ i} +\sum _{ j=1}^{p}r_{ j} = 2\nu.& & {}\\ \end{array}$$

Then,

$$\displaystyle\begin{array}{rcl} & & \hspace{-13.20007pt}E\ \prod _{i=1}^{m}{(a_{ n,\xi _{i}}Z_{n\xi _{i}})}^{q_{i} }\prod _{j=1}^{p}{(a_{ n,\eta _{j}}Z_{n,\eta _{j}})}^{r_{j} } \\ & & \leq {(1 + c)}^{\nu }\prod _{i=1}^{m}\vert a_{ n,\xi _{i}}{\vert }^{q_{i} }\prod _{j=1}^{p}{\left \vert a_{ n,\eta _{j}}\right \vert }^{1+\frac{1} {\gamma } }{(2{n}^{-\alpha })}^{\left (r_{j}-1-\frac{1} {\gamma } \right )} \\ & & \leq {(1 + c)}^{\nu }\prod _{i=1}^{m}\vert a_{ n,\xi _{i}}\vert \prod _{j=1}^{p}\vert a_{ n,\eta _{j}}\vert {(B{n}^{-\gamma })}^{\sum _{i=1}^{m}(q_{ i}-1+\frac{p} {\gamma } )}{\left ( \frac{2} {{n}^{\alpha }}\right )}^{\sum _{j=1}^{\nu }(r_{ j}-1-\frac{1} {\gamma } )}\!\!,\qquad {}\end{array}$$

(5.3.7)

where c is the upper bound for $E\vert Z_{nk}{\vert }^{1+\frac{1} {\gamma } }$ mentioned above. Now, the power to which n is raised is the negative of

$$\displaystyle\begin{array}{rcl} \gamma \sum _{i=1}^{m}(q_{ i} - 1) + p +\alpha \sum _{ j=1}^{p}\left (r_{ j} - 1 -\frac{1} {\gamma } \right ).& & {}\\ \end{array}$$

Now, if p is one (or larger),

$$\displaystyle\begin{array}{rcl} p +\alpha \sum _{ j=1}^{p}\left (r_{ j} - 1 -\frac{1} {\gamma } \right ) \geq 1 +\alpha \left (\nu -\frac{1} {\gamma } \right ),& & {}\\ \end{array}$$

while if p = 0, 

$$\displaystyle\begin{array}{rcl} \gamma \sum _{i=1}^{m}(q_{ i} - 1) =\gamma (2\nu - m) {\geq \gamma }^{\nu } = 1 +\gamma \left (\nu -\frac{1} {\gamma } \right ) \geq 1 +\alpha \left (\nu -\frac{1} {\gamma } \right );& & {}\\ \end{array}$$

the first inequality being a result of

$$\displaystyle\begin{array}{rcl} m \leq \frac{1} {2}\sum _{i=1}^{m}q_{ i} =\nu.& & {}\\ \end{array}$$

Therefore the expectation in (5.3.7) is bounded by

$$\displaystyle\begin{array}{rcl} k_{1}\prod _{i=1}^{m}\vert a_{ n,\xi _{i}}\vert \prod _{j=1}^{p}\vert a_{ n,\eta _{j}}\vert {n}^{-1-\alpha (\nu -\frac{1} {\gamma } )}& & {}\\ \end{array}$$

and k ₁ depends only on c, γ, and B. It follows that

$$\displaystyle\begin{array}{rcl} E{\left (\sum _{k=0}^{\infty }a_{ nk}Z_{nk}\right )}^{2\nu } \leq k_{ 2}{n}^{-1-\alpha (\nu -\frac{1} {\gamma } )}& & {}\\ \end{array}$$

for some k ₂ which may depend on c, γ, B, and M but is independent of n. An application of the Markov’s inequality now yields (5.3.6).

This completes the proof of Lemma 5.3.5. □ 

Proof of Theorem 5.3.2.

Observe that

$$\displaystyle\begin{array}{rcl} \sum _{k=0}^{\infty }a_{ nk}X_{k} =\sum _{ k=0}^{\infty }a_{ nk}(X_{k}-\mu ) +\mu \sum _{ k=0}^{\infty }a_{ nk}& & {}\\ \end{array}$$

and the last term converges to μ by (1.3.3). Therefore, we may consider only the case μ = 0. By the Borel-Cantelli Lemma, it suffices to show that for every ε > 0,

$$\displaystyle\begin{array}{rcl} \sum _{n=0}^{\infty }P\left (\left \vert \sum _{ k=0}^{\infty }a_{ nk}X_{k}\right \vert \geq \epsilon \right ) < \infty.& &{}\end{array}$$

(5.3.8)

But

$$\displaystyle\begin{array}{rcl} \left (\left \vert \sum _{k=0}^{\infty }a_{ nk}X_{k}\right \vert \geq \epsilon \right )& \subset & \left (\left \vert \sum _{k=0}^{\infty }a_{ nk}X_{k}\right \vert \geq \frac{\epsilon } {2}\right ) {}\\ & \cup & \left (\left \vert a_{nk}X_{k}\right \vert \geq \frac{\epsilon } {2}\ \mbox{ for some}\ k\right ) {}\\ & \cup & \left (\left \vert a_{nk}X_{k}\right \vert \geq {n}^{-\alpha }\ \mbox{ for at least two}\ k\right ). {}\\ \end{array}$$

Now if $0 <\alpha <\gamma /2(\gamma +1)$, then α < γ also and the series (5.3.8) converges as a consequence of Lemma 5.3.3–5.3.5.

This completes the proof of Theorem 5.3.2. □ 

5.4 Strong Law of Large Numbers

In the next theorem, we study the problems arising out of the strong law of large numbers.

In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment in a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value and will tend to become closer as more trials are performed.

The strong law of large numbers states that the sample average converges almost surely to the expected value (X _n  → μ(C, 1) a.s., as n → ∞), i.e.,

$$\displaystyle\begin{array}{rcl} P\left [\lim _{n\rightarrow \infty }\frac{X_{1} + X_{2} + \cdots + X_{n}} {n} =\mu \right ] = 1.& & {}\\ \end{array}$$

Kolmogorov’s strong law of large numbers asserts that EX ₁ → μ if and only if ∑ W _i is a.e. (C, 1)-summable to μ, i.e., the (C, 1)-limit of (X _n ) is μ a.e. By the well-known inclusion theorems involving Cesàro and Abel summability (cf. [41], Theorems 43 and 55), this implies that $\sum W_{i}$ is a.e. (C, α)-summable to μ for any α ≥ 1 and that $\sum W_{i}$ is a.e. (A)-summable to μ; where $W_{n} = X_{n} - X_{n-1}\ \ (X_{0} = W_{0} = 0).$ In fact, the converse also holds in the present case and we have the following theorem.

Theorem 5.4.1.

If $X_{1},X_{2},X_{3},\ldots$ is a sequence of i.i.d. random variables and α ≥ 1 and are given real numbers, then the following statements are equivalent:

$$\displaystyle\begin{array}{rcl} & & E\ X_{1} =\mu {}\end{array}$$

(5.4.1)

$$\displaystyle\begin{array}{rcl} & & \lim _{n\rightarrow \infty }\frac{X_{1} + X_{2} + \cdots + X_{n}} {n} =\mu \ \ a.e.{}\end{array}$$

(5.4.2)

$$\displaystyle\begin{array}{rcl} & & \text{ where }\binom{j+\beta }{j} = \frac{(\beta +1)\cdots (\beta +j)} {j!} \\ & & \lim _{\lambda \rightarrow 1-}(1-\lambda )\sum _{i=1}^{\infty }{\lambda }^{i}X_{ i} =\mu \ \ a.e.{}\end{array}$$

(5.4.3)

$$\displaystyle\begin{array}{rcl} & & \text{ where }\binom{j+\beta }{j} = \frac{(\beta +1)\cdots (\beta +j)} {j!} \\ & & \lim _{\lambda \rightarrow 1-}(1-\lambda )\sum _{i=1}^{\infty }{\lambda }^{i}X_{ i} =\mu \ \ a.e.{}\end{array}$$

(5.4.4)

Proof.

The implications (5.4.2) ⇒  (5.4.3) ⇒  (5.4.4) are well known (cf. [41]). We now prove that (5.4.4) implies (5.4.1). By (5.4.4)

$$\displaystyle\begin{array}{rcl} \lim _{m\rightarrow \infty } \frac{1} {m}\sum _{n=1}^{\infty }{e}^{-n/m}X_{ n}^{s} = 0\text{ }(a.e.),& & {}\\ \end{array}$$

where $X_{n}^{s} = X_{n} - X_{n}^{{}^{{\prime}} }$ with $X_{n}^{{}^{{\prime}} },$ n ≥ 1, and X _n , n ≥ 1, being i.i.d. Let

$$\displaystyle\begin{array}{rcl} Y _{m} = \frac{1} {m}\sum _{n=1}^{m}{e}^{-n/m}X_{ n}^{s},\text{ }Z_{ m} = \frac{1} {m}\sum _{n=m+1}^{\infty }{e}^{-n/m}X_{ n}^{s}.& & {}\\ \end{array}$$

Then $Y _{m} + Z_{m}\stackrel{P}{\rightarrow }0$, as m → ∞, Y _m and Z _m are independent and symmetric. Therefore it follows easily from the Levy’s inequality [57, p. 247] that $Z_{m}\stackrel{P}{\rightarrow }0$. Since Z _m and $(Y _{1},\ldots,Y _{m})$ are independent and $Y _{m} + Z_{m} \rightarrow 0$ a.e., $Z_{m}\stackrel{P}{\rightarrow }0$, we obtain by Lemma 3 of [23] that $Y _{m} \rightarrow 0$ a.e. Letting $Y _{m}^{(1)} = Y _{m} - {e}^{({m}^{-1}X_{ m}^{s}) }$, since ${e}^{({m}^{-1}X_{ m}^{s}) }\stackrel{P}{\rightarrow }0$, we have by Lemma 3 of [10] that $X_{m}^{s}/m \rightarrow \infty$ a.e. By the Borel-Cantelli lemma, this implies that E | X ₁ |  < 1. As established before, we then have $X_{n} \rightarrow EX_{1}(A)$ and so by (5.4.4), μ = EX ₁. 

This completes the proof of Theorem 5.4.1. □ 

Remark 5.4.2.

Chow [22] has shown that unlike the Cesàro and Abel methods which require E | X ₁ |  < ∞ for summability, the Euler and Borel methods require $EX_{1}^{2} < \infty$ for summability. Specifically, if $X_{1},X_{2},\ldots$ are i.i.d., then the following statements are equivalent:

$$\displaystyle\begin{array}{rcl} & & \hspace{62.39996pt} EX_{1} =\mu,\ EX_{1}^{2} < \infty, {}\\ & & X_{n} \rightarrow \mu (E,q),\ \text{for some or equivalently for every }q > 0,\text{ i.e.,} {}\\ & & \lim _{n\rightarrow \infty } \frac{1} {{(q + 1)}^{n}}\sum _{k=1}^{n}\binom{n}{k}{q}^{n-k}X_{ k} =\mu \ \ \ \text{a.e.,} {}\\ & & \lim _{n\rightarrow \infty }X_{n} =\mu (B),\ \ \text{i.e.}\ \ \lim _{\lambda \rightarrow \infty }\frac{1} {{e}^{\lambda }}\sum _{k=1}^{\infty }\frac{{\lambda }^{k}} {k!}X_{k} =\mu \ \ \text{a.e..} {}\\ \end{array}$$