1 Introduction

The problem of determining the probability of crossing a threshold by a random process has a wide range of applications, and a considerable history (Castillo et al. 2005; Leadbetter et al. 2011). The use of control charts or the study of k-of-n systems in reliability theory (Barlow and Proschan, 1965) are a classical example. Recent examples of considerable current interest include modeling flooding in hydrology (Huang et al. 2020) and the frequency of large forest fires (Alvarado et al. 1998) in climate science.

In many applications, (systems of) differential equations model the processes that underlie the exceedances. For example, stochastic differential equations with Brownian motion as the underlying driver in fields as diverse as mathematical finance (prices of equities) (Steele, 2000), hydrology (groundwater flow) (Cushman, 1987), and neuroscience (spike generation of a neuron) (Tuckwell, 1988). In other cases, the critical events are modeled as exceedances of rather simple probabilistic models, such as the multivariate normal.

The problem that we address here arose from a discussion with Professor Shun’ichi Amari about a paper by Shadlen and Newsome (1998) which dealt with the problem of modeling the natural variability of cortical neurons using a simple integrate-and-fire model. In this model, a neuron receives thousands of synaptic inputs. The magnitudes of the inputs could be viewed as i.i.d. Gaussian random variables, with positive (negative) values corresponding to excitatory (inhibitory). If the inputs were independent, the number of excitatory inputs would have a binomial distribution. However, the independence of the inputs is often not reasonable, so a natural generalization is to consider exchangeable random variables instead: the inputs may be dependent, but they are sense stationary in some sense. In short, we are interested in the structure of the probabilities of exceedances.

In Section 2 we define the exceedance statistic and its distribution. Although our primary interest is on an underlying Gaussian model, we state some of our results more generally. We show that the exceedance probabilities are expressible in terms of a single integral. We use it to describe the different shapes of the exceedance distribution and prove a majorization ordering for it.

2 Properties of the Exceedance Distribution

Consider i.i.d. symmetric random variables Z0,…,Zn with cdf F, pdf f, with f(x) = f(−x) > 0 or all x, and \(E({Z_{i}^{2}}) = 1\). Many of the results below are for the Gaussian, for which we denote the cdf and pdf by Φ and ϕ, respectively. For ρ ≥ 0, let \(X_{i} = \sqrt {1-\rho } Z_{i} + \sqrt {\rho } Z_{0}\), so that Xn = (X1,…,Xn) has mean 0 and covariance matrix Σ = (σij) with σii = 1 and σij = ρ for i ≠ j. For a constant c the exceedance statistic is

$$ S_{n} = \sum\limits_{i=1}^{n} I(X_{i} \geq c). $$

By conditioning on Z0, we get the exceedance distribution,

$$ p_{n,k}(c|\rho) = P(S_{n} {}={} k) {}={} {n \choose k} {\int}_{-\infty}^{\infty} F\left( -\frac{c+t\sqrt{\rho}}{\sqrt{1-\rho}} \right)^{k} F\left( \frac{c+t\sqrt{\rho}}{\sqrt{1-\rho}} \right)^{n-k} f(t) dt. $$
(2.1)

Several special cases of this expression are elementary (David, 1953): for example, for the Gaussian

$$ p_{2,2}(0|\rho) = \frac{1}{2} - \frac{1}{2\pi} \cos^{-1} \rho \quad \text{and} \quad p_{3,3}(0|\rho) = \frac{1}{2} - \frac{3}{4\pi} \cos^{-1} \rho, $$
(2.2)

which also hold for − 1/(n − 1) ≤ ρ < 0; these expressions also hold for elliptically contoured distributions (Iyengar and Tong, 1989). And for any such F,

$$ p_{n,k}(0|1/2) = \frac{1}{n+1}, $$
(2.3)

is the probability that Z0 is the (n − k + 1)st order statistic among (Z0,…,Zn). More generally, the connection to order statistics of equicorrelated random variables is clear: writing F(−x) = 1 − F(x), we have

$$ \begin{array}{@{}rcl@{}} p_{n,k}(c|\rho) & = & {n \choose k} {\int}_{-\infty}^{\infty} F\left( -\frac{c+t\sqrt{\rho}}{\sqrt{1-\rho}} \right)^{k} F\left( \frac{c+t\sqrt{\rho}}{\sqrt{1-\rho}} \right)^{n-k} f(t) dt \\ & = & {n \choose k} \sum\limits_{i=0}^{k} (-1)^{k-i} {k \choose i} {\int}_{-\infty}^{\infty} F\left( \frac{c+t\sqrt{\rho}}{\sqrt{1-\rho}} \right)^{n-i} f(t) dt \\ & = & {n \choose k} \sum\limits_{i=0}^{k} (-1)^{k-i} {k \choose i} P(M_{n-i} \leq c), \end{array} $$
(2.4)

where Mn is the largest order statistic in an equicorrelated sample of size n.

For large n there is an easily derived approximation.

Theorem 2.1.

For 0 ≤ t ≤ 1, as \(n \rightarrow \infty \)

$$ P\left( \frac{S_{n}}{n} < t \right) \rightarrow F\left( \frac{c + \sqrt{1-\rho}F^{-1}(t)}{\sqrt{\rho}} \right), $$
(2.5)

or with a slight abuse of notation,

$$F^{-1}\left( \frac{S_{n}}{n} \right) \overset{d} \longrightarrow \frac{c}{\sqrt{1-\rho}} + \sqrt{\frac{\rho}{1-\rho}} Z_{0}. $$

Proof.

By the strong law,

$$ \frac{1}{n} \sum\limits_{i=1}^{n} I \left( Z_{i} \geq \alpha \right) \rightarrow F(-\alpha) \text{almost surely.} $$

Thus, by dominated convergence

$$ \begin{array}{@{}rcl@{}} P\left( \frac{S_{n}}{n} < t \right) & = & {\int}_{\mathbb{R}} \left( \frac{S_{n}}{n} < t | Z_{0} = u \right) f(u) du \\ & = & {\int}_{\mathbb{R}} P\left( \frac{1}{n} \sum\limits_{i=1}^{n} \left[Z_{i} \geq \frac{c+u\sqrt{\rho}}{\sqrt{1-\rho}} \right] < t \right) f(u) du \\ & \rightarrow & {\int}_{\mathbb{R}} I \left[F \left( -\frac{c+u\sqrt{\rho}}{\sqrt{1-\rho}} \right) < t \right] f(u) du \\ & = & P\left[Z_{0} > -\frac{c + \sqrt{1-\rho} F^{-1}(t)}{\sqrt{\rho}} \right] \\ & = & F \left( \frac{c + \sqrt{1-\rho} F^{-1}(t)}{\sqrt{\rho}} \right) \end{array} $$

For the Gaussian, the following properties of \(G_{a,b}(t) = {\Phi }(a + b {\Phi }^{-1}(t))\) for \(a \in \mathbb {R}\) and b > 0, and its density

$$ g_{a,b}(t) = \frac{b \phi(a + b{\Phi}^{-1}(t))}{\phi({\Phi}^{-1}(t))} $$
(2.6)

are easy to verify.

  1. (a)

    G0,1 is the uniform.

  2. (b)

    If b = 1 and a > 0 (a < 0), the density decreases (increases) from \(\infty \) to 0 (0 to \(\infty \)).

  3. (c)

    If b > 1 the density is bounded with ga,b(0) = ga,b(1) = 0 and it is unimodal with mode at t = Φ(ab/(1 − b2)).

  4. (d)

    If b < 1 the density is bounded with \(g_{a,b}(0)=g_{a,b}(1)=\infty \) and it is U-shaped with minimum at t = Φ(ab/(1 − b2)).

  5. (e)

    The raw moments of this distribution are

    $$ {{\int}_{0}^{1}} t^{k} g_{a,b}(t) dt = {\int}_{\mathbb{R}} {\Phi} \left( \frac{x-a}{b} \right)^{k} \phi(x) dx; $$

    The first moment is \(1 - {\Phi }(a/\sqrt {1+b^{2}})\); the rest are easily computed, but not expressible in elementary terms.

Thus, the family (2.6) of limiting distributions resembles the Beta(α, β) family. For the special case c = 0 that resemblance holds for finite n for not only the Gaussian, but other latent distributions that satisfy a certain likelihood ratio ordering with respect to the scale parameter. □

Definition 2.2.

We say that the cdf F and its pdf f satisfy the LR condition if f(x) = f(−x) and that the ratio

$$ L(x|\sigma) = \frac{1}{\sigma} \frac{f(x/\sigma)}{f(x)} $$

is decreasing (increasing) in |x| for 0 < σ ≤ 1 (\(1 \leq \sigma < \infty \)). The Gaussian, Laplace, and t-distributions all satisfy this LR condition.

We next prove the intuitively clear result that for c = 0 the exceedance distribution is either U-shaped or unimodal with mode or minimum at the middle. The proof is rather involved, requiring a detailed study of the integrands and the use of the likelihood ratio method which transfers attention from any ρ to ρ = 1/2, for which the exceedance distribution is uniform from (2.3).

Theorem 2.3.

Suppose that F and f satisfy the LR condition in (2.2), and that c = 0, so that the exceedance distribution is symmetric. Then for 0 ≤ ρ ≤ 1/2 the exceedance distribution is unimodal with probabilities decreasing away from the mode. And for 1/2 ≤ ρ ≤ 1 it is U-shaped with probabilities increasing away from the minimum. The mode or minimum is at n/2 for n even and at (n ± 1)/2 for n odd.

Proof.

We prove this result for 0 ≤ ρ ≤ 1/2; the the proof for 1/2 ≤ ρ ≤ 1 is similar, so we omit it. We first show that if 0 ≤ ρ ≤ 1/2, then \(p_{1}^{(n)} - p_{0}^{(n)} \geq 0\), and that for 2 ≤ k ≤ (n + 1)/2 we have \(p_{k}^{(n)} - p_{k-1}^{(n)} \geq 0\). Let \(\alpha = \sqrt {\rho /(1-\rho )}\). Then

$$ \begin{array}{@{}rcl@{}} p_{1}^{(n)} - p_{0}^{(n)} & = & n {\int}_{-\infty}^{\infty} F(-\alpha t) F(\alpha t)^{n-1} f(t) dt - {\int}_{-\infty}^{\infty} F(-\alpha t)^{n} f(t) dt \\ & = & {\int}_{-\infty}^{\infty} [n F(-\alpha t)^{n-1} - (n+1) F(t)^{n}] L(t|\alpha) f(t) dt \\ & = & {\int}_{0}^{\infty} [g_{n}(F(t)) + g_{n}(F(-t))] L(t|\alpha) f(t) dt \\ & = & {\int}_{0}^{\infty} h_{n}(F(t)) L(t|\alpha) f(t) dt, \end{array} $$
(2.7)

where gn(y) = nyn− 1 − (n + 1)yn and hn(y) = gn(y) + gn(1 − y). We need the following facts, all of which are derived by a close examination of these polynomials. First, gn has roots at 0 and n/(n + 1); its boundary values are gn(0) = 0, gn(1) = − 1, \(g_{n}^{\prime }(0) = 0\), and \(g_{n}^{\prime }(1) = -n\); gn has its maximum value at y = (n − 1)/(n + 1). Next, hn is strictly positive for 1/2 ≤ y ≤ n/n + 1, and is strictly decreasing for n/(n + 1) ≤ y ≤ 1; thus, there is a unique y∗ between n/(n + 1) and 1 such that hn(y∗) = 0. Now let F(t∗) = y∗. Then

$$ \begin{array}{@{}rcl@{}} p_{1}^{(n)} - p_{0}^{(n)} & = & {\int}_{0}^{\infty} h_{n}(F(t)) L(t|\alpha) f(t) dt \\ & = & {\int}_{0}^{t^{*}} h_{n}(F(t)) L(t|\alpha) f(t) dt + {\int}_{t^{*}}^{\infty} h_{n}(F(t)) L(t|\alpha) f(t) dt \\ & \geq & L(t^{*}|\alpha) \left[{\int}_{0}^{t^{*}} h_{n}(F(t)) f(t) dt + {\int}_{t^{*}}^{\infty} h_{n}(F(t)) f(t) dt \right] \\ & = & L(t^{*} | \alpha) {\int}_{0}^{\infty} h_{n}(F(t)) f(t) dt = 0. \end{array} $$
(2.8)

The proof of \(p_{k}^{(n)} - p_{k-1}^{(n)} \geq 0\) for 2 ≤ k ≤ (n + 1)/2 and 0 ≤ ρ ≤ 1/2 is similar, but the functions corresponding to gn and hn are more involved. Start with

$$ \begin{array}{@{}rcl@{}} p_{k}^{(n)} - p_{k-1}^{(n)} & = & {n \choose k} {\int}_{-\infty}^{\infty} F(-\alpha t)^{k} F(\alpha t)^{n-k} f(t) dt\\ &&- {n \choose k-1} F(-\alpha t)^{k-1} F(\alpha t)^{n-k+1} f(t) dt \\ & = & {n \choose k} {\int}_{0}^{\infty} h_{n,k}(F(t)) L(t|\alpha) f(t) dt, \end{array} $$
(2.9)

where

$$ g_{n,k}(y) = (1-y)^{k-1} y^{n-k} \left( 1 - \frac{n+1}{n-k+1} y \right), $$

and

$$ h_{n,k}(y) = g_{n,k}(y) + g_{n,k}(1-y) \quad \text{for } \frac{1}{2} \leq y \leq 1. $$

Note that gn,k has roots at 0, 1, and (n − k + 1)/(n + 1), and that \(g_{n,k}^{\prime }\) has roots at

$$ \frac{n-k+1}{n+1} \pm \frac{\sqrt{k[1 - (k-1)/n]}}{n+1}. $$

Next, we show that hn,k has a unique root y∗ between 1/2 and 1, with hn,k positive (negative) to the left (right) of y∗, so we can then use the same proof as before. Using the properties of gn,k, we see that hn,k is strictly positive for 1/2 ≤ y ≤ (n − k + 1)/(n + 1), and strictly decreasing in the interval

$$ \left[ \frac{n-k+1}{n+1}, \frac{n-k+1}{n+1} + \frac{1}{n+1} \sqrt{k\left( 1 - \frac{k-1}{n} \right)} \right] = [L_{n,k}, U_{n,k}]. $$

We must therefore show that hn,k ≤ 0 in [Un,k,1]. This is clearly true for y = 1; for y < 1 we have

$$ h_{n,k}(y) \leq 0 \iff \left( \frac{y}{1-y} \right)^{n-2k+1} \geq 1 + \frac{n-2k+1}{(n+1)y -(n-k+1)}. $$

Since n ≥ 2k + 1, it suffices to show that hn,k(y) ≤ 0 in the interval

$$ \left[ \frac{n-k+1}{n+1} + \frac{\sqrt{k/2}}{n+1}, 1 \right]. $$

In this interval we have

$$ \left( \frac{y}{1-y} \right)^{n-2k+1} \geq \left( \frac{n-k+1+\sqrt{k/2}}{k - \sqrt{k/2}} \right)^{n-2k+1} $$

because the function on the left is strictly increasing in y. Thus, it is now enough to show that

$$ 1 + \frac{n-2k+1}{\sqrt{k/2}} \leq \left( \frac{n-k+1+\sqrt{k/2}}{k - \sqrt{k/2}} \right)^{n-2k+1} $$

for k ≥ 2 and n ≥ 2k − 1. This inequality is trivial for n = 2k − 1,2k,2k + 1. Finally, writing u = n − 2k + 1, we must verify that for all u ≥ 0,

$$ 1+ \frac{u}{\sqrt{k/2}} \leq \left( \frac{u+k+\sqrt{k/2}}{k-\sqrt{k/2}} \right)^{u}, $$

which is an easy (if tedious) verification, and our proof is complete. A small note: the details of this proof requires n ≥ 3; the result also holds for n = 2 using the expressions Eq. 2.1. □

Our next result concerns majorization properties that the exceedance distribution: see Marshall and Olkin (1979). Let \(x,y \in \mathbb {R}^{n}\) be nonincreasing sequences of numbers; that is, x1 ≥ x2 ≥⋯ ≥ xn, and similarly for y. Then x majorizes y, written x ≻ y, if for k = 1,…,n

$$ \sum\limits_{i=1}^{k} x_{k} \geq \sum\limits_{i=1}^{k} y_{k} \quad \text{and} \quad \sum\limits_{i=1}^{n} x_{k} = \sum\limits_{i=1}^{n} y_{k}. $$

Theorem 2.4.

As \(n \rightarrow \infty \), Suppose that the cdf F and its density f satisfy the LR condition, that c = 0, and p(ρ) = (p0(ρ),…,pn(ρ)) be the exceedance distribution. If 0 ≤ ρ1 < ρ2 ≤ 1/2, then p(ρ1) ≻ p(ρ2); and if 1/2 ≤ ρ1 < ρ2 ≤ 1, then p(ρ2) ≻ p(ρ1);

Proof.

We prove this result for n = 2m + 1 is odd and 0 ≤ ρ ≤ 1/2; the proof for even n and 1/2 ≤ ρ ≤ 1 is similar. Because c = 0, pn,i(ρ) = pn,2m+ 1−i(ρ), and

$$ p_{n,0}(\rho) \leq {\cdots} \leq p_{n,m}(\rho) = p_{n,m+1}(\rho) \geq {\cdots} \geq p_{n,2m+1}(\rho) \quad \text{for } 0 \leq \rho \leq 1/2. $$

Thus, it suffices to show that the functions pn,m, 2pn,m, 2pn,m + pn,m− 1, …, 2pn,m + 2pn,m− 1,…, all decrease with ρ. To do this, we will show that all the derivatives are negative. As before, let \(\alpha = \sqrt {\rho /(1-\rho )}\), and note that α is a strictly increasing function of ρ. Thus, for j < m, we have

$$ \begin{array}{@{}rcl@{}} H(\alpha) & = & \sum\limits_{i=0}^{j} p_{n,m-i}(\rho) = \sum\limits_{i=0}^{j} {2m+1 \choose m-i} {\int}_{\mathbb{R}} F(-\alpha t)^{m-i} F(\alpha t)^{m+i+1} f(t) dt \\ & = & {2m+1 \choose m} \sum\limits_{i=0}^{j} \frac{m! (m+1)!}{(m-i)! (m+i+1)!} {\int}_{\mathbb{R}} F(-\alpha t)^{m-i} F(\alpha t)^{m+i+1} f(t) dt. \end{array} $$

Next, writing h(t) = tϕ(t)ϕ(αt), and dropping the constant combinatorial coefficient

$$ {2m+1 \choose m} m! (m+1)!, $$

the derivative of H is

$$ \begin{array}{@{}rcl@{}} H^{\prime}(\alpha) & \propto & {\int}_{\mathbb{R}} \left[ \sum\limits_{i=0}^{j} \frac{F(-\alpha t)^{m-i} F(\alpha t)^{m+i}}{(m-i)! (m+i+1)!} - \sum\limits_{i=0}^{j} \frac{F(-\alpha t)^{m-i-1} F(\alpha t)^{m+i+1}}{(m-i)! (m+i+1)!} \right] h(t) dt \\ & = & {\int}_{\mathbb{R}} k(t) h(t) dt. \end{array} $$

Note that because h(t) is an odd function, we have

$$ {\int}_{\mathbb{R}} k(t) h(t) dt = {\int}_{0}^{\infty} [k(t) - k(-t)] h(t) dt. $$

Applying that to the expression for \(H^{\prime }(\alpha )\) we get a collapsing sum that leads to

$$ H^{\prime}(\alpha) = - C(m,j) {\int}_{0}^{\infty} [F(-\alpha t) F(\alpha t)]^{m-j-1} [F(\alpha t)^{2j+2} - F(-\alpha t)^{2j+2}] h(t) dt < 0, $$

where

$$ C(m,j) = (m+ 1) {2m+1 \choose m+j+ 1} {2m \choose m}^{-1}. $$

We now see that \({\sum }_{i=1}^{j} p_{n,m-i}(\rho )\) is decreasing in ρ for j < m; the case of j = m is trivial because \({\sum }_{i=1}^{m} p_{n,m-i}(\rho ) = 1/2\). Finally, the same calculations show that the same result holds for

$$ 2 \sum\limits_{i=0}^{j} p_{n,m-i}(\rho) + p_{n,m-j-1}(\rho), $$

and our proof is complete. □

3 Discussion

Numerical examples indicate that the approximation in Eq. 2.5 is good for n ≥ 20 near the mode, but that n ≥ 50 gives better results in the tails. Of course, for the neuroscience applications that motivated this work the approximation is quite good because n is in the thousands. Our main results – the shape of the exceedance distribution and the the majorization result – are limited in scope because c = 0. Extending these results to c≠ 0 requires knowledge of the location of the mode, complicating the computations considerably. Our numerical work indicate that the beta-distribution-like shapes may well hold for any c and ρ ≥ 0. However, the majorization result does not generalize to c≠ 0.