Keywords

1 Introduction

Consistent reconstruction is a method for estimating a signal \(x \in \mathbb{R}^{d}\) from a collection of linear measurements that have been corrupted by uniform noise or, more generally, bounded noise. Estimation with uniform noise arises naturally in quantization problems in signal processing, especially in connection with dithering and the uniform noise model [7, 11]. Consistent reconstruction has been used as a signal recovery method for memoryless scalar quantization [1, 2, 4, 11, 13], Sigma-Delta quantization [12], and compressed sensing [5, 6, 9]. See [10] for background and motivation on consistent reconstruction and estimation with uniform noise.

Let \(x \in \mathbb{R}^{d}\) be an unknown signal and let \(\{\varphi _{n}\}_{n=1}^{N} \subset \mathbb{R}^{d}\) be a given spanning set for \(\mathbb{R}^{d}\) that is used to make linear measurements 〈x, φ n 〉 of x. We consider the problem of recovering an estimate for x from the noisy measurements

$$\displaystyle{ q_{n} =\langle x,\varphi _{n}\rangle +\epsilon _{n},\ \ \ 1 \leq n \leq N, }$$
(1)

where {ε n } n = 1 N are independent uniform random variables on [−δ, δ]. For the setting of this chapter, the collection {φ n } n = 1 N is known but randomly generated, the noise level δ > 0 is fixed and known, whereas x and the noise {ε n } n = 1 N are both unknown. We focus on the situation when {φ n } n = 1 N are independent versions of a random vector \(\varphi \in \mathbb{R}^{d}\) whose distribution we refer to as the sampling distribution.

Consistent reconstruction seeks an estimate \(\widetilde{x}\) for the unknown signal x that is consistent with the knowledge that the noise is bounded in [−δ, δ]. Specifically, consistent reconstruction produces an estimate \(\widetilde{x} \in \mathbb{R}^{d}\) for x by selecting any solution of the linear feasibility problem

$$\displaystyle{ \vert \langle \widetilde{x},\varphi _{n}\rangle - q_{n}\vert \leq \delta,\ \ \ 1 \leq n \leq N. }$$
(2)

There are generally infinitely many solutions to this feasibility problem. In this chapter, we mainly focus on the worst case error associated to consistent reconstruction.

1.1 Worst case error

To describe the worst case error of consistent reconstruction, note that if \(\widetilde{x}\) is any solution to (2), then the error \((\widetilde{x} - x)\) lies in each of the closed convex sets

$$\displaystyle\begin{array}{rcl} E_{n} = \left \{u \in \mathbb{R}^{d}:\vert \langle u,\varphi _{ n}\rangle -\epsilon _{n}\vert \leq \delta \right \}.& &{}\end{array}$$
(3)

The intersection of the sets E n forms the following error polytope:

$$\displaystyle\begin{array}{rcl} P_{N} =\bigcap _{ n=1}^{N}E_{ n},& &{}\end{array}$$
(4)

which is the set of all possible errors associated to consistent reconstruction (2). The worst case error W N associated to consistent reconstruction is thus defined by

$$\displaystyle\begin{array}{rcl} W_{N} =\max \left \{\|u\|: u \in P_{N}\right \},& &{}\end{array}$$
(5)

where ∥⋅ ∥ denotes the Euclidean norm on \(\mathbb{R}^{d}\).

1.2 Background

The main results in [10] proved error bounds for the expected worst case error squared \(\mathbb{E}[(W_{N})^{2}]\) of consistent reconstruction when the sampling vectors {φ n } n = 1 N are drawn at random from a suitable probability distribution on the unit sphere \(\mathbb{S}^{d-1}\).

The work in [10] considered sampling vectors \(\{\varphi _{n}\}_{n=1}^{N} \subset \mathbb{S}^{d-1}\) that are independently drawn instances of a unit-norm random vector φ that satisfies the following admissibility condition:

$$\displaystyle{ \exists \,\alpha \geq 1,\exists \,0 <s \leq 1,\ \ \forall \,0 \leq t \leq 1,\ \ \forall x \in \mathbb{S}^{d-1},\ \ \ \ \ \Pr [\,\vert \langle x,\varphi \rangle \vert \leq t] \leq \alpha t^{s}. }$$
(6)

See Section 5 of [10] for further discussion of the admissibility condition (6). For example, if φ is uniformly distributed on \(\mathbb{S}^{d-1}\), then φ satisfies (6) with s = 1 and \(\alpha = \frac{2\varGamma (\frac{d} {2} )} {\sqrt{\pi }\varGamma (\frac{d-1} {2} )}\). On the other hand, if φ has a point mass, then φ does not satisfy (6).

Suppose that \(\{\varphi _{n}\}_{n=1}^{N} \subset \mathbb{S}^{d-1}\) are independently drawn at random according to a distribution that satisfies the admissibility condition (6). Theorem 5.5 and Corollary 5.6 in [10] prove that there exist absolute constants c 1, c 2 > 0 such that if

$$\displaystyle{N \geq c_{2}d\ln (32(2\alpha )^{1/s}),}$$

then the expected worst case error squared for consistent reconstruction satisfies

$$\displaystyle{\mathbb{E}[(W_{N})^{2}] \leq \frac{c_{1}\delta ^{2}d^{2}(2\alpha )^{1/s}\ln ^{2}(16(2\alpha )^{1/s})} {(N + 1)(N + 2)}.}$$

Moreover, in the special case when {φ N } n = 1 N are drawn independently at random according to the uniform distribution on \(\mathbb{S}^{d-1}\), Theorem 6.1 and Corollary 6.2 in [10] proved a refined error bound with a constant that has cubic dependence on the dimension

$$\displaystyle{\mathbb{E}[(W_{N})^{2}] \leq \frac{c\delta ^{2}d^{3}} {N^{2}}.}$$

For perspective, it is known that mean squared error rates of order 1∕N 2 are generally optimal for estimation with uniform noise, see [11].

1.3 Overview and main results

The error bounds for consistent reconstruction in [10] only considered the mean squared error \(\mathbb{E}[(W_{N})^{2}]\) and only considered the admissibility condition (6) in the setting of unit-norm random vectors (for example, this excludes the case of Gaussian random vectors). The main contributions of this chapter are two-fold:

  1. 1.

    We prove bounds on general error moments \(\mathbb{E}[(W_{N})^{p}]\) for consistent reconstruction. Our main results show that the error decreases like \(\mathbb{E}[(W_{N})^{p}]\lesssim 1/N^{p},\) as the number of measurements N increases.

  2. 2.

    We establish a general admissibility condition on the sampling distribution that does not require φ to be unit-norm.

In Section 2, we prove our first main result, Theorem 1, which gives upper bounds on \(\mathbb{E}[(W_{N})^{p}]\) for unit-norm sampling distributions. Section 3 builds on Theorem 1 and proves our second main result, Theorem 2, for general sampling distributions that need not be unit-norm.

2 Error moments for consistent reconstruction: unit-norm distributions

In this section we prove our first main result, Theorem 1. Theorem 1 extends Theorem 5.5 in [10] to the setting of general error moments \(\mathbb{E}[(W_{N})^{p}]\). In this section, we assume that the sampling vectors {φ n } n = 1 N are unit-norm and satisfy the admissibility condition (6). We shall later remove the unit-norm requirement from the admissibility condition in Section 3.

2.1 Consistent reconstruction and coverage problems

We begin by recalling a useful connection between consistent reconstruction and a problem on covering the sphere by random sets.

Definition 1

Let \(\left \{\varphi _{n}\right \}_{n=1}^{N}\) be a set of unit-norm vectors and let {ε n } n = 1 N ⊂ [−δ, δ]. For each λ > 0, define

$$\displaystyle\begin{array}{rcl} B_{n}\left (\lambda \right ) = B(\varphi _{n},\epsilon _{n},\lambda )& =& \left \{u \in \mathbb{S}^{d-1}:\langle u,\varphi _{ n}\rangle> \dfrac{\epsilon _{n}+\delta } {\lambda } \;\mbox{ or}\;\langle u,\varphi _{n}\rangle <\dfrac{\epsilon _{n}-\delta } {\lambda } \right \} \\ & =& \left \{u \in \mathbb{S}^{d-1}: \left \vert \lambda \langle u,\varphi _{ n}\rangle -\epsilon _{n}\right \vert>\delta \right \}. {}\end{array}$$
(7)

In our setting, the sets B n (λ) are random subsets of \(\mathbb{S}^{d-1}\) because {φ n } n = 1 N and {ε n } n = 1 N are random.

Note that each \(B_{n}\left (\lambda \right )\) can be expressed as a union of two (possibly empty) antipodal open spherical caps of different sizes

$$\displaystyle\begin{array}{rcl} B_{n}\left (\lambda \right ) = \text{Cap}\left (\varphi _{n},\theta _{n}^{+}\right ) \cup \text{Cap}\left (-\varphi _{ n},\theta _{n}^{-}\right ),& &{}\end{array}$$
(8)

where the angular radii θ n + and θ n − are given by

$$\displaystyle\begin{array}{rcl} \theta _{n}^{+} = \left \{\begin{array}{@{}l@{\quad }l@{}} \arccos \left (\frac{\delta +\epsilon _{n}} {\lambda } \right ),\quad & \text{if }\delta +\epsilon _{n} <\lambda, \\ 0, \quad & \text{otherwise}, \end{array} \right.& & {}\\ \end{array}$$

and

$$\displaystyle\begin{array}{rcl} \theta _{n}^{-} = \left \{\begin{array}{@{}l@{\quad }l@{}} \arccos \left (\frac{\delta -\epsilon _{n}} {\lambda } \right ),\quad & \text{if }\delta -\epsilon _{n} <\lambda, \\ 0, \quad & \text{otherwise}. \end{array} \right.& & {}\\ \end{array}$$

The following lemma shows a connection between consistent reconstruction and the problem of covering the unit sphere by the random sets B n (λ), see Lemma 4.1 in [10].

Lemma 1

For all λ > 0, the worst case error satisfies 

$$\displaystyle\begin{array}{rcl} \,\mathit{\text{Pr}}\left [W_{N}>\lambda \right ] \leq \,\mathit{\text{Pr}}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ].& &{}\end{array}$$
(9)

The following lemmas collect upper bounds on \(\,\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{n=1}^{N}B_{n}\left (\lambda \right )\right ]\) that are spread out over various parts of [10].

Lemma 2

If λ ≥ 4δ, then 

$$\displaystyle{ \,\mathit{\text{Pr}}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ] \leq 4^{d-1}(4^{s}\alpha )^{N}\left (\frac{\delta } {\lambda }\right )^{sN-d+1}. }$$
(10)

Lemma 2 was shown in equation (5.9) in [10].

Lemma 3

If 0 ≤ λ ≤ 4(2α)1∕s δ, then 

$$\displaystyle\begin{array}{rcl} & & \,\mathit{\text{Pr}}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ] \\ & & \leq \sum _{k=0}^{N}q(k,d - 1,\alpha,s)\binom{N}{k}\left (1 - \frac{\lambda } {4\delta (2\alpha )^{1/s}}\right )^{N-k}\left ( \frac{\lambda } {4\delta (2\alpha )^{1/s}}\right )^{k},{}\end{array}$$
(11)

 where q(k, d − 1, α, s) satisfies 

$$\displaystyle{ q(k,d - 1,\alpha,s) \leq 1, }$$
(12)

 and 

$$\displaystyle{ k \geq \frac{2d\ln (16(2\alpha )^{1/s})} {\ln (4/3)} \ \ \ \Rightarrow\ \ \ q(k,d - 1,\alpha,s) \leq \left (\frac{3} {4}\right )^{k/2}. }$$
(13)

The bound (11) appears in (5.12) in [10]. The bound (12) follows from (5.11) in [10], and the bound (13) appears in Step VI in the proof of Theorem 5.5 in [10].

2.2 Error moment bounds

We now prove our first main result that provides error moment bounds for consistent reconstruction.

Theorem 1

Suppose that \(\{\varphi _{n}\}_{n=1}^{N} \subset \mathbb{S}^{d-1}\) are independently drawn at random according to a distribution that satisfies the admissibility condition  (6) with parameters α ≥ 1 and 0 < s ≤ 1. If \(p \in \mathbb{N}\) and N ≥ (d + p)∕s, then the pth error moment for consistent reconstruction satisfies 

$$\displaystyle\begin{array}{rcl} \mathbb{E}[(W_{N})^{p}] \leq \ \ C^{\,{\prime}}\,\delta ^{p}\left (\prod _{ j=1}^{p}(N + j)\right )^{-1} + C^{\,{\prime\prime}}\,\delta ^{p}\left (\frac{1} {2}\right )^{N},& &{}\end{array}$$
(14)

 where 

$$\displaystyle{C^{\,{\prime}} = C_{ p,\alpha,s}^{{\prime}} = 2p(4(2\alpha )^{1/s})^{p}\left (\frac{2d\ln (16(2\alpha )^{1/s})} {\ln (4/3)} + p\right )^{p}\left (\sum _{ k=1}^{\infty }(k + 1)^{p-1}(3/4)^{k/2}\right ),}$$

 and 

$$\displaystyle{C^{\,{\prime\prime}} = C_{ p,\alpha,s,d}^{{\prime\prime}} = 2p(32(2\alpha )^{1/s})^{p+d-1}.}$$

Proof

We proceed by directly building on the proof of Theorem 5.5 in [10].

Step 1. We need to compute

$$\displaystyle\begin{array}{rcl} \mathbb{E}[(W_{N})^{p}] = p\int _{ 0}^{\infty }\lambda ^{p-1}\Pr [W_{ N}>\lambda ]d\lambda.& &{}\end{array}$$
(15)

By Lemma 1, we have

$$\displaystyle\begin{array}{rcl} \mathbb{E}[(W_{N})^{p}]& \leq p\int _{ 0}^{\infty }\lambda ^{p-1}\,\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ]d\lambda.&{}\end{array}$$
(16)

Thus, it suffices to bound the integral on right side of (16).

Step 2. We shall bound the integral in (16) by breaking it up into three separate integrals. We begin by estimating the integral in the range 0 ≤ λ ≤ 4δ(2α)1∕s. 

Using (11) and a change of variables gives

$$\displaystyle\begin{array}{rcl} & & p\int _{0}^{4\delta (2\alpha )^{1/s} }\lambda ^{p-1}\,\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ]d\lambda \\ & & \quad \leq p\sum _{k=0}^{N}q(k,d-1,\alpha,s)\binom{N}{k}\int _{ 0}^{4\delta (2\alpha )^{1/s} }\lambda ^{p-1}\left (1- \dfrac{\lambda } {4\delta (2\alpha )^{1/s}}\right )^{N-k}\left ( \dfrac{\lambda } {4\delta (2\alpha )^{1/s}}\right )^{k}d\lambda \\ & & \quad = p\sum _{k=0}^{N}q(k,d - 1,\alpha,s)\binom{N}{k}\left (4\delta (2\alpha )^{1/s}\right )^{p}\int _{ 0}^{1}v^{k+p-1}(1 - v)^{N-k}dv \\ & & \quad = p\left (4\delta (2\alpha )^{1/s}\right )^{p}\sum _{ k=0}^{N}q(k,d - 1,\alpha,s)\binom{N}{k}\dfrac{(N - k)!(k + p - 1)!} {(N + p)!} \\ & & \quad = p\left (4\delta (2\alpha )^{1/s}\right )^{p}\left (\prod _{ j=1}^{p}(N + j)\right )^{-1}\left [\sum _{ k=0}^{N}\frac{(k + p - 1)!} {k!} \ q(k,d - 1,\alpha,s)\right ].{}\end{array}$$
(17)

Here, we used the property of the beta function that

$$\displaystyle{ \int _{0}^{1}v^{k+p-1}(1 - v)^{N-k}dv = \dfrac{(N - k)!(k + p - 1)!} {(N + p)!}. }$$
(18)

It remains to bound the sum \(\sum _{k=0}^{N}\frac{(k+p-1)!} {k!} q(k,d - 1,\alpha,s)\) in (17). We will bound this sum by breaking it up into two separate sums, in an analogous manner to Step VI in the proof of Theorem 5.5 in [10]. Let

$$\displaystyle{ K =\Bigg \lfloor \frac{2d\ln (16(2\alpha )^{1/s})} {\ln (4/3)} \Bigg\rfloor. }$$
(19)

Since q(k, d − 1, α, s) ≤ 1, we have

$$\displaystyle\begin{array}{rcl} \sum _{k=0}^{K}\dfrac{(k + p - 1)!} {k!} q(k,d - 1,\alpha,s) \leq \sum _{k=0}^{K}(K + p - 1)^{p-1} \leq (K + p)^{p}.& &{}\end{array}$$
(20)

Using (13) we have

$$\displaystyle\begin{array}{rcl} \sum _{k=K+1}^{N}\dfrac{(k + p - 1)!} {k!} q(k,d - 1,\alpha,s)& \leq & \sum _{k=K+1}^{\infty }\dfrac{(k + p - 1)!} {k!} \left (\frac{3} {4}\right )^{k/2} \\ & \leq & \sum _{k=K+1}^{\infty }(k + p - 1)^{p-1}\left (\dfrac{3} {4}\right )^{k/2} \\ & =& \sum _{k=1}^{\infty }(k + K + p - 1)^{p-1}\left (\dfrac{3} {4}\right )^{(k+K)/2} \\ & \leq & (K + p)^{p-1}\sum _{ k=0}^{\infty }(k + 1)^{p-1}\left (\dfrac{3} {4}\right )^{k/2} \\ & =& (K + p)^{p-1}S_{ p}, {}\end{array}$$
(21)

where S p = ∑ k = 1 ∞(k + 1)p−1(3∕4)k∕2 satisfies 1 < S p < ∞.

By (20) and (21) we have

$$\displaystyle{ \sum _{k=0}^{N}\frac{(k + p - 1)!} {k!} q(k,d - 1,\alpha,s) \leq (K + p)^{p}(1 + S_{ p}) \leq 2(K + p)^{p}\,S_{ p}. }$$
(22)

Combining (17) and (22) yields

$$\displaystyle\begin{array}{rcl} & & p\int _{0}^{4\delta (2\alpha )^{1/s} }\lambda ^{p-1}\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ]d\lambda \\ & & \phantom{dfldgglkdflkgfkg} \leq 2p(4\delta (2\alpha )^{1/s})^{p}(K + p)^{p}S_{ p}\left (\prod _{j=1}^{p}(N + j)\right )^{-1}.{}\end{array}$$
(23)

Step 3. Next, we bound the integral (16) in the range 4δ(2α)1∕s ≤ λ ≤ 8δ(2α)1∕s. By Lemma 2 we know that in this range of λ,

$$\displaystyle\begin{array}{rcl} \,\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ] \leq (16(2\alpha )^{1/s})^{d-1}\left (\dfrac{1} {2}\right )^{N}.& & {}\\ \end{array}$$

Thus

$$\displaystyle\begin{array}{rcl} & & p\int _{4\delta (2\alpha )^{1/s}}^{8\delta (2\alpha )^{1/s} }\lambda ^{p-1}\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ]d\lambda \\ & & \phantom{sfdffflkdgfgkg} \leq p(16(2\alpha )^{1/s})^{d-1}\left (\dfrac{1} {2}\right )^{N}\int _{ 4\delta (2\alpha )^{1/s}}^{8\delta (2\alpha )^{1/s} }\lambda ^{p-1}d\lambda \\ & & \phantom{sfdffflkdgfgkg} \leq \delta ^{p}(16(2\alpha )^{1/s})^{d+p-1}\left (\dfrac{1} {2}\right )^{N}. {}\end{array}$$
(24)

Step 4. We next bound the integral (16) in the range λ ≥ 8δ(2α)1∕s. By Lemma 2 we know that in this range of λ,

$$\displaystyle\begin{array}{rcl} \,\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ] \leq 4^{d-1}(4^{s}\alpha )^{N}\left (\frac{\delta } {\lambda }\right )^{sN-d+1}.& & {}\\ \end{array}$$

It follows that when N ≥ (d + p)∕s,

$$\displaystyle\begin{array}{rcl} & & p\int _{8\delta (2\alpha )^{1/s}}^{\infty }\lambda ^{p-1}\text{Pr}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B_{ n}\left (\lambda \right )\right ]d\lambda \\ & & \phantom{sfdffflkdgfgk} \leq p \cdot 4^{d-1}(4^{s}\alpha )^{N}\delta ^{sN-d+1}\int _{ 8\delta (2\alpha )^{1/s}}^{\infty }\lambda ^{p-sN+d-2}d\lambda \\ & & \phantom{sfdffflkdgfgk} = p \cdot 4^{d-1}(4^{s}\alpha )^{N}\delta ^{sN-d+1}\left (\frac{(8\delta (2\alpha )^{1/s})^{p-sN+d-1}} {sN - p - d + 1} \right ) \\ & & \phantom{sfdffflkdgfgk} \leq p \cdot \delta ^{p}(32(2\alpha )^{1/s})^{p+d-1}\left (\frac{1} {2}\right )^{N}. {}\end{array}$$
(25)

Combining (16), (23), (24), and (25) completes the proof.

Theorem 1 yields the following corollary.

Corollary 1

Suppose that \(\{\varphi _{n}\}_{n=1}^{N} \subset \mathbb{S}^{d-1}\) are independently drawn at random according to a distribution that satisfies the admissibility condition  (6) with parameters α ≥ 1 and 0 < s ≤ 1. If \(p \in \mathbb{N}\) and 

$$\displaystyle{ N \geq \max \left \{\frac{2} {\ln 2}\left [\ln \left (\frac{C^{{\prime\prime}}} {C^{{\prime}}}\right ) + 2p\ln \left (\frac{4p} {e\ln 2}\right )\right ], \frac{d + p} {s} \right \}, }$$
(26)

 then 

$$\displaystyle{ \mathbb{E}[(W_{N})^{p}] \leq 2C^{{\prime}}\delta ^{p}\left (\prod _{ j=1}^{p}(N + j)\right )^{-1}, }$$
(27)

 where C ′, C ″ are as in Theorem  1 .

Proof

In view of Theorem 1, it suffices to show that if N satisfies (26) then

$$\displaystyle{C^{{\prime\prime}}\left (\frac{1} {2}\right )^{N} \leq C^{{\prime}}\left (\prod _{ j=1}^{p}(N + j)\right )^{-1}.}$$

Equivalently, it suffices to show

$$\displaystyle{ \ln \left (\frac{C^{{\prime\prime}}} {C^{{\prime}}}\right ) +\sum _{ j=1}^{p}\ln (N + j) \leq N\,\ln 2. }$$
(28)

To begin, note that

$$\displaystyle{\forall x> 0,\ \ \ \ln (x) \leq x - 1,}$$

gives

$$\displaystyle\begin{array}{rcl} \ln (N)& =& \ln \left (\frac{N\ln 2} {4p} \right ) +\ln \left (\frac{4p} {\ln 2} \right ) \\ & \leq & \frac{N\ln 2} {4p} - 1 +\ln \left (\frac{4p} {\ln 2} \right ) \\ & =& \frac{N\ln 2} {4p} +\ln \left (\frac{4p} {e\ln 2}\right ). {}\end{array}$$
(29)

Next, use (29) and N ≥ (d + p)∕s ≥ max{p, 2} to obtain

$$\displaystyle\begin{array}{rcl} \sum _{j=1}^{p}\ln (N + j)& =& \sum _{ j=1}^{p}\left [\ln (N) +\ln \left (1 + \frac{j} {N}\right )\right ] \\ & \leq & p\ln (N) + p\,\ln 2 \\ & \leq & 2p\ln (N) \\ & \leq & \frac{N\ln 2} {2} + 2p\ln \left (\frac{4p} {e\ln 2}\right ). {}\end{array}$$
(30)

In view of (30), to show (28) it suffices to have

$$\displaystyle{ \ln \left (\frac{C^{{\prime\prime}}} {C^{{\prime}}}\right ) + \frac{N\,\ln 2} {2} + 2p\ln \left (\frac{4p} {e\ln 2}\right ) \leq N\,\ln 2. }$$
(31)

Since (31) holds by the assumption (26), this completes the proof.

We conclude this section with some perspective on the dimension dependence of the constant C ′ in Theorem 1 and Corollary 1. We consider the special case when φ is uniformly distributed on the unit-sphere \(\mathbb{S}^{d-1}\) with d ≥ 3. In this case, one may take s = 1 and \(\alpha = \frac{2\varGamma (d/2)} {\sqrt{\pi }\varGamma ((d-1)/2)}\) in (6), see Example 5.1 in [10], and the constant C ′ is of order \(\left (d^{\frac{3} {2} }\ln d\right )^{p}\). Here, the logarithmic factor lnd is an artifact of the general setting of Theorem 1. In particular, for p = 2 the refined analysis in Theorem 6.1 and Corollary 6.2 of [10] shows that the factor lnd can be removed when φ is uniformly distributed on the unit-sphere \(\mathbb{S}^{d-1}\). A similar analysis extends to moments with general values of \(p \in \mathbb{N}\) and shows that the factor lnd can be replaced by an absolute constant that is independent of d.

3 Error moments for consistent reconstruction: general distributions

In Section 2 we proved bounds on the pth error moment for consistent reconstruction when the measurements are made using i.i.d. copies of a unit-norm random vector \(\varphi \in \mathbb{S}^{d-1}\). In this section, we relax the unit-norm constraint to accommodate more general distributions.

3.1 General admissibility condition

Definition 2

We shall say that a random vector \(\varphi \in \mathbb{R}^{d}\) satisfies the general admissibility condition if the following conditions hold:

  • φ = aψ, where a is a non-negative random variable, ψ is a unit-norm random vector, and a and ψ are independent.

  • ψ satisfies the admissibility condition (6).

  • \(\exists C> 0\) such that

    $$\displaystyle{ \forall \lambda> 0,\ \ \ \lambda \Pr [a\lambda \leq 1] \leq C. }$$
    (32)

  • \(r_{a} =\Pr [a> 1]\) satisfies 0 < r a < 1.

Example 1

A sufficient condition for the small-ball inequality (32) to hold is when a is an absolutely continuous random variable whose probability density function f is in \(L^{\infty }(\mathbb{R})\). In this case, for each λ > 0,

$$\displaystyle\begin{array}{rcl} \,\text{Pr}\left [a\lambda \leq 1\right ] =\, \text{Pr}\left [a \leq \frac{1} {\lambda } \right ] =\int _{ 0}^{1/\lambda }f(a)\,da \leq \frac{\|f\|_{\infty }} {\lambda }.& & {}\\ \end{array}$$

This shows that a large class of probability distributions satisfy the conditions in Definition 2. For example, if φ is a random vector whose entries are i.i.d zero mean Gaussian random variables, then φ satisfies the conditions in Definition 2.

In Definition 2, there would be no loss of generality if a were scaled differently so that \(0 <\Pr [a> T] <1\) for some T > 0. In particular, suppose that φ n = a n ψ n with \(0 <\Pr [a_{n}> T] <1\), and q n = 〈x, φ n 〉 + ε n with ε n uniformly distributed on [−δ, δ]. Then \(\widetilde{x} \in \mathbb{R}^{d}\) satisfies

$$\displaystyle{\vert \langle \widetilde{x},\varphi _{n}\rangle - q_{n}\vert \leq \delta \ \ \ \ \ \text{if and only if} \ \ \ \ \ \vert \langle \widetilde{x},\varphi _{n}^{{\prime}}\rangle - q_{ n}^{{\prime}}\vert \leq \delta ^{{\prime}},}$$

where φ n ′ = φ n ∕T = a n ′ ψ n and a n ′ = a n ∕T and q n ′ = 〈x, φ n ′〉 + ε n ′, where ε n ′ = ε n ∕T is uniformly distributed on [−δ ′, δ ′] with δ ′ = δ∕T.

3.2 Coverage problems revisited

Suppose that {φ n } n = 1 N are i.i.d. versions of a random vector φ that satisfies the conditions of Definition 2. In particular, φ n = a n ψ n , where {a n } n = 1 N i.i.d. versions of a random variable a, and {ψ n } n = 1 N are i.i.d. versions of a random vector ψ. Similar to Lemma 1, the worst case error W N for consistent reconstruction can be bounded by

$$\displaystyle{ \Pr [W_{N}>\lambda ] \leq \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ], }$$
(33)

where B(ψ n , ε n , a n λ) is defined using (7).

3.2.1 Conditioning and a bound by caps with a n = 1

The following lemma bounds (33) by coverage probabilities involving caps with a n = 1.

Lemma 4

Suppose {φ n } n = 1 N , with φ n = a n ψ n , are i.i.d. versions of a random vector φ that satisfies the conditions of Definition  2 . Then 

$$\displaystyle\begin{array}{rcl} & & \mathit{\text{Pr}}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ] \\ & & \leq \sum \limits _{j=1}^{N}\,\mathit{\text{Pr}}\left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]\,\mathrm{bino}(j,N,r) + (1 - r)^{N},{}\end{array}$$
(34)

 where 

$$\displaystyle{\mathrm{bino}(j,N,r) = \binom{N}{j}r^{j}(1 - r)^{N-j},}$$

 and \(r = r_{a} =\Pr [a> 1]\) is as in Definition  2 .

Proof

Let \(\mathscr{J}_{j,N}\) denote the event that exactly j elements of {a n } n = 1 N satisfy a n > 1. Since the {a n } n = 1 N are independent versions of the random variable a,

$$\displaystyle\begin{array}{rcl} \Pr [\mathscr{J}_{j,N}]& =& \binom{N}{j}(\Pr [a> 1])^{j}(1 -\Pr [a> 1])^{N-j} {}\\ & =& \binom{N}{j}r^{j}(1 - r)^{N-j} =\mathrm{ bino}(j,N,r). {}\\ \end{array}$$

Thus,

$$\displaystyle\begin{array}{rcl} & & \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ] \\ & & =\sum _{ j=0}^{N}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\,\middle\vert \,\,\mathscr{J}_{j,N}\right ]\,\,\text{Pr}\left [\,\mathscr{J}_{j,N}\right ] \\ & & =\sum _{ j=0}^{N}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\,\middle\vert \,\,\mathscr{J}_{j,N}\right ]\,\mathrm{bino}(j,N,r).{}\end{array}$$
(35)

By (7), when a n > 1 we have B(ψ n , ε n , a n λ) ⊃ B(ψ n , ε n , λ). Thus for 1 ≤ j ≤ N,

$$\displaystyle\begin{array}{rcl} \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\,\middle\vert \,\,\mathscr{J}_{j,N}\right ]& \leq & \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{\{ n:a_{n}>1\}}B(\psi _{n},\epsilon _{n},a_{n}\lambda )\,\middle\vert \,\,\mathscr{J}_{j,N}\right ] \\ & \leq & \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{\{ n:a_{n}>1\}}B(\psi _{n},\epsilon _{n},\lambda )\,\middle\vert \,\,\mathscr{J}_{j,N}\right ] \\ & =& \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ], {}\end{array}$$
(36)

where the last equality holds because \(\left \{a_{n}\right \}_{n=1}^{N}\) are i.i.d. random variables that are independent of the i.i.d. random vectors {ψ n } n = 1 N. For j = 0, we use the trivial bound

$$\displaystyle{\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{\{ n:a_{n}>1\}}B(\psi _{n},\epsilon _{n},\lambda )\,\middle\vert \,\,\mathscr{J}_{j,N}\right ] \leq 1.}$$

Combining (35) and (36) completes the proof.

To bound the binomial terms in Lemma 4 it will be useful to recall Hoeffding’s inequality for Bernoulli random variables. If 0 < p < 1 and m ≤ Np, then

$$\displaystyle\begin{array}{rcl} \sum _{j=0}^{m}\mathrm{bino}(j,N,p) \leq \text{exp}\left (-2\left (Np - m\right )^{2}/N\right ).& &{}\end{array}$$
(37)

3.2.2 Covering and discretization

A useful technique for bounding coverage probabilities such as (33) is to discretize the problem by discretizing the sphere \(\mathbb{S}^{d-1}\) with an ε-net, see [3]. In this section, we briefly recall necessary aspects of this discretization method as used in [10].

Recall that a set \(\mathscr{N}_{\epsilon } \subset \mathbb{S}^{d-1}\) is a geodesic ε-net for \(\mathbb{S}^{d-1}\) if

$$\displaystyle{\forall x \in \mathbb{S}^{d-1},\ \exists \,z \in \mathscr{ N}_{\epsilon },\ \ \ \text{such that }\ \ \ \mathrm{arccos}(\langle x,z\rangle ) \leq \epsilon.}$$

For the remainder of this section, let \(\mathscr{N}_{\epsilon }\) be a geodesic ε- net of cardinality

$$\displaystyle{\#\left (\mathscr{N}_{\epsilon }\right ) \leq \left (\frac{8} {\epsilon } \right )^{d-1}.}$$

It is well known that geodesic ε-nets of such cardinality exist, e.g., see Lemma 13.1.1 in [8] or Section 2.2 in [10].

Recall that a set \(\mathscr{N}_{\epsilon } \subset \mathbb{S}^{d-1}\) is a geodesic ε-net for \(\mathbb{S}^{d-1}\) if

$$\displaystyle{\forall x \in \mathbb{S}^{d-1},\ \exists \,z \in \mathscr{ N}_{\epsilon },\ \ \ \text{such that }\ \ \ \mathrm{arccos}(\langle x,z\rangle ) \leq \epsilon.}$$

For the remainder of this section, let \(\mathscr{N}_{\epsilon }\) be a geodesic ε- net of cardinality

$$\displaystyle{\#\left (\mathscr{N}_{\epsilon }\right ) \leq \left (\frac{8} {\epsilon } \right )^{d-1}.}$$

It is well known that geodesic ε-nets of such cardinality exist, e.g., see Lemma 13.1.1 in [8] or Section 2.2 in [10].

Recalling (8), define the shrunken bi-cap \(T_{\epsilon }\left [B(\psi _{n},\epsilon _{n},a_{n}\lambda )\right ]\) by

$$\displaystyle{T_{\epsilon }\left [B(\psi _{n},\epsilon _{n},a_{n}\lambda )\right ] = \text{Cap}\left (\psi _{n},T_{\epsilon }(\theta _{n}^{+})\right ) \cup \text{Cap}\left (-\psi _{ n},T_{\epsilon }(\theta _{n}^{-})\right ),}$$

where

$$\displaystyle{T_{\epsilon }(\theta ) = \left \{\begin{array}{@{}l@{\quad }l@{}} \theta -\epsilon,\quad & \text{if }\theta \geq \epsilon;\\ 0, \quad & \text{if }0 \leq \theta \leq \epsilon. \end{array} \right.}$$

Similar to equations (5.4) and (5.5) in [10], the coverage probability (33) can be discretized as follows:

$$\displaystyle\begin{array}{rcl} & & \Pr \Bigg[\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\Bigg] \leq \Pr \left [\mathscr{N}_{\epsilon }\not\subset \bigcup _{n=1}^{N}T_{\epsilon }\left [B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]\right ] \\ & & \phantom{dglgldglkdlgdsf} \leq \left (\frac{8} {\epsilon } \right )^{d-1}\left (\sup _{ z\in \mathbb{S}^{d-1}}\Pr \Big[z\not\in T_{\epsilon }\left [B(\psi _{n},\epsilon _{n},a_{n}\lambda )\right ]\Big]\right )^{N}.{}\end{array}$$
(38)

Similar to equation (5.6) in [10], one has that

$$\displaystyle{B(\psi _{n},\epsilon _{n},a_{n}\lambda ) \supset \left \{u \in \mathbb{S}^{d-1}: \vert \langle u,\psi _{ n}\rangle \vert> \frac{2\delta } {a_{n}\lambda }\right \}}$$

and

$$\displaystyle{T_{\epsilon }\left [B(\psi _{n},\epsilon _{n},a_{n}\lambda )\right ] \supset \left \{u \in \mathbb{S}^{d-1}: \vert \langle u,\psi _{ n}\rangle \vert> \frac{2\delta } {a_{n}\lambda }+\epsilon \right \}.}$$

This gives

$$\displaystyle{ \Pr \bigg[z\not\in T_{\epsilon }\left [B(\psi _{n},\epsilon _{n},a_{n}\lambda )\right ]\bigg] \leq \Pr \bigg [\vert \langle z,\psi _{n}\rangle \vert \leq \frac{2\delta } {a_{n}\lambda }+\epsilon \bigg]. }$$
(39)

3.3 Moment bounds for general distributions

We now state our next main theorem.

Theorem 2

Suppose that \(\left \{\varphi _{n}\right \}_{n=1}^{N}\) are i.i.d. versions of a random vector φ that satisfies the conditions of Definition  2 . Let \(r = r_{a} =\Pr [a> 1]\) be as in Definition  2 . If 

$$\displaystyle{ N \geq \frac{2(d + p)} {sr}, }$$
(40)

 then the pth error moment for consistent reconstruction satisfies 

$$\displaystyle\begin{array}{rcl} \mathbb{E}\left [\left (W_{N}\right )^{p}\right ] \leq pC^{{\prime}}\left ( \frac{2\delta } {Nr}\right )^{p} + pC^{{\prime\prime}}\delta ^{p}\left (\frac{1} {2}\right )^{Nr/2} +\delta ^{p}\varLambda ^{p}e^{-Nr^{2}/2 } +\delta ^{p}C^{{\prime\prime\prime}}\left (\frac{1} {2}\right )^{N},& & {}\\ \end{array}$$

 where C ′, C ″ are as in Theorem  1 , Λ is defined by  (42) and  (57), and C ‴ is defined by  (60) and  (57).

Proof

As in Theorem 1 we shall use (15). In view of (33), we need to estimate

$$\displaystyle{ \mathbb{E}[(W_{N})^{p}] \leq p\int _{ 0}^{\infty }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]\,d\lambda. }$$
(41)

 Step 1. We begin by estimating the integral in (41) over the range 0 ≤ λ ≤ Λδ, where

$$\displaystyle{ \varLambda =\max \{\varLambda _{0},\varLambda _{1}\},\ \ \ \ \text{with} \ \ \ \ \varLambda _{0} = \frac{2^{s+3}C} {\alpha } \ \ \ \ \ \text{and} \ \ \ \ \ \varLambda _{1} = 4\left (2K^{{\prime\prime}}\right )^{\frac{s+1} {s} }, }$$
(42)

and K ″ is defined in (57).

By Lemma 4 we have

$$\displaystyle\begin{array}{rcl} & & p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]d\lambda \\ & & \quad \leq p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\sum _{ j=0}^{N}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]\mathrm{bino}(j,N,r)d\lambda \\ & & \quad = p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\sum _{ j=0}^{\lfloor Nr/2\rfloor }\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]\mathrm{bino}(j,N,r)d\lambda {}\end{array}$$
(43)
$$\displaystyle\begin{array}{rcl} \phantom{=} +p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\sum _{ j=\lceil Nr/2\rceil }^{N}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]\mathrm{bino}(j,N,r)d\lambda.& &{}\end{array}$$
(44)

Hoeffding’s inequality and the trivial bound \(\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{n=1}^{j}B(\psi _{n},\epsilon _{n},\lambda )\right ] \leq 1\) can be used to bound (43) as follows:

$$\displaystyle\begin{array}{rcl} & & p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\sum _{ j=0}^{\lfloor Nr/2\rfloor }\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]\mathrm{bino}(j,N,r)d\lambda \\ & & \phantom{fdgdlgkfjg} \leq p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\left (\sum _{ j=0}^{\lfloor Nr/2\rfloor }\mathrm{bino}(j,N,r)\right )d\lambda \\ & & \phantom{fdgdlgkfjg} \leq p\left (e^{-Nr^{2}/2 }\right )\int _{0}^{\varLambda \delta }\lambda ^{p-1}d\lambda \\ & & \phantom{fdgdlgkfjg} =\delta ^{p}\varLambda ^{p}e^{-Nr^{2}/2 }. {}\end{array}$$
(45)

To bound the integral in (44), recall (40) and note that if j satisfies (d + p)∕s ≤ ⌈Nr∕2⌉ ≤ j ≤ N, then the bounds on (16) obtained in the proof of Theorem 1 give that

$$\displaystyle\begin{array}{rcl} & & p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]d\lambda \\ & & \phantom{dfgdsflgflg} \leq p\int _{0}^{\infty }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]d\lambda \\ & & \phantom{dfgdsflgflg} \leq C^{{\prime}}\delta ^{p}\left (\prod _{ l=1}^{p}(j + l)\right )^{-1} + C^{{\prime\prime}}\delta ^{p}\left (\frac{1} {2}\right )^{j} \\ & & \phantom{dfgdsflgflg} \leq \frac{C^{{\prime}}\delta ^{p}} {j^{p}} + C^{{\prime\prime}}\delta ^{p}\left (\frac{1} {2}\right )^{j}, {}\end{array}$$
(46)

where C ′ and C ″ are as in Theorem 1.

Using (46), along with ∑ j = 0 Nbino(j, N, r) = 1, one may bound (44) as follows:

$$\displaystyle\begin{array}{rcl} & & p\sum _{j=\lceil Nr/2\rceil }^{N}\int _{ 0}^{\varLambda \delta }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{j}B(\psi _{ n},\epsilon _{n},\lambda )\right ]\mathrm{bino}(j,N,r)d\lambda \\ & & \phantom{sfdlgfdlg} \leq p\sum _{j=\lceil Nr/2\rceil }^{N}\mathrm{bino}(j,N,r)\left [\dfrac{C^{{\prime}}\delta ^{p}} {j^{p}} + C^{{\prime\prime}}\delta ^{p}\left (\dfrac{1} {2}\right )^{j}\right ] \\ & & \phantom{sfdlgfdlg} \leq p\delta ^{p}\left [ \frac{2^{p}C^{{\prime}}} {(Nr)^{p}} + C^{{\prime\prime}}\left (\frac{1} {2}\right )^{Nr/2}\right ]\sum _{ j=\lceil Nr/2\rceil }^{N}\mathrm{bino}(j,N,r) \\ & & \phantom{sfdlgfdlg} \leq p\delta ^{p}\left [C^{{\prime}}\left ( \frac{2} {Nr}\right )^{p} + C^{{\prime\prime}}\left (\frac{1} {2}\right )^{Nr/2}\right ]. {}\end{array}$$
(47)

Applying the bounds (45) and (47) to (43) and (44) gives

$$\displaystyle\begin{array}{rcl} & & p\int _{0}^{\varLambda \delta }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]\,d\lambda \\ & & \phantom{dfldggfgklk} \leq \delta ^{p}\varLambda ^{p}e^{-Nr^{2}/2 } + pC^{{\prime}}\left ( \frac{2\delta } {Nr}\right )^{p} + pC^{{\prime\prime}}\delta ^{p}\left (\frac{1} {2}\right )^{Nr/2}.{}\end{array}$$
(48)

Step 2. We next estimate the integral in (41) over the range λ ≥ Λδ. By (38) and (39) we have

$$\displaystyle\begin{array}{rcl} & & \Pr \left [\mathbb{S}^{d-1}\not\subset \right.\left.\bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ] \\ & & \phantom{fdjdkgjkdj} \leq \left (\frac{8} {\epsilon } \right )^{d-1}\left (\sup _{ z\in \mathbb{S}^{d-1}}\Pr \bigg[\vert \langle z,\psi _{n}\rangle \vert \leq \frac{2\delta } {a_{n}\lambda }+\epsilon \bigg]\right )^{N}.{}\end{array}$$
(49)

We therefore need to bound \(\Pr [\vert \langle z,\psi _{n}\rangle \vert \leq \frac{2\delta } {a_{n}\lambda }+\epsilon ]\).

For the remainder of this step set

$$\displaystyle{ A = \left ( \frac{\alpha } {C}\right )^{ \frac{1} {s+1} }\left (\frac{4\delta } {\lambda } \right )^{ \frac{s} {s+1} }\ \ \ \ \text{and} \ \ \ \ \ \epsilon = \frac{2\delta } {A\lambda } = \left (\frac{1} {2}\right )\left (\frac{4\delta C} {\lambda \alpha } \right )^{ \frac{1} {s+1} }, }$$
(50)

where C, α, s are the parameters in (6) and Definition (2). By (42), note that λ ≥ Λδ ≥ Λ 0 δ implies that 0 < ε ≤ 1∕4.

For any \(z \in \mathbb{S}^{d-1}\) we have

$$\displaystyle\begin{array}{rcl} \Pr \left [\vert \langle z,\psi _{n}\rangle \vert \leq \frac{2\delta } {a_{n}\lambda }+\epsilon \right ] =\, \text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \dfrac{2\delta } {a_{n}\lambda }+\epsilon \,\middle\vert \,a_{n}> A\right ]\,\text{Pr}\left [a_{n}> A\right ]& &{}\end{array}$$
(51)
$$\displaystyle\begin{array}{rcl} \phantom{=} +\,\text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \dfrac{2\delta } {a_{n}\lambda }+\epsilon \,\middle\vert \,a_{n} \leq A\right ]\,\text{Pr}\left [a_{n} \leq A\right ].& &{}\end{array}$$
(52)

We now bound the terms appearing in (51). Recall that λ ≥ Λδ implies that 4δ∕(Aλ) = 2ε ≤ 1∕2. By our choice of ε in (50), and using the admissibility assumption (6), for each λ ≥ Λδ one has

$$\displaystyle\begin{array}{rcl} & & \text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \dfrac{2\delta } {a_{n}\lambda }+\epsilon \,\middle\vert \,a_{n}> A\right ]\,\text{Pr}\left [a_{n}> A\right ] \\ & & \leq \,\text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \dfrac{2\delta } {A\lambda }+\epsilon \,\middle\vert \,a_{n}> A\right ]\,\text{Pr}\left [a_{n}> A\right ] \\ & & =\, \text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \dfrac{4\delta } {A\lambda }\,\middle\vert \,a_{n}> A\right ]\,\text{Pr}\left [a_{n}> A\right ] \\ & & \leq \,\text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \dfrac{4\delta } {A\lambda }\right ] \\ & & \leq \alpha \left ( \dfrac{4\delta } {A\lambda }\right )^{s}. {}\end{array}$$
(53)

To bound (52), note that by (32) one has \(\,\text{Pr}\left [a_{n} \leq A\right ] \leq CA\), and thus

$$\displaystyle\begin{array}{rcl} \,\text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \dfrac{2\delta } {a_{n}\lambda }+\epsilon \,\middle\vert \,a_{n} \leq A\right ]\,\text{Pr}\left [a_{n} \leq A\right ] \leq \,\text{Pr}\left [a \leq A\right ] \leq CA.& &{}\end{array}$$
(54)

Using the bounds (53) and (54) in (51) and (52) gives

$$\displaystyle\begin{array}{rcl} \,\text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \frac{2\delta } {a_{n}\lambda }+\epsilon \right ] \leq \alpha \left ( \dfrac{4\delta } {A\lambda }\right )^{s} + CA.& &{}\end{array}$$
(55)

Since our choice of A in (50) gives

$$\displaystyle{\alpha \left ( \dfrac{4\delta } {A\lambda }\right )^{s} = CA,}$$

we have

$$\displaystyle\begin{array}{rcl} \,\text{Pr}\left [\vert \langle z,\psi _{n}\rangle \vert \leq \frac{2\delta } {a_{n}\lambda }+\epsilon \right ] \leq 2CA = 2C\left ( \frac{\alpha } {C}\right )^{ \frac{1} {s+1} }\left (\frac{4\delta } {\lambda } \right )^{ \frac{s} {s+1} }.& &{}\end{array}$$
(56)

Thus, combining (49) and (56) gives

$$\displaystyle\begin{array}{rcl} \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]& \leq & \left (\frac{8} {\epsilon } \right )^{d-1}\left [2C\left ( \frac{\alpha } {C}\right )^{ \frac{1} {s+1} }\left (\frac{4\delta } {\lambda } \right )^{ \frac{s} {s+1} }\right ]^{N} {}\\ & =& \left (16\left ( \frac{\alpha \lambda } {4\delta C}\right )^{ \frac{1} {s+1} }\right )^{d-1}\left [2C\left ( \frac{\alpha } {C}\right )^{ \frac{1} {s+1} }\left (\frac{4\delta } {\lambda } \right )^{ \frac{s} {s+1} }\right ]^{N}. {}\\ \end{array}$$

To simplify notation, let

$$\displaystyle{ K^{{\prime}} = \left (16\left ( \frac{\alpha } {C}\right )^{ \frac{1} {s+1} }\right )^{d-1}\ \ \ \ \ \text{and} \ \ \ \ \ K^{{\prime\prime}} = 2C\left ( \frac{\alpha } {C}\right )^{ \frac{1} {s+1} }, }$$
(57)

so that

$$\displaystyle\begin{array}{rcl} \Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]& \leq & K^{{\prime}}\left ( \frac{\lambda } {4\delta }\right )^{\frac{d-1} {s+1} }\left [K^{{\prime\prime}}\left (\frac{4\delta } {\lambda } \right )^{ \frac{s} {s+1} }\right ]^{N} \\ & =& K^{{\prime}}\,(K^{{\prime\prime}})^{N}\left (\frac{4\delta } {\lambda } \right )^{\left (\frac{sN-d+1} {s+1} \right )}.{}\end{array}$$
(58)

Since 0 < s ≤ 1 and 0 < r < 1, note that (40) implies \(\left (\frac{sN-d+1} {s+1} - p + 1\right ) \geq 2\). By (58) we have

$$\displaystyle\begin{array}{rcl} & & p\int _{\varLambda \delta }^{\infty }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]d\lambda {}\\ & & \phantom{dflflglfkg} \leq pK^{{\prime}}(K^{{\prime\prime}})^{N}\int _{ \varLambda \delta }^{\infty }\lambda ^{p-1}\left (\frac{4\delta } {\lambda } \right )^{\left (\frac{sN-d+1} {s+1} \right )}d\lambda {}\\ & & \phantom{dflflglfkg} = pK^{{\prime}}(K^{{\prime\prime}})^{N}(4\delta )^{p-1}\int _{ \varLambda \delta }^{\infty }\left (\frac{4\delta } {\lambda } \right )^{\left (\frac{sN-d+1} {s+1} -p+1\right )}d\lambda {}\\ & & \phantom{dflflglfkg} = pK^{{\prime}}(K^{{\prime\prime}})^{N}(4\delta )^{p}\int _{ \varLambda /4}^{\infty }\left (\frac{1} {\lambda } \right )^{\left (\frac{sN-d+1} {s+1} -p+1\right )}d\lambda {}\\ & & \phantom{dflflglfkg} = pK^{{\prime}}(K^{{\prime\prime}})^{N}(4\delta )^{p}\left ( \frac{\varLambda } {4}\right )^{p-\frac{sN-d+1} {s+1} }\left (\frac{sN - d + 1} {s + 1} - p\right )^{-1}. {}\\ & & \phantom{dflflglfkg} \leq pK^{{\prime}}(K^{{\prime\prime}})^{N}(4\delta )^{p}\left ( \frac{\varLambda } {4}\right )^{p-\frac{sN-d+1} {s+1} } {}\\ & & \phantom{dflflglfkg} = pK^{{\prime}}(4\delta )^{p}\left ( \frac{\varLambda } {4}\right )^{p+\frac{d-1} {s+1} }\left [K^{{\prime\prime}}\left (\frac{4} {\varLambda } \right )^{ \frac{s} {s+1} }\right ]^{N}. {}\\ \end{array}$$

Since (42) implies that \(K^{{\prime\prime}}\left (\frac{4} {\varLambda } \right )^{ \frac{s} {s+1} } \leq 1/2\), it follows that

$$\displaystyle\begin{array}{rcl} p\int _{\varLambda \delta }^{\infty }\lambda ^{p-1}\Pr \left [\mathbb{S}^{d-1}\not\subset \bigcup _{ n=1}^{N}B(\psi _{ n},\epsilon _{n},a_{n}\lambda )\right ]d\lambda \leq \delta ^{p}C^{{\prime\prime\prime}}\left (\frac{1} {2}\right )^{N},& &{}\end{array}$$
(59)

where

$$\displaystyle{ C^{{\prime\prime\prime}} = pK^{{\prime}}4^{p}\left ( \frac{\varLambda } {4}\right )^{p+\frac{d-1} {s+1} }. }$$
(60)

Combining (41), (48) and (59) completes the proof.

Similar to Corollary 1, the following corollary of Theorem 2 shows that \(\mathbb{E}[(W_{N})^{p}]\) is at most of order 1∕N p when N is sufficiently large.

Corollary 2

Let {φ n } n = 1 N be as in Theorem  2 . There exist constants C 1, C 2 > 0 such that 

$$\displaystyle\begin{array}{rcl} \forall N \geq C_{1},\ \ \ \ \ \mathbb{E}\left [\left (W_{N}\right )^{p}\right ] \leq \dfrac{C_{2}\delta ^{p}} {N^{p}}.& &{}\end{array}$$
(61)

 The constants C 1, C 2 depend on α, s, C, p, d.

3.4 Numerical experiment

This section illustrates Theorem 2 with a numerical experiment.

Let x = (2, π) and \(\delta = \frac{1} {10}\). Given N ≥ 3, let \(\{e_{n}\}_{n=1}^{N} \subset \mathbb{R}^{2}\) be independent random vectors with i.i.d. N(0, 1) entries. Let {q n } n = 1 N be defined as in (1). Since there infinitely many different solutions \(\widetilde{x}\) to the consistent reconstruction condition (2), we select the minimal norm estimate by

$$\displaystyle{ \widetilde{x} =\ \text{arg min}\ _{z\in \mathbb{R}^{2}}\|z\|^{2}\ \ \ \ \ subject to \ \ \ \ \ \vert \langle z,\varphi _{ n}\rangle - q_{n}\vert \leq \delta,\ \ \ 1 \leq n \leq N. }$$
(62)

We repeat this experiment 20 times and let E(N, p) denote the average value of \(\|\widetilde{x} - x\|^{p}\). Figures 1 and 2 show log-log plots of E(N, p) versus N for p = 2 and p = 5. For comparison, these respective figures also show log-log plots of 3∕N 2 and 20∕N 5 versus N. In particular, E(N, p) appears to decay like 1∕N p, as predicted by the worst case error bounds in Theorem 2.

Fig. 1
figure 1

Log-log plot of E(N, 2) versus N, see Section 3.4.

Full size image
Fig. 2
figure 2

Log-log plot of E(N, 5) versus N, see Section 3.4.

Full size image