1 Introduction

The basic principle of GNSS positioning is the method of distance intersection. There are two ways to measure the distances between the receiver and the GNSS satellites: one is via pseudo-range code and the other is via carrier phase. Because the measurement accuracy of the carrier phase is far better than that of the pseudo-range, the observations of carrier phase are indispensable data in GNSS precision positioning. However, to obtain the distance between the receiver and the satellite, the result measured via carrier phase needs to add an unknown integer multiplied by the wavelength. The unknown integer is called ambiguity.

The observation equations of GNSS precise positioning can be expressed as the following Gauss–Markov models:

$$\begin{aligned} \left\{ {\begin{array}{l} \varepsilon =Aa+Bb-L \\ \varepsilon \sim N\left( {0,D_{LL} } \right) \\ \end{array}} \right. \end{aligned}$$
(1)

where L is the observation vector containing the double-difference (DD) observations of carrier phase; \(\varepsilon \) is the random error vector; a is the DD ambiguity vector of order n; b is all other unknown parameter vector; A and B are the corresponding design matrices; and \(D_{LL} \) is the variance matrix of L.

Solving Eq. (1) by least-squares, the float solution \(\hat{{a}}\), \(\hat{{b}}\) and the corresponding variance and covariance matrix of \(D_{\hat{{a}}\hat{{a}}} \), \(D_{\hat{{b}}\hat{{b}}} \) and \(D_{\hat{{a}}\hat{{b}}} \) can be obtained. Since ambiguity vector a, in theory, should be an integer vector, a further work should be done to obtain the fixed solution \({\mathop {a}\limits ^{\smile }}\) and \({\mathop {b}\limits ^{\smile }}\) based on the integer nature. Since \({\mathop {b}\limits ^{\smile }}\) can be obtained based on \({\mathop {a}\limits ^{\smile }}\), it is the key how to obtain \({\mathop {a}\limits ^{\smile }}\) based on the integer nature of a. The methods to obtain \({\mathop {a}\limits ^{\smile }}\) can be divided into three types: integer estimation, integer aperture estimation and integer equivariant estimation (Teunissen 2003a).

The method of integer estimation is that all \(\hat{{a}}\) in a space defined by a mapping method have the same \({\mathop {a}\limits ^{\smile }}\). The general mapping methods are rounding, bootstrapping (Blewitt 1989; Dong and Bock 1989), and integer least-squares (Teunissen 1993). Different mapping methods may have different \({\mathop {a}\limits ^{\smile }}\). To evaluate the properties of the three integer estimators, Teunissen (1998) proposed the concept of the success rate. Teunissen (1999) has proved the success rate of the integer least-squares estimator \({\mathop {a}\limits ^{\smile }}_\mathrm{ILS}\) is the largest one among the three integer estimators. Moreover, it is very expedite to obtain the integer least-squares estimator \({\mathop {a}\limits ^{\smile }}_\mathrm{ILS}\) (Li et al. 2013; Verhagen et al. 2013a). So, \({\mathop {a}\limits ^{\smile }}_\mathrm{ILS}\) is generally adopted as \({\mathop {a}\limits ^{\smile }}\).

Integer aperture estimation is a class of estimation that the result may be \({\mathop {a}\limits ^{\smile }}_\mathrm{ILS}\) or remain as the same as \(\hat{{a}}\) (Teunissen 2003b; Verhagen and Teunissen 2006). Because \({\mathop {a}\limits ^{\smile }}_\mathrm{ILS}=a\) is satisfied in probability, to control the risk of wrong ambiguities, it is necessary to validate \({\mathop {a}\limits ^{\smile }}_\mathrm{ILS}=a\) . Many validation methods have been proposed, the early methods were mainly to compare the minimum quadratic form of the residuals and the second minimum one, such as F-ratio test (Frei and Beutler 1990), R-ratio test (Euler and Schaffrin 1991), difference test (Tiberius and Jonge 1995; Zhang et al. 2015) and W-ratio test (Wang et al. 1998). Based on the success rate and the failure rate, Teunissen (2004) proposed penalized method. After controlled failure-rate method (Teunissen 2005a) was proposed, a series of validation methods of combination with controlled failure-rate were proposed, such as combined R-ratio test (Teunissen and Verhagen 2009; Verhagen and Teunissen 2013b; Wang and Feng 2013; Li et al. 2014), combined W-ratio test (Li and Wang 2014) and combined difference test (Wang and Verhagen 2015). An evaluation to some combination validation methods can be seen in Verhagen and Teunissen (2006). Specifically, Wu and Bian (2015) proposed a noteworthy validation method based on the posterior probability of \({{\mathop {a}\limits ^{\smile }}_\mathrm{ILS}=a}\).

Integer equivariant estimation is to obtain the best integer equivariant estimator \({\mathop {a}\limits ^{\smile }}_\mathrm{BIE}\) using every integer vector \(z_k \in Z^{n}\) in probability (Teunissen 2003c). Integer equivariant estimation can avoid the risk of wrong ambiguities, but before calculating \({\mathop {a}\limits ^{\smile }}_\mathrm{BIE}\) , the posterior probability of \(z_k =a\) need to be worked out in advance.

It can be seen that it is of great significance to rapidly and accurately calculate the posterior probability. For integer aperture estimation, the posterior probability of \({{\mathop {a}\limits ^{\smile }}_\mathrm{ILS}=a}\) can provide a very intuitive indicator to validate \({{\mathop {a}\limits ^{\smile }}_\mathrm{ILS}=a}\) (Wu and Bian 2015). For integer equivariant estimation, the posterior probability of \(z_k =a\) is the premise to calculate the best integer equivariant estimator \({\mathop {a}\limits ^{\smile }}_\mathrm{BIE}\) (Teunissen 2003c).

Blewitt (1989), based on Bayes’ formula, have given out the theoretical equation to calculate the posterior probability. However, because of the infinite number of integer vectors in the equation, it is impossible to use the equation directly. To have a practical method for calculating the posterior probability, Lacy et al. (2002) presented using Monte Carlo method to obtain a finite number of integer vectors for calculating the posterior probability. Because of the complex simulation, the method has the difficulty to be widely used. Teunissen (2005b) presented a searching method to obtain a finite number of integer vectors based on Chi-square distribution, which is built on the basis of explicit theory and convenient to work out the approximate value of the posterior probability of \(z_k =a\). Wu and Bian (2015) presented another method to search the integer vectors based on the lower limit defined by exponential function.

In this paper, an alternative approach to calculate the posterior probability is proposed. The proposed approach has the advantages of small amount of computations, high calculation accuracy and strong adaptability. The remaining parts of the contribution are organized as follows: in Sect. 2, the related theories and methods are briefly introduced; In Sect. 3, through dividing the infinite number of integer vectors into two subsets, the idea which aims to rapidly and accurately calculate the posterior probability is given out; In Sect. 4, the method to obtain the first subset containing a finite number of integer vectors is detailed; In Sect. 5, the method, based on the first subset, to calculate the correction factor is detailed; In Sect. 6, to be understood easily, the calculating process and formulas of the proposed approach are reviewed; In Sect. 7, several examples are given to demonstrate the performance of the proposed approach.

2 Related theories and methods

2.1 The theoretical formula for calculating posterior probability

Bayes’ formula: Suppose that \(\Omega \) is the sample space of a test, K is an event of the test, and P(K) is the probability that the event K will occur, then \(0\le P(K)\le 1\), \(P(\Omega )=1\). M is another event of the test; under the circumstance that M has occurred, the probability that K will occur is written as \(P(K\left| M \right. )\). Suppose that \(\Omega \) can be compartmentalized to \(M_1 \), \(M_2 ,\ldots , M_t \), Bayes’ formula is stated mathematically as the following equation:

$$\begin{aligned} P\left( {M_k \left| K \right. } \right) =\frac{P\left( {K\left| {M_k } \right. } \right) P\left( {M_k } \right) }{\sum \nolimits _{i=1}^t {P\left( {K\left| {M_i } \right. } \right) P\left( {M_i } \right) } } \end{aligned}$$
(2)

where \(M_k \in \left( {M_1 ,M_2 ,\ldots ,M_t } \right) \).

When carrying out precise positioning via the carrier phase of GNSS, the set that every element is probably the ambiguity a is \(\left( {z_1 ,z_2 ,\ldots ,z_\infty } \right) \), (\(z_i \in Z^{n})\) . Before the observation vector L is known, the probability of \(z_i =a\) is written as \(P\left( {z_i =a} \right) \). When \(z_k =a\), \(z_k \in \left( {z_1 ,z_2 ,\ldots ,z_\infty } \right) \), the probability that \(\hat{{a}}\) will occur is written as \(P(\hat{{a}}\left| {z_k =} \right. a)\).

According to Eq. (2), the posterior probability of \(z_k =a\) under the circumstance that \(\hat{{a}}\) has occurred can be obtained by

$$\begin{aligned} P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) =\frac{P\left( {\hat{{a}}\left| {z_k =a} \right. } \right) P\left( {z_k =a} \right) }{\sum \nolimits _{i=1}^\infty {P\left( {\hat{{a}}\left| {z_i =a} \right. } \right) P\left( {z_i =a} \right) } } \end{aligned}$$
(3)

Since \(\hat{{a}}\sim N_n \left( {a,D_{\hat{{a}}\hat{{a}}} } \right) \), Eq. (3) can be written as (Betti et al. 1993)

$$\begin{aligned} P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) = \frac{P\left( {z_k =a} \right) \exp \left[ {-\frac{1}{2} \left\| {\hat{{a}}-z_k } \right\| _{D_{\hat{a}\hat{a}}}^2 } \right] }{\sum \nolimits _{i=1}^\infty {P\left( {z_i =a} \right) \exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{a}\hat{a}}}^2 } \right] }}\nonumber \\ \end{aligned}$$
(4)

where \(\left\| \bullet \right\| _{D_{\hat{a}\hat{a}}}^2 \) stands for \(\left( \bullet \right) ^{T}{D_{\hat{a}\hat{a}}^{-1}} \left( \bullet \right) \).

Before we know the observation vector L, for \(z_1 =a\), \(z_2 =a,\ldots ,z_\infty =a\), we do not know which one is more likely to be true. So we have to believe

$$\begin{aligned} P\left( {z_1 =a} \right) =P\left( {z_2 =a} \right) =\cdots =P\left( {z_\infty =a} \right) \end{aligned}$$
(5)

Inserting Eqs. (5) into (4), the theoretical equation to calculate the posterior probability can be expressed as (Blewitt 1989; Zhu et al. 2001)

$$\begin{aligned} P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) =\frac{\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_k } \right\| _{D_{\hat{a}\hat{a}}}^2} \right] }{\sum \nolimits _{i=1}^\infty {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] } } \end{aligned}$$
(6)

Because of the infinite number of integer vectors, it is impossible to directly calculate \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) based on Eq. (6). It is the key, for working out \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \), how to deal with the infinite number of integer vectors.

2.2 Existing major practical methods

The first method introduced is Chi-square searching method, which was presented by Teunissen (2005b). The method is to define a space based on Chi-square distribution, and only the integer vectors within the space are taken to calculate the posterior probability \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \).

Given the confidence level \(1-\alpha \), a probability equation can be expressed as

$$\begin{aligned} P\left( {\left\| {\hat{a}-a} \right\| _{D_{\hat{a}\hat{{a}}}}^2 \le \lambda _n^2 } \right) =1-\alpha \end{aligned}$$
(7)

According to Chi-square distribution, \(\lambda _n^2 \) can be determined. An inequation is given as

$$\begin{aligned} \left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}} }^2 \le \lambda _n^2 \end{aligned}$$
(8)

Using searching technique, the integer vectors \(z_1 \), \(z_2, \ldots , z_t \) that satisfy Eq. (8) can be obtained. Based on the integer vectors and Eq. (6), the approximate value of \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) can be obtained by (Teunissen 2005b)

$$\begin{aligned} \hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) =\frac{\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_k } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] }{\sum \nolimits _{i=1}^t {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{a}\hat{a}}}^2}\right] }} \end{aligned}$$
(9)

where \(z_k \in \left\{ {z_1 ,z_2 ,\ldots ,z_t } \right\} \).

The second method introduced was presented by Wu and Bian (2015). In which, the space of integer vectors that are used to calculate \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) is defined as

$$\begin{aligned} S\left( z \right) =\left\{ {z_i \left| {\begin{array}{l} \exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] \\ \ge 10^{-8}\cdot \displaystyle \sum _{j=1}^{i-1} {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_j } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] } \\ \end{array}} \right. } \right\} \end{aligned}$$
(10)

Searching method is used to find \(z_i \) that satisfies Eq. (10). Using the integer vectors within \(S\left( z \right) \), similarly, \(\hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) can be calculated based on Eq. (9).

However, since the denominator in Eq. (9) is a little smaller than the one in Eq. (6), \(\hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) >P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \).

2.3 Success rate

Success rate can be expressed as (Teunissen 1999)

$$\begin{aligned}&P_{S_{\mathop {a}\limits ^{\smile }}} \left( {\mathop {a}\limits ^{\smile }=a} \right) \nonumber \\&\quad =\int _{S_{\mathop {a}\limits ^{\smile }}} {(2\pi )^{-\frac{n}{2}}\left| {D_{\hat{{a}}\hat{{a}}} } \right| ^{-\frac{1}{2}}\cdot \exp \left\{ {-\frac{1}{2}\left\| {X-a} \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right\} \mathrm{d}X} \nonumber \\&\quad =\int _{S_{\mathop {a}\limits ^{\smile }}} {(2\pi )^{-\frac{n}{2}}\left| {D_{\hat{{a}}\hat{{a}}} } \right| ^{-\frac{1}{2}}\cdot \exp \left\{ {-\frac{1}{2}\left\| {X-{\mathop {a}\limits ^{\smile }}} \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right\} \mathrm{d}X}\nonumber \\ \end{aligned}$$
(11)

where \(X=\left( {{\begin{array}{lll} {x_1 }&{} \cdots &{} {x_n } \\ \end{array} }} \right) ^{T}\), \(S_{\mathop {a}\limits ^{\smile }}\) is the space defined by the mapping method of integer estimation.

Three integer estimators have different \(S_{\mathop {a}\limits ^{\smile }}\). The relationship among their corresponding success rates can be expressed as (Teunissen 1999):

$$\begin{aligned} P_\mathrm{R} \left( {{\mathop {a}\limits ^{\smile }}_\mathrm{R} =a} \right) \le P_\mathrm{B} \left( {{\mathop {a}\limits ^{\smile }}_\mathrm{B} =a} \right) \le P_{\mathrm{ILS}} \left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}} =a}\right) \end{aligned}$$
(12)

where \({{\mathop {a}\limits ^{\smile }}_\mathrm{R}}\) is the fixed solution of a obtained by rounding, \({{\mathop {a}\limits ^{\smile }}_\mathrm{B} =a}\) is obtained by bootstrapping, and \({{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}}\) is obtained by integer least squares.

Only when \(D_{\hat{{a}}\hat{{a}}} \) is a diagonal matrix, these success rates are equal. Now, only \(P_B \left( {{\mathop {a}\limits ^{\smile }}_B =a} \right) \) can be worked out explicitly (Teunissen 1999, 2000; Verhagen 2003), and the corresponding formula reads

$$\begin{aligned} P_\mathrm{B} \left( {{\mathop {a}\limits ^{\smile }}_\mathrm{B} =a} \right) =\prod _{j=1}^n {\left[ {2\Phi \left( {\frac{1}{2\sqrt{d_{jj\left| J \right. } }}} \right) -1} \right] } \end{aligned}$$
(13)

where \(J=\left\{ {1,2,\ldots ,j-1} \right\} \), and \(d_{jj\left| J \right. } \) is the variance of the jth ambiguity based on the case that the first \(j-1\) ambiguities are known.

Posterior probability \(P(z_k =a\left| {\hat{{a}}} \right. )\) and success rate \(P_{S_{{\mathop {a}\limits ^{\smile }}}} \left( {{\mathop {a}\limits ^{\smile }}=a} \right) \) are different. On the one hand, the result of \(P(z_k =a\left| {\hat{{a}}} \right. )\) depends on the float solution \(\hat{{a}}\), but the result of \(P_{S_{{\mathop {a}\limits ^{\smile }}}} \left( {{\mathop {a}\limits ^{\smile }}=a} \right) \) does not; on the other hand, for any integer vector \(z_k \in Z^{n}\), it has the corresponding \(P(z_k =a\left| {\hat{{a}}} \right. )\), but only \({{\mathop {a}\limits ^{\smile }}_\mathrm{R}}\), \({{{\mathop {a}\limits ^{\smile }}_\mathrm{B}}}\) and \({{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}}\) have the corresponding \(P_{S_{{\mathop {a}\limits ^{\smile }}}} \left( {{\mathop {a}\limits ^{\smile }}=a} \right) \). In Sect. 7, a numerical comparison between \(P(z_k =a\left| {\hat{{a}}} \right. )\) and \(P_{S_{\mathop {a}\limits ^{\smile }}} \left( {{\mathop {a}\limits ^{\smile }}=a} \right) \) will be carried out to show their differences.

3 Calculation strategy

The set of integers \(Z^{n}\) is divided into two subsets, one consists of \(z_1 \), \(z_2 , \ldots z_m \) and the other consists of \(z_{m+1} \), \(z_{m+2},\ldots z_\infty \). Equation (6) can be expressed as

$$\begin{aligned} P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) =\frac{\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_k } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] }{\sum \nolimits _{i=1}^m {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] } }P_{m/\infty } \end{aligned}$$
(14)

with

$$\begin{aligned} P_{m/\infty } =\frac{\sum \nolimits _{i=1}^m {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] } }{\sum \nolimits _{i=1}^\infty {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] } } \end{aligned}$$
(15)

\(P_{m/\infty } \) is called as correction factor. For convenient expression, let

$$\begin{aligned} K_{1\sim m} =\sum _{i=1}^m {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}} }^2 } \right] } \end{aligned}$$
(16)

and

$$\begin{aligned} K_{\left( {m+1} \right) \sim \infty } =\sum _{i=m+1}^\infty {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}} }^2 } \right] } \end{aligned}$$
(17)

Eq. (15) can be written as

$$\begin{aligned} P_{m/\infty } =\frac{K_{1\sim m} }{K_{1\sim m} +K_{\left( {m+1} \right) \sim \infty } } \end{aligned}$$
(18)

Obviously, if \(z_1 \), \(z_2,\ldots z_m \) and \(K_{\left( {m+1} \right) \sim \infty }\) are known, \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) can be easily worked out based on the above formulas. A flexible and straightforward method to obtain \(z_1 \), \(z_2,\ldots z_m \) and a matched method to calculate \(P_{m/\infty } \) based on \(z_1 \), \(z_2, \ldots z_m \) will be given out.

4 Determining the m integer vectors

4.1 Ambiguity decorrelation

In the least-squares ambiguity decorrelation approach (LAMBDA) (Teunissen 1993, 1995a), to decrease the search space, the variance matrix \(D_{\hat{{a}}\hat{{a}}} \) is transformed via decorrelation. Similarly, to instantaneously obtain \(z_1 \), \(z_2 ,\ldots z_m \), the decorrelation approach will also be adopted.

According to \(D_{\hat{{a}}\hat{{a}}} \), an integer square matrix Z can be obtained, every element of Z and \(Z^{T}\) is an integer, and \(\left| Z \right| =\pm 1\) (Teunissen 1995b). There are many methods to obtain Z, for example, Liu et al. (1999), Xu (2001), Lou and Grafarend (2003), Liu and He (2007), Zhou (2011), Zhang et al. (2011), and Zhou and He (2014). Using Z to transform \(D_{\hat{{a}}\hat{{a}}} \), a real symmetric matrix whose main diagonal is dominant can be obtained by

$$\begin{aligned} D_{\hat{{a}}\hat{{a}},Z} =Z^{T}D_{\hat{{a}}\hat{{a}}} Z \end{aligned}$$
(19)

Accordingly, several vectors can be transformed by

$$\begin{aligned} \hat{{a}}_Z =Z^{T}\hat{{a}} \end{aligned}$$
(20)
$$\begin{aligned} a_Z =Z^{T}a \end{aligned}$$
(21)

and

$$\begin{aligned} z_{i,Z} =Z^{T}z_i \end{aligned}$$
(22)

Since \(\hat{{a}}\sim N_n \left( {a,D_{\hat{{a}}\hat{{a}}} } \right) \), furthermore, we have

$$\begin{aligned} \hat{{a}}_Z \sim N_n \left( {a_Z ,D_{\hat{{a}}\hat{{a}},Z} } \right) \end{aligned}$$
(23)

4.2 Determining the integer subset

Let

$$\begin{aligned} X_Z \sim N_n \left( {a_Z ,D_{\hat{{a}}\hat{{a}},Z} } \right) \end{aligned}$$
(24)

where \(X_Z =\left( {{\begin{array}{lll} {x_{Z1} }&{} \cdots &{} {x_{Zn} } \\ \end{array} }} \right) ^{T}\).

Under the circumstance that \(\hat{{a}}\) has occurred, \(\hat{{a}}\) and \(\hat{{a}}_Z \) are two constant vectors. A following space is assigned as

$$\begin{aligned} S_{\hat{{a}}_Z } =\left\{ {\begin{array}{l} \left[ {\hat{{a}}_Z (1)} \right] -\frac{1}{2}\le x_{Z1} \le \left[ {\hat{{a}}_Z (1)} \right] +\frac{1}{2} \\ \cdots \\ \left[ {\hat{{a}}_Z (n)} \right] -\frac{1}{2}\le x_{Zn} \le \left[ {\hat{{a}}_Z (n)} \right] +\frac{1}{2} \\ \end{array}} \right\} \end{aligned}$$
(25)

where \(\hat{{a}}_Z (j)\) is the j th component of \(\hat{{a}}_Z \), \(\left[ \bullet \right] \) denotes rounding.

Let

$$\begin{aligned} Y=a_Z +\left[ {\hat{{a}}_Z } \right] -X_Z \end{aligned}$$
(26)

where \(Y=\left( {{\begin{array}{lll} {y_1 }&{} \cdots &{} {y_n } \\ \end{array} }} \right) ^{T}\).

Inserting Eqs. (26) into (24), we have

$$\begin{aligned} Y\sim N_n \left( {\left[ {\hat{{a}}_Z } \right] ,D_{\hat{{a}}\hat{{a}},Z} } \right) \end{aligned}$$
(27)

When \(z_{i,Z} =a_Z \), inserting Eqs. (26) into (25), the space \(S_{\hat{{a}}_Z}\) can be expressed as

$$\begin{aligned} \begin{array}{l} S_{z_{i,Z} } \,=\left\{ {S_{\hat{{a}}_Z } \left| {z_{i,Z} =a_Z } \right. } \right\} \\ \quad \qquad =\left\{ {\begin{array}{l} z_{i,Z} (1)-\frac{1}{2}\le y_1 \le z_{i,Z} (1)+\frac{1}{2} \\ \cdots \\ z_{i,Z} (n)-\frac{1}{2}\le y_n \le z_{i,Z} (n)+\frac{1}{2} \\ \end{array}} \right\} \\ \end{array} \end{aligned}$$
(28)

Noting that \(S_{z_{i,Z} } \) stands for \(S_{\hat{{a}}_Z } \) with \(z_{i,Z} =a_Z \). According to Eq. (28), a larger space can be expressed as

$$\begin{aligned}&S_{1\sim m} =\displaystyle \bigcup _{i=1}^m {S_{z_{i,Z} } } \nonumber \\&\qquad =\left\{ {\begin{array}{l} \min \left[ {z_{i,Z} \left( 1 \right) } \right] -\frac{1}{2}\le y_1 \le \max \left[ {z_{i,Z} \left( 1 \right) } \right] +\frac{1}{2} \\ \cdots \\ \min \left[ {z_{i,Z} \left( n \right) } \right] -\frac{1}{2}\le y_n \le \max \left[ {z_{i,Z} \left( n \right) } \right] +\frac{1}{2} \\ \end{array}} \right\} \nonumber \\ \end{aligned}$$
(29)

The space \(S_{1\sim m} \) is designated as a centrosymmetric space to \(\left[ {\hat{{a}}_Z } \right] \), we have

$$\begin{aligned} \max \left[ {z_{i,Z} \left( j \right) } \right] =2\left[ {\hat{{a}}_Z \left( j \right) } \right] -\min \left[ {z_{i,Z} \left( j \right) } \right] \end{aligned}$$
(30)

According to Eq. (30), Eq. (29) can be written as

$$\begin{aligned}&S_{1\sim m} \nonumber \\&\quad =\left\{ {\begin{array}{l} \min \left[ {z_{i,Z} \left( 1 \right) } \right] -\frac{1}{2}\le y_1 \le 2\left[ {\hat{{a}}_Z \left( 1 \right) } \right] -\min \left[ {z_{i,Z} \left( 1 \right) } \right] +\frac{1}{2} \\ \cdots \\ \min \left[ {z_{i,Z} \left( n \right) } \right] -\frac{1}{2}\le y_n \le 2\left[ {\hat{{a}}_Z \left( n \right) } \right] -\min \left[ {z_{i,Z} \left( n \right) } \right] +\frac{1}{2} \\ \end{array}} \right\} \nonumber \\ \end{aligned}$$
(31)

To guarantee the representative of \(z_1 \), \(z_2 ,\ldots z_m \), according to Eq. (27), the first constraint condition can be expressed as

$$\begin{aligned} \min \left[ {z_{i,Z} \left( j \right) } \right] \le \left[ {\hat{{a}}_Z \left( j \right) } \right] +\frac{1}{2}-c\sqrt{d_{jj,Z} } \end{aligned}$$
(32)

with

$$\begin{aligned} \int _{-c}^c {\frac{1}{\sqrt{2\pi }}} \exp \left( {-\frac{x^{2}}{2}} \right) \mathrm{d}x=1-\alpha \end{aligned}$$
(33)

where \(1-\alpha \) denotes the confidence level, \(d_{jj,Z} \) is the \(j\hbox {th}\) diagonal element of \(D_{\hat{{a}}\hat{{a}},Z} \).

According to Eq. (32), when \(c\sqrt{d_{jj,Z} }\!\le \! 0.5\), \(\min \left[ {z_{i,Z} \left( j \right) } \right] \) may be equal to \(\left[ {\hat{{a}}_Z \left( j \right) } \right] \). If \(\min \left[ {z_{i,Z} \left( j \right) } \right] =\left[ {\hat{{a}}_Z \left( j \right) } \right] \), according to Eq. (31), the \(j\hbox {th}\) component of \(S_{1\sim m} \) has only one integer \(\left[ {\hat{{a}}_Z \left( j \right) } \right] \). Under the circumstances, it is difficult to accurately calculate the correction factor \(P_{m/\infty } \), which may lead to an inaccurate result of \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) based on Eq. (14). To avoid the problem, the second constraint condition can be written as

$$\begin{aligned} \min \left[ {z_{i,Z} \left( j \right) } \right] \le \left[ {\hat{{a}}_Z \left( j \right) } \right] -1 \end{aligned}$$
(34)

The number of \(\min \left[ {z_{i,Z} \left( j \right) } \right] \) that satisfy Eqs. (32) and (34) is infinite. To keep the minimal calculation workload of \(K_{1\sim m}\), the third constraint condition reads

$$\begin{aligned} \min \left[ {z_{i,Z} \left( j \right) } \right] =\arg \mathop {\min }\limits _{1\le i\le m} \left\{ {\left[ {\hat{{a}}_Z \left( j \right) } \right] -\min \left[ {z_{i,Z} \left( j \right) } \right] } \right\} \end{aligned}$$
(35)

Based on Eqs. (32), (34) and (35), \(\min \left[ {z_{i,Z} \left( j \right) } \right] \) in Eq. (31) can be obtained by

$$\begin{aligned} \left\{ \!\begin{array}{ll} {\min \left[ {z_{i,Z} \left( j \right) } \right] =\left\lfloor {\left[ {\hat{{a}}_Z \left( j \right) } \right] +\frac{1}{2}-c\sqrt{d_{jj,Z} }} \right\rfloor }&{}\quad {c\sqrt{d_{jj,Z} }>\frac{1}{2}} \\ \\ {\min \left[ {z_{i,Z} \left( j \right) } \right] =\left[ {\hat{{a}}_Z \left( j \right) } \right] -1}&{}\quad {c\sqrt{d_{jj,Z} }\le \frac{1}{2}} \\ \end{array} \right. \nonumber \\ \end{aligned}$$
(36)

where \(\left\lfloor \bullet \right\rfloor \) denotes rounding down.

4.3 Integer combination

According to the value range defined by Eqs. (31) and (36), the integer vectors \(z_{1,Z} \), \(z_{2,Z},\ldots ,z_{m,Z} \) can be obtained via combination. Furthermore, the integer vectors that are probably DD ambiguity vector a can be obtained by

$$\begin{aligned} \left\{ {\begin{array}{l} z_1 =\left( {Z^{T}} \right) ^{-1}z_{1,Z} \\ \cdots \\ z_m =\left( {Z^{T}} \right) ^{-1}z_{m,Z} \\ \end{array}} \right. \end{aligned}$$
(37)

5 Calculating the correction factor

5.1 Equation transformation

To calculate the correction factor \(P_{m/\infty } \) expediently, Eq. (18) need to be transformed from based on \(z_1 \), \(z_2 ,\ldots z_\infty \) to only based on \(z_1 \), \(z_2 ,\ldots z_m \). The complementary space of \(S_{1\sim m} \) can be expressed as

$$\begin{aligned} S_{\left( {m+1} \right) \sim \infty } =\sum _{i=m+1}^\infty {S_{z_{i,Z} } } =R^{n}-S_{1\sim m} \end{aligned}$$
(38)

For \(\forall z_{i,Z} \), obviously, the volume of \(S_{z_{i,Z} } \) is equal to the one of \(S_{\hat{{a}}_Z } \), which can be expressed as

$$\begin{aligned} V_S =\int _{S_{\hat{{a}}_Z } } \mathrm{d} X=\int _{\left[ {\hat{{a}}_Z (1)} \right] -\frac{1}{2}}^{\left[ {\hat{{a}}_Z (1)} \right] +\frac{1}{2}} \cdots \int _{\left[ {\hat{{a}}_Z (n)} \right] -\frac{1}{2}}^{\left[ {\hat{{a}}_Z (n)} \right] +\frac{1}{2}} {\mathrm{d}x_1 \cdots \mathrm{d}x_n } =1 \end{aligned}$$
(39)

The occurring probability of \(S_{z_{i,Z} } \) can be expressed as

$$\begin{aligned} P\left( {S_{z_{i,Z} } } \right) =\int _{S_{z_{i,Z} } } {R\cdot \exp \left\{ {-\frac{1}{2}\left\| {Y-\left[ {\hat{{a}}_Z } \right] } \right\| _{D_{\hat{{a}}\hat{{a}},Z} }^2 } \right\} dY} \end{aligned}$$
(40)

where \(R=(2\pi )^{-\frac{n}{2}}\left| {D_{\hat{{a}}\hat{{a}},Z} } \right| ^{-\frac{1}{2}}\).

For \(\forall S_{z_{i,Z} } \in S_{\left( {m+1} \right) \sim \infty } \), the approximate probability that \(S_{z_{i,Z} } \) will occur can be expressed as

$$\begin{aligned} \bar{{P}}\left( {S_{z_{i,Z} } } \right) =V_S \cdot R\cdot \exp \left[ {-\frac{1}{2}\left\| {z_{i,Z} -\hat{{a}}_Z } \right\| _{D_{\hat{{a}}\hat{{a}},Z}}^2 } \right] \end{aligned}$$
(41)

When the space \(S_{1\sim m} \) is large enough, based on Eqs. (38), (40) and (41), an approximate equation can be expressed as

$$\begin{aligned} \sum _{i=m+1}^\infty {\bar{{P}}\left( {S_{z_{i,Z} } } \right) }= & {} \sum _{i=m+1}^\infty {P\left( {S_{z_{i,Z} } } \right) } \nonumber \\= & {} P\left( {S_{\left( {m+1} \right) \sim \infty } } \right) =1-P\left( {S_{1\sim m} } \right) \end{aligned}$$
(42)

where \(P\left( {S_{\left( {m+1} \right) \sim \infty } } \right) \) denotes the occurring probability of \(S_{\left( {m+1} \right) \sim \infty } \), and \(P\left( {S_{1\sim m} } \right) \) denotes the one of \(S_{1\sim m} \).

Proof see Appendix.

According to Eqs. (39), (41) and (42), Eq. (17) can be expressed as

$$\begin{aligned} K_{\left( {m+1} \right) \sim \infty }= & {} \sum _{i=m+1}^\infty {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}_Z -z_{i,Z} } \right\| _{D_{\hat{{a}}\hat{{a}},Z}}^2 } \right] } \nonumber \\= & {} \sum _{i=m+1}^\infty {\frac{1}{V_S \cdot R}\bar{{P}}\left( {S_{z_{i,Z} } } \right) } =\frac{1}{R}\left[ {1-P\left( {S_{1\sim m} } \right) } \right] \nonumber \\ \end{aligned}$$
(43)

Inserting Eqs. (43) into (18), yields

$$\begin{aligned} P_{m/\infty } =\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P\left( {S_{1\sim m} } \right) } \end{aligned}$$
(44)

When \(1-\alpha \ge 0.997\) in Eq. (33), namely \(c\ge 3\), the error of Eq. (44), which is caused by the approximation of Eq. (42), can be neglected in general.

5.2 Valuation and error

According to Eq. (44), for calculating \(P_{m/\infty } \), \(P\left( {S_{1\sim m} } \right) \) should be worked out in advance, which can be expressed as

$$\begin{aligned} P\left( {S_{1\sim m} } \right) =\int _{S_{1\sim m} } {R\cdot \exp \left\{ {-\frac{1}{2}\left\| {Y-\left[ {\hat{{a}}_Z } \right] } \right\| _{D_{\hat{{a}}\hat{{a}},Z}}^2 } \right\} \mathrm{d}Y} \end{aligned}$$
(45)

Since \(D_{\hat{{a}}\hat{{a}},Z} \) is not a diagonal matrix, it is discommodious to work out \(P\left( {S_{1\sim m} } \right) \) based on Eq. (45). Referring to Eq. (14) in Teunissen (1998), the lower bound of \(P\left( {S_{1\sim m} } \right) \) can be obtained by

$$\begin{aligned}&P\left( {S_{1\sim m} } \right) \ge P_\Lambda \left( {S_{1\sim m} } \right) \nonumber \\&\quad = \prod _{j=1}^n {\left[ {2\Phi \left( {\frac{\left[ {\hat{{a}}_Z \left( j \right) } \right] -\min \left[ {z_{i,Z} \left( j \right) } \right] +\frac{1}{2}}{\sqrt{d_{jj,Z} }}} \right) -1} \right] } \end{aligned}$$
(46)

where \(\Phi \left( s \right) =\int _{-\infty }^s {\frac{1}{\sqrt{2\pi }}} \exp \left( {-\frac{x^{2}}{2}} \right) \mathrm{d}x\).

According to Eq. (44), the lower bound of the correction factor \(P_{m/\infty } \) can be obtained by

$$\begin{aligned} P_{m/\infty } \ge \frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_\Lambda \left( {S_{1\sim m} } \right) } \end{aligned}$$
(47)

Referring to Eq. (20) in Teunissen (1998), an upper bound of \(P\left( {S_{1\sim m} } \right) \) can be obtained by

$$\begin{aligned}&P\left( {S_{1\sim m} } \right) \le P_B \left( {S_{1\sim m} } \right) \nonumber \\&\quad =\prod _{j=1}^n {\left[ {2\Phi \left( {\frac{\left[ {\hat{{a}}_Z \left( j \right) } \right] -\min \left[ {z_{i,Z} \left( j \right) } \right] +\frac{1}{2}}{\sqrt{d_{jj\left| {J,Z} \right. } }}} \right) -1} \right] } \end{aligned}$$
(48)

where \(J=\left\{ {1,2,\ldots ,j-1} \right\} \), and \(d_{jj\left| J \right. ,Z} \) is the variance of \(y_j \) based on the case that \(\left( {{\begin{array}{lll} {y_1 }&{} \cdots &{} {y_{j-1} } \\ \end{array} }} \right) ^{T}\) are known.

Accordingly, the upper bound of \(P_{m/\infty } \) can be obtained by

$$\begin{aligned} P_{m/\infty } \le \frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_B \left( {S_{1\sim m} } \right) } \end{aligned}$$
(49)

According to Eqs. (47) and (49), the correction factor \(P_{m/\infty } \) can be estimated by

$$\begin{aligned}&\hat{{P}}_{m/\infty }\nonumber \\&\quad = \frac{1}{2}\left[ {\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_\Lambda \left( {S_{1\sim m} } \right) }+\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_B \left( {S_{1\sim m} } \right) }} \right] \nonumber \\ \end{aligned}$$
(50)

The error upper bound of \(\hat{{P}}_{m/\infty } \) can be obtained by

$$\begin{aligned}&\Delta \hat{{P}}_{m/\infty }\le \bar{\Delta } \hat{{P}}_{m/\infty } \nonumber \\&\quad = \frac{1}{2}\left( {\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_B \left( {S_{1\sim m} } \right) }-\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_\Lambda \left( {S_{1\sim m} } \right) }} \right) \nonumber \\ \end{aligned}$$
(51)

6 The calculation process in practice

(a) The first step is to realize the ambiguity decorrelation. Based on Eq. (1), the float solution \(\hat{{a}}\) of DD ambiguity and the corresponding variance matrix \(D_{\hat{{a}}\hat{{a}}} \) are obtained. Further, an integer transformation matrix Z is obtained based on \(D_{\hat{{a}}\hat{{a}}} \). Then, \(\hat{{a}}\) and \(D_{\hat{{a}}\hat{{a}}} \) can be transformed by

$$\begin{aligned} \hat{{a}}_Z =Z^{T}\hat{{a}} \end{aligned}$$
(52)

and

$$\begin{aligned} D_{\hat{{a}}\hat{{a}},Z} =Z^{T}D_{\hat{{a}}\hat{{a}}} Z \end{aligned}$$
(53)

(b) The second step is to determine m integer vectors. Adopting an appropriate c (in general, \(c=3\) is adopted), the lower bound of every component can be calculated by

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} \min \left[ {z_{i,_Z} \left( j \right) } \right] =\left\lfloor {\left[ {\hat{{a}}_{_Z} \left( j \right) } \right] +\frac{1}{2}-c\sqrt{d_{jj,Z} }} \right\rfloor &{}{c\sqrt{d_{jj,Z} }>\frac{1}{2}}\\ \\ \min \left[ {z_{i,_Z} \left( j \right) } \right] =\left[ {\hat{{a}}_{_Z} \left( j \right) } \right] -1&{} {c\sqrt{d_{jj,Z} }\le \frac{1}{2}} \end{array}\right. \nonumber \\ \end{aligned}$$
(54)

where \(\left\lfloor \bullet \right\rfloor \) denotes rounding down.

The upper bound of every component can be obtained by

$$\begin{aligned} \max \left[ {z_{i,Z} \left( j \right) } \right] =2\left[ {\hat{{a}}_Z \left( j \right) } \right] -\min \left[ {z_{i,Z} \left( j \right) } \right] \end{aligned}$$
(55)

According to the range of every component, the integer vectors \(z_{1,Z} \), \(z_{2,Z}, \ldots , z_{m,Z} \) can be easily obtained via combination. Furthermore, the adopted integer vectors that are probably DD ambiguity vector can be obtained by

$$\begin{aligned} z_i =\left( {Z^{T}} \right) ^{-1}z_{i,Z} \quad z_{i,Z} \in \left( {z_{1,Z} ,z_{2,Z} ,\ldots ,z_{m,Z} } \right) \end{aligned}$$
(56)

(c) The third step is to calculate the correction factor \(P_{m/\infty } \). Using \(z_1 ,z_2 ,\ldots ,z_m \), \(K_{1\sim m} \) can be obtained by

$$\begin{aligned} K_{1\sim m} =\sum _{i=1}^m {\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_i } \right\| _{D_{\hat{{a}}\hat{{a}}} }^2 } \right] } \end{aligned}$$
(57)

The lower bound and the upper bound of \(P\left( {S_{1\sim m} } \right) \) can be obtained by

$$\begin{aligned}&P_\Lambda \left( {S_{1\sim m} } \right) \nonumber \\&\quad =\prod _{j=1}^n {\left[ {2\Phi \left( {\frac{\left[ {\hat{{a}}_Z \left( j \right) } \right] -\min \left[ {z_{i,Z} \left( j \right) } \right] +\frac{1}{2}}{\sqrt{d_{jj,Z} }}} \right) -1} \right] } \end{aligned}$$
(58)

and

$$\begin{aligned}&P_B \left( {S_{1\sim m} } \right) \nonumber \\&\quad = \prod _{j=1}^n {\left[ {2\Phi \left( {\frac{\left[ {\hat{{a}}_Z \left( j \right) } \right] -\min \left[ {z_{i,Z} \left( j \right) } \right] +\frac{1}{2}}{\sqrt{d_{jj\left| {J,Z} \right. } }}} \right) -1} \right] } \end{aligned}$$
(59)

The estimation value of \(P_{m/\infty } \) can be worked out by

$$\begin{aligned} \begin{array}{l} \displaystyle \hat{{P}}_{m/\infty } \\ \displaystyle \quad = \frac{1}{2}\left[ {\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_\Lambda \left( {S_{1\sim m} } \right) }+\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_B \left( {S_{1\sim m} } \right) }} \right] \\ \end{array}\nonumber \\ \end{aligned}$$
(60)

The error upper bound of \(\hat{{P}}_{m/\infty } \) can be obtained by

$$\begin{aligned} \begin{array}{l} \displaystyle \bar{\Delta } \hat{{P}}_{m/\infty } \\ \displaystyle \quad = \frac{1}{2}\left( {\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_B \left( {S_{1\sim m} } \right) }-\frac{R\cdot K_{1\sim m} }{R\cdot K_{1\sim m} +1-P_\Lambda \left( {S_{1\sim m} } \right) }} \right) \\ \end{array}\nonumber \\ \end{aligned}$$
(61)

(d) The fourth step is to calculate the posterior probabilities. The probability of \(z_k =a\) with the known \(\hat{{a}}\) can be calculated by

$$\begin{aligned} \hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) =\frac{\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_k } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] }{K_{1\sim m} }\hat{{P}}_{m/\infty } \end{aligned}$$
(62)

The error upper bound of \(\hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) can be obtained by

$$\begin{aligned} \Delta \hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \le \frac{\exp \left[ {-\frac{1}{2}\left\| {\hat{{a}}-z_k } \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 } \right] }{K_{1\sim m} }\bar{\Delta } \hat{{P}}_{m/\infty } \end{aligned}$$
(63)

Specially, because \(\left\| {\hat{{a}}-{{{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}}}} \right\| _{D_{\hat{{a}}\hat{{a}}}}^2 \) is the smallest one, \(\hat{{P}}\left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}=a\left| {\hat{{a}}} \right. } \right) \) is the largest one. Obviously, \(\hat{{P}}\left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}=a\left| {\hat{{a}}} \right. } \right) \) and \(\Delta \hat{{P}}\left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}=a \left| {\hat{{a}}} \right. } \right) \) can be calculated using Eqs. (62) and (63). Because \({{\mathop {a}\limits ^{\smile }}_R} ,{{\mathop {a}\limits ^{\smile }}_B} \in Z^{n}\), obviously, \(\hat{{P}}\left( {{\mathop {a}\limits ^{\smile }} _{R}=a\left| {\hat{{a}}} \right. } \right) \), \(\hat{{P}}\left( {{\mathop {a}\limits ^{\smile }}_{B}=a\left| {\hat{{a}}} \right. } \right) \) and their corresponding error upper bounds can also be expediently worked out based on Eqs. (62) and (63).

7 Example of verification

In this case, the baseline is about 500 m long. Static relative measurement was implemented by two GPS receivers, and the epoch interval was 5 s. Referring to Eq. (1), observation equations were established, and based on least-square, the float solution \(\hat{{a}}\) of DD ambiguity and its variance matrix \({D_{\hat{{a}}\hat{{a}}}} \) were worked out.

7.1 Computation process and performance

(a) Taking the DD observations of the carrier phase of 120 epochs in 10 min as L, the float solution vector and its variance matrix are

$$\begin{aligned} \hat{{a}}= & {} \left( 6.404272\quad 14.122925\quad 12.627364\quad -0.001465\quad \right. \\&\left. -3.447006\quad 2.309204\right) ^{T} \end{aligned}$$

and

$$\begin{aligned}&D_{\hat{{a}}\hat{{a}}}\\&\quad \!=\!\left( {{\begin{array}{r@{\quad }r@{\quad }r@{\quad }r@{\quad }r@{\quad }r@{\quad }l} {{373.294}}&{} {{257.933}}&{} {{-195.394}}&{} {\hbox {290.875}}&{} {{200.985}}&{} {{-152.257}} \\ {\hbox {257.933}}&{} {\hbox {178.269}}&{} {{-134.960}}&{} {\hbox {200.985}}&{} {\hbox {138.908}}&{} {{-105.165}} \\ {{-195.394}}&{} {{-134.960}}&{} {\hbox {103.973}}&{} {{-152.257}}&{} {{-105.165}}&{} {\hbox {81.015}} \\ {\hbox {290.875}}&{} {\hbox {200.985}}&{} {{-152.257}}&{} {\hbox {226.660}}&{} {\hbox {156.614}}&{} {{-118.640}} \\ {\hbox {200.985}}&{} {\hbox {138.908}}&{} {{-105.165}}&{} {\hbox {156.614}}&{} {\hbox {108.244}}&{} {{-81.945}} \\ {{-152.257}}&{} {{-105.165}}&{} {\hbox {81.015}}&{} {{-118.640}}&{} {{-81.945}}&{} {\hbox {63.133}} \\ \end{array} }} \right) \end{aligned}$$

To clearly illustrate the proposed approach, the calculation process was given step by step as follows.

The first step is to realize the ambiguity decorrelation. Based on \(D_{\hat{{a}}\hat{{a}}} \), we gained the integer transformation matrix

$$\begin{aligned} Z=\left( {{\begin{array}{r@{\quad }r@{\quad }r@{\quad }r@{\quad }r@{\quad }r} {{-3}}&{} {2}&{} {0}&{} {1}&{} {1}&{} {2} \\ {2}&{} {{-2}}&{} {{-2}}&{} {{-3}}&{} {2}&{} {1} \\ {{-1}}&{} {{-1}}&{} {0}&{} {0}&{} {0}&{} {{-3}} \\ {4}&{} {{-3}}&{} {{-1}}&{} {0}&{} {{-1}}&{} {{-2}} \\ {{-3}}&{} {3}&{} {4}&{} {2}&{} {{-3}}&{} {{-2}} \\ {1}&{} {1}&{} {0}&{} {0}&{} {0}&{} {4} \\ \end{array} }} \right) \end{aligned}$$

Then, the decorrelated ambiguity vector \(\hat{{a}}_Z \) and the corresponding variance matrix \(D_{\hat{{a}}\hat{{a}},Z} \) were obtained by Eqs. (52) and (53).

The second step is to determine m integer vectors. Adopting \(c=3\), according to Eqs. (54) and (55), the minimum integer value and the maximum one of every component of \(S_{1\sim m} \) can be obtained, which are

$$\begin{aligned} \begin{array}{l} \min \left[ {z_{i,Z} \left( 1 \right) } \right] =8, \;\;\;\;\;\;\;\;\;\;\;\;\max \left[ {z_{i,Z} \left( 1 \right) } \right] =10 \\ \min \left[ {z_{i,Z} \left( 2 \right) } \right] =-37, \;\;\;\;\;\;\;{}\max \left[ {z_{i,Z} \left( 2 \right) } \right] =-35 \\ \min \left[ {z_{i,Z} \left( 3 \right) } \right] =-43, \;\;\;\;\;\;\;\;\max \left[ {z_{i,Z} \left( 3 \right) } \right] =-41 \\ \min \left[ {z_{i,Z} \left( 4 \right) } \right] =-44, \;\;\;\;\;\;\;{}{}\max \left[ {z_{i,Z} \left( 4 \right) } \right] =-42 \\ \min \left[ {z_{i,Z} \left( 5 \right) } \right] =44, \;\;\;\;\;\;\;\;\;\;{}\max \left[ {z_{i,Z} \left( 5 \right) } \right] =46 \\ \min \left[ {z_{i,Z} \left( 6 \right) } \right] =4, \;\;\;\;\;\;\;\;\;\;\;\;\max \left[ {z_{i,Z} \left( 6 \right) } \right] =6 \\ \end{array} \end{aligned}$$

Via combination, 729 integer vectors, namely \(z_{1,Z} \), \(z_{2,Z},\ldots ,z_{729,Z} \), were obtained. Furthermore, the adopted integer vectors that are probably DD ambiguity vector, namely \(z_1 \), \(z_2, \ldots , z_{729} \), can be obtained by Eq. (56).

Table 1 The top ten integer vectors and their posterior probabilities based on the observations of 120 epochs
Full size table
Table 2 The top ten integer vectors and their posterior probabilities based on the observations of 384 epochs
Full size table

The third step is to calculate the correction factor \(P_{m/\infty } \). Based on Eq. (57), we have

$$\begin{aligned} K_{1\sim m} =\hbox {0.987995391211628} \end{aligned}$$

According to Eq. (58), the lower bound of \(P\left( {S_{1\sim m} } \right) \) reads

$$\begin{aligned} P_\Lambda \left( {S_{1\sim m} } \right) \hbox {=0.998813975481466} \end{aligned}$$

According to \(D_{\hat{{a}}\hat{{a}},Z} \), the variance of \(y_j \), based on the case that \(\left( {{\begin{array}{lll} {y_1 }&{} \cdots &{} {y_{j-1} } \\ \end{array} }} \right) ^{T}\) are known, can be obtained as follows

$$\begin{aligned}&d_{11\left| {J,Z} \right. } =\hbox {0.190000000001078}\quad d_{22\left| {J,Z} \right. } =\hbox {0.173263157895045}\\&d_{33\left| {J,Z} \right. } =\hbox {0.129425577156756}\quad d_{44\left| {J,Z} \right. } =\hbox {0.116314834921560}\\&d_{55\left| {J,Z} \right. } =\hbox {0.085889475191953}\quad d_{66\left| {J,Z} \right. } =\hbox {0.092514336521617} \end{aligned}$$

According to Eq. (59), the upper bound of \(P\left( {S_{1\sim m} } \right) \) is

$$\begin{aligned} P_B \left( {S_{1\sim m} } \right) =\hbox {0.999064762343891} \end{aligned}$$

According to Eq. (60), the estimation value of \(P_{m/\infty } \) is

$$\begin{aligned} \hat{{P}}_{m/\infty } =\hbox {0.999471860778787} \end{aligned}$$

According to Eq. (61), the error upper bound of \(\hat{{P}}_{m/\infty } \) is

$$\begin{aligned} \bar{\Delta } \hat{{P}}_{m/\infty } = 0.0000\hbox {62406902962} \end{aligned}$$

The fourth step is to calculate the posterior probabilities of \(z_1 =a\), \(z_2 =a, \ldots , z_{729} =a\) using Eqs. (62) and (63). The top ten integer vectors and their corresponding posterior probabilities are listed in Table 1.

(b) Taking the DD observations of the carrier phase of 384 epochs in 32 min as L, the float solution vector and its variance matrix are

$$\begin{aligned} \hat{{a}}= & {} \left( {\hbox {8.514432}}\quad {\hbox {15.653847}}\quad {\hbox {11.231013 }}\quad {\hbox {1.632104}}\quad \right. \\&\left. -2\,.256995 \quad {\hbox {1.196251}} \right) ^{T} \end{aligned}$$

and

$$\begin{aligned} D_{\hat{{a}}\hat{{a}}}=\left( {{\begin{array}{r@{\quad }r@{\quad }r@{\quad }r@{\quad }r@{\quad }r} {{9.885}}&{} {{6.960}}&{} {{ -5.315}}&{} {{ 7.702}}&{} {{5.423}}&{} {{-4.142}} \\ {{6.960}}&{} {{4.902}}&{} {{-3.744}}&{} {{5.423}}&{} {{3.819}}&{} {{ -2.918}} \\ {{-5.315}}&{} {{-3.744}}&{} {{ 2.901}}&{} {{-4.142}}&{} {{-2.918}}&{} {{2.259}} \\ {{7.702}}&{} {{5.423}}&{} {{-4.142}}&{} {{6.003}}&{} {{ 4.226}}&{} {{-3.227}} \\ {{5.423}}&{} {{3.819}}&{} {{-2.918}}&{} {{4.226}}&{} {{2.977}}&{} {{-2.273}} \\ {{ -4.142}}&{} {{-2.918}}&{} {{2.259}}&{} {{-3.227}}&{} {{-2.273}}&{} {{1.762}} \\ \end{array} }} \right) \end{aligned}$$

Adopting \(c=3\), similar to the calculation steps above, 729 integer vectors were obtained. The estimation result of \(P_{m/\infty } \) is \(\hat{{P}}_{m/\infty } \approx 1\) and the corresponding error upper bound is \(\bar{\Delta } \hat{{P}}_{m/\infty } \approx \hbox {0}\). The top ten integer vectors and their corresponding posterior probabilities are listed in Table 2.

According to the above-mentioned calculation process and the results in Tables 1 and 2, we can conclude: (a) the adopted integer vectors can be directly obtained via formulas, without the searching step; (b) using the correction factor \(P_{m/\infty } \), the accuracy of \(\hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) can be improved; (c) the error upper bound of \(\hat{{P}}\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) can be obtained and the numerical example shows which is so small that it can almost be neglected; and (d) for both the high-precision float solution and the low-precision one, all the estimators of the posterior probabilities are very accurate.

7.2 Comparison with the success rate

To explain the differences between the posterior probability \(P(z_k =a\left| {\hat{{a}}} \right. )\) and the success rate \(P_{S_{\mathop {a}\limits ^{\smile }}} \left( {{\mathop {a}\limits ^{\smile }}=a} \right) \), the posterior probabilities and the success rates of \({{\mathop {a}\limits ^{\smile }}_{\mathrm{R}}=a}\), \({{\mathop {a}\limits ^{\smile }}_{\mathrm{B}}=a}\) and \({{\mathop {a}\limits ^{\smile }}=a}\) were calculated based on the experiment data.

Based on the DD observations of the carrier phase of 384 epochs in 32 min, the posterior probabilitis and the success rates of \({\mathop {a}\limits ^{\smile }}_{\mathrm{R}}=a\), \({{\mathop {a}\limits ^{\smile }}_{\mathrm{B}}}=a\) and \({{{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}}=a}\) are listed in Table 3.

Table 3 Comparison between the posterior probability and the success rate based on the observations of 384 epochs
Full size table

As can be seen from Table 3, under the current data,\({{\mathop {a}\limits ^{\smile }}_{\mathrm{R}}} ={{\mathop {a}\limits ^{\smile }}_{\mathrm{B}}} ={{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}}\) and \(\hat{{P}}\left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{R}} ={\mathop {a}\limits ^{\smile }}_{\mathrm{B}}=a\left| {\hat{{a}}} \right. } \right) =\hat{{P}}\left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{B}}=a\left| {\hat{{a}}} \right. } \right) =\hat{{P}}\left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}} ={{\mathop {a}\limits ^{\smile }}_{\mathrm{B}}=a\left| {\hat{{a}}} \right. } \right) \). However, based on the theory of success rate, under the current data, \(P_\mathrm{R} \left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{R}}={\mathop {a}\limits ^{\smile }}_{\mathrm{B}}=a}\right) \) < \(P_\mathrm{B} \left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{B}}=a}\right) \) < \({P_{\mathrm{ILS}}} \left( {{\mathop {a}\limits ^{\smile }}_{\mathrm{ILS}}={\mathop {a}\limits ^{\smile }}_{\mathrm{B}}=a} \right) \), which seem to be paradoxical. So, the posterior probability can reflect the probability of \({{\mathop {a}\limits ^{\smile }}=a}\) under the current data.

8 Concluding remarks

The float solution \(\hat{{a}}\) of DD ambiguity is a real vector of order n, but the DD ambiguity vector a should be an integer vector. Since a is unknown, any integer vector of order n is a probably. It is necessary to obtain the probability that an integer vector is DD ambiguity a based on probability and statistics theory. In this paper, an approach to calculate the posterior probability of \(z_k =a\), using the float solution \(\hat{{a}}\) and the corresponding variance matrix \(D_{\hat{{a}}\hat{{a}}}\), is proposed.

In the proposed approach, first, the space of integers \(Z^{n}\) is divided into two subspaces, one subspace consists of \(z_1 \), \(z_2 , \ldots z_m \), and the complementary space consists of \(z_{m+1} \), \(z_{m+2} , \ldots z_\infty \). Using the integer vectors \(z_1 \), \(z_2 , \ldots ,z_m \), a symmetric space \(S_{1\sim m} \in R^{n}\) can be structured, and the corresponding complementary space is \(S_{\left( {m+1} \right) \sim \infty } \). It should be noted that to avoid losing the strict dependence between the posterior probability \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) and the float solution \(\hat{{a}}\), every component of \(S_{1\sim m} \) should consist of more than one integer. The upper and lower bounds of \(S_{1\sim m} \) can be directly worked out by formulas, and the integer vectors \(z_1 \), \(z_2 , \ldots , z_m \) can be obtained by combination. The formula to calculate the correction factor \(P_{m/\infty } \), which involves the space \(S_{\left( {m+1} \right) \sim \infty } \), is transformed to only involve the space \(S_{1\sim m} \). Then, the estimator of \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) and the corresponding upper bound of the estimator error can be obtained. In this paper, the process of calculating \(P\left( {z_k =a\left| {\hat{{a}}} \right. } \right) \) and the derivations of the formulas have been presented in detail.

The proposed approach has the following advantages: (a) less calculation workload, which benefits from the ability that the integers of every component can be immediately obtained by calculation without searching, (b) high accuracy of result benefitting from the correction factor, and (c) strong adaptability for any float solutions with various levels of precision.